Unsupervised particle image velocity measurement method based on cross-correlation and H-RAMi attention mechanism

Abstract

The invention belongs to the field of particle image velocity measurement, and particularly relates to an unsupervised particle image velocity measurement method based on cross-correlation and H-RAMi attention mechanism. The model is structurally optimized on the basis of LiteFlowNet3, and a layered interactive attention mixing module and cross-correlation feature priori are introduced to enhance the feature extraction and matching capability of a network to a complex flow structure. Meanwhile, a boundary loss function is designed, and the estimation precision of a marginal region velocity field is remarkably improved. The model is trained through unsupervised loss, gets rid of dependence on large-scale labeled data, shows higher precision and robustness, and particularly has advantages in the aspects of small-scale vortex structure reconstruction and Gaussian noise interference resistance. The method can be applied to high-precision real-time measurement of a complex flow field, and has important application value for ocean exploration and fluid mechanics research.

Description

Technical Field

[0001] This invention belongs to the field of flow field measurement technology, specifically relating to an unsupervised particle image velocimetry (PIV) model that integrates cross-correlation algorithms. Based on LiteFlowNet3, the model undergoes structural optimization, introducing a hierarchical interactive attention hybrid module and cross-correlation feature priors to enhance the network's ability to extract and match features from complex flow structures. Simultaneously, a boundary loss function is designed to significantly improve the estimation accuracy of the velocity field in edge regions. This model is trained using unsupervised loss, eliminating its dependence on large-scale labeled data, and exhibits higher accuracy and robustness in both synthetic and real PIV experiments, particularly excelling in small-scale vortex structure reconstruction and resistance to Gaussian noise interference. This technology can be widely applied to high-precision real-time measurement of complex flow fields, and has significant application value in fluid mechanics research, marine exploration, and engineering monitoring. Background Technology

[0002] Flow visualization techniques play a crucial role in fluid mechanics research and the revelation of complex flow phenomena. Particle image velocimetry (PIV), as a typical non-contact quantitative measurement method for flow fields, has been widely used in experimental fluid mechanics. However, the quality of velocity field estimation in PIV (including resolution, accuracy, and computational efficiency) largely depends on the velocimetry algorithm employed.

[0003] Traditional PIV methods mainly fall into two categories: cross-correlation (CC) and optical flow (OF). The cross-correlation method assumes that the velocity and direction of local particles are consistent, obtaining the velocity vector by finding the maximum correlation within a fixed window. However, this assumption often fails to hold in regions with large velocity gradients or complex flow structures, resulting in sparse velocity field results and easily obscuring detailed structures. Furthermore, the matching window mechanism of this method has limited ability to capture small-scale vortex structures, and its computational speed is relatively slow, restricting its application in real-time measurements. The optical flow method, based on the assumption of constant brightness, can provide pixel-level velocity resolution, but it is sensitive to image noise, prone to local outliers, and performs poorly in large displacement scenarios. It also suffers from complex parameter tuning and high computational cost.

[0004] In recent years, deep learning technology has made groundbreaking progress in the application of PIV (Particle Image Processing). Supervised deep learning models (such as PIV-DCNN, CC-FCNN, PIV-NetS, PIV-LiteFlowNet, and LightPIVNet) outperform traditional methods in both accuracy and computational efficiency. However, these models typically rely on synthetic ground data generated by computational fluid dynamics (CFD) simulations for training. Synthetic data often fails to fully reflect the complex features of real particle images, causing biases in actual complex flow fields. Furthermore, the inconsistency between the distribution of training and testing data limits their generalization ability in diverse PIV scenarios.

[0005] To overcome the dependence of supervised learning methods on real-world labeled data, unsupervised deep learning methods (Unsup-DL) have gained increasing attention in the field of PIV (Portable Flow Analysis). These methods do not rely on real-world labels and are trained using only the information from the image itself, effectively mitigating the domain offset problem and exhibiting greater adaptability and practicality. A typical example is UnLiteFlowNet-PIV, which introduces a mature unsupervised optical flow estimation framework from computer vision into the PIV task and combines photometric consistency and spatial smoothness constraints to achieve relatively accurate flow field estimation without labeled data. However, it is still prone to matching errors in areas with particle occlusion, boundary motion, and low-texture or high-noise regions, and its ability to capture small-scale or rapidly changing flows is limited. Summary of the Invention

[0006] This invention is based on an unsupervised PIV model that integrates a cross-correlation algorithm and a deep optical flow network. The model is based on LiteFlowNet3 with structural optimizations, and introduces a hierarchical interactive attention hybrid (H-RAMi) module to enhance multi-scale feature fusion capabilities. The coarse velocity field calculated by the cross-correlation method is used as the initial velocity of the network decoding layer and fused into the costvolume to improve reconstruction accuracy. A new boundary loss function is designed to effectively alleviate the reconstruction uncertainty caused by particles flying out of the boundary, and the structural features of the image edge are preserved through a reflection filling strategy. Thus, the estimation accuracy, robustness and spatial resolution of the velocity field are significantly improved in complex flow fields.

[0007] While existing unsupervised particle image velocimetry methods overcome the shortcomings of supervised learning methods that rely on large-scale real flow field data, they still suffer from insufficient reconstruction accuracy and robustness when dealing with particle occlusion, edge mismatches, and complex small-scale flow structures, and their performance degrades significantly under noise interference. To address these issues, this invention proposes a high-resolution unsupervised particle image velocimetry model that integrates a cross-correlation algorithm and a deep optical flow network.

[0008] This invention optimizes the structure of the LiteFlowNet3 optical flow network. In the feature extraction stage, a hierarchical interactive attention hybrid module is introduced to effectively fuse multi-scale features, enhancing the perception of fine-grained features in particle images. In the decoding stage, a sparse velocity field calculated using traditional cross-correlation methods is incorporated as prior information into the cost volume for collaborative velocity field estimation, thereby improving reconstruction accuracy under complex motion patterns. Furthermore, this invention designs a boundary consistency loss function and integrates photometric loss, smoothness loss, and consistency loss to alleviate the flow field discontinuity problem caused by particles flying out of image boundaries. A reflection-filling strategy is employed in the convolution operation to preserve edge structure information. Through these improvements, this invention can achieve high-precision, high-resolution, and highly robust velocity field reconstruction without relying on real flow field labels. This makes it widely applicable in fields such as turbulence measurement, ocean flow field monitoring, and industrial flow detection.

[0009] To address the shortcomings of existing technologies, the present invention aims to propose an unsupervised particle image velocimetry method based on cross-correlation and H-RAMi attention mechanism. This method includes the following steps: S1. Obtain particle images from two adjacent frames of flow field data to form a dataset; S2. Construct the UnLECNet-PIV neural network based on the LiteFlowNet3 network, including an encoder network and a decoder network, denoted as NetC and NetE respectively; The NetC is built on the encoder of the LiteFlowNet3 network, which includes 6 cascaded convolutional modules. The difference between it and the encoder of the LiteFlowNet3 network is that a hierarchical interactive attention hybrid module, denoted as H-RAMi module, is set between the 3rd and 4th convolutional modules. The H-RAMi module upsamples the features obtained from the first three convolutional modules out of the six cascaded convolutional modules to a uniform resolution using a pixel shuffler. Then, it uses a lightweight convolutional structure to fuse the features, which alleviates the loss of particle displacement details caused by continuous downsampling during the hierarchical feature extraction process and fuses flow field information at different scales to maintain the consistency between local particle motion details and the overall flow field structure. The fused features are then input into the fourth convolutional module. The NetE is built based on the LiteFlowNet3 network decoder and is used to further process the features of the two images extracted by NetC. The difference between the decoder of NetE and LiteFlowNet3 is that two more layers of flow inference modules are added after the original pyramid structure of the LiteFlowNet3 decoder, as the 5th and 6th layers, so that NetE contains 6 layers of flow inference modules; the 6 flow inference modules correspond one-to-one with the 6 convolutional modules in NetC. In each level of decoding, the streaming inference module first performs deconvolution and upsampling on the optical flow prediction obtained from the previous layer, then concatenates it with the features output by the corresponding convolution module in NetC, and then performs feature matching, cost volume modulation and convolution operations to generate the optical flow estimate at the current resolution. The input to the 5th layer flow inference module is the output of the 4th layer flow inference module and the feature vector of the 2nd layer output of NetC. The input to the 6th layer flow inference module is the output of the 5th layer flow inference module and the feature vector of the 1st layer output of NetC. The difference between the decoders of the NetE and LiteFlowNet3 networks also lies in: The initial velocity field in the LiteFlowNet3 network is changed from zero to a sparse velocity field; the sparse velocity field is obtained by performing a cross-correlation algorithm on two adjacent frames of particle images that serve as inputs to the UnLECNet-PIV neural network. S3. Input the particle images of two adjacent frames from S1 into the UnLECNet-PIV neural network in the order of the first frame first and the second frame last to predict the positive flow field from the first frame to the second frame. ; The UnLECNet-PIV neural network is then input into the second frame first, followed by the first frame, to predict the reverse flow field from the second frame to the first frame. ; in These represent the horizontal velocity components in the forward and reverse flow fields, respectively. These represent the vertical velocity components in the forward and reverse flow fields, respectively; Based on the forward and reverse flow fields, an unsupervised loss function is constructed to train the UnLECNet-PIV neural network.

[0010] Preferably, the unsupervised loss function It consists of a weighted average of photometric loss, smoothing loss, consistency loss, and boundary loss. ; in, Represents luminosity loss, Represents smoothing loss. Represents consistency loss, Represents boundary loss; The construction of the boundary loss includes the following steps: The boundary region refers to the area formed by the four outermost pixel bands on the top, bottom, left, and right sides in the final PIV image output by NetE; The four outermost pixel bands and the four inner pixel bands of the same size as the corresponding pixel bands in the boundary area form an adjacent region; Boundary loss is defined as the loss in the four directions: up, down, left, and right. Average value: ; The loss in each direction is used to measure the optical flow in the boundary region. Optical flow in its adjacent inner region The differences between them, among which The optical flow vectors of the corresponding boundary regions are extracted directly from the PIV image output by NetE. Similarly... The optical flow vectors of corresponding neighboring regions are directly extracted from the PIV image output by NetE, and then calculated using the generalized Charbonnier loss function. ; The hyperparameters are set to , . Preferably, the lightweight convolutional structure is the MobileNet Variants network.

[0011] Preferably, the construction of the photometric loss includes: First frame Second frame Input UnLECNet-PIV to obtain the forward flow field. Based on the forward flow field Second frame Perform warping transformation (Warp) The operation yields the predicted value for the first frame; similarly, the predicted value for the second frame after changing its order is obtained. and the first frame Input UnLECNet-PIV to obtain the prediction value for the second frame; Based on the predicted value of the first frame, the forward luminance loss is obtained; based on the predicted value of the second frame, the reverse luminance loss is obtained; the bidirectional luminance loss is the sum of the two, as shown in the following formula: ; Where P is the set of pixel coordinates, and X represents pixel coordinates, and... Alignment of coordinate grids, It is a generalized Charbonnier penalty function. ;Will Set to 0.45, Set as .

[0012] Preferably, the smoothing loss employs a second-order smoothing constraint, which is defined as follows: ;

[0013] Where P is the set of pixel coordinates, X is the coordinate of the current pixel, the outer summation in the above formula represents the calculation for each pixel X in the image domain P consisting of the set of pixel coordinates, N(X) is the neighborhood of each pixel X, and s and r represent two adjacent pixels of X, s and r are located on both sides of X respectively. This is the Charbonnier penalty function.

[0014] Preferably, the consistency loss is defined as follows: ;

[0015] Where P is the set of pixel coordinates, and X represents the pixel coordinates. For each pixel, the forward optical flow is first used... Map it to the corresponding position in the second frame. Then locate the reverse optical flow at that position. and Add; This represents the Charbonnier penalty function.

[0016] The beneficial effects of this invention are as follows: This invention proposes an unsupervised particle image velocimetry model (UnLECNet-PIV) that integrates cross-correlation algorithms and hierarchical interactive attention mechanisms. Based on LiteFlowNet3, it optimizes the structure by introducing cross-correlation feature priors and a multi-level feature fusion strategy, significantly improving the accuracy and robustness of velocity field reconstruction under complex flow conditions. This method addresses the shortcomings of traditional cross-correlation methods in high-velocity gradient regions and the high dependence on labeled data in supervised learning methods, providing a novel approach for high-precision flow field estimation without requiring ground-based labels.

[0017] By fusing the coarse velocity field with the cost volume of the deep optical flow network's decoding layer, the model effectively improves feature matching quality and physical consistency. Simultaneously, the hierarchical interactive attention hybrid module enhances the perception of fine-grained features in particle images, mitigating information loss caused by pyramid downsampling. The newly designed boundary loss function reduces velocity field discontinuities caused by particles flying out of image boundaries, resulting in better performance in flow field prediction at edge regions. Compared to the traditional PIV algorithm, this invention significantly reduces the average endpoint error on both synthetic and real datasets and exhibits higher stability and spatial resolution in complex scenarios such as small-scale vortex structures, Gaussian noise interference, and particle occlusion. In real-world experimental environments, the reconstructed velocity field demonstrates higher physical consistency, showcasing superior performance compared to some supervised learning methods and strong application potential, particularly suitable for practical applications such as ocean observation, turbulence research, and industrial flow field measurement. Attached Figure Description

[0018] Figure 1 This is the definition of the boundary region and adjacent region in the PIV image used for boundary loss calculation in this invention; Figure 2 This is the layered interactive attention hybrid module used in this invention; Figure 3 This is the unsupervised loss calculation process in this invention. Detailed Implementation

[0019] The invention will now be further described with reference to the accompanying drawings.

[0020] This invention employs unsupervised deep learning to address the lack of real ground flow field data in real particle image velocimetry (PIV) tasks. It proposes an unsupervised particle image velocimetry model (UnLECNet-PIV) that integrates cross-correlation feature priors and a hierarchical interactive attention mechanism. This model optimizes the LiteFlowNet3 network structure, utilizes multi-scale feature fusion to enhance fine-grained feature extraction capabilities for complex flow fields, and combines cross-correlation calculation results to improve the accuracy and physical consistency of velocity field estimation. Simultaneously, a boundary loss function is introduced to effectively mitigate the flow field discontinuity caused by particles flying out of image boundaries. This method demonstrates strong robustness and spatial resolution in complex environments such as small-scale vortex structures, high noise interference, and particle occlusion, and can be widely applied in fields such as ocean flow field monitoring, turbulence research, and industrial flow detection. The method includes the following steps: Step 1: Prepare the dataset The dataset used in this invention originates from the simulated particle image velocimetry dataset provided by Cai et al. This dataset is based on computational fluid dynamics methods, artificially generating particle image pairs under various flow field conditions. The dataset consists of particle image pairs under multiple flow field conditions, covering flow field types including: uniform flow, horizontal flow, backward step flow, flow around a cylinder, DNS turbulence, SQG surface flow, JHTDB-channel flow, JHTDB-mhd1024 turbulence, and JHTDB-isotropic1024 turbulence. Specifically, DNS turbulence is a uniform and isotropic flow; SQG surface flow is an ocean current driven by a quasi-geostrophic model; JHTDB-channel flow data comes from channel flow data in the Johns Hopkins Turbulence Database; JHTDB-mhd1024 turbulence originates from magnetohydrodynamic turbulence in this database; and JHTDB-isotropic1024 turbulence data is forced isotropic turbulence data provided by this database.

[0021] In the experiment, 90% of the dataset was used for model training, and the remaining 10% was used for model testing, in order to ensure the independence of training and testing and the reliability of the results.

[0022] Step 2: Define the unsupervised loss function Two adjacent particle image pairs are used as input to a neural network to predict the forward flow field from the first frame to the second frame, which includes velocity components in the horizontal and vertical directions. To improve estimation accuracy, the reverse flow field prediction from the second frame to the first frame is also calculated.

[0023] (2.1) Photometric loss Photometric loss is used to measure the difference between the first frame after forward flow field prediction and the second frame after warping transformation, as well as the difference between the second frame after reverse flow field prediction and the first frame after warping transformation.

[0024] Specifically, the first frame Second frame Input UnLECNet-PIV to obtain the forward flow field. Based on the forward flow field Second frame A warp transformation is performed to obtain the predicted value for the first frame; similarly, the predicted value for the second frame after changing its order is obtained. and the first frame Input UnLECNet-PIV to obtain the prediction value for the second frame.

[0025] Based on the predicted value of the first frame, the forward luminance loss is obtained; based on the predicted value of the second frame, the reverse luminance loss is obtained; the bidirectional luminance loss is the sum of the two, as shown in the following formula: ; Where P is a set of pixel coordinates, and X represents pixel coordinates, usually related to... Alignment of coordinate grids, It is a generalized Charbonnier penalty function. In this invention we will Set to 0.45, Set as .

[0026] (2.2) Smoothing loss Because some particles may have missing boundaries or appear newly in two frames, the velocity field can easily become discontinuous locally. Therefore, a smoothing loss constraint is introduced to enhance the spatial consistency of the flow field prediction. This paper adopts a second-order smoothing constraint, defined as follows: ;

[0027] This second-order smoothness loss function constrains the spatial variation of the optical flow field, ensuring its smoothness even in areas with missing particles or boundary regions. The outer summation of the formula represents the calculation for each pixel position X in the image domain P, where P is the set of pixel coordinates and X is the coordinate of the current pixel. For each pixel coordinate X, we consider its neighborhood N(X), which is the set of pixel directions defined by four convolutional filters: the horizontal direction (x-direction), the vertical direction (y-direction), and two diagonal directions. In each direction, two adjacent pixels s and r, located on either side of X, are selected to calculate the second-order difference. The second-order difference term F(s)−2F(X)+F(r) in the formula measures the curvature or trend of the optical flow value in a certain direction. If the optical flow is smooth in that direction, this value will be close to zero; if the change is drastic (e.g., particle disappearance or boundary movement), this value will be large. To suppress the effects of large errors and maintain sensitivity to small errors when calculating the loss, we substitute the results of the second-order difference into the generalized Charbonnier penalty function. The first part of the formula Corresponding to the smoothness constraint of the forward optical flow, the latter part The smoothness constraint corresponds to the reverse optical flow. Finally, the losses in the four directions are summed to obtain the overall second-order smoothness loss.

[0028] (2.3) Consistency loss Ideally, the forward estimate and the backward estimate should be consistent, i.e. and Since the absolute values ​​of the numbers are equal and opposite in sign, their sum should be zero. Based on this, a consistency loss function can be constructed: ;

[0029] In the formula, P represents the set of pixel coordinates in the image domain. For each pixel X, we first utilize the forward optical flow... Map it to the corresponding position in the second frame. Then locate the reverse optical flow at that position. and Add them together. If the two are exactly the same (opposite directions and equal values), the sum should be zero; otherwise, it will produce a non-zero value. To quantify this discrepancy, we substitute it into the generalized Charbonnier penalty function.

[0030] The first term of the formula The calculation is based on the consistency error obtained from the forward optical flow; the second term This is a symmetrical calculation derived from the reverse optical flow. By adding these two results and summing them over the entire image, the global consistency loss value is obtained. During training, this loss makes the forward and backward optical flows output by the network more symmetrical and consistent, thereby improving the accuracy and robustness of optical flow estimation.

[0031] (2.4) Boundary loss This paper proposes a boundary loss mechanism to constrain the flow consistency in image boundary regions, thereby effectively mitigating the particle fly-out problem and improving the accuracy of optical flow estimation in boundary regions. This is achieved by explicitly delineating the boundary region with its adjacent neighboring regions, such as... Figure 1 As shown. The boundary region refers to the PIV image, i.e. and The outermost pixel bands (top, bottom, left, right) surround the boundary region, while the adjacent regions are the inner pixel bands immediately following the boundary. This design aims to encourage the optical flow vector in the boundary region to maintain continuity and smoothness with its neighboring regions in both magnitude and direction, thereby mitigating discontinuities caused by missing contextual information or edge padding. The boundary loss is defined as the average of the losses in the four directions: ;

[0032] The loss in each direction is used to measure the optical flow in the boundary region. Optical flow in its adjacent inner region The differences between them, among which Similarly, the optical flow vectors of the corresponding boundary regions are directly extracted from the optical flow predicted by the network. It directly extracts the optical flow vectors of corresponding neighboring regions from the optical flow predicted by the network and calculates them using the generalized Charbonnier loss function: ;

[0033] The hyperparameters are set to , .

[0034] (2.5) Final loss function The final loss function is the sum of the three loss functions multiplied by their corresponding weights, and is defined as follows: ;

[0035] Step 3: Image feature extraction using UnLECNet-PIV neural network (3.1) Attention mechanism After the particle image pairs undergo the first few layers of feature extraction, the multi-scale features extracted from the first three layers are input into the Hierarchical Reciprocal Attention Mixing (H-RAMi) module, such as... Figure 2 As shown, this module aims to mitigate the loss of particle displacement details caused by continuous downsampling during hierarchical feature extraction, while fusing flow field information at different scales to maintain consistency between local particle motion details and the overall flow field structure. Specifically, H-RAMi first uses a pixel shuffler to upsample the first three levels of features to a uniform high resolution (256 × 256), and then uses MobiVari (MobileNet Variants) to effectively aggregate and fuse them. The fused features not only preserve the fine-grained spatial structure but also enhance high-level semantic expression, and are finally input into the fourth-layer feature extraction module as a key support for subsequent feature learning and optical flow estimation.

[0036] (3.2) Internal structure of unsupervised loss Compared to traditional supervised learning-based loss functions, the model proposed in this paper predicts not only the velocity field from the first frame to the second frame, but also the velocity field from the second frame to the first frame for input neighboring particle image pairs, thus achieving bidirectional prediction in both forward and backward directions. Figure 3 As shown, the light purple part represents forward prediction and the red part represents backward prediction. The model uses a backbone network with shared weights and constructs an unsupervised loss function by combining the forward and backward prediction results.

[0037] (3.3) Network Structure The model presented in this paper is named UnLECNet-PIV, and the backbone network used is LiteFlowNet3. It uses a feature extraction neural network (NetC) with shared weights to extract features from two images. NetC has a pyramid structure. In the decoder (NetE), unlike LiteFlowNet3 which sets the initial velocity field to zero, this paper introduces a sparse velocity field calculated based on a cross-correlation algorithm as the initial flow field. This design provides the decoder with prior information, significantly reducing the learning difficulty of the decoder, enabling it to more accurately propagate and correct optical flow, and effectively improving the accuracy and robustness of the final optical flow estimation. After obtaining the initial velocity field, the network enters the flow inference module in the optical flow estimation module to perform progressive upsampling and optimization of the velocity field, thereby gradually improving its accuracy and spatial resolution. To obtain the final high-resolution velocity field output, this invention extends the structure based on the original LiteFlowNet3 NetE module. Since the original NetE module only contains a four-level pyramid structure, the output resolution is 64×64, which cannot meet the requirements for fine estimation of the particle image velocity field. To address this, this paper adds two new flow inference modules at the end of NetE, namely the 5th and 6th layers. The input of the 5th layer flow inference module is the output of the 4th layer flow inference module and the feature vector of the 2nd layer of NetC. The input of the 6th layer flow inference module is the output of the 5th layer flow inference module and the feature vector of the 1st layer of NetC. This ultimately improves the output resolution to 256×256, making it more suitable for high-precision particle image velocimetry (PIV) tasks.

[0038] The input for the forward flow estimation is (I1, I2), and the input for the backward flow is (I2, I1).

[0039] Step 4: Comparative Analysis of Experimental Results (4.1) Evaluation indicators To quantitatively evaluate the model's performance in the velocity field estimation task, this paper uses the Average Endpoint Error (AEE) as the primary evaluation metric. This metric measures the average Euclidean distance between the reconstructed velocity field and the true velocity field, and its calculation formula is as follows: ;

[0040] in, This indicates that the model reconstructs the velocity field at the pixel location. The horizontal and vertical components at that location, , representing the component of the real velocity field at the corresponding location. It is a norm 2. Let N be the set of all valid pixels, and N be the total number of pixels. Considering that the AEE value of a single pixel is usually small and not easy to compare intuitively, this paper further uses the average endpoint error per 100 pixels for statistical analysis and visualization to enhance the readability of the error analysis, thereby more clearly reflecting the model's estimation accuracy in space. The smaller the AEE value, the closer the reconstructed velocity field is to the true velocity field, and the better the model performance.

[0041] (4.2) Comparison between the model of this invention and the PIV method based on unsupervised learning To further verify the performance advantages of the proposed UnLECNet-PIV network, this paper compares it with several existing PIV neural network methods based on unsupervised learning. The results are summarized in Table 1.

[0042] Table 1: Experimental results compared with unsupervised deep learning methods

[0043] Experimental results show that UnLECNet-PIV achieves the lowest average endpoint error (AEE) across five typical test datasets (Cylinder, Back-step, JHTDB_channel, DNS_turbulence, and SQG), demonstrating excellent overall performance. Compared to UnLiteFlowNet-en, UnLECNet-PIV achieves an accuracy improvement of approximately 12%–30% across various flow fields, fully demonstrating its robustness and generalization ability under different flow conditions. This performance improvement is mainly attributed to several optimizations in the network structure. First, UnLECNet-PIV introduces a hierarchical interactive attention hybrid module in the encoder, significantly enhancing the perception and representation of fine-grained features in particle images. Second, the decoder incorporates prior information from the traditional cross-correlation method in PIV, which helps improve the physical rationality and robustness of displacement estimation. Furthermore, the improved unsupervised loss function further enhances the constraint on edge structures, effectively improving training stability and estimation accuracy. Compared to the state-of-the-art unsupervised PIV network URAFT-PIV, the proposed method exhibits lower average endpoint error (AEE) in the Cylinder and Back-step flow fields, while its error is slightly higher in other flow fields. However, it's worth noting that URAFT-PIV is highly computationally dependent, typically requiring tens of hours or even longer to train in a multi-GPU parallel computing environment, which is costly in practical applications. Therefore, the network proposed in this paper shows greater potential for real-world applications.

Claims

1. An unsupervised particle image velocimetry method based on cross-correlation and H-RAMi attention mechanism, characterized in that, The method includes the following steps: S1. Obtain particle images from two adjacent frames of flow field data to form a dataset; S2. Construct the UnLECNet-PIV neural network based on the LiteFlowNet3 network, including an encoder network and a decoder network, denoted as NetC and NetE respectively; The NetC is built on the encoder of the LiteFlowNet3 network, which includes 6 cascaded convolutional modules. The difference between it and the encoder of the LiteFlowNet3 network is that a hierarchical interactive attention hybrid module, denoted as H-RAMi module, is set between the 3rd and 4th convolutional modules. The H-RAMi module upsamples the features obtained from the first three convolutional modules out of the six cascaded convolutional modules to a uniform resolution using a pixel shuffler. Then, it uses a lightweight convolutional structure to fuse the features, which alleviates the loss of particle displacement details caused by continuous downsampling during the hierarchical feature extraction process and fuses flow field information at different scales to maintain the consistency between local particle motion details and the overall flow field structure. The fused features are then input into the fourth convolutional module. The NetE is built based on the LiteFlowNet3 network decoder and is used to further process the features of the two images extracted by NetC. The difference between the decoder of NetE and LiteFlowNet3 is that two more layers of flow inference modules are added after the original pyramid structure of the LiteFlowNet3 decoder, as the 5th and 6th layers, so that NetE contains 6 layers of flow inference modules; the 6 flow inference modules correspond one-to-one with the 6 convolutional modules in NetC. In each level of decoding, the streaming inference module first performs deconvolution and upsampling on the optical flow prediction obtained from the previous layer, then concatenates it with the features output by the corresponding convolution module in NetC, and then performs feature matching, cost volume modulation and convolution operations to generate the optical flow estimate at the current resolution. The input to the 5th layer flow inference module is the output of the 4th layer flow inference module and the feature vector of the 2nd layer output of NetC. The input to the 6th layer flow inference module is the output of the 5th layer flow inference module and the feature vector of the 1st layer output of NetC. The difference between the decoders of the NetE and LiteFlowNet3 networks also lies in: The initial velocity field in the LiteFlowNet3 network is changed from zero to a sparse velocity field; the sparse velocity field is obtained by performing a cross-correlation algorithm on two adjacent frames of particle images that serve as inputs to the UnLECNet-PIV neural network. S3. Input the particle images of two adjacent frames from S1 into the UnLECNet-PIV neural network in the order of the first frame first and the second frame last to predict the positive flow field from the first frame to the second frame. ; The UnLECNet-PIV neural network is then input into the second frame first, followed by the first frame, to predict the reverse flow field from the second frame to the first frame. ; in These represent the horizontal velocity components in the forward and reverse flow fields, respectively. These represent the vertical velocity components in the forward and reverse flow fields, respectively. Based on the forward and reverse flow fields, an unsupervised loss function is constructed to train the UnLECNet-PIV neural network.

2. The unsupervised particle image velocimetry method based on cross-correlation and H-RAMi attention mechanism as described in claim 1, characterized in that, The unsupervised loss function Weighted by photometric loss, smoothing loss, uniformity loss, and boundary loss composition: ； in, Represents luminosity loss, Represents smoothing loss. Represents consistency loss, Represents boundary loss; The construction of the boundary loss includes the following steps: The boundary region refers to the area formed by the four outermost pixel bands on the top, bottom, left, and right sides in the final PIV image output by NetE; The four outermost pixel bands and the four inner pixel bands of the same size as the corresponding pixel bands in the boundary area form an adjacent region; Boundary loss is defined as the loss in the four directions: up, down, left, and right. Average value: ； The loss in each direction is used to measure the optical flow in the boundary region. Optical flow in its adjacent inner region The differences between them, among which The optical flow vectors of the corresponding boundary regions are extracted directly from the PIV image output by NetE. Similarly... The optical flow vectors of corresponding neighboring regions are directly extracted from the PIV image output by NetE, and then calculated using the generalized Charbonnier loss function. ； The hyperparameters are set to = 0.45、 .

3. The unsupervised particle image velocimetry method based on cross-correlation and H-RAMi attention mechanism as described in claim 1, characterized in that, The lightweight convolutional structure is the MobileNet Variants network.

4. The unsupervised particle image velocimetry method based on cross-correlation and H-RAMi attention mechanism as described in claim 2, characterized in that, The construction of the photometric loss includes: First frame Second frame Input UnLECNet-PIV to obtain the forward flow field. Based on the forward flow field Second frame Perform warping transformation (Warp) The operation yields the predicted value for the first frame; similarly, the predicted value for the second frame after changing its order is obtained. and the first frame Input UnLECNet-PIV to obtain the prediction value for the second frame; Based on the predicted value of the first frame, the forward luminance loss is obtained; based on the predicted value of the second frame, the reverse luminance loss is obtained; the bidirectional luminance loss is the sum of the two, as shown in the following formula: ； Where P is the set of pixel coordinates, and X represents pixel coordinates, and... Alignment of coordinate grids, It is a generalized Charbonnier penalty function. ;Will Set to 0.45, Set as .

5. The unsupervised particle image velocimetry method based on cross-correlation and H-RAMi attention mechanism as described in claim 2, characterized in that, The smoothing loss employs a second-order smoothing constraint, defined as follows: ； Where P is the set of pixel coordinates, X is the coordinate of the current pixel, the outer summation in the above formula represents the calculation for each pixel X in the image domain P consisting of the set of pixel coordinates, N(X) is the neighborhood of each pixel X, and s and r represent two adjacent pixels of X, s and r are located on both sides of X respectively. This is the Charbonnier penalty function.

6. The unsupervised particle image velocimetry method based on cross-correlation and H-RAMi attention mechanism as described in claim 2, characterized in that, The consistency loss is defined as follows: ； Where P is the set of pixel coordinates, and X represents the pixel coordinates. For each pixel, the forward optical flow is first used... Map it to the corresponding position in the second frame. Then locate the reverse optical flow at that position. and Add; This represents the Charbonnier penalty function.

Images