Multi-scale Res2RAFT particle image velocity measurement method based on global attention mechanism optimization

Abstract

The invention relates to the field of particle image velocimetry (PIV), in particular to a multi-scale Res2RAFT particle image velocimetry method based on global attention mechanism optimization. The invention aims to solve the problem that an existing optical flow network is insufficient in prediction of a fine-grained displacement field in a complex flow field, especially in application in multi-scale vortex and turbulent flow fields. By designing an optical flow estimation structure with multi-scale feature extraction and global information interaction, Res2RAFT can more accurately capture fine flow features in a complex flow field, and the flow field prediction precision and adaptability in a PIV task are improved. According to the Res2RAFT particle image velocity measurement method provided by the invention, the defects of a traditional PIV method in processing an underwater complex flow field are overcome, high-resolution displacement field prediction of the flow field is realized through fine multi-scale feature extraction and a global attention mechanism, and the Res2RAFT particle image velocity measurement method has a wide application prospect.

Description

Technical Field

[0001] The present invention relates to the field of particle image velocimetry (PIV), and in particular to a multi-scale Res2RAFT particle image velocimetry method based on global attention mechanism optimization. Background Art

[0002] Particle image velocimetry (PIV) is a commonly used flow field measurement method in experimental fluid mechanics and is widely used to reveal the details of fluid motion. However, traditional PIV methods have limitations in measurement accuracy and adaptability in complex flow fields, especially in underwater or turbulent environments. Due to insufficient illumination and irregular flow fields, traditional cross-correlation methods and variational optical flow methods are difficult to capture subtle flow structures.

[0003] The cross-correlation method is one of the most commonly used methods of PIV. It estimates the displacement field of the fluid by finding the maximum correlation of the local window in the continuous image pair. This method works well in simple flow fields, but due to the fixed window size, it is difficult to adapt to flow structures of different scales, resulting in the inability to effectively capture the subtle flow structures in complex flow fields. At the same time, the velocity field generated by this method is relatively sparse, making it difficult to achieve fine-grained tracking of the fluid. The variational optical flow method solves the flow field displacement through partial differential equations and can generate a dense velocity field. However, the calculation process of this method is complex, the computational cost is high, and errors are prone to occur in areas where the gradient changes dramatically in the flow field. The variational optical flow method usually relies on physical assumptions and prior knowledge. This limitation is difficult to meet the dynamic characteristics of complex flow fields in practical applications, especially in high-gradient and nonlinear flow fields, and it is easy to lose accuracy.

[0004] In recent years, with the development of deep learning, optical flow networks have gradually been introduced into PIV tasks, but existing networks still face many challenges in fine-grained prediction under complex flow fields.

[0005] With the development of computer vision and deep learning, convolutional neural networks (CNNs) have been widely used in PIV tasks to process particle images in an end-to-end manner. These deep learning models, such as optical flow estimation networks such as FlowNet and LiteFlowNet, have improved optical flow prediction to a certain level, but there are still some shortcomings: the FlowNet series of networks introduced an end-to-end convolutional network structure in optical flow estimation, which has a high dense optical flow prediction capability. However, FlowNet2 has a complex structure, high computing resource consumption, and a long model training time. At the same time, its ability to capture multi-scale information is limited, resulting in insufficient prediction accuracy and generalization ability when dealing with complex turbulent flow fields. LiteFlowNet adopts a lightweight design to reduce the computational cost, but due to the reduction in the number of parameters, the model performs poorly in high-resolution fine-grained predictions, and it is difficult to take into account both prediction accuracy and model efficiency at the same time, especially in complex flow fields, where the ability to capture fine structures is weak. As a recursive optical flow estimation network, RAFT calculates the correlation volume through full-pair feature matching to achieve efficient and accurate flow field estimation. Although RAFT performs well in dense optical flow estimation, its multi-scale feature extraction capability is limited and there is still room for improvement in the prediction of complex and fine-grained flow structures, especially in flow fields with multi-scale vortex structures, where the local features of the model are insufficiently expressed.

[0006] The Res2Net network is a convolutional neural network with multi-scale feature extraction capabilities. Its Bottle2neck module achieves layer-by-layer extraction and fusion of multi-scale features through fine-grained feature division and parallel convolution operations. The structural innovation of Bottle2neck is reflected in the following two points. First, multi-scale parallel convolution: The Bottle2neck module divides the input features into several scales, performs parallel convolution operations on each scale, and fuses features of different scales. This method improves the model's ability to capture fine-grained flow information, and can still retain rich multi-scale features even in the case of low image resolution. The second is feature hierarchical combination: The module gradually combines feature maps of different scales in the convolution operation, so that the network maximizes the use of multi-scale information while maintaining computational efficiency. This feature combination method can help the model maintain sensitivity to local and global information when processing multi-scale motion information in particle images, and adapt to the prediction needs of complex flow fields in PIV tasks.

[0007] GAM is a global attention mechanism that mixes channels and spaces, which can enhance the model's sensitivity to feature selection. Its core lies in the selective enhancement of input features through channel attention and spatial attention. The channel attention mechanism strengthens the channel dependency between features by weighting the channel dimension of the feature map. In the PIV task, channel attention can effectively distinguish the flow field features in different particle images and suppress unimportant feature information. Spatial attention weights the spatial dimension of the feature map, and further optimizes the model's focus on flow characteristics by focusing on the significant feature areas in the flow field. GAM's channel and spatial attention work together to effectively focus global information on important features, significantly improving the ability to capture details in irregular areas such as eddies and turbulence in complex flow fields.

[0008] Existing optical flow network models, such as FlowNet, LiteFlowNet, and RAFT, have demonstrated certain optical flow estimation performance, but there is still room for improvement in accuracy and multi-scale feature extraction capabilities. Therefore, it is of great significance to propose a multi-scale optical flow estimation network based on deep learning to better handle complex flow fields. Summary of the invention

[0009] The purpose of the present invention is to provide a high-precision optical flow estimation method for particle image velocimetry (PIV), namely, a Res2RAFT model based on Res2Net multi-scale feature extraction and GAM global attention mechanism. This method aims to solve the problem that the existing optical flow network is insufficient in predicting fine-grained displacement fields in complex flow fields, especially in applications in multi-scale vortex and turbulent flow fields. By designing an optical flow estimation structure with multi-scale feature extraction and global information interaction, Res2RAFT can more accurately capture subtle flow features in complex flow fields and improve the flow field prediction accuracy and adaptability in PIV tasks.

[0010] Particle image velocimetry (PIV) is an important method for studying flow field distribution in experimental fluid mechanics, and provides key technical support for revealing fluid dynamics and velocity field distribution. Especially in complex underwater flow field environments such as deep-sea plumes, the PIV method helps to observe particle flow and diffusion behavior, which is of great significance for studying the underwater particle flow field state and its impact on the deep-sea ecological environment. The particle motion characteristics in deep-sea plumes are complex and the flow structure is subtle. The traditional PIV method has limitations in resolution and accuracy when observing such fine-grained dynamics.

[0011] The Res2RAFT model of the present invention enhances the multi-scale feature extraction capability by introducing the Bottle2neck module in the Res2Net network, and combines the GAM global attention mechanism to achieve accurate capture of complex underwater flow fields. This method can effectively extract flow features in particle images at different scales, thereby more accurately predicting the displacement field distribution in the flow field. Res2RAFT will provide an efficient and reliable particle image velocimetry solution for complex underwater flow fields including deep-sea plumes, meet the needs of detailed observation of the underwater particle flow field state, and provide technical support for marine environmental monitoring and protection.

[0012] The present invention provides a multi-scale Res2RAFT particle image velocimetry method based on global attention mechanism optimization, comprising the following steps:

[0013] S1. Obtain PIV dataset and data preprocessing:

[0014] Each data sample in the PIV data set includes two consecutive frames of particle images and the actual particle velocity corresponding to the particle image;

[0015] S2. Construct a particle image velocimetry model, wherein the particle image velocimetry model is based on a RAFT optical flow network.

[0016] The RAFT optical flow network includes a feature extraction network, a context network, a full pair correlation layer, and a GRU recursive update module;

[0017] Replace the feature extraction network and context network of the RAFT optical flow network with the Res2encoder network.

[0018] Obtain a particle image velocimetry model;

[0019] The Res2Encoder network includes: convolutional layer 1, RA-Bottle2neck module, RB-

[0020] Bottle2neck module, RA-Bottle2neck module, RB-Bottle2neck module, RA-Bottle2neck module, RB-Bottle2neck module, convolutional layer 2;

[0021] The convolution layer 1 uses a 7×7 convolution layer to generate an initial feature map of 64 channels;

[0022] The convolutional layer 2 uses a 1×1 convolution to generate an output feature map of 128 channels;

[0023] The RA-Bottle2neck module is a bottleneck structure with residual learning based on the Bottle2neck structure, which is used for downsampling to maintain multi-scale information. A multi-scale feature processing method is introduced in each residual block, and the input feature map of the Res2Encoder module is divided into s subsets. As shown in the following formula, each subset performs a convolution operation independently, and extracts features at several scales to increase the diversity of the feature space:

[0024]

[0025] For the first subset x i , without convolution operation, directly retain the original input as output y i ; The second subset x2 undergoes a standard 3×3 convolution operation K2(x2) to obtain the output y2; starting from the third subset x3, the convolution operation not only depends on the input x3, but also combines the output y2 of the previous group, that is, K3(x3+y2), thereby fusing information from different scales layer by layer; the layer-by-layer accumulation and recursive feature extraction method continues until the last subset x s The processing is completed;

[0026] The RB-Bottle2neck module is a bottleneck structure with a global attention mechanism based on the Bottle2neck structure, which is used to maintain resolution; the RB-Bottle2neck module adds a global attention mechanism GAM on the basis of the RA-Bottle2neck module, and the global attention mechanism GAM is added after the last layer of the RA-Bottle2neck module; and the global attention mechanism GAM in the RB-Bottle2neck module includes channel attention and spatial attention modules, which utilize the interaction of channel and spatial dimensions to enable the particle image velocimetry model to focus on particle flow information in two consecutive frames;

[0027] S3 constructs a loss function and uses the PIV dataset described in step S1 to train the particle image velocimetry model through supervised learning.

[0028] Preferably, in step S1, the data preprocessing specifically includes: uniformly processing the particle image size in the PIV data set, setting the length and width to 256×256, and then regularizing the image to ensure that it is compatible with the input of Res2Encoder.

[0029] Preferably, in step S2, s=4, and the stride and baseWidth parameters of the RA-Bottle2neck module are set to 2 and 26 respectively.

[0030] Preferably, in step S2, the stride and baseWidth parameters of the RB-Bottle2neck module are set to 1 and 26 respectively.

[0031] Preferably, in step S3, the loss function formula is as follows:

[0032]

[0033] Where n is the number of samples in the PIV dataset, F i is the optical flow prediction value obtained by inputting two consecutive frames of particle images of the sample in the PIV dataset into the particle image velocimetry model, and F gt It is the real optical flow value reflecting the real speed of particles in the samples in the PIV dataset;

[0034] Preferably, i =0.8 n-i It is the weight of each prediction, giving greater weight to later predictions to improve the accuracy of the final result.

[0035] Preferably, the all-pair correlation layer is used to perform all-pair correlation calculation on the features extracted by the feature extraction network to obtain a four-layer correlation pyramid;

[0036] The all-pair correlation layer establishes the association between pixels by calculating the similarity between each pixel in the two frames. The similarity calculation is performed by constructing a 4D correlation volume according to the following formula, which stores the inner product of the feature vectors between all pixel pairs:

[0037] C(g θ (I1),g θ (I2))∈R H×W×H×W ,C ijkl =∑hg θ (I1)ijh·g θ (I2)klh

[0038] In the formula, C is the correlation volume, g θ (I1) and g θ (I2) is the feature map extracted by the feature encoder of the two frames;

[0039] C ijkl It represents the weight coefficients of the i-th, j-th spatial dimensions (such as the width and height of the convolution kernel) and the k-th, l-th channel dimensions in the convolution kernel.

[0040] h is the input of the convolution calculation, which is the value of the local area of ​​the input image or the feature map of the previous layer. It is part of the convolution operation, and the convolution kernel extracts features through it.

[0041] This formula indicates that each pixel pair in the two images (i.e., (i, j) and (k, l)) obtains their similarity score by calculating the inner product;

[0042] The four-layer correlation pyramid is constructed by performing a pooling operation on the correlation volume.

[0043] Preferably, the GRU recursive update module uses convolutional GRU and separation convolutional GRU to recursively update features and gradually adjust the flow field prediction. In each recursion, the GRU updates the features according to the input correlation features,

[0044] The current optical flow estimation and hidden state retain important historical information through the update gate and reset gate, and at the same time, the optical flow increment is calculated based on the current features, and the optical flow is iteratively updated;

[0045] The separable convolution GRU captures more fine-grained feature information through separable convolution in both horizontal and vertical directions, making the optical flow estimation more accurate.

[0046] The present invention has the beneficial effects:

[0047] The present invention provides a high-precision optical flow estimation method for particle image velocimetry (PIV), which enhances the model's perception of complex flow fields through multi-scale feature extraction and global attention mechanism. The method includes processes such as data preprocessing, Res2Encoder feature extraction, full-pair correlation calculation, GRU recursive optimization, upsampling, and high-resolution displacement field output. It is particularly suitable for particle displacement field prediction in complex underwater environments, and can cope with common challenges such as insufficient underwater illumination, impurity interference, and flow irregularity, providing a reliable solution for high-precision observation of complex flow fields. Through this method, the model can more accurately capture the motion state of particles when processing complex underwater flow fields (such as deep-sea plumes, etc.), effectively improving the resolution and detail retention capabilities of flow field prediction. The present invention is suitable for application in the fields of ecological monitoring and flow field research in underwater environments, providing higher observation accuracy and reliability.

[0048] The present invention uses the Bottle2neck module of Res2Net to construct Res2Encoder, and uses multi-scale feature extraction to perform hierarchical processing on the flow information in the particle image. By dividing the input features into different scales and performing parallel convolution processing, the module can effectively extract fine-grained features in the flow field at different scales. This multi-scale structure can ensure that the model can capture multi-level information from large-scale overall flow trends to small-scale eddies when dealing with complex flows. The fusion of multi-scale features enables the model to have higher adaptability and accuracy when dealing with complex flow fields, and effectively improves the ability to capture detailed information.

[0049] The GAM global attention mechanism is introduced into the model. By combining channel and spatial attention, the selectivity of the model for important features is improved. In the PIV task, the particle image contains a lot of redundant information. The GAM mechanism can focus on the key flow area and suppress irrelevant features, thereby avoiding information dispersion. The channel attention mechanism weights the channel dimension of the feature, enhancing the dependency between features of different channels; while the spatial attention focuses on the salient areas in the image by weighting the features of the spatial dimension. The introduction of GAM enables the Res2RAFT model to better adapt to the changes in flow patterns in highly complex flow fields, especially in dynamic and violent scenes such as turbulence and vortices.

[0050] The Res2RAFT model of the present invention significantly improves the practicality of PIV tasks in complex underwater environments through the synergy of multi-scale feature extraction and global attention mechanism. The model is suitable for a variety of application scenarios in fluid mechanics, such as simulated flow field tests in laboratory environments, deep-sea particle flow observations in marine ecological monitoring, and complex flow field evaluations in industrial applications. Especially in scenes with complex flow characteristics such as deep-sea plumes and offshore floating particle flows, the present invention can provide accurate flow field observation data, which helps to reveal detailed changes in fluid motion and provide technical support for scientific research and environmental protection. Overall, the Res2RAFT particle image velocimetry method proposed in the present invention overcomes the shortcomings of traditional PIV methods in dealing with complex underwater flow fields. Through sophisticated multi-scale feature extraction and global attention mechanisms, it achieves high-resolution displacement field prediction of the flow field, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 : Overall structure diagram of Res2RAFT.

[0052] Figure 2 : The hierarchical structure of Res2Encoder.

[0053] Figure 3 : Multi-scale convolutional structure diagram of the RA-Bottle2neck module.

[0054] Figure 4 : Multi-scale convolutional structure diagram of the RB-Bottle2neck module.

[0055] Figure 5 : Structural diagram of the GAM global attention mechanism in the RB-Bottle2neck module.

[0056] Figure 6 : Prediction results of Reynolds number 200.

[0057] Figure 7: Comparison of Res2RAFT experimental results with traditional methods.

[0058] Figure 8 : Experimental results of Res2RAFT and comparison with deep learning methods. DETAILED DESCRIPTION

[0059] A multi-scale Res2RAFT particle image velocimetry method based on global attention mechanism optimization, the steps of the present invention: prepare a data set and perform preprocessing; introduce the Bottle2neck module in the Res2Net network to enhance the multi-scale feature extraction capability, and introduce GAM (global attention mechanism) to enhance the global dimensional interaction of features and reduce information diffusion. Subsequently, through the correlation calculation layer, GRU recursive update module, upsampling module and other steps and methods, the improved model Res2RAFT optical flow prediction network can effectively recover high-resolution optical flow fields from low-resolution features, thereby improving the accuracy and adaptability of flow field measurement. It includes the following steps:

[0060] Step 1: Dataset selection and preprocessing:

[0061] This study selected a dataset specifically used to evaluate the performance of particle image velocimetry, namely the PIV Dataset, which was artificially synthesized by Cai et al. using computational fluid dynamics technology. As shown in Table 1, it contains a total of 11,650 pairs of PIV particle images and standard flow field results in five categories. Each pair of data sets contains two consecutive frames of particle images and the actual velocity results corresponding to the particle image pair. This dataset generates images of the motion process of various flows based on computational fluid dynamics. Detailed information on the PIVDataset dataset is shown in Table 1.

[0062] Table 1 Dataset composition

[0063]

[0064] By obtaining a PIV dataset containing two consecutive frames of particle images, the image size of the dataset is unified, the length and width are all set to 256×256, and the image is regularized to ensure that it is suitable for the input of Res2Encoder. The experiment adopts the cross-validation method, dividing the dataset into ten parts, nine of which are training sets and one is a test set, and the performance of the algorithm is analyzed by calculating various classification indicators.

[0065] The present invention uses the average endpoint error to measure the accuracy of the model flow field prediction. The calculation formula of the average endpoint error (AEE) is as follows:

[0066]

[0067] In the formula, N represents the total number of pixels. fgt represents the ground truth optical flow of each pixel. fes,i represents the estimated optical flow at the i-th iteration. ∥.∥ represents the L1 norm, which is the absolute error of the optical flow. Directly taking the AEE value appears to be too small and difficult to compare, so this article takes the error value of every 100 pixels for comparison, which is clearer.

[0068] Regarding loss calculation, we use supervised learning to calculate the loss of the optical flow prediction results and the actual displacement field, and update the model parameters through back propagation. The loss calculation uses a sequence loss function. Specifically, the difference between the optical flow prediction sequence {F1, F2, …, Fn} and the actual optical flow Fgt is measured by the weighted L1 loss.

[0069]

[0070] In the formula, λ i =0.8 n-i It is the weight of each prediction, giving greater weight to later predictions to improve the accuracy of the final result.

[0071] Step 2: Feature extraction network Res2Encoder:

[0072] The present invention mainly proposes a new Res2Encoder feature encoder, which acts on feature extraction and context network at the same time. Res2Encoder uses a unified structural design in the feature encoder and context network, and is composed of multiple RA-Bottle2neck and RB-Bottle2neck modules. Among them, the RA-Bottle2neck module mainly achieves resolution reduction through the design of stride=2, and enhances the expression ability of multi-scale features. The RB-Bottle2neck module enhances feature selectivity and the ability to understand global information by retaining resolution and combining GAM (global attention mechanism). This modular design makes the functional implementation of Res2Encoder in the two sub-networks consistent and scalable.

[0073] The task of the feature encoder is to extract rich multi-scale features from the input image pair. Its specific functions include:

[0074] (1) Multi-scale feature extraction

[0075] Through the RA-Bottle2neck module, Res2Encoder is able to downsample the input image pairs layer by layer to capture features at different spatial scales. This is particularly important for the multi-scale flow patterns unique to particle images, as the feature encoder needs to provide comprehensive feature information for optical flow estimation.

[0076] (2) Enhanced feature expression capabilities

[0077] The RB-Bottle2neck module further screens and optimizes multi-scale features through the GAM global attention mechanism to highlight key features. This process can not only enhance the model's understanding of global information, but also retain necessary detail information as the feature resolution decreases layer by layer.

[0078] (3) Providing high-quality features for optical flow estimation

[0079] The final output of the feature encoder is a multi-channel, high-dimensional feature map. These feature maps are input into the correlation layer and subsequent modules to provide strong support for the optical flow estimation task.

[0080] The task of the context network is to use global context information to further optimize the optical flow estimation results. The role of Res2Encoder is mainly reflected in:

[0081] (1) Maintaining high-resolution features

[0082] The Res2Encoder in the context network keeps the spatial resolution of the feature map unchanged (stride=1 in RA-Bottle2neck). This design ensures that the model can better capture local detail features and provide support for fine-grained estimation of optical flow.

[0083] (2) Strengthening global information exchange

[0084] The GAM mechanism in the RB-Bottle2neck module can strengthen the global semantic relationship of feature expression through channel attention and spatial attention. The context network relies on this ability to model the potential global dependency in the optical flow estimation process, thereby improving the estimation accuracy.

[0085] (3) Providing global optimization capabilities

[0086] The context network further processes the feature maps from the feature encoder to provide global optimization support for the optical flow estimation process, making the model more robust when dealing with complex flow fields.

[0087] Although the structure of the feature encoder and the Res2Encoder in the context network are the same, their functional positioning is different: in the feature encoder, the core of Res2Encoder is to extract multi-scale features and downsample layer by layer to obtain more efficient feature representation. In the context network, Res2Encoder further optimizes high-resolution features through a global attention mechanism, enhancing the model's ability to capture global dependencies and complex flow field patterns. This unified but complementary design makes Res2Encoder the core module that connects feature extraction and global optimization, providing an important guarantee for improving the overall performance of the model.

[0088] In the design of the feature extraction network Res2Encoder of the present invention, the network architecture is mainly composed of three levels of modules, such as Figure 2 As shown in the figure. Each level is composed of RA-Bottle2neck and RB-Bottle2neck modules, which respectively undertake different feature processing tasks. The former is used for downsampling to maintain multi-scale information, and the latter is used to maintain resolution and introduce global attention. This enables the effective extraction of multi-scale features of particle images. In the overall architecture, the input image first passes through a 7×7 convolution layer to generate an initial feature map of 64 channels, and then passes through three layers of RA-Bottle2neck and RB-Bottle2neck modules in turn, and finally generates a 128-channel output feature map through 1×1 convolution. This hierarchical structure is downsampled through the RA-Bottle2neck module, while the RB-Bottle2neck module is responsible for retaining resolution and introducing a global attention mechanism, forming a gradual refinement and enhancement of features.

[0089] For Res2Encoder, the core design idea is to enhance the network's feature extraction capability through multi-scale feature representation. Traditional convolutional networks usually use fixed-size convolution kernels to extract features in each layer, while the Res2Encoder module introduces a multi-scale feature processing method in each residual block. Specifically, the input feature map is divided into multiple subsets, each of which can be independently convolved and features can be extracted at different scales, which greatly increases the diversity in the feature space.

[0090]

[0091] As shown in the formula, the input feature map is divided into s subsets x i For the first subset x i , without convolution operation, directly retain the original input as output y iThe next subset x2 undergoes a standard 3×3 convolution operation K2(x2) to obtain the output y2. Starting from the third subset x3, the convolution operation not only depends on the input x3, but also combines the output y2 of the previous group, that is, K3(x3+y2), thereby fusing information from different scales layer by layer. This layer-by-layer accumulation and recursive feature extraction method continues until the last subset x s The processing is completed.

[0092] The key advantage of this design is that each 3×3 convolution kernel not only performs a simple convolution operation on the current input features, but also incorporates the features of the previous layer, thereby effectively expanding the receptive field. The expansion of the receptive field means that the network can capture richer spatial information at different scales, especially when dealing with complex visual tasks. This multi-scale feature expression capability enables the network to better adapt to different object sizes, motion patterns, and structural complexities.

[0093] In addition, through this step-by-step transmission of features, multiple "equivalent receptive fields" are formed, that is, the network can capture and fuse features at different scales. This combined effect effectively improves the modeling ability of input information. For example, when dealing with particle image velocimetry (PIV) tasks in fluid mechanics, the multi-scale characteristics of the flow field (from large-scale overall flow to small-scale turbulent structures) can be captured and modeled in this way.

[0094] From the perspective of parameter optimization, the number of network parameters is significantly reduced by reducing the number of direct 3×3 convolution kernel operations. Compared with directly using complete convolution operations in each layer, Res2Net adopts a layer-by-layer feature transfer method, which greatly reduces the computational burden while maintaining the integrity of the information. Due to the close connection between feature groups, input features are efficiently converted into output features, thereby improving the overall performance of the network without increasing too much computational overhead.

[0095] Scale is a new dimension proposed by Bottle2neck, which is orthogonal to traditional dimensions such as depth, width, and cardinality. By processing multi-scale features at a fine-grained level, Res2Encoder not only improves the performance of the network, but also enables it to better handle complex visual tasks.

[0096] (1)RA-Bottle2neck

[0097] The design of the RA-Bottle2neck module is as follows Figure 3As shown. Its scale parameter is 4 and baseWidth is set to 26. This module divides the input features into multiple scales (such as x1 to x4 in the figure) and performs parallel convolution operations on these scales. The output of each scale is convolved and fused to form the output features (y1 to y4), and finally these features are re-aggregated to the main branch. The RA-Bottle2neck module reduces the resolution of the input feature map by setting stride=2, so that multi-scale information can still be retained during the downsampling process. This multi-scale convolution structure can enhance the model's feature expression ability at different scales, ensuring that even when the resolution is reduced, sufficient local and global information can still be captured.

[0098] Unlike ordinary residual connections, the RA-Bottle2neck module introduces a more fine-grained feature segmentation and processing method. Through multi-scale processing, the module can maximize the model's ability to capture information at all scales in the input image while maintaining computational efficiency. This multi-scale feature extraction is very important for flow information in particle images, because in particle images, flow information often presents a variety of different scale changes.

[0099] (2) RB-Bottle2neck

[0100] The design of the RB-Bottle2neck module is as follows Figure 4 As shown in , its characteristic is to enhance the selectivity of features by introducing the GAM global attention mechanism. This module extracts local features through multi-scale convolution operations while keeping the resolution of the feature map unchanged, and finally uses the GAM attention mechanism to optimize the features at the end. Figure 5 As shown in the figure, the GAM mechanism first strengthens the dependency of features between channels through channel attention, and then focuses on important spatial regions through the spatial attention module.

[0101] The introduction of the GAM mechanism in RB-Bottle2neck improves the ability to understand global information, allowing the model to better capture the complex flow patterns in particle images. Since the information in the particle image flow field has complex global dependencies, the role of the GAM global attention mechanism is to suppress unimportant features through the interaction of channels and spatial dimensions, so that the model can focus on the most critical flow information in the image.

[0102] In the RA-Bottle2neck and RB-Bottle2neck modules designed by the present invention, the scale parameter is set to 4 and the baseWidth is set to 26. This design has sufficient theoretical basis and experimental verification support. Specifically, multi-scale feature extraction is the key to improving model performance, and the size of the scale parameter directly determines the granularity of feature segmentation and the effectiveness of multi-scale convolution operations. However, too large a scale value will significantly increase the computational complexity of the model, not only reducing the training and reasoning efficiency, but also may introduce too much redundant information, resulting in the overfitting problem of the model on a specific data set. At the same time, if the scale value is too small, the role of multi-scale feature extraction cannot be fully utilized, thereby failing to effectively improve the performance of the model.

[0103] After many experimental comparisons, we chose to set the scale to 4, which strikes a good balance between the effectiveness of feature extraction and the complexity of the model. In addition, the baseWidth parameter is set to 26. This smaller base width further controls the computational burden of each convolution layer, making the model more efficient while maintaining the ability to extract multi-scale features.

[0104] To further reduce the risk of overfitting, we introduced the Dropout mechanism in the design process and set its value to 0.2. Dropout, as a commonly used regularization method, can randomly discard some neuron connections to prevent the model from overfitting the training data. This setting not only meets the requirements of improving the robustness of the model in theory, but also verifies its effectiveness through a large number of experiments. Experimental results show that when the scale parameter is 4, the baseWidth is 26, and the Dropout value is 0.2, the model performs best on different types of data sets. This design enables the model to achieve the best balance between multi-scale feature extraction and computational efficiency, while significantly reducing the possibility of overfitting, laying the foundation for accurate prediction of complex flow fields.

[0105] Step 3: Correlation calculation layer

[0106] All-pair Correlation Layer: Perform all-pair correlation calculations on the extracted features to generate a four-layer correlation pyramid (such as Figure 1 The four-layer structure in Figure 1 is used to capture feature similarities at different scales. The correlation layer plays a vital role. It establishes associations between pixels by calculating the similarity between each pixel in two frames. This process is done by constructing a 4D correlation volume that stores the inner product of the feature vectors between all pixel pairs.

[0107] As shown in the following formula:

[0108] C(g θ (I1),g θ (I2))∈R H×W×H×W ,C ijkl =∑hg θ (I1)ijh·g θ (I2)klh (4)

[0109] In the formula, C is the correlation volume, g θ (I1) and g θ (I2) is the feature map extracted by the feature encoder of the two frames. This formula indicates that each pixel pair in the two images (i.e. (i,j) and (k,l)) is scored by calculating the inner product to obtain their similarity scores.

[0110] The multi-scale correlation pyramid is constructed by pooling the correlation volume, specifically by averaging the last two dimensions of the volume to obtain volumes of multiple resolutions. In this way, both large-scale and small-scale displacement information can be captured simultaneously. During the calculation process, the initial optical flow field (f1, f2) is first estimated and used for local search at multiple levels of the pyramid.

[0111] The lookup operation can be defined as:

[0112] LC={x′+dx∣dx∈Z2,∣∣dx∣1≤r} (5)

[0113] In the formula, x′=(u+f1(u),v+f2(v)), this local grid will be sampled in the correlation volume according to the optical flow estimation value to obtain a new feature map for the next step of optical flow estimation. This full pixel correlation method significantly enhances the ability of the RAFT model to capture large displacement and complex motion scenes, and is particularly suitable for complex flow field prediction tasks in particle image velocimetry.

[0114] Step 4: GRU recursive update module

[0115] GRU recursive update module (Conv-GRU Update Block): This module uses convolutional GRU to iteratively update the optical flow estimate, and gradually approximates the real optical flow field by recursively optimizing the correlation and features. Each iteration can further improve the optical flow prediction accuracy based on the previous round of estimation.

[0116] Specifically, the optical flow prediction is gradually optimized through the gated recurrent unit (GRU). This module uses the convolutional GRU (ConvGRU) and the separated convolutional GRU (SepConvGRU) to recursively update the features and gradually adjust the flow field prediction. In each recursion, the GRU effectively retains important historical information through the update gate and reset gate based on the input correlation features, the current optical flow estimate and the hidden state, and calculates the optical flow increment (delta_flow) based on the current features, and iteratively updates the optical flow.

[0117] In addition, the action feature encoder encodes the correlation features and the initial optical flow into action features, further improving the effect of recursive updating. SepConvGRU captures more fine-grained feature information through separate convolutions in the horizontal and vertical directions, making the optical flow estimation more accurate. Through this recursive optimization, the GRU recursive update module can refine the flow field prediction frame by frame in the complex particle image velocimetry task, thereby gradually improving the overall accuracy of the model.

[0118] Step 5: Upsampling module

[0119] In the upsampling process of the Res2RAFT model, a convex combination upsampling method is used, which is an important step to improve the resolution of optical flow.

[0120] Specifically, the model first generates low-resolution optical flow prediction results through a recursive update module. In order to restore these low-resolution optical flow fields to high-resolution flow fields with the same resolution as the original image, the convex upsampling method interpolates the low-resolution optical flow after convolution operations to generate higher-resolution prediction results.

[0121] In the specific implementation, convex upsampling constructs multiple masks for interpolation and then performs convolution operations on the local area of ​​each flow field. These masks are normalized by the softmax function to ensure that the contribution of each pixel in the interpolation process can be reasonably distributed. Finally, the low-resolution flow field after multiple convolutions is combined with the generated mask to complete the upsampling process of the optical flow. This not only maintains the details of the original optical flow, but also effectively improves the spatial resolution of the optical flow prediction, making the prediction result more accurate. The advantage of this method is that it can achieve accurate upsampling of the optical flow field from low resolution to high resolution without significantly increasing the computational cost, which is suitable for the prediction of complex fluid fields.

[0122] Step 6: Output displacement field

[0123] After processing through multiple layers of modules, the predicted displacement field can be finally obtained and a displacement field image can be output. Figure 6As shown, the left figure shows the real displacement field in the particle image, where the arrow indicates the direction of flow and the color represents the magnitude of displacement. The darker the color, the larger the displacement. These real values ​​are used to measure the accuracy of model predictions and are our benchmark for evaluating model performance. The middle figure shows the flow field predicted by the Res2RAFT model, where arrows also indicate the direction of flow and colors indicate the magnitude of displacement. By comparing with the real flow field on the left, we can intuitively observe the performance of the model at different Reynolds numbers. The right side of the figure shows the error between the model-predicted flow field and the real flow field, using the average endpoint error (EPE) as the evaluation criterion. The darker the color, the larger the error. Through these error graphs, we can clearly see in which areas the model's prediction accuracy is low, which provides a reference for further improving the model.

[0124] Experimental verification:

[0125] As shown in Table 2, multiple datasets (such as Cylinder, DNS_turbulence, and SQG) are used to test the model performance. The results show that the proposed method has high accuracy and robustness in complex flow fields, especially in Cylinder and DNS_turbulence flow fields, with an accuracy improvement of more than 20%.

[0126] Table 2 Comparison of prediction values ​​of different methods

[0127]

[0128] It can be clearly seen that Res2RAFT performs significantly better than traditional methods in most flow fields (especially complex flow fields such as Cylinder and DNSTurbulence), while there are some fluctuations in errors in the JHTDB Channel flow field. Overall, Res2RAFT has obvious advantages in multi-scale feature extraction for complex flow fields.

[0129] Experimental results analysis:

[0130] Overall, Res2RAFT, as an improved deep learning model, not only performs stably in simple flow fields, but also performs well in complex flow fields (such as DNS Turbulence and SQG). Figure 7 , Figure 8 As shown in the figure, there is a significant performance improvement compared to RAFT and other traditional methods, deep learning models, etc. In the most challenging flow fields, Res2RAFT provides a powerful combination of multi-scale feature extraction and global attention mechanism, enabling it to more accurately capture the complex nonlinear features in the flow field. Although there is still room for improvement in the JHTDB Channel flow field, the performance of Res2RAFT in most flow fields is sufficient to demonstrate its strong ability in optical flow prediction tasks.

[0131] Experimental conclusion:

[0132] The Res2RAFT model significantly outperforms traditional methods and existing deep learning models on multiple complex flow field datasets (including Cylinder, DNS_turbulence, and SQG), especially in terms of accuracy improvement, showing an advantage of more than 20%. By introducing multi-scale feature extraction and global attention mechanisms, the Res2RAFT model's ability to extract global and local information in complex flow fields has been significantly enhanced, and it shows higher robustness and accuracy when dealing with complex nonlinear flows (such as eddies and turbulence). The Res2RAFT model of the present invention provides a more accurate and efficient solution for optical flow estimation in PIV tasks through innovative designs in feature extraction and feature selection.

[0133] The above descriptions are only some specific implementation methods of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any person familiar with the art within the technical scope disclosed in the present invention should be covered within the protection scope of the present invention.

Claims

1. A multi-scale Res2RAFT particle image velocimetry method based on global attention mechanism optimization, characterized in that: The following steps are involved: S1. Obtain PIV dataset and data preprocessing: Each data sample in the PIV data set includes two consecutive frames of particle images and the actual particle velocity corresponding to the particle image; S2. Constructing a particle image velocimetry model, wherein the particle image velocimetry model is based on a RAFT optical flow network, wherein the RAFT optical flow network includes a feature extraction network, a context network, a full pair correlation layer, and a GRU recursive update module; The feature extraction network and context network of the RAFT optical flow network are replaced by the Res2encoder network to obtain the particle image velocimetry model; The Res2Encoder network includes, in sequence: a convolutional layer 1, a RA-Bottle2neck module, a RB-Bottle2neck module, a RA-Bottle2neck module, a RB-Bottle2neck module, a RA-Bottle2neck module, a RB-Bottle2neck module, and a convolutional layer 2; The convolution layer 1 uses a 7×7 convolution layer to generate an initial feature map of 64 channels; The convolutional layer 2 uses a 1×1 convolution to generate an output feature map of 128 channels; The RA-Bottle2neck module is a bottleneck structure with residual learning based on the Bottle2neck structure, which is used for downsampling to maintain multi-scale information. A multi-scale feature processing method is introduced in each residual block, and the input feature map of the Res2Encoder module is divided into s subsets. As shown in the following formula, each subset performs a convolution operation independently, and extracts features at several scales to increase the diversity of the feature space: For the first subset x i , without convolution operation, directly retain the original input as output y i ; The second subset x2 undergoes a standard 3×3 convolution operation K2(x2) to obtain the output y2; starting from the third subset x3, the convolution operation not only depends on the input x3, but also combines the output y2 of the previous group, that is, K3(x3+y2), thereby fusing information from different scales layer by layer; the layer-by-layer accumulation and recursive feature extraction method continues until the last subset x s The processing is completed; The RB-Bottle2neck module is a bottleneck structure with a global attention mechanism based on the Bottle2neck structure, which is used to maintain resolution; the RB-Bottle2neck module adds a global attention mechanism GAM on the basis of the RA-Bottle2neck module, and the global attention mechanism GAM is added after the last layer of the RA-Bottle2neck module; and the global attention mechanism GAM in the RB-Bottle2neck module includes channel attention and spatial attention modules, which utilize the interaction of channel and spatial dimensions to enable the particle image velocimetry model to focus on particle flow information in two consecutive frames; S3 constructs a loss function and uses the PIV dataset described in step S1 to train the particle image velocimetry model through supervised learning.

2. A multi-scale Res2RAFT particle image velocimetry method based on global attention mechanism optimization as claimed in claim 1, characterized in that: In step S1, the data preprocessing specifically includes: uniformly processing the particle image size in the PIV data set, setting the length and width to 256×256, and then regularizing the image.

3. A multi-scale Res2RAFT particle image velocimetry method based on global attention mechanism optimization as claimed in claim 1, characterized in that: In step S2, s=4, and the stride and baseWidth parameters of the RA-Bottle2neck module are set to 2 and 26 respectively.

4. A multi-scale Res2RAFT particle image velocimetry method based on global attention mechanism optimization as claimed in claim 3, characterized in that: In step S2, the stride and baseWidth parameters of the RB-Bottle2neck module are set to 1 and 26 respectively.

5. A multi-scale Res2RAFT particle image velocimetry method based on global attention mechanism optimization as claimed in claim 1, characterized in that: In step S3, the loss function formula is as follows: Where n is the number of samples in the PIV dataset, F i is the optical flow prediction value obtained by inputting two consecutive frames of particle images of the sample in the PIV dataset into the particle image velocimetry model, and F gt It is the real optical flow value reflecting the real speed of particles in the samples in the PIV dataset.

6. A multi-scale Res2RAFT particle image velocimetry method based on global attention mechanism optimization as claimed in claim 5, characterized in that: λ i =0.8 n-i It is the weight of each prediction, giving greater weight to later predictions to improve the accuracy of the final result.

7. A multi-scale Res2RAFT particle image velocimetry method based on global attention mechanism optimization as claimed in claim 1, characterized in that: The all-pair correlation layer is used to perform all-pair correlation calculation on the features extracted by the feature extraction network to obtain a four-layer correlation pyramid; The all-pair correlation layer establishes the association between pixels by calculating the similarity between each pixel in the two frames of images; the similarity calculation is performed by constructing a 4D correlation volume according to the following formula, and the 4D correlation volume stores the inner product of the feature vectors between all pixel pairs: <h2 style=";text-align:left;direction:ltr">C(g<h2 style=";text-align:left;direction:ltr"> θ <h2 style=";text-align:left;direction:ltr"> (I1),g<h2 style=";text-align:left;direction:ltr"> θ <h2 style=";text-align:left;direction:ltr"> (I2))∈R<h2 style=";text-align:left;direction:ltr"> H×W×H×W <h2 style=";text-align:left;direction:ltr"> ,C<h2 style=";text-align:left;direction:ltr"> ijkl <h2 style=";text-align:left;direction:ltr"> =∑hg<h2 style=";text-align:left;direction:ltr"> θ <h2 style=";text-align:left;direction:ltr"> (I1)ijh·g<h2 style=";text-align:left;direction:ltr"> θ <h2 style=";text-align:left;direction:ltr"> (I2)klh In the formula, C is the correlation volume, g θ (I1) and g θ (I2) is the feature map extracted by the feature encoder of the two frames; Among them, C ijkl represents the weight coefficients of the i-th, j-th spatial dimension and the k-th, l-th channel dimension in the convolution kernel; h is the input of the convolution calculation; The four-layer correlation pyramid is constructed by performing a pooling operation on the correlation volume.

8. A multi-scale Res2RAFT particle image velocimetry method based on global attention mechanism optimization as claimed in claim 1, characterized in that: The GRU recursive update module uses convolutional GRU and separated convolutional GRU to recursively update features and gradually adjust the flow field prediction; in each recursion, the GRU retains important historical information through update gates and reset gates based on the input correlation features, current optical flow estimation and hidden state, and calculates the optical flow increment in combination with the current features, and iteratively updates the optical flow; The separable convolution GRU captures more fine-grained feature information through separable convolution in both horizontal and vertical directions.

Images