A method for unsupervised learning particle image modeling and velocity measurement based on neural network

Abstract

The present invention discloses an unsupervised learning particle image modeling and velocity measurement method based on a neural network, including: using a cross-correlation algorithm to obtain a rough velocity field; using two adjacent frames of particle images in a particle image sequence and a rough velocity field obtained by the cross-correlation algorithm as input layers respectively, and merging the results of the two input layers after shrinking through a shrinkage network structure; using the merged result as the input of a neural network for unsupervised learning, using optical flow estimation to evaluate the neural network training results, and the unsupervised learning loss function includes photometric loss, flow smoothness loss, and divergence-free constraints; generating a PIV data set for neural network training. The present invention uses the fusion data of the input particle image and the rough initial velocity field to make the neural network closer to the target during training compared to the unsupervised learning PIV, the initial value of the loss function is smaller, the particle calculation speed is faster than the traditional PIV, the speed is increased by about 30%, the resolution is greatly improved, and a huge data set is effectively saved compared to the supervised learning PIV.

Description

Technical Field

[0001] The present invention relates to the technical field of fluid mechanics, and in particular to an unsupervised learning particle image modeling and velocity measurement method based on a neural network. Background Art

[0002] Particle image velocimetry (PIV) is a non-contact, image-based velocity measurement method. A certain number of tiny particles that do not affect the flow are placed in the flow field. These particles are illuminated by a laser, and a camera captures the scattered light from the particles to form an image of the particles. The general idea is to calculate the velocity field information at that moment by taking the time difference between two frames of image and the distance the corresponding particles have traveled. The velocity field effects obtained by different methods vary significantly, which is largely related to the algorithm design and network structure. This method can be used to measure velocity and optimize flow fields for specific flow scenarios in contexts such as process industries.

[0003] Deep learning methods are currently the mainstream method for processing PIV. Due to their high computational efficiency and high velocity field resolution, they are widely used in computational fluid dynamics. In recent years, research has proposed network structures such as FlowNet, LiteFlowNet, and UnLiteFlowNet, corresponding to supervised learning and unsupervised learning. Among them, supervised learning PIV requires a huge data set, and the data set needs to contain multiple flow states and multiple noise conditions. There is also a large gap between the actual flow and the theoretical and simulation values. Unsupervised learning PIV can effectively solve the defects of supervised learning, but there are also some problems. For example, in actual operation, the simplified optimization process will lead to the simplification of the energy function, thereby sacrificing the quality of the output. Therefore, there is an urgent need for a method that can combine the advantages of supervised learning and unsupervised learning, overcome the existing defects of both, and significantly improve the resolution. Summary of the Invention

[0004] An object of the present invention is to provide a method for unsupervised learning particle image modeling and velocimetry based on a neural network, and to provide at least the advantages to be described later.

[0005] The technical solutions of the present invention are as follows:

[0006] A method for unsupervised learning particle image modeling and velocity measurement based on a neural network comprises the following steps:

[0007] 1) Use cross-correlation algorithm to obtain rough velocity field;

[0008] 2) fusing two inputs, where the neural network comprises two input layers and one output layer, and merging the results of the two input layers after shrinking them through a shrinking network structure;

[0009] 3) using the result of the fusion of the two inputs in step 2) as the input of the neural network for unsupervised learning, and using optical flow estimation to evaluate the neural network training result;

[0010] 4) Generate PIV data set for neural network training and obtain particle image velocity results.

[0011] Preferably, the two input layers in step 2) are specifically: the first input layer inputs two adjacent frames of particle images in the particle image sequence, and the second input layer inputs the rough velocity field in step 1).

[0012] Preferably, in step 3), the loss function of the unsupervised learning includes photometric loss and flow smoothness loss, specifically:

[0013] L(I1,I2,F 1 ,F 2 )=λ p L p +λ s L s

[0014] Among them, I is the image input, F is the rough velocity field change value input, L p is the luminosity loss, L s is the loss of flow smoothness.

[0015] Preferably, the step 1) is specifically as follows: using a cross-correlation algorithm to process the particle images of two adjacent frames in the particle image sequence, and calculating the following: Sk+1=Sk+Vk*Δt based on the position of the same particle and the time interval between the two frames, where S is the particle position at a certain moment, V is the corresponding velocity, and Δt is the time interval between the two frames. Then, a suitable window size is selected to generate a rough initial velocity field.

[0016] Preferably, in step 2), the two input layer structures are exactly the same, both including convolutional pooling layers, and the contraction network structure as a whole forms a network structure consisting of alternating convolutional layers and pooling layers.

[0017] Preferably, in step 2), the first input layer is subjected to a multi-scale shrinkage operation through a first shrinkage network structure to extract features, and the second input layer is subjected to a shrinkage operation through a second shrinkage network structure to extract features, and the output layer is the predicted particle motion field.

[0018] Preferably, in step 2), the output layer is a high-precision and high-resolution velocity field.

[0019] Preferably, the luminosity loss Lp is the sum of two parts, specifically:

[0020] Lp (I1,I2,F 1 ,F 2 )=∑ρ(I1(x)-I2(x+F 1 (x))+∑ρ(I2(x)-I1(x+F 2 (x))

[0021] Among them, ∑ρ(I1(x)-I2(x+F 1 (x)) is the difference between the first frame image and the second frame image after distortion, ∑ρ(I2(x)-I1(x+F 2 (x)) is the difference between the second frame image and the first frame image after distortion;

[0022] The flow smoothness loss is calculated using a second-order smoothness constraint:

[0023] L s (F f ,F b )=∑ (s,r)∈N ∑ρ(F f (s)-2F f (x)+F f (r))+ρ(F b (s)-2F b (x)+F b (r))

[0024] Among them, N is a four-channel filter, F f is the forward flow field, F b is the backward flow field;

[0025] First, the values ​​of each component of the two flows in the four-channel filter are calculated, and then their photometric loss function and flow smoothness loss function are calculated. On this basis, a divergence-free constraint is added to constrain the divergence of the velocity field to offset the errors caused by erroneous and missing velocity vectors.

[0026] Preferably, the PIV dataset contains 15050 frames of data images, covering 8 different types of flow states, 2 / 3 of the data are used for training, and the remaining 1 / 3 of the data are used for testing.

[0027] Beneficial effects

[0028] The present invention provides an unsupervised learning particle image modeling and velocity measurement method based on a neural network, which combines the advantages of traditional supervised learning and traditional unsupervised learning, and can perform super-resolution calculations, so that the neural network can ultimately achieve the characteristics of high resolution, high precision and high robustness. At the same time, by inputting the fusion data of the particle image and the rough initial velocity field, the neural network of the present invention is closer to the target during training than the traditional unsupervised learning PIV, and the initial value of the loss function is smaller. The particle velocity measurement calculation speed of the present invention is faster than that of the traditional PIV, with a speed increase of about 30%, and the resolution is greatly improved. Compared with the traditional supervised learning PIV, the present invention effectively saves a huge data set. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A flow chart of an embodiment of the unsupervised learning particle image modeling and velocity measurement method based on a neural network provided by the present invention;

[0030] Figure 2 Schematic diagram of processing adjacent particle images using a cross-correlation algorithm in step 1) of the unsupervised learning particle image modeling and velocimetry method based on a neural network provided by the present invention;

[0031] Figure 3 This is a flowchart of the fusion of two input layers in step 2) of the unsupervised learning particle image modeling and speed measurement method based on a neural network provided by the present invention. DETAILED DESCRIPTION

[0032] The present invention will be described in further detail below in conjunction with the accompanying drawings so that those skilled in the art can implement the invention with reference to the description.

[0033] It should be understood that terms such as “having,” “including,” and “comprising” used herein do not prescribe the existence or addition of one or more other elements or combinations thereof.

[0034] like Figures 1 to 3 As shown, the present invention provides an unsupervised learning particle image modeling and speed measurement method based on a neural network, which includes:

[0035] 1) Obtain a rough velocity field using a cross-correlation algorithm: In a particle image sequence, a cross-correlation algorithm is used to process particle images of two adjacent frames in the particle image sequence, and a suitable window size is selected to generate a rough velocity field. Specifically, the cross-correlation algorithm calculates the position of the same particle and the time interval between the two frames:

[0036] Sk+1=Sk+Vk*Δt

[0037] Among them, S is the particle position at a certain moment, V is the corresponding velocity, and Δt is the time interval between two frames of images.

[0038] 2) Fusion of two inputs: The neural network contains two input layers and one output layer. The results of the two input layers after shrinkage operation through the shrinkage network structure are merged.

[0039] The first input layer receives two adjacent particle images from the particle image sequence; the second input layer receives the rough velocity field generated by the cross-correlation algorithm in step 1). This rough velocity field is used to provide initial values ​​for unsupervised learning. Compared to traditional neural network input, this input is equivalent to providing an initial sample of particle motion, giving the neural network input some supervised information, thereby helping the neural network obtain more information, allowing it to find the trajectory of particle motion more quickly and directly during training, and improving the training speed of unsupervised learning; the output layer is the predicted particle motion field. The output of the first input layer is passed through a first contraction network structure for multi-scale contraction and feature extraction. Similarly, the second input layer is contracted through a second contraction network structure for feature extraction. The results of the contraction operations at each scale obtained during the feature extraction process are then merged.

[0040] 3) The fused information from step 2) is fed into the unsupervised learning neural network as input. This information does not contain the actual velocity field, so the training results of this network are evaluated using optical flow. The loss function of unsupervised learning consists of photometric loss and flow smoothness loss, specifically:

[0041] L(I1,I2,F 1 ,F 2 )=λ p L p +λ s L s

[0042] Among them, I is the image input, F is the rough velocity field change value input, L p is the luminosity loss, L s is the loss of flow smoothness.

[0043] In addition, the luminosity loss L p It is based on the difference between the first frame image estimated using the forward flow field and the distorted second frame image, as well as the difference between the second frame image and the distorted first frame image, and uses reverse estimation to perform unsupervised learning on the particle image. The photometric loss is defined as the sum of the two differences:

[0044] L p (I1,I2,F 1 ,F 2 )=∑ρ(I1(x)-I2(x+F 1 (x))+∑ρ(I2(x)-I1(x+F2 (x))

[0045] In addition, the flow smoothness loss L s There are some areas in the image that lack necessary information. For example, there may be insufficient information at the image boundary because the particle may have moved out of the image area in the second frame or the particle has not yet entered the image in the first frame. Therefore, in order to avoid ambiguity in these areas with insufficient information, the flow smoothness loss is included in the total unsupervised learning loss. This part is calculated using a second-order smoothness constraint:

[0046] L s (F f ,F b )=∑ (s,r)∈N ∑ρ(F f (s)-2F f (x)+F f (r))+ρ(F b (s)-2F b (x)+F b (r))

[0047] Among them, N is a four-channel filter, F f is the forward flow field, F b is the backward flow field;

[0048] Here, we first calculate the values ​​of each component of the two flows in the four-channel filter, and then calculate their photometric loss function and flow smoothness loss function. On this basis, we add a divergence-free constraint to constrain the divergence of the velocity field. Due to factors such as shooting errors and laser reflections, the divergence obtained by shooting may not be zero. In fact, the divergence of a passive velocity field must be zero. Therefore, the error caused by erroneous and missing velocity vectors is offset by adding a divergence-free constraint.

[0049] 4) Generate a PIV dataset for neural network training and obtain particle image velocity results: Using a PIV dataset containing 15,050 frames of data images covering eight different flow regimes, the present invention uses two-thirds of this data for training and one-third for testing. Testing shows that this method can improve training speed by approximately 30% compared to a network without incorporating coarse velocity field information, while maintaining the same level of velocity field accuracy. Experiments were conducted on each of the eight flow regimes in the PIV dataset, and the results show that this method is applicable to all eight flow regimes, achieving speed improvements for all of them.

Claims

1. A method for unsupervised learning particle image modeling and velocity measurement based on neural network, characterized in that: The following steps are involved: 1) Use the cross-correlation algorithm to obtain the rough velocity field; 2) fusing two inputs, the neural network comprising two input layers and one output layer, and merging the results of the two input layers after shrinking through a shrinking network structure; 3) using the result of the fusion of the two inputs in step 2) as the input of the neural network for unsupervised learning, and using optical flow estimation to evaluate the training result of the neural network; 4) Generate PIV data set for neural network training and obtain particle image velocity results; The two input layers in step 2) are specifically: the first input layer inputs two adjacent frames of particle images in the particle image sequence, and the second input layer inputs the rough velocity field in step 1); In step 3), the loss function of the unsupervised learning includes photometric loss, flow smoothness loss and no divergence constraint, specifically: L(I1,I2,F 1 ,F 2 )=λ p L p +λ s L s +λ f L f Among them, I is the image input, F is the rough velocity field change value input, L p is the luminosity loss, L s is the flow smoothness loss, L f There is no divergence constraint; The step 1) is specifically as follows: using a cross-correlation algorithm to process particle images of two adjacent frames in a particle image sequence, and calculating by the position of the same particle and the time interval between the two frames: Sk+1=Sk+Vk*Δt, Among them, S is the particle position at a certain moment, V is the corresponding velocity, Δt is the time interval between two frames of images, and then a suitable window size is selected to generate a rough initial velocity field; The PIV dataset contains 15050 frames of data images, covering 8 different types of flow states, 2 / 3 of the data are used for training, and the remaining 1 / 3 of the data are used for testing.

2. The unsupervised learning particle image modeling and velocity measurement method based on neural network according to claim 1, characterized in that: In the step 2), the two input layer structures are exactly the same, both including convolutional pooling layers, and the contraction network structure as a whole forms a network structure consisting of alternating convolutional layers and pooling layers.

3. The unsupervised learning particle image modeling and velocity measurement method based on neural network according to claim 2, characterized in that: In the step 2), the first input layer is subjected to a multi-scale shrinkage operation through a first shrinkage network structure to extract features, and the second input layer is subjected to a shrinkage operation through a second shrinkage network structure to extract features, and the output layer is a predicted particle motion field.

4. The unsupervised learning particle image modeling and velocity measurement method based on neural network according to claim 1, characterized in that: In the step 2), the output layer is a high-precision and high-resolution velocity field.

5. The unsupervised learning particle image modeling and velocity measurement method based on neural network according to claim 4, characterized in that: The luminosity loss L p It is the sum of two parts, specifically: L p (I1,I2,F 1 ,F 2 )=∑ρ(I1(x)-I2(x+F 1 (x))+∑ρ(I2(x)-I1(x+F 2 (x)) Among them, ∑ρ(I1(x)-I2(x+F 1 (x)) is the difference between the first frame image and the second frame image after distortion, ∑ρ(I2(x)-I1(x+F 2 (x)) is the difference between the second frame image and the first frame image after distortion; The flow smoothness loss L s Compute using a second-order smoothness constraint: L s (F f ,F b )=∑ (s,r)∈N ∑ρ(F f (s)-2F f (x)+F f (r))+ρ(F b (s)-2F b (x)+F b (r)) Where N is a four-channel filter, F f is the forward flow field, F b is the backward flow field; First, the values ​​of each component of the two flows in the four-channel filter are calculated, and then their photometric loss function and flow smoothness loss function are calculated. On this basis, a divergence-free constraint is added to constrain the divergence of the velocity field to offset the errors caused by erroneous and missing velocity vectors.

Images