Gas-liquid two-phase flow bubble identification method based on improved Mask R-CNN

By introducing an improved Mask R-CNN method in the two-phase flow of gas and liquid, using attention mechanism and deep learning technology, the problem of edge segmentation fuzzy and noise influence in traditional bubble recognition technology is solved, and efficient and accurate bubble recognition is achieved and morphological information extraction, suitable for complex gas-liquid two-phase flow environments.

CN119006980APending Publication Date: 2024-11-22SOUTHWEST PETROLEUM UNIV
View PDF 0 Cites 5 Cited by

Patent Information

Application Number
CN202411152625.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

Traditional bubble recognition technology in the two-phase flow of gas-liquid bubbles has problems such as blurred edge segmentation, high background noise influence, and high computing resources consumption, resulting in low recognition accuracy and low efficiency.

Method used

The improved Mask R-CNN method is adopted to introduce channel attention and spatial attention mechanisms into the convolutional neural network, and combine ResNet and CBAM modules to extract and recognize bubble morphological information, and use deep learning technology to improve recognition efficiency.

Benefits of technology

It improves the accuracy and efficiency of bubble recognition, reduces the cost of data set annotation and processing, adapts to bubble recognition of different types and sizes, and enhances the monitoring and analysis capabilities of electrode surface status.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006980A_ABST
    Figure CN119006980A_ABST
Patent Text Reader

Abstract

The invention discloses a gas-liquid two-phase flow bubble recognition method based on an improved Mask R-CNN, and the method comprises the steps: collecting a bubble image through a high-speed camera, carrying out the precise marking and preprocessing through Label software, and constructing a high-quality data set. In the model design, an improved Mask R-CNN introduces a channel attention mechanism and a space attention mechanism in a ResNet network, the feature extraction capability is enhanced, transfer learning is performed through pre-training weight, and a recognition result is further optimized in combination with multiple convolution operations and an up-sampling layer. According to the method, the bubbles can be efficiently and accurately recognized, key morphological information such as the diameter and the projection area of the bubbles is extracted, the bubble recognition precision and efficiency are remarkably improved, and the method is suitable for monitoring and analyzing scenes of the bubbles of various types and different sizes.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a method for identifying bubbles in gas-liquid two-phase flow based on an improved Mask R-CNN method. The method uses deep learning technology to accurately measure information such as bubble shape and size to improve the accuracy and efficiency of bubble identification, and belongs to the field of gas-liquid two-phase flow identification. Background Art

[0002] In the measurement of flow field in gas-liquid two-phase flow, due to the interaction between the gas and liquid phases, the flow process is much more complicated than single-phase flow, which is also one of the difficulties in the current research of multiphase fluid mechanics. At the same time, the velocity, size and shape distribution of bubbles determine the flow structure, mass transfer area, mixing performance and reaction efficiency. Therefore, the research on the measurement technology of bubble shape and size has a wide range of engineering application value.

[0003] In order to obtain accurate information on the shape and size of bubbles, complex experimental equipment and advanced image processing technology are required. With the continuous changes in experimental equipment and image processing technology, digital image processing (DIP), as a non-contact image measurement technology, is widely used in the measurement of bubble morphology and size because it can intuitively display the size, shape, and distribution of bubbles without interfering with the flow field. The traditional image segmentation method has the following obvious shortcomings: a. In the process of using traditional algorithms, due to insufficient or excessive segmentation of bubble images, the edge segmentation is blurred, affecting the accuracy of bubble recognition; b. Traditional segmentation algorithms are easily affected by background noise, resulting in poor robustness; c. The image pre-processing process consumes a lot of time and computing resources. Summary of the invention

[0004] The purpose of the present invention is to provide a gas-liquid two-phase flow bubble recognition method based on an improved Mask R-CNN, which utilizes a deep learning algorithm to extract bubble morphological information, thereby effectively improving the recognition efficiency.

[0005] The technical solution to achieve the purpose of the present invention is:

[0006] A bubble recognition method for gas-liquid two-phase flow based on improved Mask R-CNN includes the following steps:

[0007] Step 1. Construct a data set. First, use a high-speed camera to capture a video of the bubble image on the electrode surface, extract each frame of the video using video editing software, annotate the bubble image using Labelme software, and perform a series of preprocessing steps on the annotated images to obtain a bubble label data set. Then randomly select 80% of the images and their labels as the training data set, and the remaining 20% ​​of the images and their labels as the test data set.

[0008] Step 2. Use the labeled data set obtained in step 1 to train the convolutional neural network and obtain the convolutional neural network model. By adding the channel attention mechanism and the spatial attention mechanism to the ResNet network, the attention mechanism is added to the last layer of convolution and the first layer of convolution without changing the network structure. The initial learning rate of the model is set to 0.004, the momentum is set to 0.9, and the learning rate is adjusted using the cosine annealing algorithm. In order to speed up the network training speed and feature learning ability, the pre-trained weights are obtained from the MSCOCO dataset, and the model is transferred for learning. In order to improve the segmentation effect, four pictures are used as small batch inputs on the GPU.

[0009] The entire network training process includes the following steps:

[0010] Step 2.1: Input the RGB bubble image of size H×W×3 into ResNet-50. The input image passes through a 7x7 convolution layer with 64 output channels and a stride of 2, and the output size is H / 2×W / 2×64. The convolutional feature map is normalized to stabilize the training process, and the normalized feature map is activated by ReLU, and then passed through a 3×3 maximum pooling layer with a stride of 2, and the output size is H / 4×W / 4×64.

[0011] Step 2.2: After the first convolution layer, perform global average pooling and global maximum pooling on the feature map to obtain two 1D feature vectors with a shape of 1×1×64. Input these two feature vectors into a shared MLP, which contains a hidden layer. The hidden layer size is usually 64 / reduction. Add the two vectors output by the MLP and pass the sigmoid activation function to obtain the attention weight of each channel. Multiply each channel of the input feature map by the corresponding attention weight to obtain the enhanced feature map.

[0012] Step 2.3: Perform maximum pooling and average pooling on the channel dimension of the feature map enhanced by channel attention to obtain two 2D feature maps with a shape of H / 4×W / 4×1. Concatenate these two feature maps in the channel dimension to obtain a feature map with a shape of H / 4×W / 4×2. Use a 7×7 convolution layer to reduce the dimension of the feature map to H / 4×W / 4×1, use the sigmoid activation function to obtain the attention weight of each spatial position, and multiply each spatial position of the feature map output by the channel attention module by the corresponding attention weight to obtain the final enhanced feature map.

[0013] Step 2.4: The structure of the residual layer of ResNet-50 is divided into four stages, each of which contains several residual blocks: Stage 2 contains 3 residual blocks with an output size of H / 4×W / 4×256. Stage 3: contains 4 residual blocks with an output size of H / 8×W / 8×512. Stage 4: contains 6 residual blocks with an output size of H / 16×W / 16×1024. Stage 5 contains 3 residual blocks with an output size of H / 32×W / 32×2048.

[0014] Step 2.5: Apply the CBAM module again after the output of the last residual block of ResNet-50, and the processing process is the same as the CBAM module added after the first convolution layer. The final output is an enhanced feature map with a size of H / 32×W / 32×2048.

[0015] Step 2.6: The RPN network passes the enhanced feature map through a 3×3 convolution layer to reduce the number of channels of the feature map, but keeps the spatial size unchanged, with an output size of H / 32×W / 32×512, and then passes through two parallel 1×1 convolution layers, one of which is used for binary classification of the target and background, with an output size of H / 32×W / 32×(2*k), and the other 1×1 convolution layer is used for bounding box regression adjustment, with an output size of H / 32×W / 32×(4*k). Generate multiple anchors on the feature map, with k anchors of different scales and aspect ratios at each position.

[0016] Step 2.7: The candidate regions output by RPN, each region has a bounding box and score, and RoIAlign is applied to each candidate region to adjust the RoI feature maps of different sizes to a uniform size. The aligned RoI feature map has a fixed size of 7×7×2048.

[0017] Step 2.8: Perform object classification on each RoI, pass through a fully connected layer and softmax activation, output the probability of each category, the shape is (N, num_classes + 1), perform bounding box regression on each RoI, pass through a fully connected layer, and output the bounding box adjustment parameters of each category, the shape is (N, num_classes * 4).

[0018] Step 2.9: The aligned RoI feature map is of size 14×14×2048. It is extracted through several convolutional layers with an output size of 14×14×256. A deconvolution layer is used to enlarge the spatial size of the feature map to 28×28×256. A 1×1 convolution layer is then used to reduce the number of channels to the number of target categories. Finally, a segmentation mask of the target is generated for each RoI, and the sigmoid activation function is used to output the probability of each pixel.

[0019] Step 3. Input the bubble image to be identified into the convolutional network model obtained in step 2 for identification. Save the bounding box information of each bubble in the recognition result of Mask R-CNN, extract the values ​​of the bounding box vertices to determine the bubble aspect ratio and obtain the projection area of ​​the bubble through the mask information, and then display the bubble morphological information on each identified image. For the fully convolutional network model after training, input a picture of gas-liquid two-phase flow to be identified into the model. The network can almost accurately identify the bubbles in the picture, and calculate and obtain the accuracy of bubble identification.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] The present invention aims at the problem of bubble recognition based on gas-liquid two-phase flow, introduces an improved Mask R-CNN model, and establishes a bubble recognition model based on gas-liquid two-phase flow. The advantages of this method are: (1) the attention mechanism is introduced on the original model, and by combining channel attention and spatial attention, a more comprehensive and effective feature extraction capability is provided for the convolutional neural network. This dual attention mechanism enables the model to adaptively focus on the key channels and spatial positions in the input features, thereby improving the performance of the model. (2) The model proposed in the present invention is based on the improved model Mask R-CNN, so it does not require massive data sets to be retrained, and can be effectively played by fine-tuning on a limited data set. The requirement for the amount of data is low, and the annotation is simple, which greatly reduces the annotation cost and data processing cost. (3) The deep learning method is introduced into the bubble recognition on the electrode surface, which can automatically extract features from complex images through a multi-layer convolutional network, improve the recognition accuracy and efficiency, and adapt to bubbles of different types and sizes, so as to more effectively monitor and analyze the electrode surface state. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0023] Figure 1 It is a flow chart of bubble recognition in gas-liquid two-phase flow based on improved Mask R-CNN disclosed in the present invention;

[0024] Figure 2 It is a gas-liquid two-phase flow bubble recognition network framework based on improved Mask R-CNN disclosed in the present invention;

[0025] Figure 3 A residual module that adds a CBAM attention mechanism to the ResNet network disclosed in the present invention;

[0026] Figure 4The present invention discloses an improved Mask R-CNN gas-liquid two-phase flow bubble diameter error analysis;

[0027] Figure 5 The present invention discloses an improved Mask R-CNN based gas-liquid two-phase flow bubble projection area error analysis;

[0028] Figure 6 This is the bubble volume error analysis of gas-liquid two-phase flow based on the improved Mask R-CNN disclosed in the present invention. DETAILED DESCRIPTION

[0029] like Figure 1 As shown,

[0030] Step S1 is a data input module, which takes images of bubbles on the electrode surface through a water electrolysis experiment, and extracts each frame of the bubble image through video editing software.

[0031] Step S2 is a preprocessing module, which performs a series of preprocessing steps on the original bubble image, including random changes in image brightness, image noise increase, random flipping and mirroring of bubble positions in the image, and divides the bubble image into two parts, one for S3 and S4. Obtain a neural network model, and the remaining images can be recognized using the obtained neural network model, that is, step S5.

[0032] Step S3 is a data set production module. The bubble images are screened out, and the background area and bubble area are calibrated by Labelme software to produce the corresponding data set according to the requirements.

[0033] Step S4 is the learning module. The Mask R-CNN network is trained using the labeled image set to obtain the convolutional network model framework as follows: Figure 2 As shown in Figure 2, the model consists of a backbone network, a region proposal network (RPN) ROI Align, a recognition branch, and a mask branch. In the backbone network, Figure 3 As shown in the figure, the attention mechanism is introduced and combined with ResNet to form ResNet-CBAM.

[0034] The entire network training process includes the following steps:

[0035] Step 3.1: Input the RGB bubble image of size H×W×3 into ResNet-50. The input image passes through a 7x7 convolution layer with 64 output channels and a stride of 2, and the output size is H / 2×W / 2×64. The convolutional feature map is normalized to stabilize the training process, and the normalized feature map is activated by ReLU, and then passed through a 3×3 maximum pooling layer with a stride of 2, and the output size is H / 4×W / 4×64.

[0036] Step 3.2: After the first convolution layer, perform global average pooling and global maximum pooling on the feature map to obtain two 1D feature vectors with a shape of 1×1×64. Input these two feature vectors into a shared MLP, which contains a hidden layer. The hidden layer size is usually 64 / reduction. Add the two vectors output by the MLP and pass them through the sigmoid activation function to obtain the attention weight of each channel. Multiply each channel of the input feature map by the corresponding attention weight to obtain the enhanced feature map.

[0037] Step 3.3: Perform max pooling and average pooling on the channel dimension of the feature map enhanced by channel attention to obtain two 2D feature maps with a shape of H / 4×W / 4×1. Concatenate these two feature maps in the channel dimension to obtain a feature map with a shape of H / 4×W / 4×2. Use a 7×7 convolutional layer to reduce the dimension of the feature map to H / 4×W / 4×1. Use the sigmoid activation function to obtain the attention weight of each spatial position. Multiply each spatial position of the feature map output by the channel attention module by the corresponding attention weight to obtain the final enhanced feature map.

[0038] Step 3.4: The structure of the residual layer of ResNet-50 is divided into four stages, each of which contains several residual blocks: Stage 2 contains 3 residual blocks with an output size of H / 4×W / 4×256. Stage 3: contains 4 residual blocks with an output size of H / 8×W / 8×512. Stage 4: contains 6 residual blocks with an output size of H / 16×W / 16×1024. Stage 5 contains 3 residual blocks with an output size of H / 32×W / 32×2048.

[0039] Step 3.5: Apply the CBAM module again after the output of the last residual block of ResNet-50, and the processing process is the same as the CBAM module added after the first convolution layer. The final output is an enhanced feature map with a size of H / 32×W / 32×2048.

[0040] Step 3.7: The RPN network passes the enhanced feature map through a 3×3 convolution layer to reduce the number of channels of the feature map, but keeps the spatial size unchanged, with an output size of H / 32×W / 32×512, and then passes through two parallel 1×1 convolution layers, one of which is used for binary classification of the target and background, with an output size of H / 32×W / 32×(2*k), and the other 1×1 convolution layer is used for bounding box regression adjustment, with an output size of H / 32×W / 32×(4*k). Generate multiple anchors on the feature map, with k anchors of different scales and aspect ratios at each position.

[0041] Step 3.8: The candidate regions (RoIs) output by RPN, each region has a bounding box and score, and RoI Align is applied to each candidate region to adjust the RoI feature maps of different sizes to a uniform size. The aligned RoI feature map has a fixed size of 7×7×2048.

[0042] Step 3.9: Perform object classification on each RoI, pass through a fully connected layer and softmax activation, output the probability of each category, the shape is (N, num_classes + 1), perform bounding box regression on each RoI, pass through a fully connected layer, and output the bounding box adjustment parameters of each category, the shape is (N, num_classes * 4).

[0043] Step 3.10: The aligned RoI feature map is of size 14×14×2048. It is subjected to several convolutional layers for feature extraction, and the output size is 14×14×256. A deconvolution layer is used to enlarge the spatial size of the feature map to 28×28×256. A 1×1 convolution layer is then used to reduce the number of channels to the number of target categories. Finally, a segmentation mask of the target is generated for each RoI, and the sigmoid activation function is used to output the probability of each pixel.

[0044] Step 3.11: Calculate and output the network's precision, recall, and F1 score.

[0045] In order to verify the detection effect of the improved Mask R-CNN model proposed in this invention, 100 bubble images in the test set are used to perform detection and evaluation on the six models respectively. The calculation of the evaluation index parameters is shown in Table 1.

[0046] Table 1 Evaluation indicators of five models on the test set

[0047]

[0048]

[0049] By comparison, we can see that the improved Mask R-CNN network has an accuracy of 91.49% in bubble segmentation, which is 0.84%, 3.16% and 2.58% higher than the PA-net, SOLO and YOLOACT networks respectively.

[0050] Step S5 is a recognition module, which calculates the diameter of the bubble, the bubble projection area and the volume of the bubble by identifying the bounding box data and mask pixels.

[0051] In order to verify the effect of the improved algorithm on the extraction of key parameters of bubble morphology, the parameters obtained by the algorithm model are compared with the parameters obtained from the real bubble data, and the relative error is calculated to verify the accuracy of the algorithm. Four typical bubbles are randomly selected to calculate the error of morphological parameters. The calculation results Figure 4 , Figure 5 and Figure 6 shown.

[0052] It can be seen that the relative errors are controlled within 10%, and the relative error of the bubble diameter is within 5%, with high detection accuracy.

[0053] Step S6 is the output module.

Claims

1. A gas-liquid two-phase flow bubble recognition method based on improved Mask R-CNN, characterized in that: The steps include: Step 1. Take an image of bubbles on the electrode surface using a high-speed camera; Step 2. Label each frame of the image extracted by the video editing software. In order to enhance the generalization ability of the model, a series of preprocessing steps are performed on the labeled images to finally obtain the bubble label dataset; Step 3. Use the label data set obtained in step 2 to train the convolutional neural network to obtain a convolutional neural network model; Step 4. Input the bubble image to be identified into the convolutional network model obtained in step 3 for identification, and output the image segmentation and identification results.

2. The gas-liquid two-phase flow bubble recognition method based on improved Mask R-CNN according to claim 1, characterized in that: In step 1, the bubbles are captured by a high-speed camera on the electrode surface during the water electrolysis experiment.

3. The gas-liquid two-phase flow bubble recognition method based on improved Mask R-CNN according to claim 2, characterized in that: In step 2, Labelme software is used to calibrate the background area and bubble area of ​​the image, and data enhancement is performed by randomly changing the image brightness, increasing image noise, and randomly flipping the bubble image position.

4. The gas-liquid two-phase flow bubble recognition method based on improved Mask R-CNN according to claim 3, characterized in that: In step 3, by adding channel attention mechanism and spatial attention mechanism to ResNet, ResNet-CBAM is constructed. The entire network training process includes the following steps: S1: Input the H×W×3 RGB bubble image into ResNet-50, pass through the 7×7 convolution layer, output H / 2×W / 2×64, normalize the feature map, activate ReLU, and then pass through the 3×3 maximum pooling layer to output H / 4×W / 4×64; S2: After the first layer of convolution, two 1×1×64 feature vectors are obtained through global average pooling and global maximum pooling, which are input into the shared MLP. The MLP outputs are added, and the attention weight is obtained through sigmoid. Each channel feature map of the input feature map is multiplied by the attention weight to obtain the enhanced feature map. S3: Perform maximum pooling and average pooling on the enhanced feature map to obtain two feature maps of H / 4×W / 4×1. These two feature maps are concatenated in the channel dimension to obtain a feature map of shape H / 4×W / 4×2. The dimension is reduced to H / 4×W / 4×1 through a 7×7 convolutional layer. The sigmoid activation is used to obtain the attention weight. The output feature map is multiplied by the corresponding attention weight to obtain the final enhanced feature map. S4: After the last residual block of ResNet-50 is output, the CBAM module is applied to output an enhanced feature map with a size of H / 32×W / 32×2048; S5: The RPN network passes the enhanced feature map through a 3×3 convolutional layer, outputs H / 32×W / 32×512, and then passes it through two parallel 1×1 convolutional layers for binary classification and bounding box regression; S6: Each candidate region (RoIs) output by RPN has a bounding box and score. RoI Align is applied to adjust the RoI feature map to 7×7×2048; S7: Classify each RoI, pass the fully connected layer and softmax activation, output the probability of each category (N, num_classes + 1), and regress the bounding box through the fully connected layer, output the bounding box adjustment parameters (N, num_classes * 4); S8: Aligned RoI features: After several convolutional layers, the features are extracted and the output is 14×14×256. The size is enlarged to 28×28×256 through the deconvolution layer, and then the number of channels is reduced through the 1×1 convolution layer to generate the target segmentation mask, use sigmoid activation, and output the probability of each pixel.

5. The gas-liquid two-phase flow bubble recognition method based on improved Mask R-CNN according to claim 4, characterized in that: In step 3, the channel attention module (CAM) and the spatial attention module (SAM) are combined to obtain the convolutional block attention network (CBAM).

6. The gas-liquid two-phase flow bubble recognition method based on improved Mask R-CNN according to claim 5, characterized in that: In step 3, adding CBAM to the first and last convolution layers does not change the network, and the pre-trained parameters can be used.

7. The gas-liquid two-phase flow bubble recognition method based on improved Mask R-CNN according to claim 6, characterized in that: In step 3, the network's precision Precision and recall Recall are calculated and output, and the accuracy of bubble recognition is calculated separately.

8. The gas-liquid two-phase flow bubble recognition method based on improved Mask R-CNN according to claim 7, characterized in that: In step 4, the bounding box information of each bubble is saved in the recognition result of Mask R-CNN, the values ​​of the bounding box vertices are extracted to determine the aspect ratio of the bubble and the projection area of ​​the bubble is obtained through the mask information, and then the bubble morphological information is displayed on each recognized image.

9. The method for identifying bubbles in gas-liquid two-phase flow based on improved Mask R-CNN according to claim 8, characterized in that: In step 4, for the fully convolutional network model after training, a picture of gas-liquid two-phase flow to be identified is input into the model. The network can almost accurately identify the bubbles in the picture, and calculate and obtain the accuracy of bubble identification.

Citation Information

Cited By

  • High-precision gas-liquid two-phase flow pattern identification method based on data conversion and multi-head attention mechanism

    CN119337776A

  • Multidimensional information parallel targeting extraction method based on multiphase flow droplet / bubble spatial characteristics

    CN120126115A

  • A multi-dimensional information parallel targeting extraction method based on spatial characteristics of droplets / bubbles of multiphase flow

    CN120126115B

  • Image bubble noise removal method and system based on dual-channel characteristics

    CN120219230A

  • A method and system for removing image bubble noise based on dual-channel features

    CN120219230B