Optical image detection method for bubble plumes

Abstract

The present application relates to the technical field of submarine cold spring bubble plume detection, and specifically relates to a bubble plume optical image detection method. The method comprises the following steps: S1, obtaining a basic optical image containing a bubble plume, performing preprocessing, labeling the data image, assigning the labeled data image to a training set, a validation set and a test set, and obtaining a bubble plume optical image sample data set; S2, constructing an improved YOLOv5 model and initializing model parameters; S3, using a pseudo-label semi-supervised learning method to train the improved YOLOv5 model, and obtaining a final bubble plume detection model; S4, inputting a bubble plume optical image to be detected into the bubble plume detection model obtained in step S3, and outputting the position of the plume in the image and the corresponding category. The method can better extract complex target features from insufficient training samples, and more accurately detect and study cold spring bubble plumes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of submarine cold spring bubble plume detection, and particularly relates to a bubble plume optical image detection method. BACKGROUND

[0002] Submarine natural gas hydrate is an important natural resource, and a bubble plume refers to a gas column formed by submarine natural gas hydrate decomposition and leakage into seawater. Observation and detection of the bubble plume are of great significance for discovering natural gas hydrate.

[0003] At present, the commonly used detection methods for bubble plume recognition include acoustic detection, submarine in-situ detection, and seismic wave detection technology. Since each of the above methods has its own advantages and disadvantages, a multi-technology joint detection method is used in actual marine surveys. However, due to the scarcity of bubble plume optical image samples and the complexity of the background, the traditional manual feature extraction method faces challenges in detection effect and thus receives less research attention.

[0004] With the continuous development of machine vision technology, the use of deep learning methods for bubble plume optical image target detection has gradually increased. Deep learning models can automatically learn features, thus avoiding the need for manual feature extraction in traditional methods. In the task of bubble plume optical image target detection, deep learning models can directly learn features from raw images, better adapting to the characteristics of the data set.

[0005] However, there are still some challenges in using deep learning for bubble plume optical image target detection. For example, due to the limited number of bubble plume optical image samples, a large-scale training set needs to be constructed to avoid overfitting problems. Meanwhile, for the detection of small size targets or approximately overlapping targets in bubble plume optical images, the network structure and training strategy need to be further optimized.

[0006] Currently, the detection of bubble plume optical images has the following deficiencies:

[0007] (1) The number of sample images is limited, and they come from different sea areas, backgrounds, and lighting conditions, resulting in large cross-domain differences in the data set. This makes it difficult to train the model due to the scarcity of samples and the complexity of the background, and a larger training set needs to be constructed to improve the performance of the model;

[0008] (2) Sample videos are usually taken by ROV-mounted cameras, and the camera angle is large, resulting in a large proportion of the background occupying the entire image, while the bubble plume only occupies a small part of the image. This makes the detection of small targets (such as bubbles) less effective. In addition, due to the special nature of the underwater environment, the bubble plume image often appears blurred, further increasing the difficulty of target detection;

[0009] (3) The top layer of the bubble plume contains a large number of isolated bubbles, which are very similar to the seabed sediments in the background. This similarity makes it difficult for existing methods to accurately distinguish bubbles from the background, resulting in difficult reproducibility of performance and indicators. SUMMARY

[0010] The purpose of the present application is to overcome the above-mentioned defects existing in the prior art, and to provide a bubble plume optical image detection method which can better extract complex target features from insufficient training samples and more accurately detect and study cold spring bubble plumes.

[0011] The technical scheme of the present application is: a bubble plume optical image detection method, characterized by comprising the following steps:

[0012] S1, obtaining a basic optical image containing a bubble plume, pre-processing the obtained image, labeling the data image, assigning the labeled data image to a training set, a validation set and a test set, and thus obtaining a bubble plume optical image sample data set;

[0013] S2, constructing an improved YOLOv5 model and initializing the model parameters;

[0014] S3, training the improved YOLOv5 model using the pseudo-label semi-supervised learning method, and obtaining the final bubble plume detection model;

[0015] S4, inputting the bubble plume optical image to be detected into the bubble plume detection model obtained in step S3, outputting the position of the plume in the image and the corresponding class, and evaluating the model.

[0016] In the present application, the improved YOLOv5 model comprises a BackBone network, a Neck network and a Head module.

[0017] The BackBone network comprises a first module, a second module, a third module, a fourth module and a fifth module connected in sequence, and the size of the convolution kernel in the module is 1x1 or 3x3;

[0018] An input image with a size of 3x640x640 is input into the first module, the first module outputs a feature map with a size of 64x320x320 and is input into the second module; the second module outputs a feature map with a size of 256x160x160 and is input into the third module; the third module outputs a feature map with a size of 512x80x80 and is input into the fourth module; the fourth module outputs a feature map with a size of 1024x40x40 and is input into the fifth module; the fifth module outputs a feature map with a size of 2048x20x20 and is input into the pyramid pooling SPPF module; and a feature map with a size of 1024x20x20 is output to the Neck network.

[0019] The Neck network comprises a first Conv module, a first Upsample module, a first BiFPN_Concat2 module, a first C3 module, a second Conv module, a second Upsample module, a second BiFPN_Concat2 module, a second C3 module, an SENet attention module, a third Conv module, a third BiFPN_Concat3 module, a fourth Conv module, a fourth BiFPN_Concat2 module, and a fourth C3 module connected in sequence.

[0020] The SENet attention module comprises the following two steps:

[0021] S2.1, compression processing is performed on the feature map U, the features in the spatial dimension HxW are superimposed, and the feature map U is reduced to z in the spatial dimension HxW,

[0022]

[0023] wherein z c is the cth element of z;

[0024] S2.2, excitation operation is performed, the compressed and aggregated input information is subjected to dimension increasing and dimension decreasing operations:

[0025] s=F ex (z,W)=σ(g(z,W))=σ(W2δ(W1z)),

[0026] wherein σ represents a Sigmoid function, δ represents a ReLU function, W1 and W2 represent full connection operations;

[0027] The generated weight vector s is used to assign weights to the feature map U, and the required feature map

[0028]

[0029] The HxW number of each channel in the feature map U is multiplied by the weight value of the corresponding channel in s, and finally the feature map with channel attention is obtained Output to the next layer.

[0030] The first BiFPN_Concat2 module receives the feature map from the first Upsample module and the feature map of the fourth module in the BackBone network;

[0031] The second BiFPN_Concat2 module receives the feature map from the second Upsample module and the feature map of the third module in the BackBone network;

[0032] The third BiFPN_Concat3 module receives the feature map from the third Conv module, the feature map from the second Conv module, and the feature map of the fourth module in the BackBone network;

[0033] The fourth BiFPN_Concat2 module receives the feature map from the fourth Conv module and the feature map from the first Conv module, respectively.

[0034] Step S3 includes the following specific steps:

[0035] S3.1, data enhancement is performed on the plume optical image intercepted in the video, and the image data sample is a data set without labels;

[0036] S3.2, the bubble plume optical image sample data set in step S1 is sent to the improved YOLOv5 model constructed in step S2 to obtain a pre-training model capable of effectively extracting bubble plume features;

[0037] S3.3, the pre-training model trained in step S3.2 is used to label the unlabeled data set generated in step S3.1, and the newly labeled data set is combined with the basic optical image containing the bubble plume obtained in step S1;

[0038] S3.4, the data set obtained by combining in step S3.3 is divided into a training set, a validation set and a test set, and is re-input into the improved YOLOv5 model obtained in step S2 for training to obtain a final training model;

[0039] The final training model is used to verify the validation set, and the bubble plume target detection frame in each image is output.

[0040] The beneficial effects of the present application are:

[0041] (1) The application can better extract complex target features from insufficient training samples, and compared with traditional image detection methods, it can more accurately detect cold spring plume, in addition, the deep learning model can also improve its performance and accuracy over time through continuous learning and optimization;

[0042] (2) The trained model can be deployed on ROV or AUV to realize automatic and real-time detection of cold spring plume without human intervention, greatly reducing the workload of manual monitoring and analysis, and also reducing the possibility of human error;

[0043] (3) Compared with sonar equipment, optical equipment is generally cheaper, easier to obtain and maintain, and data processing speed is faster; compared with the target detection of cold spring plume sonar image, optical image can provide more detailed information such as flow rate, shape, etc., which helps to more accurately detect and study cold spring bubble plume. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 is the flow chart of the present application;

[0045] Figure 2 is the structure diagram of SENet attention module;

[0046] Figure 3 is the structure diagram of the improved YOLOv5 model;

[0047] Fig. 4(a) is a plume detection diagram of the existing YOLOv5 model;

[0048] Fig. 4(b) is a plume detection diagram of the improved YOLOv5 model using step S2;

[0049] Fig. 4(c) is a plume detection diagram of the bubble plume detection model of step S3. DETAILED DESCRIPTION

[0050] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0051] In the following description, specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be practiced in a variety of ways other than those described herein, and those skilled in the art can make similar extensions without departing from the scope of the present application. Therefore, the present application is not limited by the specific embodiments disclosed below.

[0052] The present application proposes a bubble plume optical image detection method, which comprises the following steps.

[0053] First, obtain the basic optical image containing bubble plumes, pre-process the obtained image, label the plume with a labeling software, and distribute the labeled data image according to a certain proportion into a training set, a validation set and a test set, thereby obtaining the optical image sample data set of bubble plumes.

[0054] Obtain the cold spring plume image data, which can be collected and processed through two ways.

[0055] The first way is to use open source video resources, select open source videos containing bubble plumes, and perform frame capture operation on the videos to extract the required image frames. These image frames may contain noise or other unnecessary information, so subsequent data processing and screening are required.

[0056] The second way is to obtain image data through a simulation experiment table. In the simulation experiment table, appropriate conditions such as bright background and clear plume can be set to ensure that high-quality sample images are obtained. Similarly, the image data obtained from the simulation experiment table also needs to be screened and pre-processed. For the obtained image data, in order to improve the quality and usability of the data, the CLAHE image enhancement method is used to smooth the image, which can enhance the contrast of the image while suppressing the noise of the image.

[0057] After obtaining the optical image, the bubble plume is labeled by using the rolabelimg labeling software. rolabelimg is different from other labeling software. It not only can realize the labeling frame of the target, but also supports the angle transformation of the labeling frame. The top bubble of the bubble plume in the real environment will drift with the direction of the ocean current, and the feature of rolabelimg that can transform the angle of the labeling frame is very suitable for the labeling of bubble plume, which can more accurately label the whole bubble plume area. After completing the labeling, the data set can be divided into training set, validation set and test set according to the ratio of 8:1:1.

[0058] In this embodiment, the bubble plume in the downloaded open source video is selected, and the segment containing the target video frame is intercepted. If there is no plume in a certain segment, the segment does not need to be selected. For the video with plume wake, every ten frames are saved as an image to reduce the data redundancy in the data set. The intercepted image data may contain noise, in order to improve the quality of the data, the CLAHE image enhancement method is used to enhance the image.

[0059] Second, build an improved YOLOv5 model and initialize the model parameters.

[0060] In this embodiment, the improved YOLOv5 model includes a BackBone network, a Neck network and a Head module. The improvements to the YOLOv5 model include the following aspects:

[0061] (1) The structure of the BackBone network of the YOLOv5 model is improved, and the depth of the network structure is increased. The input image is down-sampled by 32 times and the feature map is output, and the ability of the model to extract features is increased.

[0062] (2) The SENet attention module is introduced into the Neck network to construct an improved deep learning network architecture.

[0063] (3) The BiFPN_Concat splicing cross-layer connection method is introduced into the Neck network, which realizes the feature fusion method of adding feature maps from different layers. BiFPN is a feature pyramid network that introduces bidirectional connection, which can interact and fuse information at different scales, thereby better capturing semantic information at different scales. BiFPN_concat network includes multiple BiFPN modules, each module contains two main parts: upward and downward feature pyramid paths. In the upward path, BiFPN_concat converts low-resolution feature maps into high-resolution feature maps through upsampling and feature fusion operations. In the downward path, BiFPN_concat converts high-resolution feature maps into low-resolution feature maps through pooling operations and fuses them with the feature maps in the upward path.

[0064] (4) Update the loss function: the Alpha-EIoU loss function is used in the YOLOv5 model, which can better target positioning ability.

[0065] The BackBone network in this embodiment selects a residual network with fifty convolutional layers, divides the BackBone into multiple modules, each module has a series of residual blocks, each residual block includes multiple convolutional layers with a convolution kernel size of 1x1 or 3x3 and batch normalization layers. Each module receives the feature map output from the previous module and performs convolution operation, changes the size of the feature map and outputs to the next module. The improved BackBone network enhances the network's perception of different regions in the image through the added segmentation attention mechanism. The segmentation attention mechanism divides the attention into several subspaces, and learns different attention weights in each subspace, which can make the BackBone network better capture fine-grained features related to the classification task.

[0066] In this embodiment, the BackBone network includes a first module, a second module, a third module, a fourth module and a fifth module connected in sequence, and the size of the convolution kernel in each module is 1x1 or 3x3. An image with a size of 3x640x640 is input into the first module, and after the convolution operation of the first module, a feature map with a size of 64x320x320 is obtained. The feature map output by the first module is input into the second module, and after the convolution operation of the second module, a feature map with a size of 256x160x160 is obtained. The feature map output by the second module is input into the third module, and after the convolution operation of the third module, a feature map with a size of 512x80x80 is obtained. The feature map output by the third module is input into the fourth module, and after the convolution operation of the fourth module, a feature map with a size of 1024x40x40 is obtained. The feature map output by the fourth module is input into the fifth module, and after the convolution operation of the fifth module, a feature map with a size of 2048x20x20 is obtained.

[0067] The BackBone network in this embodiment can perform sampling at 32 times and output a five-dimensional feature map. An input image with a size of 3x640x640 is input into a residual network with 50 convolution layers and a segmentation attention mechanism, and a feature map with a size of 2048x20x20 is output at the fifth layer. Then, the feature map is input into a pyramid pooling SPPF module and a feature map with a size of 1024x20x20 is output to the Neck network.

[0068] The Neck network includes a first Conv module, a first Upsample module, a first BiFPN_Concat2 module, a first C3 module, a second Conv module, a second Upsample module, a second BiFPN_Concat2 module, a second C3 module, an SENet attention module, a third Conv module, a third BiFPN_Concat3 module, a fourth Conv module, a fourth BiFPN_Concat2 module, and a fourth C3 module connected in sequence.

[0069] As Figure 2 , for any kind of transformation, such as convolution, a corresponding SENet attention module is constructed to perform feature recalibration. The input of the SENet attention module is transformed by convolution to generate a feature map U, U∈HxWxC, where W represents the width, H represents the height, and C represents the number of channels. The SENet attention module in this embodiment includes a compression and excitation part.

[0070] First, the global average pooling operation is used to compress the feature map U, and the feature mapping in the spatial dimension (H x W) is stacked to generate a channel symbol. Reducing the feature map U to its spatial dimension H x W produces z, which produces an overall distribution of embedded channel direction characteristics, so that the generated channel-level statistics z contains context information, which alleviates the channel dependency problem.

[0071]

[0072] where z c is the cth element of z.

[0073] Second, the excitation operation is performed to compress the input information after aggregation, and dimensionality reduction and dimensionality increase operations are performed:

[0074] s = F ex (z, W) = σ(g(z, W)) = σ(W2δ(W1z)),

[0075] where σ represents the Sigmoid function, δ represents the ReLU function, and W1 and W2 represent the full connection operation.

[0076] The implementation of the excitation operation mainly includes two full connection layers and a ReLU activation function layer. The first full connection layer is used to compress the features after the compression operation in the channel dimension; then a ReLU activation function layer is used to keep the dimension of the output features unchanged; then the second full connection layer is used to restore the dimension of the output features to the channel number of the features after the compression operation, and then the Sigmoid function is used to output a weight vector s with the same number of input features.

[0077]

[0078] As shown in the formula, the generated feature vector s is multiplied by the corresponding channel of the feature map U, that is, the H x W number of each channel in the feature map U is multiplied by the weight of the corresponding channel in s. Finally, the feature map with channel attention is output to the next layer, and its size is exactly the same as that of the feature map U.

[0079] The feature map from the second C3 module is compressed in the compression step of the SENet attention module, and the input feature map is compressed into a scalar value by the global average pooling operation, thereby obtaining a channel importance vector. In the excitation step of the SENet attention module, the SENet attention module performs nonlinear transformation and activation on the channel importance vector through an MLP to obtain a channel weight vector. Finally, the channel weight vector is multiplied by the input feature map to obtain a weighted feature map, where the channel weight vector controls the importance of each channel.

[0080] The first BiFPN_Concat2 module receives the feature map from the first Upsample module and the feature map of the fourth module in the BackBone network. The second BiFPN_Concat2 module receives the feature map from the second Upsample module and the feature map of the third module in the BackBone network. The third BiFPN_Concat3 module receives the feature map from the third Conv module, the feature map of the second Conv module, and the feature map of the fourth module in the BackBone network. The fourth BiFPN_Concat2 module respectively receives the feature map from the fourth Conv module and the feature map of the first Conv module. The feature maps of different scales are spliced in the channel dimension for feature fusion, which can enable the model to utilize low-level and high-level feature information simultaneously, thereby improving the detection and positioning ability of the target, obtaining a network with stronger feature extraction and expression ability for target detection task.

[0081] In this embodiment, the loss function is selected as Alpha-EIoU for target detection. Alpha-EIoU is a loss function used in target detection algorithms, which can effectively solve the problems of class imbalance and difficult sample imbalance. Traditional IOU loss function treats all targets as having the same importance when training the target detection model, and cannot handle class imbalance and difficult sample imbalance problems. The Alpha-EIoU loss function associates the importance of the target with its difficulty by introducing a weight coefficient alpha, thereby better handling class imbalance and difficult sample imbalance problems.

[0082] The calculation formula of the Alpha-EIoU loss function is:

[0083] L = -log(IOU) + alpha*log(1-IOU),

[0084] where IOU is the IOU value between the predicted bounding box of the target detection model and the real bounding box.

[0085] alpha is a weight coefficient used to adjust the importance of the target. When alpha is 0, the Alpha-EIoU loss function is equivalent to the traditional IOU loss function; when alpha is 1, the Alpha-EIoU loss function will pay more attention to difficult samples.

[0086] Alpha-EIoU, compared with other loss functions, not only fully considers the three geometric factors of bounding box regression: overlapping area, center point distance, and aspect ratio. It also calculates the difference value of width and height instead of aspect ratio, and introduces FocalLoss to solve the problem of difficult sample imbalance.

[0087] The advantage of the Alpha-EIoU loss function is that it can adjust the weight of the loss function according to the difficulty of the target, so as to better train the model. For example, for difficult samples, their weights can be increased, so that the target detection model pays more attention to these difficult-to-detect targets, improving the detection performance and effect.

[0088] In this embodiment, initializing the model parameters includes:

[0089] (1) Initialize the network weights, learning rate, batch size, and iteration number of the improved YOLOv5 model.

[0090] (2) Introduce multi-scale training to make the YOLOv5 model more powerful and have generalization ability, better adapting to the needs of different scenarios and tasks.

[0091] In this embodiment, the initial epochs is set to 100, the batch size is set to 16, the input image size is set to 3x640x640, the optimizer is set to SGD, and the initial learning rate is set to 0.01. Because the improved YOLOv5 model changes the BackBone structure, the yolov5s.pt pre-trained model provided by YOLOv5 cannot be loaded, and thus loses its function. Therefore, this training does not use the pre-trained model of the.pt type provided by the official. According to the characteristics of the bubble plume, within a reasonable range, 285 samples from open source videos are selected, and the rolabelimg software is used to label the 285 samples.

[0092] To solve the problem of performance degradation caused by the inconsistency of target size distribution in the training set and the test set, this embodiment adopts the method of random multi-scale training. In the actual bubble plume detection task, since the size and proportion of the target are uncertain, random multi-scale training can alleviate the problems of data imbalance and class imbalance. By training with different scales, the model can better capture bubble plume targets of various sizes and proportions, thereby improving the detection performance of these targets. During training, the resolution of the input image is randomly changed within the range of [0.6p, 1.1p], where p represents the original input resolution of the model.

[0093] Thirdly, the improved YOLOv5 model is trained using the pseudo-label semi-supervised learning method, and the final bubble plume detection model is obtained. The specific steps include:

[0094] (1) The plume optical image intercepted in the video is subjected to data augmentation methods such as cropping and flipping to generate more diverse augmented image data samples. This image data sample is a data set without labels.

[0095] (2) The data of the training set, validation set and test set that have been labeled and assigned using rolabelimg in the first step are input into the improved YOLOv5 model in the second step to obtain a pre-trained model that can effectively extract the features of bubble plumes.

[0096] (3) The pre-trained model trained in step (2) is used to label the unlabeled data set generated after data augmentation in step (1), and the newly labeled data set is merged with the basic optical images containing bubble plumes obtained in the first step.

[0097] (4) Finally, the data set obtained by merging in step (3) is divided into training set, validation set and test set, and is re-input into the improved YOLOv5 model obtained in the second step for training to obtain the final training model. The final training model is used to verify on the validation set, output the bubble plume target detection frame in each image, and evaluate the detection ability of the model. Among them, the training set is used for model training and parameter optimization, the validation set is used for adjusting the hyperparameters of the model and evaluating the performance of the model, and the test set is used for evaluating the accuracy and generalization ability of the final model. By dividing the data set, the training process of the model can be better monitored and the effect of the model can be better evaluated to ensure the reliability and effectiveness of the model.

[0098] In this embodiment, 285 basic optical images are obtained by intercepting video frames in the first step. Each of the 285 basic optical images is respectively subjected to cropping, translation, changing brightness, adding noise, rotating angle, mirroring, and random occlusion operation by data augmentation method to generate ten new sample data, and finally 2850 enhanced image data samples are obtained. The 2850 enhanced image data samples are unlabeled data sets.

[0099] The 285 basic optical image samples are input into the improved YOLOv5 model obtained in step two for pre-training to generate a pre-trained model that can effectively extract the features of bubble plumes. Then, the pre-trained model is used to label and annotate the 2850 enhanced sample images. The 285 samples and the 2850 annotated samples are merged and divided into training set, validation set and test set according to the ratio of 8:1:1, and input into the improved YOLOv5 model for retraining to obtain detection images and print training results such as precision, recall, mAP@0.5, mAP@0.5_0.9, etc.

[0100] Precision (Precision), also known as recall, is a measure of the accuracy of a prediction. It is the probability that a sample that is predicted to be positive is actually positive. Its formula is as follows:

[0101]

[0102] Recall, also called the recall rate, is for the original sample, which means the probability of being predicted as positive in the actual positive sample, and its formula is as follows:

[0103]

[0104] Where TP represents the number of features predicted as positive and actually positive; FP represents the number of features predicted as positive and actually negative; TN represents the number of features predicted as negative and actually negative; and TN represents the number of features predicted as negative and actually negative.

[0105] Intersection over Union (IoU) is a concept used in target detection, and IoU calculates the intersection rate of the "predicted bounding box" and the "real bounding box", that is, the ratio of their intersection and union.

[0106] mAP represents the overall average precision.

[0107] mAP@0.5 represents the average precision of the model when the IoU threshold is 0.5.

[0108] mAP@0.5_0.95 represents the average precision of the model when the IoU threshold is 0.5 to 0.95.

[0109] Table 1 Comparison of parameters before and after improvement of YOLOv5 model

[0110]

[0111] The results show that the improved YOLOv5 model has progress in precision, recall, mAP@0.5 and mAP@0.5_0.95. Among them, the precision, mAP@0.5, mAP@0.5_0.95 are increased by 2.7%, 0.9%, 4.1% respectively. The bubble and feather flow detection model after pseudo-label semi-supervised learning has an increase of 0.2% and 0.6% in precision and mAP@0.5 respectively, and an increase of 2.5% in mAP@0.5_0.95, reaching 68.6%, that is, the average precision of the model when the IoU threshold is 0.5 to 0.95 is high.

[0112] Fourthly, the optical image of the bubble and feather flow to be detected is input into the bubble and feather flow detection model obtained in the third step, and the position of the feather flow in the image and the corresponding category are output.

[0113] In this embodiment, the image to be detected is input into the existing YOLOv5 model, the improved YOLOv5 model obtained in the second step, and the final bubble plume detection model obtained in the third step, respectively. Fig. 4(a) is a plume physical detection image output by the existing YOLOv5 model, Fig. 4(b) is a plume physical detection image output by the improved YOLOv5 model in the second step, and Fig. 4(c) is a plume physical detection image output by the bubble plume detection model in the third step. According to the detection results in Figs. 4(a), 4(b) and 4(c), it can be seen that the confidence of the detection result in Fig. 4(c) is the largest, which is 0.94. Therefore, it can be seen that the performance of the final bubble plume detection model obtained in the third step is higher. Figures 4(a) to 4(c)

[0114] The above two kinds of data prove that the model trained by the method of the present application has good target detection performance for bubble plume, and the effectiveness of the method of the present application is verified.

[0115] The above describes in detail the bubble plume optical image detection method provided by the present application. The principles and implementation manners of the present application are described by using specific examples, and the above description of the embodiments is only used to help understand the method of the present application and its core idea. It should be pointed out that, for those skilled in the art, some improvements and modifications can be made to the present application without departing from the principles of the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application. The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.​

Claims

1. A method for detecting bubble plume flow using optical images, characterized in that, The method comprises the following steps: S1, obtaining a basic optical image containing a bubble plume, pre-processing the obtained image, labeling the data image, assigning the labeled data image to a training set, a validation set and a test set, and thus obtaining a bubble plume optical image sample data set; S2, constructing an improved YOLOv5 model and initializing model parameters; S3, training the improved YOLOv5 model using a pseudo-label semi-supervised learning method, and obtaining a final bubble plume detection model; S4, inputting the bubble plume optical image to be detected into the bubble plume detection model obtained in step S3, outputting the plume position and its corresponding category in the image, and performing model evaluation; The improved YOLOv5 model comprises a BackBone network, a Neck network and a Head module; The BackBone network comprises a first module, a second module, a third module, a fourth module and a fifth module connected in sequence, and the size of the convolution kernel in the module is 1×1 or 3×3; An input image with a size of 3×640×640 is input into the first module, the first module outputs a feature map with a size of 64×320×320, and inputs the feature map into the second module; the second module outputs a feature map with a size of 256×160×160, and inputs the feature map into the third module; the third module outputs a feature map with a size of 512×80×80, and inputs the feature map into the fourth module; the fourth module outputs a feature map with a size of 1024×40×40, and inputs the feature map into the fifth module; the fifth module outputs a feature map with a size of 2048×20×20, and inputs the feature map into a pyramid pooling SPPF module; and a feature map with a size of 1024×20×20 is output to the Neck network; The Neck network comprises a first Conv module, a first Upsample module, a first BiFPN_Concat2 module, a first C3 module, a second Conv module, a second Upsample module, a second BiFPN_Concat2 module, a second C3 module, an SENet attention module, a third Conv module, a third BiFPN_Concat3 module, a fourth Conv module, a fourth BiFPN_Concat2 module and a fourth C3 module connected in sequence.

2. The method of claim 1, wherein the method further comprises: The SENet attention module comprises the following two steps: S2.1, compressing the feature map U to reduce the spatial dimension H×W of the feature map U to z, , wherein, is the cth element of z; S2.2, performing an excitation operation on the compressed and aggregated input information to perform dimension increasing and dimension reducing operations: ， wherein denotes a Sigmoid function, and δ denotes a ReLU function, and denotes a fully connected operation; The generated weight vector s is used to weight the feature map U, to obtain the required feature map , ， The number of HxW of each channel in the feature map U is multiplied by the weight value of the corresponding channel in s, and finally the feature map with channel attention is obtained Output to the next layer.

3. The bubble plume optical image detection method according to claim 1, characterized in that, The first BiFPN_Concat2 module receives the feature map from the first Upsample module and the feature map from the fourth module in the BackBone network; The second BiFPN_Concat2 module receives the feature map from the second Upsample module and the feature map from the third module in the BackBone network; The third BiFPN_Concat3 module receives the feature map from the third Conv module, the feature map from the second Conv module, and the feature map from the fourth module in the BackBone network; The fourth BiFPN_Concat2 module receives the feature map from the fourth Conv module and the feature map from the first Conv module respectively.

4. The method of claim 1, wherein the method further comprises: Step S3 includes the following specific steps: S3.1, data enhancement is performed on the plume optical image intercepted in the video; S3.2, the bubble plume optical image sample data set in step S1 is input into the improved YOLOv5 model constructed in step S2 to obtain a pre-training model capable of effectively extracting bubble plume features, and the image sample data is a data set without labels; S3.3, the pre-training model trained in step S3.2 is used to label the label-free data set generated in step S3.1, and the newly labeled data set is combined with the basic optical image containing the bubble plume obtained in step S1; S3.4, the data set obtained by combining in step S3.3 is divided into a training set, a validation set and a test set, and is re-input into the improved YOLOv5 model obtained in step S2 for training to obtain a final training model; The final training model is used to verify the validation set, and the bubble plume target detection frame in each image is output.

Images