An underwater blurred image target detection method and device

Abstract

The application discloses an underwater blurred image target detection method and device, and the method comprises the following steps: collecting an underwater blurred image dataset, and performing category labeling and bounding box labeling on target objects in the image; a neural network model is built by improving YOLO11n, and the labeled dataset is used for training to obtain a target detection model; wherein the improvement of YOLO11n comprises the following steps: (1) replacing the third, fourth and fifth CBSs of the feature extraction network with a hybrid pooling convolution module, (2) improving the SPPF of the feature extraction network: adding a hybrid pooling operation between the first CBS and the first max pooling as a residual input branch of the Concat in the SPPF; the obtained target detection model is used for target detection on an underwater blurred image to be detected, and an image with a bounding box and a target category in the bounding box are output. The application can improve the accuracy and efficiency of blurred image detection in an underwater scene.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image recognition, and particularly relates to an underwater blurred image target detection method and device. BACKGROUND

[0002] Unlike the conventional target detection environment, due to the existence of a large amount of suspended medium in the underwater environment and the influence of light absorption and scattering under water, the underwater image has problems such as detail loss, imaging blur, low contrast, and serious image degradation. In addition, underwater organisms are often densely distributed, and have problems such as small targets, stacking, and overlapping. Therefore, the underwater image target detection has problems such as low detection accuracy, poor accuracy, false detection, and high missing detection rate.

[0003] In recent years, the deep learning technology has developed rapidly, and the target detection technology has made great progress. The YOLO series is one of the classic models in the target detection field, and is known for its high real-time performance, fast detection speed, and small parameter quantity. YOLO11 is an iterative version of YOLOV8 and is one of the most advanced YOLO models, with higher accuracy and detection efficiency. However, when it comes to the detection of underwater blurred images, the detection performance of YOLO11 is limited by the underwater environment, and cannot well play the effectiveness of detection. SUMMARY

[0004] The present application provides an underwater blurred image target detection method and device, aiming to solve the problem of blurred image detection in underwater scenes and improve the accuracy and efficiency of detection.

[0005] To achieve the above technical purpose, the present application adopts the following technical scheme:

[0006] An underwater blurred image target detection method, comprising:

[0007] Step 1, collect an underwater blurred image dataset, and perform class labeling and bounding box labeling on the target objects in the images to obtain a labeled dataset;

[0008] Step 2, build a neural network model, and train the built neural network model using the labeled dataset to obtain a target detection model;

[0009] The neural network model is improved from YOLO11n, and the improvement includes: (1) replacing the 3rd, 4th, and 5th CBS modules of the feature extraction network of YOLO11n with a hybrid pooling convolution module, and (2) improving the SPPF module of the feature extraction network of YOLO11n: adding a hybrid pooling operation between the 1st CBS module and the 1st max pooling of the SPPF module, and using the hybrid pooling operation as a residual input branch of Concat in the SPPF module;

[0010] Step 3, using the trained target detection model to detect the underwater blurred image to be detected, outputting an image with a bounding box and a target class in the bounding box.

[0011] Further, the mixed pooling convolution module is expressed as:

[0012] ;

[0013] Wherein, Y represents the output feature map of the mixed pooling convolution module, X represents the input feature map of the mixed pooling convolution module, SiLU represents an activation function, BN represents a batch normalization operation, conv2d represents a two-dimensional convolution operation, Maxpool represents a maximum pooling operation, and Avgpool represents an average pooling operation.

[0014] Further, the improvement of the neural network model on YOLO11n further includes: for the feature map P2 obtained by the first C3K2 module of the feature extraction network of YOLO11n, adding a connection path at the lowermost layer of the feature fusion network of YOLO11n: processing and splicing the feature map P2 and the last obtained feature map from top to bottom, and then performing fusion processing from bottom to top.

[0015] Further, the improved feature fusion network based on YOLO11n includes two parts of a top-down path and a bottom-up path.

[0016] The top-down path starts from the up-sampling operation of the topmost output X5 of the feature extraction network, and the up-sampling operation includes, from input to output, nearest neighbor up-sampling, a Concat module, and a C3K2 module, and generates feature maps X4, X3, and X2 through three up-sampling operations; wherein the Concat module of each up-sampling operation performs feature fusion on the feature map obtained by the nearest neighbor up-sampling and the corresponding size feature maps P4, P3, and P2 obtained by the feature extraction network through horizontal connection.

[0017] The bottom-up path starts from the down-sampling operation of the lowermost output of the top-down path, and the down-sampling operation includes, from input to output, a CBS module, a Concat module, and a C3K2 module, and generates feature maps F3, F4, and F5 through three down-sampling operations, as inputs of the detection module; wherein the Concat module of each down-sampling operation performs feature fusion on the feature map obtained by the CBS module and the corresponding size feature maps X3, X4, and X5 of the top-down path through horizontal connection.

[0018] Further, the loss function of the neural network model for obtaining the target detection model adopts BCE as the classification loss function of the target class, adopts DFL and CIOU as the loss function of the boundary box regression, and then the classification loss function and the regression loss function are weighted to obtain the overall loss function, which is represented as:

[0019] ;

[0020] In the formula, The overall loss function is represented as 、 、 BCE classification loss, DFL regression loss and CIOU regression loss are represented as 、 、 corresponding weights.

[0021] An underwater fuzzy image target detection device comprises:

[0022] A data collection and labeling module is configured to collect an underwater fuzzy image dataset, label the target objects in the images in categories and boundary boxes, and obtain a labeled dataset;

[0023] A model training module is configured to train a neural network model using the labeled dataset to obtain a target detection model;

[0024] The neural network model is improved from YOLO11n, and the improvement includes: (1) replacing the third, fourth and fifth CBS modules of the feature extraction network of YOLO11n with a hybrid pooling convolution module, and (2) improving the SPPF module of the feature extraction network of YOLO11n: adding a hybrid pooling operation between the first CBS module and the first maximum pooling of the SPPF module as a residual input branch of Concat in the SPPF module;

[0025] A target detection module is configured to configure the trained target detection model to detect the target of the underwater fuzzy image to be detected, output the image with the boundary box and the target category in the boundary box.

[0026] The underwater fuzzy image target detection method and device provided by the application have less model parameter quantity, can fully adapt to the particularity of the underwater environment, realize efficient and high-precision detection of underwater fuzzy images, have low missed detection rate, and can effectively complete the task requirements of underwater fuzzy image detection. Compared with the existing underwater image detection technology, the application has the following advantages:

[0027] (1) The network model has small parameter quantity, high model efficiency and is easy to deploy on a mobile terminal.​

[0028] (2) The application designs a mixed-pooling convolution and feature fusion path optimization, improves the feature extraction capability of the model, effectively avoids the loss of important detail information due to the complexity of the underwater environment, and further improves the performance of the target detection model.

[0029] (3) It has strong practicability and generalization ability. The application achieves good results on the public data set URPC2020, and the detection performance of underwater blurred images is fully verified. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 The figure is a network structure diagram of the underwater blurred image target detection of the embodiment of the application.

[0031] Figure 2 The figure is a mixed-pooling convolution MAConv structure diagram of the embodiment of the application.

[0032] Figure 3 The figure is a SPPF_improved structure diagram obtained by improving SPPF of the embodiment of the application.

[0033] Figure 4 The figure is a comparison diagram of the detection effect of the embodiment of the application and YOLO11n on picture one of the URPC2020 test set.

[0034] Figure 5 The figure is a comparison diagram of the detection effect of the embodiment of the application and YOLO11n on picture two of the URPC2020 test set. DETAILED DESCRIPTION

[0035] The embodiment of the application will be described in detail below. The embodiment is based on the technical scheme of the application and gives a detailed implementation mode and specific operation process, which further explains and describes the technical scheme of the application.

[0036] The embodiment provides a kind of underwater blurred image target detection method, can effectively strengthen the feature extraction capability of underwater target, fully fuse detail information, realize higher detection precision and efficiency. Referring to Figure 1 , including the following steps.

[0037] Step 1, construct real underwater scene underwater blurred image data set, including model training set, verification set and test set, the training set is used for model training, the verification set is used for verifying the precision of training model, and the test set is used for testing the generalization precision of model.

[0038] The data set constructed by the embodiment is the open-source target detection data set URPC2020 of the underwater target detection competition-optical event, which contains four main categories of sea cucumber, sea urchin, starfish and scallop, a total of 7543 pictures, including 5543 training set pictures, 800 test set A pictures and 1200 test set B pictures. During the experiment, the training set pictures are randomly divided into experimental training set and experimental validation set according to 8:2. The resolution of the image includes multiple scales such as 720*405, 1920*1080 and 3840*2160.

[0039] Step 2.1, design a neural network model for underwater blurred image target detection, which is improved from YOLO11n and includes three parts of feature extraction network, feature fusion network and detection module.

[0040] The feature extraction network is implemented as follows: for the training set data, first, the input image is preprocessed, including image scaling, normalization and mosaic data enhancement. The image scaling operation adjusts the input images of different sizes to a uniform size by the LetterBox scaling method, while maintaining the aspect ratio to avoid distortion, and the other parts are filled with gray. At the same time, the channel number is adjusted; the normalization linearly maps the image pixel value from [0, 255] to [0, 1], and normalizes it to a distribution with a mean of 0 and a standard deviation of 1, to speed up model convergence; the core idea of this method is to randomly combine multiple (usually four) pictures into a new training picture, thereby enriching the background information and target diversity and improving the generalization ability of the model. The implementation process of the Mosaic data enhancement method is as follows:

[0041] (1) Randomly select pictures: randomly select four pictures from the original data set and randomly select picture splicing reference point coordinates (x, y).

[0042] (2) Random scaling and cropping: randomly scale and crop each picture at a random position so that they can fit the splicing canvas and determine the position of each picture in the final combined picture.

[0043] (3) Picture splicing: place the four pictures in the four quadrants (top left, top right, bottom left, bottom right) of a large canvas, and adjust the position of each picture according to the randomly generated reference point position.

[0044] (4) Adjust the label: adjust the bounding box label of the target in each picture according to the coordinates of the spliced new picture.

[0045] Through the above pretreatment operation, a gray image with a size of 640*640*3 can be obtained as the input of the network, and then a shallow feature map P2 is obtained by continuously passing through two CBS convolution downsampling and a C3K2 module. At this time, if only ordinary CBS convolution is used for downsampling, the details and part of the local features of the target may be lost, and the underwater target is often similar to the background and densely distributed, and the input image noise is obvious. In order to better extract image features, the application improves and adopts a mixed pooling convolution MAConv and a C3K2 module as a group of feature extraction modules. The mixed pooling convolution processes the input feature map in parallel through maximum pooling and average pooling, which can not only retain the local features such as the edges and textures of the object, but also smooth the local fluctuations, suppress the noise, provide global smooth features, highlight the background information and form a more comprehensive feature representation, thereby improving the underwater target detection accuracy. The structure of the mixed pooling convolution MAConv is referred to as Figure 2 , which can be expressed by a formula as follows:

[0046] ;

[0047] , wherein Y represents the output feature map of the mixed pooling convolution module, X represents the input feature map of the mixed pooling convolution module, SiLU represents an activation function, BN represents a batch normalization operation, conv2d represents a two-dimensional convolution operation, Maxpool represents a maximum pooling operation, and Avgpool represents an average pooling operation.

[0048] Through the three groups of feature extraction modules, the feature maps P3, P4 and P5 containing high semantic information can be obtained in sequence, wherein P2, P3 and P4 are added to the feature fusion path of the feature fusion network through a Concat operation.

[0049] In order to make the output of the feature extraction network contain more rich positioning information and semantic information, the application further improves the spatial pyramid pooling layer SPPF. After the spatial pyramid pooling layer SPPF is reduced in channel dimension through the CBS convolution, the mixed pooling operation is introduced to increase a residual input branch of Concat, so as to integrate more fine features, make the prediction box closer to the real box, improve the detection accuracy of the model, and the structure of the improved spatial pyramid pooling layer SPPF_improved is referred to as Figure 3 . Then, P5 continues to pass through the improved spatial pyramid pooling layer SPPF_improved and the multi-head attention module C2PSA for deep feature extraction, so as to capture object information of different scales and obtain a multi-scale feature representation X5 as the output of the feature extraction network and the input of the feature fusion network.

[0050] The feature fusion network adopts a path aggregation network (PAN) as a feature extraction method, and a P2 connection path is added. The P2 feature map is obtained by processing the original input picture through two CBS convolutions and a C3K2 module, and has a large resolution and rich position information. By adding the P2 connection path, the high semantic feature map is fused with more accurate position information, thereby retaining more detailed information and effectively solving the problem of insufficient information fusion in the forward propagation process of the network. The feature fusion network is divided into a top-down path and a bottom-up path. The top-down path starts from the output X5 of the feature extraction network and performs upsampling operations, which include sequentially performing nearest neighbor upsampling, Concat, and C3K2 processing on the input. Through three upsampling operations, feature maps X4, X3, and X2 are generated in sequence, where Concat represents feature fusion through horizontal connection with feature maps P4, P3, and P2 of the corresponding size. The bottom-up path starts from the output X2 of the top-down path and performs downsampling. In this process, the information of each layer of feature map is fused with the corresponding size feature map of the top-down path, and finally feature maps F3, F4, and F5 of the same size as P3, P4, and P5 are obtained as the input of the detection module.

[0051] The detection module is implemented by taking the above-mentioned output feature maps F3, F4, and F5 as the input of three detection heads, respectively, obtaining the original prediction output of the model, and calculating the corresponding loss function to guide the optimization of the model. The loss function takes BCE as the classification loss function, and DFL and CIOU as the regression loss function. The total loss is obtained by weighted sum, which can be represented as follows: Finally, the original prediction output of the model is converted into the final underwater target detection output through post-processing operations.

[0052] The , , in the total loss respectively represent the BCE classification loss, the DFL regression loss, and the CIOU regression loss, are , , corresponding weights, which can be adjusted according to actual needs. Specifically:

[0053] ;

[0054] ;

[0055] ;

[0056] wherein, represents the number of categories of target objects, represents the the true label value of the class sample, the prediction probability of the model for the first class sample.

[0057] the true label value of the sample, , respectively represent the left and right two integers closest to the true label , the probability that the model prediction output is equal to , the probability that the model prediction output is equal to .

[0058] represent the distance between the predicted frame and the center point of the true frame, is the diagonal distance of the minimum circumscribed rectangle, represent the relative value of the length-width ratio of the frame, is a smoothing coefficient, and IOU is the intersection over union, which is an important indicator for measuring the similarity between the predicted value and the true value, representing the proportion of the common part of the predicted frame and the true frame area to the total area of the two, and the larger the IOU value, the closer the predicted structure to the true value, and the better the model performance.

[0059] The post-processing operation mainly includes three key steps:

[0060] (1) Remove low-probability prediction frames by confidence filtering;

[0061] (2) Decode the bounding box to convert the offset in distribution form to the actual coordinates;

[0062] (3) Use non-maximum suppression (NMS) to remove overlapping redundant frames, and restore the coordinates according to the original image size;

[0063] (4) Finally output the bounding box, confidence and class information of each target.

[0064] Step 2.2, training a neural network to obtain a target detection model, including training the underwater blurred image target detection network designed in step 2; the experimental environment of the embodiment is shown in Table 1.

[0065] During the training process, some hyperparameters are set as follows: the input image size is 640*640, the number of worker threads is set to 8, the batch size is 32, SGD is used as the optimizer, the initial learning rate is 0.01, the number of iterations is 150, no pre-trained weights are used, and other parameters are default values.

[0066] ;

[0067] Step 3, verify the target detection model trained on the test set, and specifically test on the URPC 2020 public data set.

[0068] In this embodiment, precision, recall, and mean average precision (mAP) are used as performance evaluation indicators of the model. The calculation formula is as follows:

[0069]

[0070]

[0071]

[0072] Among them, TP (True Positive) represents the number of samples that the model predicts as positive and is actually positive; FP (False Positive) represents the number of samples that the model predicts as positive but is actually negative; TN (True Negative) represents the number of samples that the model predicts as negative and is actually negative; FN (False Negative) represents the number of samples that the model predicts as negative but is actually positive. AP i represents the average precision of the i-th class, and the value is equal to the area surrounded by the i-th class precision-recall curve and the coordinate axis, which represents the precision performance of the model at various recall rates.

[0073] To facilitate understanding of the effectiveness of the technical effects of the present application, the ablation experiment and the comparative experiment of the present application are shown in Tables 2 and 3.

[0074]

[0075]

[0076] According to the results in Tables 2 and 3, the improved method of the present application effectively improves the performance of the model. Compared with the model detection method based on the original YOLO11n, the recall rate of the improved YOLO11n network structure model based on the present application is increased by 3.5%, mAP50 is increased by 2.3%, mAP50-95 is increased by 3%, and the parameter amount is only increased by 0.58M. The results show that the present application effectively improves the missed detection and repeated detection of targets in the underwater scene, and significantly improves the recognition accuracy and positioning precision of underwater targets. Figure 4 and Figure 5 are the effect comparison diagrams of the target detection of the present application method and YOLO11n on two different underwater blurred images, respectively, wherein, Figure 4 (a) and Figure 5 (a) is an original labeled underwater blurred image, Figure 4 ​​​​​(b) and Figure 5 (b) is the target detection result of the original underwater blurred image based on the YOLO11n network structure, Figure 4 (c) and Figure 5 (c) is the target detection result of the original underwater blurred image based on the improved YOLO11n network structure in the method of the application, wherein the boxes of different colors represent different class labels.

[0077] The above embodiments are the preferred embodiments of the application, and those skilled in the art can also make various transformations or improvements on the basis of the above, and these transformations or improvements should all belong to the scope of protection of the application without departing from the general concept of the application.

Claims

1. An underwater blurred image object detection method, characterized in that, The application relates to an underwater fuzzy image target detection method. Step 1, collecting an underwater fuzzy image dataset, performing category labeling and boundary box labeling on target objects in the images to obtain a labeled dataset; Step 2, building a neural network model and training the built neural network model using the labeled dataset to obtain a target detection model; The neural network model is improved based on YOLO11n, and the improvement includes the following two aspects: (1) replacing the third, fourth and fifth CBS modules of the feature extraction network of YOLO11n with a mixed pooling convolution module, and (2) improving the SPPF module of the feature extraction network of YOLO11n: adding a mixed pooling operation between the first CBS module and the first maximum pooling of the SPPF module and taking the mixed pooling operation as a residual input branch of Concat in the SPPF module; The improvement of the neural network model on YOLO11n also includes the following: for the feature map P2 obtained by the first C3K2 module of the feature extraction network of YOLO11n, a connection path is added at the lowermost layer of the feature fusion network of YOLO11n: the feature map P2 and the last obtained feature map are processed and spliced, and then fusion processing is performed from bottom to top; The improved feature fusion network based on YOLO11n includes two parts of a top-down path and a bottom-up path; The top-down path starts from the up-sampling operation of the topmost output X5 of the feature extraction network, and the up-sampling operation includes, from input to output, a nearest neighbor up-sampling operation, a Concat module and a C3K2 module, and the feature maps X4, X3 and X2 are generated through three up-sampling operations; the Concat module of each up-sampling operation performs feature fusion on the feature map obtained by the nearest neighbor up-sampling operation and the feature map P4, P3 and P2 of the corresponding size obtained by the feature extraction network through horizontal connection; The bottom-up path starts from the down-sampling operation of the lowermost output of the top-down path, and the down-sampling operation includes, from input to output, a CBS module, a Concat module and a C3K2 module, and the feature maps F3, F4 and F5 are generated through three down-sampling operations, and the feature maps F3, F4 and F5 are taken as the input of the detection module; the Concat module of each down-sampling operation performs feature fusion on the feature map obtained by the CBS module and the feature maps X3, X4 and X5 of the corresponding size of the top-down path through horizontal connection; Step 3, using the trained target detection model to detect the underwater fuzzy image to be detected, and outputting an image with a boundary box and a target category in the boundary box.

2. The underwater blurred image object detection method of claim 1, wherein, The mixed pooling convolution module is expressed as follows: ； wherein Y represents the output feature map of the mixed pooling convolution module, X represents the input feature map of the mixed pooling convolution module, SiLU represents an activation function, BN represents a batch normalization operation, conv2d represents a two-dimensional convolution operation, Maxpool represents a maximum pooling operation and Avgpool represents an average pooling operation. 3.The underwater blurred image object detection method of claim 1, wherein, The loss function of training the neural network model to obtain the target detection model adopts BCE as a classification loss function of a target class, adopts DFL and CIOU as a loss function of a bounding box regression, and then weights the classification loss function and the regression loss function to obtain an overall loss function, which is represented as: ； In the formula, The loss function represents the whole, 、 、 BCE classification loss, DFL regression loss, CIOU regression loss, Respectively 、 、 The corresponding weight.

4. An underwater blurred image object detection apparatus, characterized by, The method comprises the following steps: a data collection and labeling module, configured to: collect an underwater fuzzy image dataset, and label a target in the image in terms of class and bounding box to obtain a labeled dataset; a model training module, configured to: train a neural network model built to obtain a target detection model using the labeled dataset; The improvements of the neural network model to YOLO11n further comprise: for a feature map P2 obtained by the first C3K2 module of the feature extraction network of YOLO11n, a connection path is added at the lowermost layer of the feature fusion network of YOLO11n: the feature map P2 and a feature map obtained from top to bottom are processed and spliced, and then fusion processing is performed from bottom to top; The improved feature fusion network based on YOLO11n comprises two parts of a top-down path and a bottom-up path; The top-down path starts from the up-sampling operation of the output X5 of the topmost layer of the feature extraction network, and the up-sampling operation comprises, from input to output, nearest neighbor up-sampling, a Concat module and a C3K2 module, and generates feature maps X4, X3 and X2 through three up-sampling operations; the Concat module of each up-sampling operation performs feature fusion on the feature map obtained by the nearest neighbor up-sampling and a feature map P4, P3 or P2 of a corresponding size obtained by the feature extraction network through horizontal connection; The bottom-up path starts from the down-sampling operation of the output of the lowermost layer of the top-down path, and the down-sampling operation comprises, from input to output, a CBS module, a Concat module and a C3K2 module, and generates feature maps F3, F4 and F5 through three down-sampling operations, as inputs of the detection module; the Concat module of each down-sampling operation performs feature fusion on the feature map obtained by the CBS module and a feature map X3, X4 or X5 of a corresponding size of the top-down path through horizontal connection; a target detection module, configured to: configure the target detection model obtained by training to perform target detection on an underwater fuzzy image to be detected, and output an image with a bounding box and a target class in the bounding box. ​

Images