Underwater small target detection method based on feature screening and context fusion

Abstract

The invention discloses an underwater small target detection method based on feature screening and context fusion, and the method comprises the steps: improving a YOLOv3 network model, carrying out the feature screening and context fusion in a backbone network, constructing a target detection YOLOv3-CTE network model, carrying out the sparse training and optimization processing of a pruning network model of the target detection YOLOv3-CTE through a sample training set, and carrying out the detection of an underwater small target. Obtaining an optimal target detection network model; the sample verification set is adopted to verify the optimal target detection network model, a network model verification result is obtained, and the optimal target detection network model is updated according to the network model verification result; and predicting the sample test set based on the updated optimal target detection network model to obtain a detection result of the underwater small target. According to the method, in the underwater small target detection process, the parameter quantity can be reduced to a great extent, meanwhile, the calculation speed of the model is improved, and the classification and recognition accuracy of the underwater small target is further improved on the premise that the detection efficiency is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater small target detection, and in particular to an underwater small target detection method based on feature screening and context fusion. Background Art

[0002] Underwater small target recognition technology is an important part of underwater target detection, which mainly involves processing underwater images with small size, low contrast, blur and color distortion to detect and identify underwater targets. The influence of illumination in underwater environment greatly reduces the quality of images, resulting in problems such as visibility loss, weak contrast and texture distortion, color change, etc., which makes underwater target detection more difficult. At the same time, target objects in underwater environment are usually very small, such as sea urchins, scallops, sea cucumbers, etc.

[0003] The current methods for detecting small underwater targets mainly include computer vision-based methods and acoustic-based methods. Computer vision-based methods usually use image processing and machine learning techniques to analyze and process underwater images to detect and identify small targets. Among them, deep learning technologies are increasingly being used in the field of underwater target detection, such as convolutional neural networks (CNN) and transfer learning. Acoustic-based methods use the propagation characteristics of sound waves underwater to process and analyze underwater sound wave signals to detect and identify small targets. These methods include sonar, ultrasonic detection, acoustic imaging, etc.

[0004] However, existing target detection algorithms are usually developed based on images in terrestrial environments, while underwater images differ greatly from terrestrial images in terms of lighting, texture, color, etc. Therefore, directly applying terrestrial target detection algorithms to underwater environments often has poor results, which in turn leads to the inability to learn effective feature representations from a large number of underwater images, greatly reducing the accuracy and robustness of small target detection. Summary of the invention

[0005] The present invention provides an underwater small target detection method based on feature screening and context fusion to overcome the above technical problems.

[0006] In order to achieve the above object, the technical solution of the present invention is:

[0007] A method for detecting underwater small targets based on feature screening and context fusion, comprising the following steps:

[0008] S1: Obtain image data of a small underwater target containing a target category label, and perform S*S grid division on the image data to obtain a specific data set; randomly divide the specific data set into a sample training set, a sample verification set, and a sample test set;

[0009] S2: constructing a target detection YOLOv3-CTE network model, and performing sparse training on the sample training set to obtain a pruned network model of the target detection YOLOv3-CTE;

[0010] Based on the pruned network model of target detection YOLOv3-CTE, feature screening and context fusion are performed on the sample training set to obtain the contribution of sample data, so as to update the pruned network model of target detection YOLOv3-CTE;

[0011] S3: Optimize the updated pruning network model of the target detection YOLOv3-CTE to obtain the optimal target detection network model;

[0012] S4: Using a sample verification set to verify the optimal target detection network model, obtaining a network model verification result, and updating the optimal target detection network model according to the network model verification result;

[0013] The sample test set is predicted based on the updated optimal target detection network model to obtain the detection results of underwater small targets.

[0014] Furthermore, the target detection YOLOv3-CTE network model includes a plurality of CTE structural units, a feature extraction network unit, and a feature fusion network unit and a prediction network unit sequentially connected to the feature extraction network unit;

[0015] The feature extraction network unit includes a first convolution layer, a second convolution layer, a first residual module, a third convolution layer, a second residual module, a fourth convolution layer, a third residual module, a fifth convolution layer, a fourth residual module, a sixth convolution layer and a fifth residual module;

[0016] The first convolution layer is used to perform a convolution operation on the input sample data, one side of the second convolution layer is connected to the first convolution layer, and the other side is connected to one side of the first residual module, and the second convolution layer is used to perform a convolution operation on the output data of the first convolution layer to obtain feature maps of different scales;

[0017] The first residual module is used to perform a residual processing operation on the output data of the second convolutional layer, and the residual processing operation is used to fuse feature maps of different scales; one side of the third convolutional layer is connected to the first residual module, and the other side is connected to one side of the second residual module;

[0018] The third convolutional layer is used to perform a convolution operation on the output data of the first residual module, the second residual module is used to perform a residual processing operation on the output of the third convolution, and the other side of the second residual module is connected to one side of the fourth convolutional layer;

[0019] The fourth convolutional layer is used to perform a convolution operation on the output data of the second residual module, and one side of the third residual module is connected to the fourth convolutional layer, and the other side is connected to the fifth convolutional layer; the third residual module is used to perform a residual processing operation on the output data of the fourth convolutional layer;

[0020] The fifth convolutional layer is used to perform a convolution operation on the output data of the third residual module, and one side of the fourth residual module is connected to the fifth convolutional layer, and the other side is connected to one side of the sixth convolutional layer;

[0021] The fourth residual module is used to perform a residual processing operation on the output data of the fifth convolutional layer, and the sixth convolutional layer is used to perform a convolution operation on the output data of the fourth residual module, and the other side of the sixth convolutional layer is connected to one side of the fifth residual module, and the fifth residual module is used to perform a residual processing operation on the output data of the sixth convolutional layer;

[0022] The CTE structural unit is arranged between the feature extraction network unit and the feature fusion network unit; and the CTE structural unit is provided with a plurality of extended convolutional layers with different expansion rates, each extended convolutional layer is used to perform a convolution operation on the output data of the feature extraction network unit, and channel-splicing the output data of each extended convolutional layer to obtain multi-scale feature fusion data.

[0023] Furthermore, the feature fusion network unit includes a seventh convolutional layer, an eighth convolutional layer, a first upsampling layer, a first splicing layer, a ninth convolutional layer, a tenth convolutional layer, a second upsampling layer, a second splicing layer and an eleventh convolutional layer connected in sequence;

[0024] The seventh convolutional layer is used to perform a convolution operation on the output data of the fifth residual module, and transmit the output data of the seventh convolutional layer to the prediction network unit, and define the feature map output by the prediction network unit at this time as a large-scale small target feature map;

[0025] The eighth convolutional layer is used to perform a convolution operation on the output data of the seventh convolutional layer;

[0026] A first CTE structure unit is provided between the fourth residual module and the first splicing layer; the first CTE structure unit is used to extract feature context information of the output data of the fourth residual module and perform multi-scale feature fusion;

[0027] The feature context information includes a plurality of grid pixels of the output data and a spatial relationship between the grid pixels;

[0028] The first upsampling layer is used to upsample the output data of the eighth convolutional layer, and the first splicing layer is used to perform channel splicing on the output data of the first upsampling layer and the output data of the first CTE structural unit; the ninth convolutional layer is used to perform a convolution operation on the output data of the first splicing layer, and transmit the output data of the ninth convolutional layer to the prediction network unit, and define the feature map output by the prediction network unit at this time as a medium-scale small target feature map;

[0029] The tenth convolutional layer is used to perform a convolution operation on the output data of the ninth convolutional layer, and a second CTE structural unit is provided between the third residual module and the second splicing layer; the second CTE structural unit is used to extract feature context information of the output data of the third residual module and perform multi-feature feature fusion;

[0030] The second upsampling layer is used to upsample the output data of the tenth convolutional layer, and the second splicing layer is used to perform channel splicing on the output data of the second upsampling layer and the output data of the second CTE structure unit; the eleventh convolutional layer is used to perform a convolution operation on the output data of the second splicing layer, and transmit the output data of the eleventh convolutional layer to the prediction network unit, and define the feature map output by the prediction network unit at this time as a small-scale small target feature map.

[0031] Furthermore, the pruning network model for obtaining target detection YOLOv3-CTE described in S2 includes the following steps:

[0032] S21: Encode each network layer of the target detection YOLOv3-CTE network model, and input sample data of multiple scales in the sample training set into the target detection YOLOv3-CTE network model for model training;

[0033] S22: Define the input and output of the i-th convolutional layer as L i With L i+1 , the jth filter of the i-th convolutional layer is expressed as Get the geometric median of the filter of the i-th convolutional layer;

[0034] The expression of the geometric median of the i-th convolutional layer filter is

[0035]

[0036] Where: x GM represents the geometric median of the filter of the i-th convolutional layer; F i,j′ Representing multidimensional space The j′th filter in ; x represents the input tensor of the network layer;

[0037] Then, the filters that can be set to zero in each network layer are determined according to the geometric median, and a pruned network model of target detection YOLOv3-CTE is obtained;

[0038] The filter F that can be set to zero i,j* The discriminant formula is

[0039]

[0040] S23: defining the feature map corresponding to the output after the filter is set to zero as an invalid feature map; and defining the difference between the total amount of sample data at each scale and the number of invalid feature maps as the number of valid feature maps;

[0041] Acquire contribution of sample data of multiple scales in a sample training set according to the effective feature map;

[0042] The contribution is the ratio of the number of valid feature maps of sample data at each scale to the total amount of sample data at each scale;

[0043] S24: and adjusting the depth of the corresponding network layer of the pruned network model of the target detection YOLOv3-CTE according to the contribution to update the pruned network model of the target detection YOLOv3-CTE;

[0044] And the adjustment principle for adjusting the depth of the corresponding network layer according to the contribution is:

[0045] Compare the contribution of sample data at each scale with the set threshold;

[0046] If the contribution of the sample data is greater than the set threshold, the number of network layers that output the corresponding scale feature map is reduced;

[0047] If the contribution of the sample data is less than or equal to the set threshold, the number of network layers corresponding to the output of this scale feature map is increased.

[0048] Furthermore, the S3 specifically includes the following steps:

[0049] S31: In the pruning network model of target detection YOLOv3-CTE, each convolution layer is provided with a plurality of feature channels, and each of the feature channels is sequentially connected with a BN processing layer and an activation function layer;

[0050] Set the maximum number of iterations for model training, feature channel ID, and feature channel pruning threshold γ;

[0051] S32: Initialize the network parameters of the pruning network model of target detection YOLOv3-CTE; use the sample data in the sample training set as input data, and use the target category label in the sample training set as output data;

[0052] Training the pruned network model of the target detection YOLOv3-CTE to obtain the target type label prediction value;

[0053] S33: Obtain a prediction error value according to the target type label prediction value and the target type label actual value; and determine the magnitude of the prediction error value and a preset error threshold;

[0054] If the prediction error value is greater than the preset error threshold, S32 is repeatedly executed to adaptively update the network parameter weights of the pruning network model of the target detection YOLOv3-CTE;

[0055] If the prediction error value is less than or equal to the preset error threshold, it is confirmed that the currently trained target detection YOLOv3-CTE pruning network model is a network parameter optimized pruning network model;

[0056] S34: comparing the output value of each BN processing layer in the network parameter optimization pruning network model with the feature channel pruning threshold γ;

[0057] The feature channel ID corresponding to the output value of the BN processing layer that is less than the feature channel pruning threshold γ is pruned to obtain the optimal target detection network model.

[0058] Beneficial effect: The present invention provides an underwater small target detection method based on feature screening and context fusion. By improving the YOLOv3 network model, feature screening and context fusion are performed in the backbone network, a target detection YOLOv3-CTE network model is constructed, and the pruned network model of the target detection YOLOv3-CTE is sparsely trained and fine-tuned through a sample training set to obtain the optimal target detection network model; in the process of underwater small target detection, the amount of parameters can be greatly reduced, while the calculation speed of the model is improved, and under the premise of ensuring the detection efficiency, the accuracy of classification and recognition of underwater small targets is further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0060] Figure 1 This is a flow chart of a method for detecting underwater small targets based on feature screening and context fusion according to the present invention;

[0061] Figure 2Schematic diagram of the target detection YOLOv3-CTE network model constructed in this embodiment;

[0062] Figure 3 Schematic diagram of the CTE structural unit in this embodiment;

[0063] Figure 4 Flow chart of sparse training of the YOLOv3-CTE network model for target detection in this embodiment;

[0064] Figure 5 This is a flowchart for fine-tuning the pruning network model of YOLOv3-CTE for target detection in this embodiment. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] This embodiment provides a method for detecting underwater small targets based on feature screening and context fusion. Figure 1 As shown, the following steps are included:

[0067] S1: Obtain image data of a small underwater target containing a target category label, and perform S*S grid division on the image data to obtain a specific data set; randomly divide the specific data set into a sample training set, a sample verification set, and a sample test set;

[0068] Specifically, the image data of underwater small targets containing target category labels refers to the underwater data pictures provided by the 2020 National Underwater Robot Target Grasping Competition, and a specific data set obtained based on the underwater data pictures, wherein the underwater data pictures include four types of underwater targets, namely sea urchins, sea cucumbers, scallops and starfish, totaling about 6,000 pictures, and 60% of them are randomly selected as training sets, 20% as validation sets, and 20% as test sets;

[0069] S2: constructing a target detection YOLOv3-CTE network model, and performing sparse training on the sample training set to obtain a pruned network model of the target detection YOLOv3-CTE;

[0070] Based on the pruned network model of target detection YOLOv3-CTE, feature screening and context fusion are performed on the sample training set to obtain the contribution of sample data, so as to update the pruned network model of target detection YOLOv3-CTE;

[0071] In a specific embodiment, Figure 2 As shown, the target detection YOLOv3-CTE network model includes a feature extraction network unit, a plurality of CTE structure units, and a feature fusion network unit and a prediction network unit sequentially connected to the feature extraction network unit;

[0072] The feature extraction network unit includes a first convolution layer, a second convolution layer, a first residual module, a third convolution layer, a second residual module, a fourth convolution layer, a third residual module, a fifth convolution layer, a fourth residual module, a sixth convolution layer and a fifth residual module;

[0073] The first convolution layer is used to perform a convolution operation on the input sample data, one side of the second convolution layer is connected to the first convolution layer, and the other side is connected to one side of the first residual module, and the second convolution layer is used to perform a convolution operation on the output data of the first convolution layer to obtain feature maps of different scales;

[0074] The first residual module is used to perform a residual processing operation on the output data of the second convolutional layer, and the residual processing operation is used to fuse feature maps of different scales; wherein feature maps of different sizes will be generated during the convolution feature extraction process, and the feature extraction process is an existing well-known technical content, and here it is only to realize the function of obtaining the output result through the input data of the network layer of the corresponding network model according to the network structure involved in the present invention; one side of the third convolutional layer is connected to the first residual module, and the other side is connected to one side of the second residual module;

[0075] The third convolutional layer is used to perform a convolution operation on the output data of the first residual module, the second residual module is used to perform a residual processing operation on the output of the third convolution, and the other side of the second residual module is connected to one side of the fourth convolutional layer;

[0076] The fourth convolutional layer is used to perform a convolution operation on the output data of the second residual module, and one side of the third residual module is connected to the fourth convolutional layer, and the other side is connected to the fifth convolutional layer; the third residual module is used to perform a residual processing operation on the output data of the fourth convolutional layer;

[0077] The fifth convolutional layer is used to perform a convolution operation on the output data of the third residual module, and one side of the fourth residual module is connected to the fifth convolutional layer, and the other side is connected to one side of the sixth convolutional layer;

[0078] The fourth residual module is used to perform a residual processing operation on the output data of the fifth convolutional layer, and the sixth convolutional layer is used to perform a convolution operation on the output data of the fourth residual module, and the other side of the sixth convolutional layer is connected to one side of the fifth residual module, and the fifth residual module is used to perform a residual processing operation on the output data of the sixth convolutional layer;

[0079] The CTE structure unit is arranged between the feature extraction network unit and the feature fusion network unit, and includes a first CTE structure unit and a second CTE structure unit;

[0080] The CTE structural unit is provided with a plurality of extended convolutional layers with different expansion rates, each extended convolutional layer is used to perform a convolution operation on the output data of the feature extraction network unit, and channel-join the output data of each extended convolutional layer to obtain multi-scale feature fusion data.

[0081] Specifically, convolutions with different expansion rates are used to obtain contextual information of different receptive fields, and injected into FPN (feature pyramid feature fusion network unit) to enrich the contextual information. Use dilated convolutions with different expansion rates to obtain contextual information of different receptive fields, such as Figure 3 The CTE structural unit shown. Convolution is performed on the feature layer of FPN (feature pyramid feature fusion network unit) with different dilation rates to obtain semantic information of different receptive fields (referring to the image information represented by the image features extracted after convolution). The convolution kernel sizes used in this design are 1×1 and 3×3, with step sizes of 1, 3, and 5, respectively, to obtain four feature extraction branches (convolution kernel size is 1×1 and step size is 1, convolution kernel size is 3×3 and step size is 1, convolution kernel size is 3×3 and step size is 3, and convolution kernel size is 3×3 and step size is 5), and then the results of these four feature extraction branches are fused together to form a new residual block (the residual block / residual module is used to enhance the expressive power of deep neural networks: as the depth of the network increases, its expressive power will become stronger and stronger, but it will also be affected by problems such as gradient vanishing and gradient exploding. Since the residual block can achieve direct connection by skipping a certain number of layers, this helps to alleviate these problems and avoid performance degradation caused by training deep neural networks.) to achieve the purpose of fusing features of different scales.

[0082] In a specific embodiment, the feature fusion network unit includes a seventh convolutional layer, an eighth convolutional layer, a first upsampling layer, a first splicing layer, a ninth convolutional layer, a tenth convolutional layer, a second upsampling layer, a second splicing layer and an eleventh convolutional layer connected in sequence;

[0083] The seventh convolutional layer is used to perform a convolution operation on the output data of the fifth residual module, and transmit the output data of the seventh convolutional layer to the prediction network unit, and define the feature map output by the prediction network unit at this time as a large-scale small target feature map;

[0084] The eighth convolutional layer is used to perform a convolution operation on the output data of the seventh convolutional layer;

[0085] A first CTE structure unit is provided between the fourth residual module and the first splicing layer; the first CTE structure unit is used to extract feature context information of the output data of the fourth residual module and perform multi-feature fusion;

[0086] The feature context information includes a plurality of grid pixels of the output data and a spatial relationship between the grid pixels;

[0087] The first upsampling layer is used to upsample the output data of the eighth convolutional layer, and the first splicing layer is used to perform channel splicing on the output data of the first upsampling layer and the output data of the first CTE structural unit; the ninth convolutional layer is used to perform a convolution operation on the output data of the first splicing layer, and transmit the output data of the ninth convolutional layer to the prediction network unit, and define the feature map output by the prediction network unit at this time as a medium-scale small target feature map;

[0088] The tenth convolutional layer is used to perform a convolution operation on the output data of the ninth convolutional layer, and a second CTE structural unit is provided between the third residual module and the second splicing layer; the second CTE structural unit is used to extract feature context information of the output data of the third residual module and perform multi-scale feature fusion;

[0089] The second upsampling layer is used to upsample the output data of the tenth convolutional layer, and the second splicing layer is used to channel splice the output data of the second upsampling layer with the output data of the second CTE structure unit; the eleventh convolutional layer is used to perform a convolution operation on the output data of the second splicing layer, and transmit the output data of the eleventh convolutional layer to the prediction network unit, and define the feature map output by the prediction network unit at this time as a small-scale small target feature map. Specifically, the prediction network unit is composed of sequentially connected convolutional layers, which are used to perform a convolution operation on the input data and output the positioning coordinates and target category of the underwater small target; Darknet-53Improved backbone: Darknet-53 after the improved backbone network (feature extraction network unit) Type: operation type; Filters: number of filters; Size: convolution kernel size; Output: length and width of the output feature map; Conv: convolution operation; Residual: residual block; Concat: feature stacking; Predict: prediction; Upsample: upsampling;

[0090] Specifically, in a convolutional neural network, the bottom layer is a layer with a lower number of convolutional neural networks, and the top layer is a deeper layer of the convolutional neural network. The feature map obtained at the top layer has a larger receptive field and stronger semantic information due to multiple convolution operations, which is suitable for classification. However, the feature map at the top layer has a smaller size and fewer small target features, making it difficult to detect small targets. The feature map obtained at the bottom layer has a smaller receptive field and rich detail information. It is also large in size and has more small target features, but the semantic information is weak and cannot complete the recognition of small targets. Therefore, fusing feature maps of different layers and enhancing the information of small targets will be beneficial to the detection of small targets. Based on the YOLOv3 network, the target detection YOLOv3-CTE network model is constructed, and the information of the top layer is fused to the feature maps of other different layers through a top-down structure, which enhances the semantic information of different layers and improves the detection effect of small targets to a certain extent. The bottom-up structure can fuse the rich features of small targets at the bottom layer to different feature maps, making the obtained fused feature map more conducive to the detection of small targets. Specifically, the calculation methods of the convolutional layer, residual module, upsampling layer, and splicing layer in this embodiment are all existing. Here, only the network structure involved in the present invention is used to realize the function of obtaining the output result of the predicted network unit through the input data of the network model.

[0091] S3: fine-tune the pruned network model of the target detection YOLOv3-CTE to obtain the optimal target detection network model;

[0092] S4: Using a sample verification set to verify the optimal target detection network model, obtaining a network model verification result, and updating the optimal target detection network model according to the network model verification result;

[0093] Specifically, if the optimal target detection network model is verified by the sample verification set and the obtained verification accuracy does not meet the preset accuracy requirement, then repeat S1 to S3 for repeated training until the verification result of the optimal target detection network model meets the set accuracy standard;

[0094] The sample test set is predicted based on the updated optimal target detection network model to obtain the detection results of underwater small targets.

[0095] The present invention improves the YOLOv3 network model, performs feature screening and context fusion in the backbone network, constructs a target detection YOLOv3-CTE network model, and performs sparse training and fine-tuning on the pruning network model of the target detection YOLOv3-CTE through a sample training set to obtain an optimal target detection network model; in the process of underwater small target detection, the amount of parameters can be greatly reduced, while the calculation speed of the model is improved, and the accuracy of classification and recognition of underwater small targets is further improved under the premise of ensuring detection efficiency.

[0096] In a specific embodiment, Figure 4 As shown, the pruning network model for obtaining target detection YOLOv3-CTE described in S2 includes the following steps:

[0097] S21: Encode each network layer of the target detection YOLOv3-CTE network model, and input sample data of multiple scales in the sample training set into the target detection YOLOv3-CTE network model for model training;

[0098] S22: Define the input and output of the i-th convolutional layer as L i With L i+1 , the jth filter of the i-th convolutional layer is expressed as Get the geometric median of the filter of the i-th convolutional layer;

[0099] The expression of the geometric median of the i-th convolutional layer filter is

[0100]

[0101] Where: x GM represents the geometric median of the filter of the i-th convolutional layer; F i,j′ Representing multidimensional space The j′th filter in ; x represents the input tensor of the network layer;

[0102] Then, the filters that can be set to zero in each network layer are determined according to the geometric median, and a pruned network model of target detection YOLOv3-CTE is obtained;

[0103] The filter F that can be set to zero i,j* The discriminant formula is

[0104]

[0105] S23: defining the feature map corresponding to the output after the filter is set to zero as an invalid feature map; and defining the difference between the total amount of sample data at each scale and the number of invalid feature maps as the number of valid feature maps;

[0106] Acquire contribution of sample data of multiple scales in a sample training set according to the effective feature map;

[0107] The contribution is the ratio of the number of valid feature maps of sample data at each scale to the total amount of sample data at each scale;

[0108] S24: and adjusting the depth of the corresponding network layer of the pruned network model of the target detection YOLOv3-CTE according to the contribution to update the pruned network model of the target detection YOLOv3-CTE;

[0109] And the adjustment principle for adjusting the depth of the corresponding network layer according to the contribution is:

[0110] Compare the contribution of sample data at each scale with the set threshold;

[0111] If the contribution of the sample data is greater than a set threshold, the number of network layers corresponding to the output of the scale feature map is reduced to reduce the depth of the network layer corresponding to the output of the scale feature map;

[0112] If the contribution of the sample data is less than or equal to the set threshold, the number of network layers corresponding to the output of the scale feature map is increased to increase the depth of the network layer corresponding to the output of the scale feature map;

[0113] In a specific embodiment, Figure 5 As shown, S3 specifically includes the following steps:

[0114] S31: In the pruning network model of target detection YOLOv3-CTE, each convolution layer is provided with a plurality of feature channels, and each of the feature channels is sequentially connected with a BN processing layer and an activation function layer;

[0115] Set the maximum number of iterations for model training, feature channel ID, and feature channel pruning threshold γ;

[0116] S32: Initialize the network parameters of the pruned network model of target detection YOLOv3-CTE;

[0117] Using the sample data in the sample training set as input data and the target category label in the sample training set as output data; training the pruned network model of the target detection YOLOv3-CTE to obtain the target type label prediction value;

[0118] S33: Obtain a prediction error value according to the target type label prediction value and the target type label actual value; and determine the magnitude of the prediction error value and a preset error threshold;

[0119] If the prediction error value is greater than the preset error threshold, S32 is repeatedly executed to adaptively update the network parameter weights of the pruning network model of the target detection YOLOv3-CTE;

[0120] If the prediction error value is less than or equal to the preset error threshold, it is confirmed that the currently trained target detection YOLOv3-CTE pruning network model is a network parameter optimized pruning network model;

[0121] S34: comparing the output value of each BN processing layer in the network parameter optimization pruning network model with the feature channel pruning threshold γ;

[0122] The feature channel ID corresponding to the output value of the BN processing layer that is less than the feature channel pruning threshold γ is pruned to obtain the optimal target detection network model.

[0123] Specifically, there are multiple connections of residual modules in the feature extraction network unit, and the adjacent layers of the layers directly connected to the residual modules are pruned. Because the relationship between the layers connected to the residual modules is an Add operation, the layers directly connected to the residual modules cannot be pruned because the input dimensions need to be maintained the same. The present invention sets the ID and threshold γ for the feature channels of each layer, and prunes the elements whose values ​​are less than γ after the output of the BN layer (batch normalization layer), which can greatly reduce the number of parameters, improve the calculation speed of the model, and improve the accuracy of the model after removing some unnecessary weights and channels.

[0124] In this embodiment, due to the complex underwater environment and poor light, underwater targets are easily blocked and have poor distinguishability. In particular, small targets usually occupy fewer pixels in the image and lack sufficient details. Therefore, the recognition accuracy of small targets has always been low. The present invention first proposes to screen out valid feature maps. By analyzing the training set, it is found that about 70% of the 13×13 feature maps are invalid feature maps, about 52% of the 26×26 feature maps are invalid feature maps, about 15% of the 52×52 feature maps are invalid feature maps, and about 3% of the 104×104 invalid feature maps. Combined with the characteristics of YOLOv3backbone, the depth of the convolutional layer corresponding to the 13×13 scale feature is finally reduced from 4 to 2, and the depth of the convolutional layer corresponding to the 104×104 scale feature is increased from 2 to 4, so as to reconstruct the target detection YOLOv3-CTE network model. According to the reconstructed target detection YOLOv3-CTE network model, it can be determined that in order to improve the accuracy of small target detection, the proportion of shallow features needs to be increased, but the receptive field of shallow features is limited and it is impossible to obtain sufficient context information. Therefore, it is proposed to insert the CTE module before the 52×52 and 26×26 feature maps, which can further improve the mAP (average accuracy of each category of detection targets). At the same time, the feature channel Channels of each layer sets the ID and threshold γ, and prunes the elements whose values ​​after the output of the BN layer (batch normalization layer) are less than γ, which can greatly reduce the number of parameters, reduce the model size, and improve the calculation speed of the model.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for underwater small target detection based on feature screening and context fusion, characterized in that: The following steps are involved: S1: Obtain image data of a small underwater target containing a target category label, and perform S*S grid division on the image data to obtain a specific data set; randomly divide the specific data set into a sample training set, a sample verification set, and a sample test set; S2: constructing a target detection YOLOv3-CTE network model, and performing sparse training on the sample training set to obtain a pruned network model of the target detection YOLOv3-CTE; Based on the pruned network model of target detection YOLOv3-CTE, feature screening and context fusion are performed on the sample training set to obtain the contribution of sample data, so as to update the pruned network model of target detection YOLOv3-CTE; S3: Optimize the updated pruning network model of the target detection YOLOv3-CTE to obtain the optimal target detection network model; S4: Using a sample verification set to verify the optimal target detection network model, obtaining a network model verification result, and updating the optimal target detection network model according to the network model verification result; The sample test set is predicted based on the updated optimal target detection network model to obtain the detection results of underwater small targets.

2. According to claim 1, a method for detecting underwater small targets based on feature screening and context fusion is characterized in that: The target detection YOLOv3-CTE network model includes a feature extraction network unit, a plurality of CTE structure units, and a feature fusion network unit and a prediction network unit connected in sequence to the feature extraction network unit; The feature extraction network unit includes a first convolution layer, a second convolution layer, a first residual module, a third convolution layer, a second residual module, a fourth convolution layer, a third residual module, a fifth convolution layer, a fourth residual module, a sixth convolution layer and a fifth residual module; The first convolution layer is used to perform a convolution operation on the input sample data, one side of the second convolution layer is connected to the first convolution layer, and the other side is connected to one side of the first residual module, and the second convolution layer is used to perform a convolution operation on the output data of the first convolution layer to obtain feature maps of different scales; The first residual module is used to perform a residual processing operation on the output data of the second convolutional layer, and the residual processing operation is used to fuse feature maps of different scales; one side of the third convolutional layer is connected to the first residual module, and the other side is connected to one side of the second residual module; The third convolutional layer is used to perform a convolution operation on the output data of the first residual module, the second residual module is used to perform a residual processing operation on the output of the third convolution, and the other side of the second residual module is connected to one side of the fourth convolutional layer; The fourth convolutional layer is used to perform a convolution operation on the output data of the second residual module, and one side of the third residual module is connected to the fourth convolutional layer, and the other side is connected to the fifth convolutional layer; the third residual module is used to perform a residual processing operation on the output data of the fourth convolutional layer; The fifth convolutional layer is used to perform a convolution operation on the output data of the third residual module, and one side of the fourth residual module is connected to the fifth convolutional layer, and the other side is connected to one side of the sixth convolutional layer; The fourth residual module is used to perform a residual processing operation on the output data of the fifth convolutional layer, and the sixth convolutional layer is used to perform a convolution operation on the output data of the fourth residual module, and the other side of the sixth convolutional layer is connected to one side of the fifth residual module, and the fifth residual module is used to perform a residual processing operation on the output data of the sixth convolutional layer; The CTE structural unit is arranged between the feature extraction network unit and the feature fusion network unit; and the CTE structural unit is provided with a plurality of extended convolutional layers with different expansion rates, each extended convolutional layer is used to perform a convolution operation on the output data of the feature extraction network unit, and channel-splicing the output data of each extended convolutional layer to obtain multi-scale feature fusion data.

3. The underwater small target detection method based on feature screening and context fusion according to claim 2 is characterized in that: The feature fusion network unit includes a seventh convolution layer, an eighth convolution layer, a first upsampling layer, a first splicing layer, a ninth convolution layer, a tenth convolution layer, a second upsampling layer, a second splicing layer and an eleventh convolution layer connected in sequence; The seventh convolutional layer is used to perform a convolution operation on the output data of the fifth residual module, and transmit the output data of the seventh convolutional layer to the prediction network unit, and define the feature map output by the prediction network unit at this time as a large-scale small target feature map; The eighth convolutional layer is used to perform a convolution operation on the output data of the seventh convolutional layer; A first CTE structure unit is provided between the fourth residual module and the first splicing layer; the first CTE structure unit is used to extract feature context information of the output data of the fourth residual module and perform multi-scale feature fusion; The feature context information includes a plurality of grid pixels of the output data and a spatial relationship between the grid pixels; The first upsampling layer is used to upsample the output data of the eighth convolutional layer, and the first splicing layer is used to perform channel splicing on the output data of the first upsampling layer and the output data of the first CTE structural unit; The ninth convolutional layer is used to perform a convolution operation on the output data of the first concatenated layer, and transmit the output data of the ninth convolutional layer to the prediction network unit, and define the feature map output by the prediction network unit at this time as a medium-scale small target feature map; The tenth convolutional layer is used to perform a convolution operation on the output data of the ninth convolutional layer, and a second CTE structural unit is provided between the third residual module and the second splicing layer; the second CTE structural unit is used to extract feature context information of the output data of the third residual module and perform multi-feature feature fusion; The second upsampling layer is used to upsample the output data of the tenth convolutional layer, and the second splicing layer is used to perform channel splicing on the output data of the second upsampling layer and the output data of the second CTE structure unit; the eleventh convolutional layer is used to perform a convolution operation on the output data of the second splicing layer, and transmit the output data of the eleventh convolutional layer to the prediction network unit, and define the feature map output by the prediction network unit at this time as a small-scale small target feature map.

4. The underwater small target detection method based on feature screening and context fusion according to claim 3 is characterized in that: The pruning network model for obtaining target detection YOLOv3-CTE described in S2 includes the following steps: S21: Encode each network layer of the target detection YOLOv3-CTE network model, and input sample data of multiple scales in the sample training set into the target detection YOLOv3-CTE network model for model training; S22: Define the input and output of the i-th convolutional layer as L i With L i+1 , the jth filter of the i-th convolutional layer is expressed as Get the geometric median of the filter of the i-th convolutional layer; The expression of the geometric median of the i-th convolutional layer filter is Where: x GM represents the geometric median of the filter of the i-th convolutional layer; F i,j′ Representing multidimensional space The j′th filter in ; x represents the input tensor of the network layer; Then, the filters that can be set to zero in each network layer are determined according to the geometric median, and a pruned network model of target detection YOLOv3-CTE is obtained; The filter that can be set to zero The discriminant formula is S23: defining the feature map corresponding to the output after the filter is set to zero as an invalid feature map; and defining the difference between the total amount of sample data at each scale and the number of invalid feature maps as the number of valid feature maps; Acquire contribution of sample data of multiple scales in a sample training set according to the effective feature map; The contribution is the ratio of the number of valid feature maps of sample data at each scale to the total amount of sample data at each scale; S24: and adjusting the depth of the network layer of the pruning network model of the corresponding target detection YOLOv3-CTE according to the contribution, so as to update the pruning network model of the target detection YOLOv3-CTE; And the adjustment principle of adjusting the depth of the corresponding network layer according to the contribution is to compare the contribution of the sample data of each scale with the set threshold; If the contribution of the sample data is greater than the set threshold, the number of network layers that output the corresponding scale feature map is reduced; If the contribution of the sample data is less than or equal to the set threshold, the number of network layers corresponding to the output of this scale feature map is increased.

5. The underwater small target detection method based on feature screening and context fusion according to claim 3 is characterized in that: The S3 specifically includes the following steps: S31: In the pruning network model of target detection YOLOv3-CTE, each convolution layer is provided with a plurality of feature channels, and each of the feature channels is sequentially connected with a BN processing layer and an activation function layer; Set the maximum number of iterations for model training, feature channel ID, and feature channel pruning threshold γ; S32: Initializing network parameters of the pruned network model of target detection YOLOv3-CTE; using sample data in the sample training set as input data, and using target category labels in the sample training set as output data; Training the pruned network model of the target detection YOLOv3-CTE to obtain the target type label prediction value; S33: Obtain a prediction error value according to the target type label prediction value and the target type label actual value; and determine the magnitude of the prediction error value and a preset error threshold; If the prediction error value is greater than the preset error threshold, S32 is repeatedly executed to adaptively update the network parameter weights of the pruning network model of the target detection YOLOv3-CTE; If the prediction error value is less than or equal to the preset error threshold, it is confirmed that the currently trained target detection YOLOv3-CTE pruning network model is a network parameter optimized pruning network model; S34: comparing the output value of each BN processing layer in the network parameter optimization pruning network model with the feature channel pruning threshold γ; The feature channel ID corresponding to the output value of the BN processing layer that is less than the feature channel pruning threshold γ is pruned to obtain the optimal target detection network model.

Images