An underwater target detection method based on spatial feature self-supervision
Through the underwater target detection method based on space characteristics, combined with self-supervised comparison learning and feature pyramid network, the problem of poor target detection effect in underwater environment is solved, and high-precision underwater target recognition and positioning is achieved, which is suitable for Independent salvage of marine resources.
Patent Information
- Application Number
- CN202210310895.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-28
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2042-03-28
AI Technical Summary
The existing underwater target detection technology has poor effect in underwater environments, affected by light absorption and scattering, and traditional algorithms have failed to effectively deal with the characteristics and characteristics of underwater organisms, resulting in poor detection results.
Using the underwater target detection method based on spatial characteristics, through self-supervised comparison learning networks and feature pyramid networks, combining spatial attention and channel normalization, a detection framework adapted to the underwater environment is constructed, and ResNet50 is used as the feature encoder. Perform data expansion and feature redundancy suppression to achieve high-precision object detection.
It realizes high-precision target recognition and positioning in an underwater environment, adapts to the characteristics of underwater organisms, effectively suppresses feature redundancy, improves detection effect, and is suitable for independent salvage and protection of marine resources.
Smart Images

Figure CN114663683B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of self-supervision learning and underwater target detection, and specifically relates to an underwater target detection method based on spatial feature self-supervision. Background Art
[0002] In computer vision, object detection is one of the most important tasks, which can be applied to many scenarios such as marine development and marine exploration. Autonomous underwater vehicles equipped with intelligent underwater object detection systems are of great significance to the development and protection of marine resources. In recent years, seafood such as sea cucumbers and sea urchins have been loved by many people because of their high nutritional value, but the cost of manual salvage is very high, and long-term underwater operations are harmful to health and even life-threatening. Therefore, it is of practical significance to use underwater vehicles for autonomous salvage. At present, underwater object detection mainly relies on optical imaging (camera) or acoustic imaging (sonar). Sonar is very sensitive to the geometric shape of objects, but it ignores the optical color of objects and is costly. Optical images are produced by cameras and can accurately capture the shape and color of objects. Therefore, camera-based underwater images are low-cost and popular. With the development of object detection and underwater image acquisition technology, it has become possible for underwater vehicles to autonomously salvage seafood.
[0003] Underwater images are affected by light absorption and scattering, and in water, impurities introduce noise and increase the effect of scattering. These adverse effects shorten the visible distance in water and even introduce problems such as color distortion, limiting the practical application of underwater target detection technology based on images and videos. Some traditional target detection algorithms have poor detection effects in practical applications because they do not consider the characteristics of underwater environment and marine life. Underwater organisms are affected by their habits and like to live in groups, resulting in feature overlap and redundancy, which is more serious at the feature pyramid layer. Moreover, it is difficult to produce a general underwater dataset due to the difficulty of shooting and the differences in water types caused by geographical location. Therefore, the development of new underwater target detection technology is crucial to the research and application of underwater vehicles. Summary of the invention
[0004] In order to solve the above problems, the present invention provides an underwater target detection method based on spatial feature self-supervision. The method is improved on the one-stage target algorithm FCOS, can effectively perform object recognition and positioning according to the characteristics of underwater organisms, and can perfectly adapt to underwater detection tasks.
[0005] In order to achieve the above object, the present invention is achieved through the following technical solutions:
[0006] The present invention is an underwater target detection method based on spatial feature self-supervision, the method comprising the following steps:
[0007] Step 1: Collect data, determine the main categories of seafood to be detected, and use a camera to take pictures of different types of underwater creatures underwater to build the data set required for the detection task. The seafood mainly includes sea cucumbers, sea urchins, shells, and starfish;
[0008] Step 2: Data augmentation: Use image enhancement methods such as color restoration and illumination change to augment the raw data collected by the camera. After sorting, manually annotate and generate corresponding labels for the images. Finally, the training set accounts for 80% and the validation set accounts for the remaining 20%.
[0009] The color restoration adopts a white balance method, which specifically includes the following steps:
[0010] Step 2-1: After collecting pictures containing underwater creatures, place the pre-set colorimetric card in the water and use a camera to take pictures of underwater scenes with the standard colorimetric card.
[0011] Step 2-2: Use OpenCV to discretize the underwater image with the standard colorimetric card into three channels: red, green, and blue, and extract the locations of the red, green, and blue colors in the colorimetric card.
[0012] Step 2-3: Obtain pixel values according to the positions of red, green, and blue colors, and calculate the actual compensation value by comparing it with the discrete value of the standard color. The calculation of the compensation value v is as follows:
[0013] v red =c red -c′ red
[0014] v green =c green -c′ green
[0015] v blue =c blue -c′ blue
[0016] Among them, v has three values, corresponding to the three color channels of red, green and blue, respectively; c′ represents the pixel value after discretization of the standard colorimetric card; and c represents the pixel value after discretization of the extracted color;
[0017] Step 2-4: Add the three color channels of all acquired data images and the corresponding compensation values to obtain the final enhanced image.
[0018] The illumination change is achieved by randomly selecting 20% of the areas in the image and directly assigning values to these areas so that their pixel values are between 240 and 255.
[0019] Step 3: Build a self-supervised contrastive learning network, including online branches and target branches, combine spatial attention and channel normalization to model channel feature relationships, accelerate model convergence, and achieve high-precision underwater target detection. The self-supervised contrastive learning network uses spatial attention and channel normalization operations, and does not use large datasets such as ImageNet for pre-training, which effectively accelerates the convergence speed and models the spatial feature relationship of features, making the pre-trained backbone network suitable for underwater target detection tasks. Different from the supervised learning methods used by other detectors, the self-supervised network continuously optimizes the feature representation of the network output by calculating the difference between the two branches. The underwater dataset is input into the self-supervised learning network. AdamW is used as the optimization function in training, and the learning rate cosine annealing strategy is used. After training, the weight parameters are saved for initializing the target detection network parameters.
[0020] The feature encoder of the online branch is composed of ResNet50, and the online branch includes a feature encoder, a spatial feature attention, a channel normalization structure and two linear projection layers. The feature encoder is at the front, and the spatial feature attention takes the output of the feature encoder as input. The channel normalization structure takes the output of the spatial feature attention as input, and finally the output features are encoded through two linear projection layers; the feature encoder of the target branch is composed of ResNet50, and the target branch includes a feature encoder and a linear projection layer, and the output of the feature encoder is directly sent to the linear projection layer.
[0021] Step 4: Construct an underwater target detection network framework, which includes a feature extraction backbone network, a feature pyramid network, and a feature selection detection head based on feature decoupling. The feature extraction network is composed of a residual network. The feature pyramid network sets learnable parameters and adaptively fuses all features through interpolation to achieve the network's own feature selection and obtain the best representation of each layer of features. A 3DMaxPooling layer is set after the feature layer to suppress redundant features and improve feature quality. Specifically: the feature pyramid network contains 5 levels of different resolution sizes. The first three levels P 1 , P 2 , P 3 Obtained from the last three layers of the feature extraction network, P 4 , P 5 In P 3 On the basis of , downsampling was performed once and twice respectively. The output steps of the five levels in the feature pyramid network in step 4 are:
[0022] Step 4-1: Set learning parameters for the five levels, achieve fusion between different layers through bilinear interpolation, and finally achieve adaptive weighted fusion. The processing of the feature pyramid layer is expressed as the following formula:
[0023] OUT i =F(W i1 *P 1 +W i2 *P 2 +W i3 *P 3 +W i4 *P 4 +W i5 *P 5 )
[0024] Among them (w i1 ,w i2 ,w i3 ,w i4 ,w i5 ) represents the learnable weights between different layers, (P 1 ,P 2 ,P 3 ,P 4 ,P 5 ) represents the feature maps of five levels with different resolutions, and F represents the use of 3DMaxPooling;
[0025] Step 4-2: After weighted fusion, 3DMaxPooling is introduced to suppress feature redundancy. 3DMaxPooling first passes the input feature map through a 3×3 convolution kernel, and then obtains feature maps of the same size between adjacent levels through interpolation. 3Dmaxpool is used to mine potential spatial feature relationships and remove redundant features. The process of feature redundancy suppression is expressed as the following formula:
[0026] y s =max{x s,k =Bilinear(x k )}
[0027] Among them, s represents the feature map of the current layer, k represents the feature map of the adjacent layer, y represents the final output, and Bilinear represents the interpolation method;
[0028] Step 4-3: Finally, the result of adding the input feature map and 3Dmaxpool is sent to the GN-RELU layer.
[0029] A further improvement of the present invention is that the feature selection detection head includes a classification branch, a regression branch and a center degree branch, the center feature is selected in the classification branch, the boundary feature extraction module is integrated in the regression branch, the classification branch selects the middle part of the grid as a positive sample through sampling, the regression branch uses the boundary extraction module to extract boundary features for more accurate boundary positioning, the center degree branch is used to calculate the degree of sample deviation from the center, and its value is between 0 and 1, and finally the regression result is adjusted to a numerical range greater than 0 through the natural exponent.
[0030] A further improvement of the present invention is that the classification branch enhances the real box (x 1 ,y 1 ,x 2 ,y 2 ) is sampled as positive samples, and the rest are negative samples. 1 ,y 1 ) and (x 2 ,y 2 ) represent the pixel coordinate values of the upper left corner and the lower right corner of the object respectively. When a pixel point is assigned to multiple objects, only the real box with the smallest area is selected as its assigned object. The classification branch selects the middle part of the network as the positive sample through sampling, which is expressed as the following formula:
[0031] x′ 1 =c′ x -0.5wε
[0032] y′ 1 =c′ y -0.5hε
[0033] x′ 2 =c′ x +0.5wε
[0034] y′ 2 =c′ y +0.5hε
[0035] Where ε is the scaling factor, the default value is 0.3, c′ x ,c′ y is the center coordinate, w and h are the width and height of the real box respectively, [x′ 1 ,y′ 1 ,x′ 2 ,y′ 2 ] is the real position of the object in the feature map after feature extraction. It has been downsampled by different multiples and is divided by 2 from the real box. l It is obtained that l is the downsampling multiple.
[0036] The boundary positioning of the regression branch is specifically as follows: the regression branch adopts a two-regression cascade method, and a boundary feature extraction module is added after the first regression. The first regression result is used to calculate the maximum value point on the four boundaries of the regression box, and the pixel position in the original image is found by interpolation. The pixel is used for the regression task, and finally a more precise object positioning is obtained.
[0037] Step 5: Train the underwater target detection network. The collected sample data is rotated, cropped, and scaled to input the built network to obtain the weight model of the network for underwater target detection of seafood. When training the detection network, the batchsize is set to 2, and SGD is used as the optimization method. Set weight decay and momentum to 0.0001 and 0.9 respectively. In the first 500 iterations, a linear learning rate warm up strategy is used to gradually increase the learning rate from 0 to 0.005, and a 10-fold learning rate decay is performed at 80% and 90% of the total iterations. A total of 36 rounds of training are performed. During the training stage, the short side of the image is randomly changed between 640 and 800, and the long side size is always kept at 1333. Different from the general training method, the initial weight p is loaded after the basic training is completed, and then 12 rounds of tuning training are performed. During the tuning period, the learning rate decays from 0.005 to 0.00005 in each round. After completing the tuning training, the weight parameters of 12 rounds are added and averaged to obtain the final weight model.
[0038] Step 6: According to the weight model trained in step 5, the divided validation set is predicted to evaluate the detection effect in practical applications. At this stage, the input image size is always kept at 1333×800, and the NMS threshold of 0.5 is used to suppress overlapping detection frames, and the detection results with a confidence level lower than 0.05 are filtered. That is to say, the model reads the test set images into the network and generates a large number of prediction frames. In order to suppress redundant prediction frames and improve the detection effect, an IOU threshold of 0.5 is used to suppress overlapping prediction frames, and objects with a confidence level lower than 0.05 are filtered.
[0039] The present invention first builds a self-supervised contrastive learning framework so that the backbone network can better adapt to underwater scenes; then builds an underwater target detection network; crops, rotates, and scales the data input to the network to adapt to the scale change of the target; proposes adaptive weighted fusion for redundant features and adds 3DMaxPooling to obtain high-quality features; further, selects 30% of the area in the center of the object as a positive sample in the classification branch to effectively suppress the background; uses the object boundary features in the regression branch to effectively enhance the accuracy of regression positioning; finally, obtains the underwater target detection model weights through training, and evaluates the performance of the detector on the divided verification set.
[0040] The beneficial effects of the present invention are as follows: the present invention is an end-to-end underwater target detection framework, which can effectively identify and locate objects based on the characteristics of underwater organisms; through self-supervised learning, the network can adapt to specific underwater scenes; by setting learnable parameters, the network can autonomously select appropriate proportional weights. Feature redundancy is suppressed by 3DMaxPooling, which effectively selects more appropriate features, rather than being limited to fixed weight fusion between adjacent layers; the proposed feature decoupling technology selects more appropriate features for underwater organisms, which is conducive to more accurate positioning and classification; the method proposed by the present invention can perfectly adapt to underwater detection tasks. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 It is a flow chart of the present invention.
[0042] Figure 2 It is a self-supervisory network structure diagram of the present invention.
[0043] Figure 3 It is a spatial attention structure diagram of the present invention.
[0044] Figure 4 It is a normalized structure diagram of the channel of the present invention.
[0045] Figure 5 It is a structural diagram of the detection network of the present invention. DETAILED DESCRIPTION
[0046] The following will disclose the embodiments of the present invention with drawings. For the purpose of clear description, many practical details will be described together in the following description. However, it should be understood that these practical details should not be used to limit the present invention. That is to say, in some embodiments of the present invention, these practical details are not necessary.
[0047] The present invention is an underwater target detection method based on spatial feature self-supervision, which generally includes: data acquisition, using a camera with a waterproof device to shoot image data of underwater organisms and construct a data set; data preprocessing, enhancing the collected underwater data, expanding the original data and dividing it into a training set and a verification set; building a self-supervised contrast learning network for pre-training, so that the backbone network can extract features that are suitable for underwater scenes. An underwater target detection network is built, and the processed image data is input into the detection backbone network, and multi-scale features are obtained by weighted fusion at the feature pyramid layer, and feature redundancy is suppressed. In the detection head, the center feature and the boundary feature are respectively selected for classification and regression to obtain the position, category and confidence level of the target to be detected in the image, and the weight model is saved; the prediction result is evaluated on the verification set according to the saved model weights in the actual application.
[0048] The specific flow chart is as follows Figure 1As shown, the underwater target detection method includes the following steps:
[0049] Step 1: Collect data. Use an optical camera to obtain underwater photos, including the seafood to be detected, and build the data set required for the detection task. The seafood is sea cucumbers, sea urchins, shells, and starfish.
[0050] Step 2: Data preprocessing, use color restoration, illumination change and other image enhancement methods to expand the collected data, filter out pictures without objects to be detected, and use labelImg software to label pictures containing objects to be detected. Generate an xml file for each picture to store the location and category of the object to be detected. The categories that need to be labeled mainly include: sea urchin, sea cucumber, starfish and shell. After labeling, divide the training and validation sets into 80% and 20% respectively.
[0051] The color restoration adopts a white balance method, which specifically includes the following steps:
[0052] Step 2-1: After collecting pictures containing underwater creatures, place the pre-set colorimetric card in the water and use a camera to take pictures of underwater scenes with the standard colorimetric card.
[0053] Step 2-2: Use OpenCV to discretize the underwater image with the standard colorimetric card into three channels: red, green, and blue, and extract the locations of the red, green, and blue colors in the colorimetric card.
[0054] Step 2-3: Obtain pixel values according to the positions of red, green, and blue colors, and calculate the actual compensation value by comparing it with the discrete value of the standard color. The calculation of the compensation value v is as follows:
[0055] v red =c red -c′ red
[0056] v green =c green -c′ green
[0057] v blue =c blue -c′ blue
[0058] Among them, v has three values, corresponding to the three color channels of red, green and blue, respectively; c′ represents the pixel value after discretization of the standard colorimetric card; and c represents the pixel value after discretization of the extracted color;
[0059] Step 2-4: Add the three color channels of all acquired data images and the corresponding compensation values to obtain the final enhanced image.
[0060] Step 3: Use pytorch to build a self-supervised contrastive learning network on the host computer, and use spatial attention and channel normalization to make the backbone network adapt to the underwater target detection task. The self-supervised contrastive learning network includes an online branch and a target branch. The online branch includes a feature encoder, a spatial feature attention, a channel normalization structure, and two linear projection layers. The feature encoder is at the front, and the spatial feature attention takes the output of the feature encoder as input. The channel normalization structure takes the output of the spatial feature attention as input, and finally the output features are encoded through two linear projection layers; the feature encoder of the online branch and the feature encoder of the target branch are composed of ResNet50, and the target branch includes a feature encoder and a linear projection layer. The output of the feature encoder is directly sent to the linear projection layer. Build a self-supervised contrastive learning network and pre-train on the underwater data set constructed in step 2. The specific method is: adopt an online update and target momentum dual-branch network structure, select ResNet50 for the feature encoder, and output the feature vectors of the two branches through the linear projection layer. Randomly crop the image input to the self-supervised learning network to encourage the corresponding features of the same view to remain consistent and the features of different objects to be as far apart as possible. Set spatial attention and channel normalization in the online branch. In order to obtain more spatial features, the spatial attention multiplies the original feature map with its own transposed matrix, and the obtained attention weight is weighted with the original feature map, combined with the convolution kernel size of 1×1, and the final spatial feature map is added. The specific structure is as follows: Figure 3 In addition, channel normalization is set to model the channel feature relationship, as shown in Figure 4 As shown. This part sets an L2 regularization to aggregate the embedded features and sets the learnable weights. The specific process can be expressed by the formula:
[0061]
[0062] X″ i =X i +tanh(γX′ i +β)X i
[0063] Among them, δ, β, γ are learnable weights, C is the normalization factor, Xi is the input feature map, and δ is a very small number to prevent the denominator from being zero.
[0064] The loss function of the self-supervised contrastive learning network is defined as:
[0065]
[0066] Among them, q is the online view, k + is a positive sample, k iis the sample queue, n is the queue length, and t is the temperature coefficient.
[0067] Step 4: Use pytorch to write the corresponding code and build the underwater target detection network framework on the host computer, such as Figure 5 As shown in Figure 1, the underwater target detection network framework includes a feature extraction backbone network, a feature pyramid network, and a feature selection detection head based on feature decoupling. The feature pyramid network selects the mainstream ResNet50 instead of a deeper network backbone to ensure detection speed. After feature extraction, the last three layers C 3 , C 4 and C 5 .P 1 , P 2 , P 3 It is obtained from the output of the last three layers of the feature extraction backbone network. 5 Based on this, 3×3 convolution is used for downsampling to obtain P 4 .P 5 In P 4 A 3×3 convolution downsampling is added on the basis of the original image, and a ReLU layer is added. The learnable parameters are set in the feature pyramid, all feature layers are weighted and fused, and 3DMaxPoooling is used to suppress feature redundancy.
[0068] The feature pyramid network contains five levels of different resolution sizes. The first three levels P 1 , P 2 , P 3 Obtained from the last three layers of the feature extraction network, P 4 , P 5 In P 3 Based on , downsampling was performed once and twice respectively. The output steps of the five levels in the feature pyramid network are:
[0069] Step 4-1: Set learning parameters for the five levels, achieve fusion between different layers through bilinear interpolation, and finally achieve adaptive weighted fusion. The processing of the feature pyramid layer is expressed as the following formula:
[0070] OUT i =F(W i1 *P 1 +W i2 *P 2 +W i3 *P 3 +W i4 *P 4 +W i5 *P 5 )
[0071] Among them (w i1 ,w i2 ,wi3 ,w i4 ,w i5 ) represents the learnable weights between different layers, (P 1 ,P 2 ,P 3 ,P 4 ,P 5 ) represents the feature maps of five levels with different resolutions, and F represents the use of 3DMaxPooling;
[0072] Step 4-2: After weighted fusion, 3DMaxPooling is introduced to suppress feature redundancy. 3DMaxPooling first passes the input feature map through a 3×3 convolution kernel, and then obtains feature maps of the same size between adjacent levels through interpolation. 3Dmaxpool is used to mine potential spatial feature relationships and remove redundant features. The process of feature redundancy suppression is expressed as the following formula:
[0073] y s =max{x s,k =Bilinear(x k )}
[0074] Among them, s represents the feature map of the current layer, k represents the feature map of the adjacent layer, y represents the final output, and Bilinear represents the interpolation method;
[0075] Step 4-3: Finally, the result of adding the input feature map and 3Dmaxpool is sent to the GN-RELU layer.
[0076] The feature selection detection head includes a regression branch and a classification branch, and an additional center branch is set. The classification branch selects the middle part of the grid as a positive sample by sampling, and its selection process can be expressed as the following formula:
[0077] x′ 1 =c′ x -0.5wε
[0078] y′ 1 =c′ y -0.5hε
[0079] x′ 2 =c′ x +0.5wε
[0080] y′ 2 =c′ y +0.5hε
[0081] Where ε is the scaling factor, the default value is 0.3, c′ x ,c′ yis the center coordinate, w and h are the width and height of the real box respectively, [x′ 1 ,y′ 1 ,x′ 2 ,y′ 2 ] is the real position of the object in the feature map after feature extraction. It has been downsampled by different multiples and is divided by 2 from the real box. l It is obtained that l is the downsampling multiple.
[0082] The regression branch uses a boundary extraction module to extract boundary features for more accurate boundary positioning. The center degree branch is used to calculate the degree of deviation of the sample from the center, and its value is between 0 and 1.
[0083] The classification branch uses the features of the object center to greatly suppress the background class. The boundary feature extraction module is integrated in the regression branch. It finds the boundary position of the object through the result of the first regression and uses the boundary features for regression to improve the positioning accuracy. The process can be expressed as the following formula:
[0084] B 2 = Z(f(B 1 ))+f(B 1 )
[0085] Among them, B 1 represents the predicted value of the bounding box position obtained by the first regression, f(·) represents the interpolation calculation, and Z(·) represents the second regression. 2 Represents the prediction result of the final position of the bounding box.
[0086] Step 5: Train the underwater target detection network. The collected sample data is input into the established network through rotation, cropping, and scaling to obtain the weight model of the network for underwater target detection of seafood.
[0087] SGD is used as the optimizer, the batch size is set to 2, and the learning rate is set to 0.005. The loss function of the network can be expressed as:
[0088]
[0089] Where L cls and L reg Focal Loss and GIOU Loss are used for classification loss and regression loss respectively, N p represents positive samples, λ is the balance coefficient, the default value is 1, p represents the network classification prediction value, t represents the regression prediction value, c * is a positive sample among all samples.
[0090] Step 6: According to the object detector described in step 5, load the trained model weights and evaluate the divided validation set. Select the NMS threshold of 0.5 to remove overlapping bounding boxes, and filter the prediction results with confidence scores less than 0.05.
[0091] In order to verify the feasibility of this algorithm, relevant comparative experiments on multiple data sets were set up.
[0092] Experimental results:
[0093] Table 1 summarizes the mAP evaluation results of each algorithm on the underwater dataset constructed by the present invention.
[0094] Table 1
[0095]
[0096] Table 2 summarizes the mAP evaluation results of each algorithm on the existing underwater dataset URPC2018.
[0097] Table 2
[0098]
[0099] Table 3 summarizes the mAP evaluation results of each algorithm on the general object detection dataset COCO.
[0100] Table 3
[0101]
[0102] The method proposed in the present invention can be perfectly adapted to underwater detection tasks.
[0103] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.
Claims
1. An underwater target detection method based on spatial feature self-supervision, characterized in that: The underwater target detection method comprises the following steps: Step 1: Collect data. Use an optical camera to obtain underwater photos, including the seafood to be inspected, and build the data set required for the inspection task. Step 2: Data augmentation: perform data augmentation on the underwater optical data obtained in step 1, expand the original data and manually annotate it so that the training set accounts for 80% and the validation set accounts for the remaining 20%; Step 3: Build a self-supervised contrastive learning network and use spatial attention and channel normalization to adapt the backbone network to the underwater target detection task; Step 4: construct an underwater target detection network framework, wherein the underwater target detection network framework includes a feature extraction backbone network, a feature pyramid network, and a feature selection detection head based on feature decoupling; Step 5: Train the underwater target detection network, input the collected sample data into the built network by rotating, cropping, and scaling, and obtain the weight model of the network for underwater target detection of seafood; Step 6: According to the weight model trained in step 5, the results of the divided validation set are predicted to evaluate the detection effect in practical applications; Among them, the feature pyramid network in step 4 contains 5 levels with different resolution sizes. The first three levels P1, P2, and P3 are obtained by the last three layers of the feature extraction network. P4 and P5 are downsampled once and twice respectively on the basis of P3. The output steps of the 5 levels in the feature pyramid network are: Step 4-1: Set learning parameters for the five levels, achieve fusion between different layers through bilinear interpolation, and finally achieve adaptive weighted fusion. The processing of the feature pyramid layer is expressed as the following formula: OUT i =F(W i1 *P1+W i2 *P2+W i3 *P3+W i4 *P4+W i5 *P5) Among them (w i1 ,w i2 ,w i3 ,w i4 ,w i5 ) represents the learnable weights between different layers, (P1, P2, P3, P4, P5) represent the feature maps of different resolutions at five levels, and F represents the use of 3DMaxPooling; Step 4-2: After weighted fusion, 3DMaxPooling is introduced to suppress feature redundancy. 3DMaxPooling first passes the input feature map through a 3×3 convolution kernel, and then obtains feature maps of the same size between adjacent levels through interpolation. 3Dmaxpool is used to mine potential spatial feature relationships and remove redundant features. The process of feature redundancy suppression is expressed as the following formula: y s =max{x s,k =Bilinear(x k )} Among them, s represents the feature map of the current layer, k represents the feature map of the adjacent layer, y represents the final output, and Bilinear represents the interpolation method; Step 4-3: Finally, the result of adding the input feature map and 3Dmaxpool is sent to the GN-RELU layer.
2. According to claim 1, the underwater target detection method based on spatial feature self-supervision is characterized by: The feature selection detection head in step 4 includes a classification branch, a regression branch and a center branch. The classification branch selects the middle part of the grid as a positive sample by sampling. The regression branch uses a boundary extraction module to extract boundary features for more accurate boundary positioning. The center branch is used to calculate the degree of deviation of the sample from the center, and its value is between 0 and 1.
3. The underwater target detection method based on spatial feature self-supervision according to claim 2, characterized in that: The classification branch selects the middle part of the network as the positive sample by sampling, which is expressed as the following formula: x′1=c′ x -0.5wε y′1=c' y -0.5hε x′2=c′ x +0.5wε y'2=c' y +0.5hε where ε is the scaling factor, the default value is 0.3, c' x ,c' y is the center coordinate, w and h are the width and height of the real box respectively, [x′1, y′1, x′2, y′2] is the real position of the object in the feature map after feature extraction, which has been downsampled by different multiples and is the real box divided by 2 l It is obtained that l is the downsampling multiple.
4. The underwater target detection method based on spatial feature self-supervision according to claim 3, characterized in that: The boundary positioning of the regression branch is specifically as follows: the regression branch adopts a two-time regression cascade method, uses the first regression result to calculate the maximum value point on the four boundaries of the regression box, finds the pixel position in the original image by interpolation, uses the pixel to perform the regression task, and finally obtains a more precise object positioning.
5. The underwater target detection method based on spatial feature self-supervision according to claim 1, characterized in that: The self-supervised contrastive learning network in step 3 includes an online branch and a target branch. The feature encoder of the online branch is composed of ResNet50. The online branch includes a feature encoder, a spatial feature attention, a channel normalization structure and two linear projection layers. The feature encoder is at the front. The spatial feature attention takes the output of the feature encoder as input. The channel normalization structure takes the output of the spatial feature attention as input. Finally, the output features are encoded through two linear projection layers. The feature encoder of the target branch is composed of ResNet50, and the target branch includes a feature encoder and a linear projection layer, and the output of the feature encoder is directly sent to the linear projection layer.
6. The underwater target detection method based on spatial feature self-supervision according to claim 1, characterized in that: The method for enhancing the data in step 2 mainly includes color restoration and illumination change.
7. The underwater target detection method based on spatial feature self-supervision according to claim 6, characterized in that: The color restoration adopts a white balance method, which specifically includes the following steps: Step 2-1: After collecting pictures containing underwater creatures, place the pre-set colorimetric card in the water and use a camera to take pictures of underwater scenes with the standard colorimetric card. Step 2-2: Use OpenCV to discretize the underwater image with the standard colorimetric card into three channels: red, green, and blue, and extract the locations of the red, green, and blue colors in the colorimetric card. Step 2-3: Obtain pixel values according to the positions of red, green, and blue colors, and calculate the actual compensation value by comparing it with the discrete value of the standard color. The calculation of the compensation value v is as follows: v red =c red -c′ red v green =c green -c′ green v blue =c blue -c′ blue Among them, v has three values, corresponding to the three color channels of red, green and blue, respectively; c' represents the pixel value after discretization of the standard colorimetric card; and c represents the pixel value after discretization of the extracted color; Step 2-4: Add the three color channels of all acquired data images and the corresponding compensation values to obtain the final enhanced image.
8. The underwater target detection method based on spatial feature self-supervision according to claim 1, characterized in that: The training process of the underwater target detection network in step 5 is as follows: the underwater image enhanced in step 2 is input into the detection network, SGD is used as the optimization method, a linear warm up strategy is sampled in the first 500 iterations, the learning rate is gradually increased to 0.005, and a 10-fold learning rate decay is performed at 80% and 90% of the total iterations, respectively, and the initial weight p is saved after 36 rounds of basic training.
Citation Information
Patent Citations
Single-stage underwater biological target detection method based on feature enhancement and refinement
CN112417980A
Target detection method and system
CN112801146A
Collaborative learning character recognition method under attention mechanism
CN112966684A
Scene graph generation method based on self-supervised pre-training
CN112989927A
