Underwater Target Detection Method Based on Detail Enhancement

Abstract

The present invention discloses an underwater target detection method based on detail enhancement. First, underwater target data is acquired, and the data is stored, preprocessed, and dataset partitioned to form a road extraction sample dataset. Based on the YOLO target detection framework, an underwater target detection network extraction network is constructed. This algorithm is based on a detail enhancement strategy. At the feature extraction level, feature extraction is strengthened through a richer gradient flow and attention mechanism; in the feature aggregation network, a receptive field enhancement module is additionally introduced to increase the network receptive field and enhance the model's extraction of features such as texture. In addition, a detail enhancement branch is designed to enhance the underlying detail features, especially edge features, and aggregate them into the network in the form of channel weighting. The detail features, as prior knowledge, can effectively supervise the learning of the network; the underwater detection head is improved, which has more detection layers.

Description

Technical Field

[0001] The present invention relates to computer vision, deep learning, and video object detection technologies. Specifically, it relates to an object detection method based on detail enhancement. Background Art

[0002] With the continuous development of society and the increasing population, the shortage of various resources has become increasingly prominent, forcing people to turn their attention to the undeveloped ocean areas. The ocean contains rich biological and physical resources, including various fish, shellfish, kelp, seaweed, marine animals, etc. These resources are important food sources for humans. In addition, the ocean also contains rich mineral resources such as oil, natural gas, and coal, which are crucial for human economic development.

[0003] Before conducting ocean-related tasks, the primary problem is to identify underwater targets. Currently, two main techniques are used for underwater target detection: one is acoustic-based, using devices such as sonar for target detection; the other is based on optical images. Acoustic images are mainly used for long-distance and large-target detection tasks, but they cannot provide detailed information such as color and are powerless for close-range and high-precision tasks. In contrast, object detection algorithms based on optical images have higher accuracy and richer detailed information and are widely used in fields such as autonomous robots in underwater environments, underwater search and rescue, and underwater engineering. For example, in ocean resource development, underwater target detection technology is needed to determine the location, quantity, and status of mineral resources, fishery resources, etc. on the seabed and to evaluate and utilize these resources. In underwater archaeological tasks, underwater target detection technology is needed to search for and identify sunken ship remains, etc. In modern construction projects, involving the construction and inspection of undersea optical cables and oil pipelines, introducing underwater target detection technology can greatly improve work efficiency. In addition, underwater target detection technology can also be applied to fields such as ocean environmental monitoring and ocean ecological protection to help people better understand and protect ocean resources.

[0004] In the underwater environment, object detection needs to deal with complex situations such as uneven illumination, target occlusion, similar target and background colors, and image noise interference. Therefore, this patent proposes an object detection algorithm based on detail enhancement to solve the problem of detecting different-scale targets in complex underwater scenarios.

[0005] Technical Solution

[0006] The technical problem solved by the present invention is to propose an underwater object detection algorithm based on detail enhancement to achieve the detection of multi-scale targets in complex underwater scenarios and ensure accuracy and robustness.

[0007] The technical solution of the present invention is as follows: An underwater target detection algorithm based on detail enhancement. First, underwater target data is acquired, and the data is stored, preprocessed, and the dataset is divided to form a road extraction sample dataset. Based on the YOLO target detection framework, an underwater target detection network extraction network is constructed. This algorithm is based on a detail enhancement strategy. At the feature extraction level, feature extraction is strengthened through a richer gradient flow and attention mechanism; in the feature aggregation network, a receptive field enhancement module is additionally introduced to increase the network receptive field and enhance the model's extraction of features such as texture. In addition, a detail enhancement branch is designed to enhance the underlying detail features, especially edge features, and aggregate them into the network in the form of channel weighting. The detail features, as prior knowledge, can effectively supervise the learning of the network; the underwater detection head is improved, which has more detection layers. The specific steps are as follows:

[0008] (1) Acquire the underwater target dataset, store, preprocess, and divide the dataset. And clean and augment the image data.

[0009] (2) Use the established underwater dataset to train the underwater target detection algorithm. Initially, set the model learning rate to 0.01, and perform 200 rounds of training on the training set and the validation set. Observe the model training results such as precision, recall, and mAP values. On the premise of ensuring that the precision and recall values are not much different, use the method of parameter tuning to improve the mAP value. The specific steps include:

[0010] (a) The present invention adopts a C2F network that strengthens gradient shunting. The C2F class is an improvement based on CSPnet and is a module composed of two parallel convolutional blocks, and each convolutional block has an independent calculation path. Each of its bottlenecks is shunted, with a part sent to the next bottleneck and a part directly performing residual connection as a shortcut. The calculation process of the C2F class is as follows: The input feature map x is calculated through the first convolutional block to generate the feature map f1. The feature map f1 undergoes a split operation to generate the feature maps f2 and f2 i , where the feature map f2 is input into the bottleneck to obtain the feature f3, and f3 also undergoes a split operation to obtain the new features f3 and f3 i , f3 is input into the bottleneck to obtain the feature f4... and so on. Finally, the feature maps fn and fx i (x = 1, 2,... n) are concatenated together to form a feature map and output after convolution.

[0011] (b) We integrate the attention mechanism into the C2F module of the backbone and the neck. The feature output fo of the network passes through an attention mechanism module to obtain the final feature map fatt, aiming to further improve the feature extraction ability of the proposed detection model. The specific calculation is as follows:

[0012] The first branch is the channel attention calculation branch. The input feature passes through Z-Pool, then through a convolutional layer and a BN layer, and finally through a Sigmoid activation function to generate the spatial attention weight.

[0013] The second branch is the interaction branch of channel C and spatial W dimensions. The input feature is first rearranged by permute into a feature of (H, C, W) dimensions, then passes through Z-Pool in the H dimension, and then through a convolutional layer, a BN layer, and a Sigmoid activation function. For the convenience of element-wise operations. Finally, it needs to be rearranged by permuter into a feature of (C, H, W) dimensions.

[0014] The third branch is the interaction branch of channel C and spatial H dimensions. The input feature is first passed through permute to become a feature of (W, H, C) dimensions, then Z-Pool is performed in the W dimension, and then through a convolutional layer, a BN layer, and a Sigmoid activation function. For the convenience of element-wise operations. Finally, it needs to be rearranged by permuter into a feature of (C, H, W) dimensions.

[0015] Finally, the outputs of the three branches are averaged. The Z-pool layer reduces the second dimension of the tensor to 2 by concatenating the average pooling and max pooling features of each dimension. The advantage of the Z-pool layer is to obtain a detailed representation of the actual tensor, while also reducing the depth of the tensor, making the following calculations more efficient. The expression of Z-pool is:

[0016] Z-pool(χ) = [MaxPool 0d (χ), AvgPool 0d (x)]#

[0017] Among them, 0d is the 0th dimension, and the max and average set operations are performed on this dimension. Average pooling operation. For example, the Z-Pool result of a tensor with a shape of (C×H×W) is a tensor with a tensor shape of (2×H×W).

[0018] (c) A receptive field expansion module RFB is introduced between the feature extraction layer and the feature aggregation layer. The receptive field expansion module of the present invention consists of four branches Composition. To reduce the number of parameters and accelerate training, in each branch, we use a 1×1 convolutional layer to reduce the number of channels. Due to the strong correlation between adjacent units, such a dimensionality reduction operation loses very little feature information. If the output is used for spatial aggregation, dimensionality reduction even promotes faster learning. For {b m , m > 1}. We add two layers: a (2m - 1)×(2m - 1) convolutional layer and a 3×3 dilated convolutional layer with a dilation rate of (2m - 1). We connect the outputs of these branches and reduce the number of channels to 32. Large convolutional kernels can be decomposed to reduce the computational cost and the number of parameters. For example, a 5*5 convolutional kernel can be decomposed into two 3*3 convolutional kernels, and a 7*7 convolutional kernel can be decomposed into three 3*3 convolutional kernels. Furthermore, there is a way to decompose it into two asymmetric convolutional kernels. A (2m - 1)×(2m - 1) convolutional kernel can be decomposed into two convolutional kernels of (2m - 1)×1 and 1×(2m - 1). It should be noted that this way of decomposing asymmetric convolutional kernels only achieves better results in deep layers. In shallow networks, some low-level features will be lost. In the present invention, the RFB module is used after the deepest layer of the feature extraction network. Finally, there is a shortcut that directly connects to the feature map after dimensionality reduction and merging to form a residual. Another improvement point of this module in the present invention is referring to DenseNet

[67] , and taking the result of the b m branch as the input of the b n (n = m + 1) branch. The output feature map of the previous small convolutional kernel branch undergoes a larger-range dilated convolution of the subsequent branch, further expanding the receptive field, enhancing the detailed texture features, and through feature reuse, strengthening the feature transmission with a relatively small increase in the number of parameters. After adding the RFB module, the receptive field of the model is efficiently increased, effectively enhancing the detailed features, which is convenient for localization and detection. This cascading structure increases the receptive field scale to 13 types in the way of feature reuse, and can more effectively extract features at different scales. At the same time, because of a large amount of feature reuse in this cascading structure, the increase in the number of parameters is limited compared with the ordinary RFB. In short, after embedding the improved receptive field expansion module, not only the receptive field is greatly increased, but also the detailed feature extraction is effectively enhanced.

[0019] (d) The detailed enhancement module used in the present invention consists of two branches. Its input is from two low layers of the feature extraction network. f1 and f2 are respectively input into a 3×3 convolutional layer to reduce the dimension and make the number of channels of the two branches the same. Then, it undergoes a 3×3 convolutional layer for feature extraction, which can be expressed as:

[0020]

[0021] Then, an adaptive weighted switch w s is used to learn how to weigh the attention at different levels (i = 1, 2).

[0022] This process can be defined as element-wise multiplication.

[0023]

[0024] This process may simultaneously lead to the problem of valuable clues disappearing. So finally, through a splicing operation integration, two features are fused.

[0025] When the detail enhancement module extracts the features of f1 and f2 and multiplies these two features respectively, it is equivalent to performing an adaptive weighting in the channel dimension of the features, assigning different importance to the features in different directions. This can be regarded as an attention mechanism using feature multiplication, which has a low complexity and is linear. Although the model does not explicitly use the attention mechanism, it still has certain characteristics of the attention mechanism during the feature extraction process. It can well suppress background noise and, by adjusting the weights of different branches, adaptively place more attention on the areas we are concerned about.

[0026] In addition, outside the basic convolutional unit, an activation function FRelu

[69] is cascaded. The non-linear fitting ability is increased to enhance the expression ability of the network. FReLU is an activation function for visual tasks. Compared with ReLU, it extends the spatial conditions to enhance spatial sensitivity and endows it with pixelated modeling ability. The expression of FReLU is shown as follows, where T(x) represents the two-dimensional spatial condition.

[0027]

[0028]

[0029] (3) Verification of the underwater target feature model. Retain the optimal model in the training rounds of process (2), use this model to perform the first-round verification on the validation set to generate a prediction set, view the pictures with lower prediction results, add data of the same category to the training set so that the model can learn the features of this category. Repeat the training process until the model accuracy reaches an acceptable level, and retain the optimal model.

[0030] The advantages of the present invention compared with the prior art are as follows:

[0031] 1. In the feature extraction link, a convolutional layer with rich gradient flow information and integrated triple attention mechanism is reconstructed to improve the feature expression ability;

[0032] 2. In the feature fusion link, an improved receptive field expansion module is introduced. With the help of dilated convolution, without increasing the number of parameters and computational complexity, each time the convolutional kernel slides on the input tensor, it can cover a larger area and obtain more extensive context information;

[0033] 3. A detail enhancement branch is designed to better fuse the underlying feature information and use these features as prior knowledge to supervise the network learning. The detail information provides useful constraint conditions to guide feature extraction during object detection; BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is the overall flowchart of the present invention.

[0035] Figure 2 It is the feature extraction network layer of the present invention.

[0036] Figure 3 It is the receptive field expansion of the present invention.

[0037] Figure 4 It is the detail enhancement branch of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0038] In order to enable those skilled in the art to better understand the solutions of the embodiments of the present invention, the embodiments of the present invention will be further described in detail below in conjunction with the drawings and embodiments.

[0039] As Figure 1 shown, the present invention includes the following steps:

[0040] 1. Dataset preprocessing: The present invention uses RandomResize (random scale transformation) and RandomCrop (random cropping) to randomly scale and crop the images, and HSVRandomAug (color space enhancement) to enhance the color space of the images, using the HSV components to increase the brightness, saturation and hue of the images. We also use RandomFlip (random horizontal flipping) to increase the diversity of the dataset. These techniques can improve the diversity of the dataset, thereby improving the generalization ability of the model.

[0041] 2. Train the underwater target dataset with a target detection model. The specific steps include: constructing an underwater target detection network, training the target detection network, testing the target detection network training, parameter tuning, and storing the optimal model;

[0042] (e) Construct an underwater target detection network: First, construct a C2F network that strengthens gradient shunting adopted in this paper. The C2F class is an improvement based on CSPnet and is a module composed of two parallel convolutional blocks, each with an independent computational path. Each bottleneck of it is shunted, with a part sent to the next bottleneck and a part directly performing residual connection as a shortcut. The calculation process of the C2F class is as follows: The input feature map x is calculated through the first convolutional block to generate the feature map f1. The feature map f1 undergoes a split operation to generate the feature maps f2 and f2 i , where the feature map f2 is input into the bottleneck to obtain the feature f3, and f3 also undergoes a split operation to respectively obtain the new feature f3 and f3 i , f3 is input into the bottleneck to obtain the feature f4... and so on. Finally, the feature maps fn and fx i (x = 1, 2,... n) are concatenated together to form a feature map and output after convolution. Secondly, integrate the attention mechanism into the C2F modules of the backbone and neck. The feature output fo of the network passes through an attention mechanism module to obtain the final feature map fatt, aiming to further improve the feature extraction ability of the proposed detection model. The specific calculation is as follows: The first branch is the channel attention calculation branch. The input feature passes through Z-Pool, then through a convolutional layer and a BN layer, and finally through a Sigmoid activation function to generate the spatial attention weight. The second branch is the channel C and spatial W dimension interaction branch. The input feature is first rearranged through permute into a (H, C, W)-dimensional feature, then undergoes Z-Pool in the H dimension, and then through a convolutional layer, a BN layer, and a Sigmoid activation function. For the convenience of element-wise operation. Finally, it needs to be rearranged through permuter into a (C, H, W)-dimensional feature. The third branch is the channel C and spatial H dimension interaction branch. The input feature is first permuted into a (W, H, C)-dimensional feature, then undergoes Z-Pool in the W dimension, and then through a convolutional layer, a BN layer, and a Sigmoid activation function. For the convenience of element-wise operation. Finally, it needs to be rearranged through permuter into a (C, H, W)-dimensional feature. Finally, the averages of the outputs of the 3 branches are calculated. The Z-pool layer reduces the second dimension of the tensor to 2 by concatenating the average pooling and max pooling features of each dimension. The advantage of the Z-pool layer is to obtain a detailed representation of the actual tensor while also reducing the depth of the tensor, making the following calculations more efficient. The expression of Z-pool is:

[0043] Z-pool(χ) = [MaxPool 0d (χ), AvgPool0d (x)]#

[0044] Among them, 0d is the 0th dimension, and the maximum and average set operations are performed on this dimension. Average pooling operation. For example, the Z-Pool result of a tensor with a shape of (C×H×W) is a tensor with a tensor shape of (2×H×W). Then, a receptive field expansion module RFB is introduced between the feature extraction layer and the feature aggregation layer. The receptive field expansion module of the present invention consists of four branches components. In order to reduce the number of parameters and accelerate training, in each branch, we use a 1×1 convolutional layer to reduce the number of channels. Due to the strong correlation between adjacent units, very little feature information is lost in such a dimensionality reduction operation. If the output is used for spatial aggregation, dimensionality reduction even promotes faster learning. For {b m , m>1}. We add two layers: a (2m - 1)×(2m - 1) convolutional layer and a 3×3 dilated convolutional layer with a dilation rate of (2m - 1). We connect the outputs of these branches and reduce the number of channels to 32. Large convolutional kernels can be decomposed to reduce the computational amount and the number of parameters. For example, a 5*5 convolutional kernel can be decomposed into two 3*3 convolutional kernels, and a 7*7 convolutional kernel can be decomposed into three 3*3 convolutional kernels. Furthermore, there is a way to decompose it into two asymmetric convolutional kernels. A (2m - 1)×(2m - 1) convolutional kernel can be decomposed into two convolutions of (2m - 1)×1 and 1×(2m - 1). It should be noted that this method of decomposing asymmetric convolutions can only achieve better results in deep layers. In shallow networks, some low-level features will be lost. The RFB module of the present invention is used after the deepest layer of the feature extraction network. Finally, there is also a shortcut that directly connects to the feature map after dimensionality reduction and merging to form a residual. Another improvement point of this module of the present invention is to refer to DenseNet

[67] , and use the result of the b m branch as b n(n = m + 1) branch input, the output feature map of the previous small convolutional kernel branch passes through a larger receptive field dilated convolution of the subsequent branch, further expanding the receptive field, enhancing the detailed texture features, and strengthening the feature transmission with fewer additional parameters through feature reuse. After adding the RFB module, the receptive field of the model increases efficiently, effectively enhancing the detailed features for localization and detection. This cascaded structure increases the receptive field scale to 13 in the way of feature reuse, enabling more effective feature extraction at different scales. At the same time, due to a large amount of feature reuse in this cascaded structure, the increase in the number of parameters is limited compared to the ordinary RFB. In summary, after embedding the improved receptive field expansion module, not only the receptive field is significantly increased, but also the detailed feature extraction is effectively enhanced. Finally, the used detail enhancement module consists of two branches. Its input is from two low-level layers of the feature extraction network. f1 and f2 are respectively input into a 3×3 convolutional layer to reduce the dimension and make the number of channels of the two branches the same. Then, it passes through another 3×3 convolutional layer for feature extraction, which can be expressed as:

[0045]

[0046] Then, an adaptive weighted switcher w is used s to learn how to weigh the attention at different levels (i = 1, 2). This process can be defined as element-wise multiplication.

[0047]

[0048] This process may simultaneously lead to the problem of valuable clues disappearing. So finally, through a concatenation operation for integration to fuse the two features. When the detail enhancement module extracts the features of f1 and f2, and multiplies these two features respectively, it is equivalent to performing an adaptive weighting on the channel dimension of the features, assigning different importance to the features in different directions. This can be regarded as an attention mechanism using feature multiplication, with a relatively low complexity and being linear. Although this model does not explicitly use the attention mechanism, it still has certain characteristics of the attention mechanism during the feature extraction process. It can well suppress background noise and, by adjusting the weights of different branches, adaptively place more attention on the areas we are concerned about. In addition, outside the basic convolutional unit, an activation function FRelu is cascaded. To increase the non-linear fitting ability to enhance the expression ability of the network. FReLU is an activation function for visual tasks. Compared with ReLU, it expands the spatial conditions to enhance spatial sensitivity and enables pixelized modeling ability. The FReLU expression is shown as follows, where T(x) represents the two-dimensional spatial condition.

[0049]

[0050]

[0051] (3) Underwater target detection network training: Initially, the model learning rate is set to 0.01, and the training set and validation set are trained for 200 rounds. The model training results, such as precision, recall, and mAP, are observed. On the premise of ensuring that the precision and recall values are similar, the mAP value is improved by parameter tuning. The specific steps include:

[0052] Underwater object detection network training and testing: The best model in the underwater object detection network training round is retained, and the model is used to perform the first round of testing on the test set to generate a prediction set. The pictures with lower prediction results are checked, and the same category data is added to the training set so that the model can learn the features of this category.

[0053] Parameter tuning: Repeat the model training and model testing process until the model accuracy reaches a usable level, retain the optimal model, and the road extraction model is completed.

[0054] Keep the best model: Keep the best model after optimization.

[0055] It should be noted that the method of the embodiment of the present invention is suitable for target detection in complex underwater scenes.

[0056] The embodiments of the present invention are described in detail above. The present invention is explained in this article using specific implementation methods. The description of the above embodiments is only used to help understand the method of the present invention. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. Underwater target detection method based on detail enhancement, characterized in that The specific steps of this method are as follows: Step (1): Obtain an underwater image target dataset, store, preprocess, and partition the underwater image data; and clean and augment the underwater image data; Step (2): Use the established underwater dataset to train the underwater target detection algorithm. Initially, set the model learning rate to 0.01, and perform 200 rounds of training on the training set and the validation set. Observe the model training results precision, recall, and mAP; Step (3): Validate the underwater target feature model; Retain the optimal model in the training rounds of Step (2). Use this model to perform the first round of validation on the validation set to generate a prediction set. Check the pictures with lower prediction results, and add data of the same category to the training set to enable the model to learn the features of this category; Repeat the training process until the model accuracy reaches an acceptable level, and retain the optimal model; The specific steps of Step (2) include: (a) Adopt the C2F network with enhanced gradient shunting. The C2F class is an improvement based on CSPnet and is a module composed of two parallel convolutional blocks. Each convolutional block has an independent computational path; each bottleneck is shunted, with a part sent to the next bottleneck and a part directly performing residual connection as a shortcut; (b) Integrate the attention mechanism into the C2F modules of the backbone and the neck; the feature output fo of the network passes through an attention mechanism module to obtain the final feature map fatt; (c) A receptive field expansion module RFB is introduced between the feature extraction layer and the feature aggregation layer. The receptive field expansion module RFB consists of four branches ; a 1×1 convolutional layer is used to reduce the number of channels; for {b m , m > 1}; two layers are added: a (2m - 1)×(2m - 1) convolutional layer and a 3×3 dilated convolutional layer with a dilation rate of (2m - 1); the large convolutional kernel is decomposed to reduce the computational amount and the number of parameters; the receptive field expansion module RFB is used after the deepest layer of the feature extraction network; finally, the shortcut is directly connected to the feature map after dimensionality reduction and merging to form a residual, and the result of the b m branch is used as the input of the b n (n = m + 1) branch. The output feature map of the previous small convolutional kernel branch is further enlarged by the dilated convolution with a larger receptive field of the subsequent branch to enhance the detailed texture features; (d) The detail enhancement module consists of two branches. Its input is from two low-level layers of the feature extraction network. f1 and f2 are respectively input into a 3×3 convolutional layer to reduce the dimension and make the number of channels of the two branches the same. Then, it passes through another 3×3 convolutional layer for feature extraction, which is expressed as: Use an adaptive weighted switch w s to learn how to weigh different levels of attention, which is defined as element-wise multiplication; integrate through a concatenation operation to fuse two features; When the detail enhancement module extracts the features of f1 and f2, it multiplies these two features respectively, assigning different importances to the features in different directions; a activation function FRelu is cascaded outside the basic convolutional unit to increase the non-linear fitting ability.

2. The underwater target detection method based on detail enhancement according to claim 1, characterized in that The calculation process of the C2F class is as follows: The input feature map x is calculated through the first convolutional block to generate the feature map f1; The feature map f1 undergoes a split operation to generate the feature maps f2 and f2 respectively i , where the feature map f2 is input into the bottleneck to obtain the feature f3, and f3 also undergoes a split operation to obtain the new features f3 and f3 respectively i , f3 is input into the bottleneck to obtain the feature f4... and so on. Finally, the feature maps fn and fx i are concatenated together to form a feature map and output through convolution, where x = 1, 2,... n.

3. The underwater target detection method based on detail enhancement according to claim 1, characterized in that, The specific calculation of Step (b) is as follows: The first branch is the channel attention calculation branch; The input feature passes through Z-Pool, then through a convolutional layer and a BN layer, and finally through a Sigmoid activation function to generate the spatial attention weight; The second branch is the channel C and spatial W dimension interaction branch; The input feature is first rearranged to the (H, C, W) dimension feature through permute, then passes through Z-Pool in the H dimension, and then through a convolutional layer, a BN layer, and a Sigmoid activation function; for the convenience of element-wise operation; finally, it needs to be rearranged to the (C, H, W) dimension feature through permuter; The third branch is the channel C and spatial H dimension interaction branch. The input features first go through permute to become features of (W, H, C) dimension, then Z-Pool is performed on the W dimension, followed by a convolutional layer, a BN layer, and a Sigmoid activation function; for the convenience of element-wise operations; finally, it needs to go through a permuter to be rearranged into features of (C, H, W) dimension; Finally, the outputs of the 3 branches are averaged; among them, the Z-pool layer reduces the second dimension of the tensor to 2 by concatenating the average pooling and max pooling features of each dimension.

Images