Underwater small target detection system capable of reserving fine-grained information

Abstract

The invention provides an underwater small target detection system capable of reserving fine-grained information, and the system comprises a fine-grained down-sampling module which carries out the down-sampling through the linear combination between channels and employing an SPD-Conv layer, and obtains a series of sub-feature maps; the C2fSCConv module is introduced into a backbone network of YOLOv8, that is, space channel reconstruction convolution SCConv is fused into a Bottleneck bottleneck module in the C2fSCConv module to replace a second CBS module in the Bottleneck bottleneck module, and effective information of a sub-feature graph is extracted through stacking of the bottleneck module; and the deconvolution channel feature fusion module is arranged between the YOLOv8 backbone and the neck and is used for carrying out reinforced fusion on the sub-feature maps based on the effective information of the sub-feature maps. According to the method, challenges in underwater small target detection can be processed more effectively, the detection precision is improved, the feature redundancy and the calculation amount are reduced, and therefore the requirements of practical application are met.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and deep learning, and relates to an underwater small target detection system that retains fine-grained information. Background Art

[0002] Underwater images differ greatly from images taken from natural scenes in appearance, such as water mist effect, low visibility and color distortion, which make it difficult to fully utilize underwater target feature information. In addition, underwater environments often have problems such as small and densely distributed targets, leading to overlapping occlusions. The ERep-YOLOv8 underwater target detection method improves the network's feature extraction capabilities and target positioning capabilities by introducing the GAM attention mechanism, efficient generalized feature pyramid and coordinate perception head, which can reduce the interference of the environment on detection to a certain extent. However, the network does not solve the problem of small target detection in underwater scenes. Small and weak targets in underwater images usually have problems such as low contrast, mutual occlusion, dense distribution and diverse features. Therefore, the detection accuracy of existing target detection methods in such scenarios is not ideal. Summary of the invention

[0003] The purpose of the present invention is to propose an underwater small target detection system that retains fine-grained information, which can more effectively deal with the challenges in underwater small target detection and improve the accuracy and efficiency of underwater small target detection.

[0004] The technical solution to achieve the purpose of the present invention is:

[0005] An underwater small target detection system that retains fine-grained information, based on YOLOv8, is characterized by comprising:

[0006] Fine-grained downsampling module, which obtains a series of sub-feature maps through linear combination between channels and downsampling using SPD-Conv layer;

[0007] C2f_SCConv module, which is introduced into the backbone network of YOLOv8, that is, the spatial channel reconstruction convolution SCConv is integrated into the Bottleneck bottleneck module in the C2f module, replacing the second CBS module in the Bottleneck bottleneck module, and extracting the effective information of the sub-feature map through the stacking of bottleneck modules;

[0008] The deconvolution channel feature fusion module is located between the YOLOv8 backbone and the neck. It enhances the fusion of sub-feature maps based on their effective information.

[0009] Furthermore, the fine-grained downsampling module includes a cascaded CBS module, an SPD layer and a strideless convolution layer, and the CBS module includes a cascaded strideless convolution layer, a BN layer and a SILU activation function.

[0010] Furthermore, the downsampling process of the fine-grained downsampling module is as follows: assuming that the size of the input feature map is S×S×C1, the feature information of the feature map is first extracted by the CBS module, and the size of the output intermediate feature map is S×S×C1; then, the SPD layer is passed, and the intermediate feature map X of size S×S×C1 is cut to divide it into a series of sub-feature maps, and finally, a stepless convolution layer is passed to make the dimension of the feature map from S / scale×S / scale×scale 2 C1 becomes S / scale×S / scale×C2, where C2 <scale 2 C1, scale is the downsampling ratio.

[0011] Furthermore, the SPD layer is divided into a series of sub-feature graphs:

[0012] f x，y =X[x: S: scale, y: S: scale]

[0013] Among them, f x，y Represents the subgraph of the intermediate feature graph X, 0≤x, y≤, x, y are the rows and columns after downsampling, subgraph f x，y It consists of all entries X(i+j) that are divisible by i+x and j+y, where i and j represent the row index and column index of the feature map X respectively.

[0014] Furthermore, the spatial channel reconstruction convolution includes a spatial reconstruction unit and a channel reconstruction unit, which are combined in a sequential manner. For each input feature X, the spatial reconstruction unit is first operated to obtain a spatial refined feature X ω , and then use the channel reconstruction unit to obtain the channel refined feature Y, and then add each channel refined feature Y and the initial input feature X to obtain the fusion feature.

[0015] Furthermore, the output characteristics of the space reconstruction unit are:

[0016]

[0017] Here, μ and σ are the mean and standard deviation of X, ε is a stable number of settings, and γ and β are trainable parameters.

[0018] Furthermore, the channel reconstruction unit adopts a split-transform-fusion strategy, which specifically includes: input feature X is multiplied by information weight W1 and non-information weight W2 to obtain a feature with large information content and small amount of information The characteristics Divided into characteristics and Features Divided into characteristics and Features Then the features and Features Add together to get feature X ω1 ,feature and Features Add together to get feature X ω2 , and then connect the cross-reconstructed features X ω1 and X ω2 Get the spatial extraction feature X ω , X ω The upper feature X is obtained by channel compression through 1×1 convolution up and the lower feature X low , upper feature X up After K×K convolution and 1×1 PWC, the representative feature map Y1 is obtained. low and X low The generated features after 1×1 PWC operation are concatenated to obtain the representative feature map Y2. The channel soft attention mechanism is used to generate feature importance vectors β1 and β2, and the spatial channel refined feature Y is obtained by β1Y1+β2Y2.

[0019] Furthermore, the deconvolution channel feature fusion module includes a convolution layer, a deconvolution layer and a connection layer, specifically including: the last feature layer F1 in the YOLOv8 backbone network is passed through the convolution layer to obtain the feature F2, and then passed through the kernel deconvolution layer to obtain the feature F3, and finally the feature F2 and the feature F3 are connected through the connection layer to obtain the enhanced feature Output.

[0020] Furthermore, the convolutional layer is Where H and W are the height and width of the feature map, K represents the size of the convolutional layer, and G represents the number of groups.

[0021] Furthermore, the deconvolution layer is K represents the size of the deconvolution layer, i and j represent pixel coordinates, and g = 1, 2, ..., G.

[0022] Compared with the existing target detection model, the advantages and beneficial effects of the present invention are as follows:

[0023] (1) The fine-grained downsampling module of the present invention retains more detailed information through linear combinations between channels, and can flexibly adjust the number of channels to control the complexity of the model and avoid overfitting. It uses the SPD-Conv layer for downsampling and reduces the channel dimension through the strideless convolution layer to reduce the amount of calculation;

[0024] (2) The present invention integrates the spatial and channel reconstruction convolution (SCConv) into the C2f module and develops the C2f_SCConv module. By stacking bottleneck modules, the feature redundancy in the feature extraction process is fully suppressed to obtain features with more effective information.

[0025] (3) The present invention makes fuller use of the rich feature information of high-level channels in the backbone network through the deconvolution channel feature fusion module, reduces the information loss in the initial stage of the path aggregation network, and can make fuller use of semantic information, thereby obtaining high resolution and strong semantic features, and detecting small targets faster and better;

[0026] (4) The present invention can more effectively handle the challenges in underwater small target detection, improve detection accuracy, reduce feature redundancy and computational complexity, and thus meet the needs of practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Schematic diagram of the fine-grained downsampling structure in the present invention.

[0028] Figure 2 This is the structure diagram of C2f_SCConv in the present invention.

[0029] Figure 3 This is a structural diagram of the space reconstruction unit in the present invention.

[0030] Figure 4 This is a structural diagram of the channel reconstruction unit in the present invention.

[0031] Figure 5 This is a diagram of the deconvolution structure in the present invention.

[0032] Figure 6 It is a schematic diagram of the overall structure of the system of the present invention. DETAILED DESCRIPTION

[0033] The present invention will be further described below in conjunction with the accompanying drawings.

[0034] The present invention proposes an underwater small target detection system that retains fine-grained information. Firstly, considering the loss of fine-grained feature information in the downsampling process in the network, the present invention develops a fine-grained downsampling module, and fuses the CBS module on the basis of the SPD-Conv module, so as to better retain the detail information of the feature map while reducing the channel dimension and reducing the amount of calculation; secondly, there is a considerable amount of redundancy in the deep learning network, which exists not only in the dense model parameters, but also in the spatial dimension and channel dimension of the feature map. However, previous work either focuses on reducing the redundancy of the channel dimension or the redundancy of the spatial dimension, so that the network still has the problem of feature redundancy. Therefore, the present invention integrates the spatial and channel reconstruction convolution (SCConv) into the C2f module to develop the C2f_SCConv module, and fully suppresses the feature redundancy in the feature extraction process by stacking the bottleneck modules, so as to obtain features with more effective information; finally, in order to make more full use of the rich feature information of the high-level channels in the backbone network, the present invention introduces the channel feature fusion module of the deconvolution channel. Involution, CFFI), reduces the information loss in the initial stage of the path aggregation network, can make more full use of semantic information, thereby obtaining high resolution and strong semantic features, and detecting small targets faster and better. Specifically, the system is SD-YOLOv8, which improves YOLOv8, including a fine-grained downsampling module, a C2f SCConv module, and a deconvolution channel feature fusion module.

[0035] The fine-grained downsampling module retains more detail information through linear combinations between channels, and can flexibly adjust the number of channels to control the complexity of the model and avoid overfitting. It uses the SPD-Conv layer for downsampling and reduces the channel dimension through the strideless convolution layer to reduce the amount of computation. Figure 1The present invention proposes a fine-grained downsampling module (CSPDC), which includes a CBS module, an SPD layer and a strideless convolution layer. The CBS module includes a strideless ordinary convolution layer, a BN layer and a SILU activation function. The input first passes through a strideless ordinary convolution layer, and then passes through the BN layer and the SILU activation function in sequence. In this way, the feature map can first be linearly combined between channels to retain more detailed information, and the number of channels can be flexibly adjusted to control the complexity of the model and avoid overfitting. At the same time, the nonlinear expression ability of the feature can be enhanced, which is helpful for extracting richer and more abstract features; then downsampling is performed through the SPD layer, and all information in the channel dimension is retained. Finally, a strideless convolution layer is passed to reduce the channel dimension and reduce the amount of calculation. Therefore, the CSPDC module can better retain the detailed information of the feature map and is more suitable for complex underwater scenes with more small targets.

[0036] The specific process of the CSPDC module is as follows: Assuming that the input feature map size is S×S×C1, it first passes through the CBS module of YOLOv8 to extract the feature information of the feature map in a finer granularity. At this time, the output intermediate feature map size is still S×S×C1. After that, it passes through the SPD layer. For the intermediate feature map X of size S×S×C1, it can be cut to divide it into a series of sub-feature maps. Its calculation formula is shown in formula (1).

[0037] f 0，0 =X[0:S:scale,0:S:scale],...,f scale-1，0 =X[scale-1:S:scale,0:S:scale];

[0038] …

[0039] f 0，scale-1 =X[0: S: scale, scale-1: S: scale],...,

[0040] f scale-1，scale-1 =X[scale-1:S:scale,scale-1:S:scale]#(1)

[0041] where f x，y represents a sub-graph of X, and scale is the downsampling factor. For a given feature graph X, the sub-graph f x，yThe feature map is composed of all entries X(i+j) that are divisible by i+x and i+y. Therefore, each sub-map downsamples the original feature map according to the scale factor. Subsequently, the feature sub-maps are connected according to the channel dimension to obtain a new feature map X'. In the spatial dimension, X' is reduced by a scale factor scale relative to the original feature map; in the channel dimension, X' is increased by a scale factor scale relative to the original feature map 2 Therefore, the dimension of the feature map X is converted from the original S×S×C1 to S / scale×S / scale×scale 2 C l After the SPD layer completes the dimension transformation of the feature map, it passes through a stepless convolution layer, making the dimension of the feature map change from S / scale×S / scale×scale 2 C1 becomes S / scale×S / scale×C2, where C2 <scale 2 C1. By using non-strided convolution, the network can preserve feature information as much as possible while reducing the channel dimension to reduce the amount of computation.

[0042] The C2f_SCConv module integrates the spatial channel reconstruction convolution (SCConv) into the C2f module, and fully suppresses the feature redundancy in the feature extraction process by stacking bottleneck modules, thereby obtaining features with more effective information. Figure 2 The C2f_SCConv module proposed in the present invention is introduced into the backbone network, such as Figure 6 , enhance the feature representation ability of the network. Specifically, SCConv is integrated into the Bottleneck module in the original C2f module, and it is used to replace the second CBS module in Bottleneck to reduce the number of parameters and calculations of the module. The stacking of multiple SCConv modules can effectively reduce the feature redundancy in the deep learning process. At the same time, this article replaces some C2f modules in the Backbone of the original model with this module to fully suppress the feature redundancy in the feature extraction process and obtain features with more effective information.

[0043] The spatial channel reconstruction convolution consists of two modules, namely the spatial reconstruction unit (SpatialReconstruction Unit, SRU) and the channel reconstruction unit (Channel Reconstruction Unit, CRU), which are combined in a sequential manner to replace the standard convolution. Specifically, for each input feature X in the bottleneck residual block, the spatial refined feature X is first obtained by the spatial reconstruction unit operation ω, and then use the channel reconstruction unit operation to obtain the channel refined feature Y. Then add the channel refined feature Y of each bottleneck residual block and the initial input feature X. In this way, the features obtained by repeatedly stacking the bottleneck residual blocks can greatly suppress feature redundancy and reduce model parameters and calculation amount. At the same time, the initial feature X is also added and fused, which can ensure that the original detail information can be retained after the feature is refined. In this way, the fine-grained information of the underwater target can be retained to a large extent, and more effective feature information can be extracted.

[0044] Figure 3 To spatially reconstruct the unit structure map, a separation-reconstruction operation is used. The separation operation aims to separate the feature maps with large information content from the feature maps with small information content, corresponding to the spatial content. The scaling factor in the group normalization GN layer is then used to evaluate the information meaning of different feature maps. Specifically, given an intermediate feature map Where N is the batch axis, C is the channel axis, H and W are the spatial height axis and width axis respectively. It first normalizes the feature X by subtracting the mean μ and dividing by the standard deviation σ, as shown in the following formula (2):

[0045]

[0046] Where μ and σ are the mean and standard deviation of X, ε is a small stabilizing number added to ensure segmentation stability, and γ and β are trainable parameters.

[0047] Figure 4 To reconstruct the unit structure diagram of the channel, the "Split-Transform-Fuse" strategy is adopted. Usually, repeated standard K×K convolutions are used in neural networks to extract features, thereby generating some relatively redundant feature maps in the channel dimension. Assume represents the K×K convolution kernel, Represent the input and output features respectively. The standard convolution can be defined as Y = M k X. Specifically, it uses a channel reconstruction unit instead of a standard convolution, which is implemented through splitting, transformation, and fusion.

[0048] First, the input feature X is multiplied by the information weight W1 and the non-information weight W2 to obtain the feature with high information content. and small amount of information The characteristics Divide into features according to separation rules and Features Divided into characteristics and Features Then the features and Features Add together to get feature X ω1 ,feature and Features Add together to get feature X ω2 , and then connect the cross-reconstructed features X ω1 and X ω2 Get the spatial extraction feature X ω , X ω The upper feature X is obtained by channel compression through 1×1 convolution up and the lower feature X low , upper feature X up After K×KGWC (convolution) and 1×1PWC (convolution), we add them together to get the representative feature map Y1. low and X low The generated features after 1×1 PWC operation are concatenated to obtain the representative feature map Y2. The channel soft attention mechanism is used to generate feature importance vectors β1 and β2, and the spatial channel refined feature Y is obtained by the formula β1Y1+β2Y2.

[0049] The deconvolution channel feature fusion module is added between the backbone and the neck, as Figure 6 , to reduce the information loss in the path aggregation network and further improve the detection accuracy. Figure 5 This is the structure diagram of the deconvolution channel feature fusion module in the present invention, and the formula of its convolution kernel is: The transformation aims to show inverse properties in both spatial and channel domains, where H and W are the height and width of the feature map, K represents the deconvolution kernel size, and G represents the number of groups, where each group shares the same deconvolution kernel. Specifically, for the pixel corresponding to coordinate (i, j) (For brevity, the subscript of C is omitted), a specific deconvolution kernel is designed (where g = 1, 2, ..., G), and is shared between channels. Unlike the convolution kernel, the size of the deconvolution kernel depends on the size of the input feature map, so the kernel size and the input feature map are automatically aligned in the spatial dimension. The advantage of this is that the convolution kernel can adaptively distribute weights at different locations. Therefore, the information contained in the channel dimension of a single pixel is implicitly dispersed to its spatial neighborhood, which helps to obtain rich receptive field information.

[0050] The convolution kernel and deconvolution kernel are 1×1 convolution layers with 1024 channels. The deconvolution channel feature fusion method is specifically described as follows: first, the last feature layer F1 in the backbone network is subjected to a 1×1 convolution layer with 1024 channels to obtain feature F2, and then deconvolution with a kernel size of 1×1 and a channel number of 1024 is performed to obtain feature F3. Finally, the convolved feature F2 and the deconvolved feature F3 are connected to obtain the enhanced feature layer Output.

[0051] A deconvolutional channel feature fusion module is added between the backbone and the neck, which reduces the information loss in the initial stage of the path aggregation network and can make more full use of semantic information.

[0052] Table 1 Comparison of algorithm performance before and after improvement

[0053]

[0054]

[0055] As shown in Table 1, compared with the basic YOLOv8s detection network, the mAP of the present invention for sea cucumbers, sea urchins, scallops, and starfish has increased by 5.2%, 1.87%, 4.26%, and 3.53% respectively, and the precision has increased by 3.19%, 0.73%, 6.88%, and 0.11% respectively. At the same time, the present invention also has certain advantages in the recall rate indicators of various underwater targets, which have increased by 4.45%, 1.01%, 3.71%, and 1.98% respectively. It can be seen from the data that the present invention is more accurate in judging underwater targets, can more comprehensively identify underwater targets, and is more stable in negative sample prediction, reducing the false alarm rate. The improvement of precision and recall makes the algorithm in this chapter more credible and reliable in practical applications.

[0056] Experimental results show that SD-YOLOv8 has better performance when dealing with problems such as low contrast, mutual occlusion, dense distribution, and diverse features.

[0057] Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, if these modifications and variations of the embodiments of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An underwater small target detection system that retains fine-grained information, based on YOLOv8, characterized in that: include: Fine-grained downsampling module, which obtains a series of sub-feature maps through linear combination between channels and downsampling using SPD-Conv layer; C2f_SCConv module, which is introduced into the backbone network of YOLOv8, that is, the spatial channel reconstruction convolution SCConv is integrated into the Bottleneck bottleneck module in the C2f module, replacing the second CBS module in the Bottleneck bottleneck module, and extracting the effective information of the sub-feature map through the stacking of bottleneck modules; The deconvolution channel feature fusion module is located between the YOLOv8 backbone and the neck. It enhances the fusion of sub-feature maps based on their effective information.

2. According to claim 1, the underwater small target detection system retaining fine-grained information is characterized in that: The fine-grained downsampling module includes a cascaded CBS module, an SPD layer and a strideless convolution layer, and the CBS module includes a cascaded strideless convolution layer, a BN layer and a SILU activation function.

3. The underwater small target detection system retaining fine-grained information according to claim 2, characterized in that: The downsampling process of the fine-grained downsampling module is as follows: assuming that the size of the input feature map is S×S×C1, the feature information of the feature map is first extracted by the CBS module, and the size of the output intermediate feature map is S×S×C1; then, the SPD layer is passed to cut the intermediate feature map X of size S×S×C1 to divide it into a series of sub-feature maps, and finally, a stepless convolution layer is passed to make the dimension of the feature map from S / ccale×S / ccale×ccale 2 C1 becomes S / scale×S / ccale×C2, where C2 <scale 2 C1, scale is the downsampling ratio.

4. The underwater small target detection system retaining fine-grained information according to claim 3 is characterized in that: The SPD layer is divided into a series of sub-feature graphs: f x,y =X[x:S:scale,y:S:scale] Among them, f x,y Represents the subgraph of the intermediate feature graph X, 0≤x,y≤, x, y are the rows and columns after downsampling, subgraph f x,y It consists of all entries X(i+j) that are divisible by i+x and j+y, where i and j represent the row index and column index of the feature map X respectively.

5. The underwater small target detection system retaining fine-grained information according to claim 1, characterized in that: The spatial channel reconstruction convolution includes a spatial reconstruction unit and a channel reconstruction unit, which are combined in a sequential manner. For each input feature X, the spatial reconstruction unit is first operated to obtain a spatial refined feature X. ω , and then use the channel reconstruction unit to obtain the channel refined feature Y, and then add each channel refined feature Y and the initial input feature X to obtain the fusion feature.

6. The underwater small target detection system retaining fine-grained information according to claim 5, characterized in that: The output characteristics of the spatial reconstruction unit are: Here, μ and σ are the mean and standard deviation of X, ε is a stable number of settings, and γ and β are trainable parameters.

7. The underwater small target detection system retaining fine-grained information according to claim 5, characterized in that: The channel reconstruction unit adopts a split-transform-fusion strategy, which specifically includes: multiplying the input feature X by the information weight W1 and the non-information weight W2 to obtain the feature with large information content and small amount of information The characteristics Divided into characteristics and Features Divided into characteristics and Features Then the features and Features Add together to get feature X ω1 ,feature and Features Add together to get feature X ω2 , and then connect the cross-reconstructed features X ω1 and X ω2 Get the spatial extraction feature X ω , X ω The upper feature X is obtained by channel compression through 1×1 convolution up and the lower feature X low , upper feature X up After K×K convolution and 1×1 PWC, the representative feature map Y1 is obtained. low and X low The generated features after 1×1 PWC operation are concatenated to obtain the representative feature map Y2. The channel soft attention mechanism is used to generate feature importance vectors β1 and β2, and the spatial channel refined feature Y is obtained by β1Y1+β2Y2.

8. The underwater small target detection system retaining fine-grained information according to claim 1, characterized in that: The deconvolution channel feature fusion module includes a convolution layer, a deconvolution layer and a connection layer, specifically including: the last feature layer F1 in the YOLOv8 backbone network is passed through the convolution layer to obtain the feature F2, and then passed through the kernel deconvolution layer to obtain the feature F3, and finally the feature F2 and the feature F3 are connected through the connection layer to obtain the enhanced feature Output.

9. The underwater small target detection system retaining fine-grained information according to claim 8, characterized in that: The convolutional layer is Where H and W are the height and width of the feature map, K represents the size of the convolutional layer, and G represents the number of groups.

10. The underwater small target detection system retaining fine-grained information according to claim 8, characterized in that: The deconvolution layer is K represents the size of the deconvolution layer, i and j represent pixel coordinates, and g = 1, 2, ..., G.

Images