Small target detection method, device and terminal equipment

Abstract

The present invention is applicable to the field of image recognition technology, and provides a small target detection method, device and terminal equipment. The method comprises: obtaining training image data, annotating the training image data, and converting the annotated training image data into a preset format. An improved YOLOv8 model is constructed, and a small target detection layer is added to the detection head network and the feature enhancement extraction network; a GAM attention mechanism is added to the first C2f module of the trunk feature extraction network to form a C2fG module, which is connected to the small target detection layer; the first Conv module in the Bottleneck module in the last two C2f modules in the trunk feature extraction network is adjusted to a DSConv module to form a C2fD module; each detection head of the detection head network uses a structure connected with Conv and Dyhead. The improved YOLOv8 model is trained using training image data in a preset format; the image data to be detected is input into the YOLOv8 model after training to obtain the category and position of the small target. The present invention can improve the accuracy of small target detection.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a small target detection method, device and terminal equipment. Background Art

[0002] Object detection is an important task in the field of image processing, which aims to accurately locate and identify various types of target objects in images or videos. Object detection mainly includes target positioning and target classification. How to achieve accurate target detection is a research hotspot in the field of image processing. Small targets are objects with a pixel size smaller than 32×32 or an image size smaller than 0.1 times the size of the entire image. Compared with large objects, small targets occupy fewer pixels in the image, their features are not obvious, and they are easily affected by occlusion and noise. They involve complex environments and dynamic backgrounds. Therefore, the effective recognition of small targets is crucial to the performance of the overall system.

[0003] Although traditional target detection algorithms have certain effects on small target detection, their performance in small target detection is still not ideal due to the relatively small size of small targets, unclear features, susceptibility to background interference, complex target background and mutual occlusion of targets. Summary of the invention

[0004] In view of this, the embodiments of the present invention provide a small target detection method, apparatus and terminal device, which can improve the detection accuracy of small targets.

[0005] In order to achieve the above object, the present invention adopts the following technical scheme:

[0006] In a first aspect, an embodiment of the present invention provides a small target detection method, comprising:

[0007] Acquiring training image data, annotating the training image data, and converting the annotated training image data into a preset format;

[0008] An improved YOLOv8 model is constructed. The improved YOLOv8 model includes a backbone feature extraction network, a feature enhancement extraction network and a detection head network. The detection head network includes four detection heads. A small target detection layer is added to the detection head network and the feature enhancement extraction network. A GAM attention mechanism is added to the first C2f module of the backbone feature extraction network to form a C2fG module. The C2fG module is connected to the small target detection layer. The first Conv module in the Bottleneck module in the last two C2f modules in the backbone feature extraction network is adjusted to a DSConv module to form a C2fD module. Each detection head of the detection head network uses a structure connected with Conv and Dyhead to replace the original Detect detection head of YOLOv8.

[0009] Divide the training image data in a preset format into a training data set, a verification data set, and a test data set;

[0010] The improved YOLOv8 model is trained using the training data set and the validation data set, and the improved YOLOv8 model is evaluated using the test data set;

[0011] Input the image data to be detected into the trained YOLOv8 model to obtain the category and position of the small target.

[0012] Based on the first aspect, in some embodiments, the structure of the GAM attention mechanism includes a channel attention module and a spatial attention module; the data input to the GAM attention mechanism is feature F1, and the data output by the GAM attention mechanism is feature F3; the channel attention module is used to retain the three-dimensional information of the input feature, and the spatial attention module is used to perform spatial information fusion on the input feature;

[0013] Feature F1 is divided into two paths, one path is processed by the channel attention module to retain the three-dimensional information of feature F1, and the other path is feature F1 that has not been processed by the channel attention module. The two paths are then processed by a multiplier to obtain feature F2; feature F2 is divided into two paths, one path is processed by the spatial attention module, and the other path is feature F2 that has not been processed by the spatial attention module. The two paths are then processed by a multiplier to obtain feature F3.

[0014] Based on the first aspect, in some embodiments, the channel attention module includes a three-dimensional combination unit, a multi-layer perceptron, an inverse combination unit and a first Sigmoid activation function connected in sequence, the three-dimensional combination unit is used to retain the three-dimensional information of the feature F1; the multi-layer perceptron is used to calculate the weight of each channel, and weight the different channel information of the feature F1 output by the three-dimensional combination unit according to the weight of each channel; the inverse combination unit is used to adjust the number of channels of the feature F1 output by the multi-layer perceptron to the number of channels before the feature F1 enters the channel attention module; the first Sigmoid activation function is used to perform Sigmoid function processing on the feature F1 output by the inverse combination unit;

[0015] The spatial attention module includes a first 7×7 convolutional layer, a second 7×7 convolutional layer and a second Sigmoid activation function connected in sequence, the first 7×7 convolutional layer is used to reduce the number of channels of feature F2 and perform feature extraction, the second 7×7 convolutional layer is used to restore the number of channels of feature F2 to the number of channels when it is input into the spatial attention module and perform feature extraction, and the second Sigmoid activation function is used to perform Sigmoid function processing on the feature F2 output by the second 7×7 convolutional layer.

[0016] Based on the first aspect, in some embodiments, the C2fG module is located between the second convolution module and the third convolution module in the backbone feature extraction network, the small target detection layer includes a first Upsample module, a first Concat module, a second C2f module, a sixth Conv module, a second Concat module, a third C2f module and the first detection head in the detection head network, the first Upsample module, the first Concat module, the second C2f module, the sixth Conv module, the second Concat module and the third C2f module are connected in sequence, and the output end of the third C2f module is connected to the second detection head in the detection head network.

[0017] Based on the first aspect, in some embodiments, the DSConv module uses a convolution kernel K of size 9×9;

[0018] In the x-axis direction, the position of each grid in the convolution kernel K is K i±c , where c = 0, 1, 2, 3, 4 represents the horizontal distance from the center grid; the position K of each grid i±c The selection is an iterative process, starting from the center position K i Initially, the distance of each subsequent position from the center grid depends on the position of the previous grid, K i+1 With K i Compared with the increase of offset Δ = {δ|δ∈[-1,1]},

[0019]

[0020] Δy is the offset Δ in the y-axis direction;

[0021] In the y-axis direction, the position of each grid in the convolution kernel K is K j±c , where c = 0, 1, 2, 3, 4 represents the vertical distance from the center grid; the position of each grid K j±c The selection is an iterative process, starting from the center position K j Initially, the distance of each subsequent position from the center grid depends on the position of the previous grid, K j+1 With K jCompared with the increase of offset Δ = {δ|δ∈[-1,1]},

[0022]

[0023] Δx is the offset Δ in the x-axis direction.

[0024] Based on the first aspect, in some embodiments, since the offset Δ is usually a fraction, but the coordinates are usually in integer form, a bilinear difference is used to calculate the coordinate value of DSConv, and the bilinear difference calculation formula is: K = ∑ K’ B(K,K')×K', K represents K i±c Formula and K j±c The fractional position in the formula, B is the bilinear interpolation kernel, K' enumerates all the integral space positions, B(K,K') = b(K x ,K' x )×b(K y ,K' y ), b is the single linear difference kernel, K x K i±c The x-axis fractional position in the formula, K' x K i±c The integral space position in the formula, K y K j±c The y-axis fractional position in the formula, K' y K j±c The spatial location of the integral in Eq.

[0025] Based on the first aspect, in some embodiments, each detection head of the detection head network includes a Conv module and a DyHead module connected in sequence, an input end of the Conv module is connected to the feature enhancement extraction network, and an output end of the Conv module is connected to the DyHead module;

[0026] The DyHead module includes a scale-aware module, a space-aware module and a task-aware module connected in sequence. The scale-aware module is used to dynamically adjust the importance of features of different scales; the space-aware module is used to obtain offsets and offset weights through deformable convolution, and then aggregate features across layers at the same spatial position according to the offsets and offset weights; the task-aware module is used to dynamically adjust the activation function so that the model can adaptively adjust parameters according to different input features.

[0027] Based on the first aspect, in some embodiments, the scale perception module includes a first average pooling module, a 1×1 convolution module, a first relu function, a hard sigmoid activation function and a dot product module; the first average pooling module, the 1×1 Conv module, the first relu function and the hard sigmoid activation function are connected in sequence, and the first feature input to the scale perception module is processed in sequence by the first average pooling module, the 1×1 convolution module, the first relu function, and the hard sigmoid activation function to obtain a second feature, and then the first feature is multiplied by the second feature through the dot product module to obtain a third feature;

[0028] The spatial perception module includes an index module, a 3×3 convolution module, a sigmoid activation function and a polymerize module. The index module, the 3×3 convolution module and the sigmoid activation function are connected in sequence. The third feature input to the spatial perception module is processed in sequence by the index module, the 3×3 convolution module and the sigmoid activation function to obtain a fourth feature. The third feature and the fourth feature are cross-layer aggregated by the polymerize module to obtain a fifth feature.

[0029] The task perception module includes a second average pooling module, a first fully connected layer, a second relu function, a second fully connected layer, a normalization module, a summation module and a max module. The second average pooling module, the first fully connected layer, the second relu function, the second fully connected layer, the normalization module and the summation module are connected in sequence. The fifth feature input to the task perception module is processed in sequence by the second average pooling module, the first fully connected layer, the second relu function, the second fully connected layer, the normalization module and the summation module to obtain a sixth feature, and then the maximum value of the fifth feature and the sixth feature is selected and output through the max module.

[0030] In a second aspect, an embodiment of the present invention provides a small target detection device, including:

[0031] A data acquisition module, used to acquire training image data, annotate the training image data, and convert the annotated training image data into a preset format;

[0032] A model building module is used to build an improved YOLOv8 model, which includes a backbone feature extraction network, a feature enhancement extraction network and a detection head network. The detection head network includes four detection heads, and a small target detection layer is added to the detection head network and the feature enhancement extraction network; a GAM attention mechanism is added to the first C2f module of the backbone feature extraction network to form a C2fG module, and the C2fG module is connected to the small target detection layer; the first Conv module in the Bottleneck module in the last two C2f modules in the backbone feature extraction network is adjusted to a DSConv module to form a C2fD module; each detection head of the detection head network uses a structure connected with Conv and Dyhead to replace the original Detect detection head of YOLOv8;

[0033] A data partitioning module, used to partition the training image data in a preset format into a training data set, a verification data set, and a test data set;

[0034] A model training module, using the training data set and the validation data set data to train the improved YOLOv8 model, and using the test data set data to evaluate the improved YOLOv8 model;

[0035] The target detection module is used to input the image data to be detected into the trained YOLOv8 model to obtain the category and position of small targets.

[0036] In a third aspect, an embodiment of the present invention provides a terminal device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the small target detection method as described in any one of the first aspects is implemented.

[0037] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0038] In an embodiment of the present invention, after the training image data is annotated and converted into a preset format, the training image data in the preset format is used to train the constructed improved YOLOv8 model. After the training is completed, the image data to be detected is input into the trained YOLOv8 model, and the category and position of the small target can be obtained. The improved YOLOv8 model adds a small target detection layer in the detection head network and the feature enhancement extraction network. The small target detection layer can output predictions from the shallower layers in the network, thereby extracting more detailed features of small targets. The GAM attention mechanism is added to the first C2f module of the backbone feature extraction network to form a C2fG module. The C2fG module can reduce information loss, better retain input information, and improve the model's detection ability for small targets. The first Conv module in the Bottleneck module in the last two C2f modules in the backbone feature extraction network is adjusted to a DSConv module to form a C2fD module, which can enhance the model's detection ability for complex targets. Each detection head of the detection head network uses a structure connected with Conv and Dyhead to replace the original Detect detection head of YOLOv8, which can fuse information from different feature layers, so that the detection network can consider features of multiple scales at the same time, thereby improving the accuracy and robustness of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0040] Figure 1 is a flowchart of a small target detection method provided by an embodiment of the present invention;

[0041] Figure 2 It is the structure of the traditional YOLOv8 model;

[0042] Figure 3 It is the structure of the improved YOLOv8 model provided by an embodiment of the present invention;

[0043] Figure 4 This is the structural diagram of the traditional C2f module;

[0044] Figure 5 is a structural diagram of a C2fG module provided in an embodiment of the present invention;

[0045] Figure 6 is an overall structural diagram of a GAM module provided by an embodiment of the present invention;

[0046] Figure 7 is a structural diagram of a channel attention module provided by an embodiment of the present invention;

[0047] Figure 8 is a structural diagram of a spatial attention module provided by an embodiment of the present invention;

[0048] Fig. 9 It is the traditional YOLOv8 detection layer network structure;

[0049] Fig.10 The detection layer network structure after adding a small target detection layer provided by an embodiment of the present invention;

[0050] Fig.11 It is the traditional Bottleneck structure of YOLOv8 and the improved Bottleneck structure of YOLOv8 in the embodiment of the present application;

[0051] Fig.12 is the coordinate calculation strategy of the DSConv module provided in the embodiment of the present invention;

[0052] Fig.13 is the receptive field of the DSConv module provided in the embodiment of the present invention when processing an image;

[0053] Fig.14 is a schematic diagram of the structure of a DyHead module in a detection head provided by an embodiment of the present invention;

[0054] Fig.15 is a schematic diagram of the structure of a small target detection device provided by an embodiment of the present invention;

[0055] Fig.16 It is a schematic diagram of a terminal device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The present invention will be described more clearly below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the effects of the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0057] In order to make the purpose, technical solutions and advantages of the present invention more clear, specific embodiments will be described below in conjunction with the accompanying drawings.

[0058] Figure 1 A schematic diagram showing a flow chart of a small target detection method provided by an embodiment of the present invention is shown in FIG. Figure 1 , the small target detection method is described in detail as follows:

[0059] Step 101: acquiring training image data, annotating the training image data, and converting the annotated training image data into a preset format.

[0060] In this embodiment, the Visdrone2019 dataset is used as training image data.

[0061] Since each line of the annotation file of the Visdrone2019 dataset is bbox_left, bbox_top, bbox_width, bbox_height, score, object_category, truncation and occlusion, and YOLOv8 can only process annotation files in the format of object-class, x, y, width and height, it is necessary to convert the annotation data format and then perform subsequent processing based on the converted data in the preset format.

[0062] Step 102, construct an improved YOLOv8 model.

[0063] See also Figure 2 The traditional YOLOv8 model consists of three parts: the backbone feature extraction network (Backbone), the feature enhancement extraction network (Neck), and the detection head network (Head). The Backbone part adopts the C2f structure, which is richer than the C3 structure of YOLOv5, and adjusts the number of channels for different scale models. The Neck part refers to the design idea of ​​the efficient layer aggregation network in YOLOv7 to enhance the feature fusion ability of the network. The Head part consists of three standard convolution detectors, replacing the coupling head in the original YOLOv5 with a decoupling head to separate classification and detection.

[0064] The improved YOLOv8 model in the embodiment of the present invention includes a backbone feature extraction network, a feature enhancement extraction network and a detection head network. The detection head network includes four detection heads, a small target detection layer is added to the detection head network and the feature enhancement extraction network, and a GAM attention mechanism is added to the first C2f module of the backbone feature extraction network to form a C2fG module, which is connected to the small target detection layer. The first Conv module in the Bottleneck module in the last two C2f modules in the backbone feature extraction network is adjusted to a DSConv module to form a C2fD module. Each detection head of the detection head network uses a structure connected by Conv and Dyhead to replace the original Detect detection head of YOLOv8.

[0065] See also Figure 3The improvements of the improved YOLOv8 model in the embodiment of the present invention compared with the traditional YOLOv8 model include: 1. Adding a GAM attention mechanism to the first C2f module of the backbone feature extraction network, Figure 3 C2fG in the figure; 2. Add a small target detection layer in the detection head network and the feature enhancement extraction network; 3. Adjust the first Conv in the Bottleneck module in the last two C2f modules in the backbone feature extraction network to a DSConv module to form a C2fD module, as shown in Figure 3 C2fD shown in ; 4. Each detection head of the detection head network uses a structure connected with Conv and Dyhead to replace the original Detect detection head of YOLOv8.

[0066] The above improvements are described in detail below with reference to the accompanying drawings.

[0067] 1. C2fG module

[0068] In order to reduce information loss, better retain input information, and improve the model's ability to detect small targets, the present invention integrates GAM into the first C2f structure in the Backbone part, adds the GAM module after Concat, and the improved C2f forms a C2fG module.

[0069] Specifically, the backbone feature extraction network, feature enhancement extraction network and detection head network of the improved YOLOv8 model are as follows: Figure 3 shown.

[0070] The backbone feature extraction network includes a first Conv, a second Conv, C2fG, a third Conv, a first C2f, a fourth Conv, a first C2fD, a fifth Conv, a second C2fD and SPPF which are connected in sequence.

[0071] The feature enhancement extraction network includes the third Upsample, the fifth Concat, the sixth C2f, the second Upsample, the third Concat, the fourth C2f, the first Upsample, the first Concat, the second C2f, the sixth Conv, the second Concat, the third C2f, the seventh Conv, the fourth Concat, the fifth C2f, the eighth Conv, the sixth Concat and the seventh C2f connected in sequence. The output end of C2FG is also connected to the first Concat, the output end of the first C2f is connected to the third Concat, the output end of the first C2fD is also connected to the fifth Concat, the output end of SPPF is connected to the third Upsample and the sixth Concat, the output end of the sixth C2f is also connected to the fourth Concat, and the output end of the fourth C2f is also connected to the second Concat.

[0072] The detection head network includes four detection heads, each of which includes a Conv and a DyHead connected in sequence. In addition, the output end of the second C2f is also connected to the first detection head, the output end of the third C2f is also connected to the second detection head, the output end of the fifth C2f is also connected to the third detection head, and the output end of the seventh C2f is also connected to the fourth detection head.

[0073] like Figure 4 As shown in the figure, the C2f module in the traditional YOLOv8 model includes Conv, Split, n Bottlenecks, Concat and Conv connected in sequence, that is, the end of the C2f module is the Concat module and the Conv module. Figure 5 ,The C2fG module in the improved YOLOv8 model includes Conv, Split, n Bottleneck, Concat, GAM and Conv connected in sequence, that is, a GAM module is added between the Concat module and the Conv module at the end of the C2fG module.

[0074] like Figure 6 As shown, the GAM module (i.e., the GAM attention mechanism) includes a channel attention module and a spatial attention module. The data input to the GAM attention mechanism is feature F1, and the data output by the GAM attention mechanism is feature F3. The channel attention module is used to retain the three-dimensional information of the input features, and the spatial attention module is used to perform spatial information fusion on the input features.

[0075] Feature F1 is divided into two paths. One path is processed by the channel attention module to retain the three-dimensional information of feature F1, in which the importance of each channel has been dynamically adjusted; the other path is feature F1 that has not been processed by the channel attention module. The two paths are then processed by a multiplier to obtain feature F2. Feature F2 is divided into two paths. One path is processed by the spatial attention module, and the other path is feature F2 that has not been processed by the spatial attention module. The two paths are then processed by a multiplier to obtain feature F3. In this way, in subsequent processing, both features can be used at the same time to capture useful information in the input data.

[0076] See also Figure 7 The channel attention module includes a three-dimensional combination unit, a multi-layer perceptron, an inverse combination unit and a first Sigmoid activation function connected in sequence. The three-dimensional combination unit is used to retain the three-dimensional information of the feature F1. The multi-layer perceptron is used to calculate the weight of each channel, and weight the different channel information of the feature F1 output by the three-dimensional combination unit according to the weight of each channel. The inverse combination unit is used to adjust the number of channels of the feature F1 output by the multi-layer perceptron to the number of channels before the feature F1 enters the channel attention module. The first Sigmoid activation function is used to perform Sigmoid function processing on the feature F1 output by the inverse combination unit.

[0077] See also Figure 8 The spatial attention module includes a first 7×7 convolutional layer, a second 7×7 convolutional layer, and a second Sigmoid activation function connected in sequence. The first 7×7 convolutional layer is used to reduce the number of channels of feature F2 and perform feature extraction. The second 7×7 convolutional layer is used to restore the number of channels of feature F2 to the number of channels when it is input into the spatial attention module and perform feature extraction. The second Sigmoid activation function is used to perform Sigmoid function processing on the feature F2 output by the second 7×7 convolutional layer to obtain feature F3. The spatial attention module can fuse the spatial information of the input features, reduce the discreteness of information, and enhance the global interaction capability.

[0078] The calculation process formula of the channel attention module is The calculation process formula of the spatial attention module is: Among them, F1 is the feature input to the channel attention module, F2 is the feature output by the channel attention module, F3 is the feature output by the spatial attention module, and M C is the channel attention coefficient, M S is the spatial attention coefficient, is element-wise multiplication.

[0079] By introducing the attention mechanism, the neural network can automatically learn and selectively focus on important information in the input, giving different weights to different positions of the feature map to represent different attention levels. Traditional attention mechanisms usually weight the information at different positions, which may cause some information to be ignored or lost during the weighting process, while GAM can globally consider all position information.

[0080] 2. Small target detection layer

[0081] In some embodiments, see Figure 3 The C2fG module is located between the second convolution module and the third convolution module in the backbone feature extraction network, and the small target detection layer includes the first Upsample, the first Concat, the second C2f, the sixth Conv, the second Concat, the third C2f, and the first detection head in the detection head network. The first Upsample, the first Concat, the second C2f, the sixth Conv, the second Concat, and the third C2f are connected in sequence, and the output end of the third C2f module is connected to the second detection head in the detection head network.

[0082] After the input image passes through multiple convolution layers in the Backbone part, feature maps of different scales will be obtained according to different convolution depths. The feature map output by the deep network has low resolution but contains rich semantic information. The feature map output by the shallow network has high resolution but has more detailed information such as position and texture. YOLOv8 has three detection heads by default, which can detect targets of different scales. The corresponding feature map sizes are 80×80, 40×40 and 20×20, which are used to detect targets with sizes larger than 8×8, 16×16 and 32×32 respectively.

[0083] The sample size of small targets is small, while YOLOv8 has a large downsampling. After deep convolution and multiple pooling, the spatial resolution of the feature map will be reduced. As the number of network layers increases, the final feature map is greatly reduced relative to the input image. Therefore, the feature map may lose a lot of important information, and the features of large targets may cover up the information of small targets, resulting in false detection and missed detection of small targets. For images containing many small targets, the traditional YOLOv8 network structure cannot play its best role, and the original three-scale feature maps are difficult to detect small targets. Therefore, taking advantage of the fact that shallow feature maps contain a large amount of contour information and position information, the present invention newly adds a small target detection layer, and outputs predictions from the shallower layers in the network. The corresponding feature map size is 160×160, which is used to detect targets larger than 4×4, thereby extracting more detailed features of small targets. The traditional YOLOv8 detection layer network structure and the detection layer network structure after adding the small target detection layer are shown as follows. Fig. 9 and Fig.10 shown.

[0084] 3. C2fD module

[0085] In some embodiments, the first Conv module in the Bottleneck module in the last two C2f modules in the backbone feature extraction network is adjusted to a DSConv module to form a C2fD module, that is, Figure 3 The first C2fD and the second C2fD are shown in . The DSConv module adaptively adjusts the shape of the convolution kernel according to different parts of the input data by introducing deformation displacement. The adjusted convolution kernel is convolved on the feature map so that the network can better capture the information of the input data.

[0086] The present invention replaces the first Conv in the Bottleneck of the last two C2f modules in the backbone feature extraction network with the DSConv module. DSConv allows the shape of the convolution kernel to be dynamically adjusted as the features of the input data change. At different positions and scales, the network can adaptively change the shape of the convolution kernel. For complex targets, their shapes and scales may vary greatly in the image, so DSConv can better adapt to such changes, thereby enhancing the model's ability to detect complex targets. The traditional YOLOv8 Bottleneck structure and the improved YOLOv8 Bottleneck structure are shown in Figure 2. Fig.11 shown.

[0087] In traditional convolution, the receptive field is fixed, so the YOLOv8 model may find it difficult to capture some detailed features. The essence of the DSConv module is to introduce deformation offsets, which makes it more flexible than traditional convolution. The DSConv module achieves feature extraction of small targets through the following process. Let the center coordinate of the standard two-dimensional convolution kernel be R i =(x i ,y i ), the expansion coefficient is 1, then the 3×3 convolution kernel can be expressed as: R = {(x-1,y-1), (x-1,y), …, (x+1,y+1)}.

[0088] In order to make the convolution kernel adapt to the characteristics of small targets, a deformation offset is introduced, but if the model randomly learns the deformation offset, the receptive field may offset the target. The DSConv module adopts an iterative strategy to match each target to an observable position in turn, which can ensure continuous attention to the target features.

[0089] Specifically, the DSConv module can use a convolution kernel K of size 9×9.

[0090] In the x-axis direction, the position of each grid in the convolution kernel K is K i±c ,

[0091] K i±c The coordinate calculation strategy is as follows Fig.12 As shown in the upper middle part, c = 0, 1, 2, 3, 4 represents the horizontal distance from the center grid. The position K of each grid in the convolution kernel K i±c The selection is an iterative process, starting from the center position K i Initially, the position of each subsequent position from the center grid depends on the position of the previous grid. i+1 With K iCompared with the added offset Δ={δ|δ∈[-1,1]}, the offset needs to be accumulated to ensure that the convolution kernel conforms to the linear morphological structure, and Δy is the offset Δ in the y-axis direction.

[0092] In the y-axis direction, the position of each grid in the convolution kernel K is K j±c ,

[0093]

[0094] K j±c like Fig.12 The lower middle part shows that c = 0, 1, 2, 3, 4 represents the vertical distance from the center grid. The position K of each grid in the convolution kernel K j±c The selection is an iterative process, starting from the center position K j Initially, the distance of each subsequent position from the center grid depends on the position of the previous grid, K j+1 With K j Compared with the previous example, the offset Δ={δ|δ∈[-1,1]} is added, which needs to be accumulated to ensure that the convolution kernel conforms to the linear morphological structure. Δx is the offset Δ in the x-axis direction.

[0095] Since the offset Δ is usually a fraction, but the coordinates are usually in integer form, bilinear interpolation is used to calculate the coordinate values ​​of DSConv. The bilinear interpolation calculation formula is: K = ∑ K’ B(K,K')×K', K represents K i±c Formula and K j±c The fractional position in the formula, B is the bilinear interpolation kernel, K' enumerates all the integral space positions, B(K,K') = b(K x ,K' x )×b(K y ,K' y ), b is the single linear difference kernel, K x K i±c The x-axis fractional position in the formula, K' x K i±c The integral space position in the formula, K y K j±c The y-axis fractional position in the formula, K' y K j±c The spatial location of the integral in Eq.

[0096] After the two-dimensional transformation, DSConv covers a range of 9×9, expanding the receptive field area of ​​the model. The receptive field of DSConv when processing images is as follows: Fig.13 shown.

[0097] 4. The detection head adopts the structure of connecting Conv and Dyhead

[0098] In some embodiments, each detection head of the detection head network includes a Conv module and a DyHead module connected in sequence, the input end of the Conv module is connected to the feature enhancement extraction network, and the output end of the Conv module is connected to the DyHead module.

[0099] The DyHead module includes a scale-aware module, a space-aware module and a task-aware module, which are connected in sequence. The scale-aware module is used to dynamically adjust the importance of features of different scales; the space-aware module is used to obtain offsets and offset weights through deformable convolution, and then aggregate features across layers at the same spatial position according to the offsets and offset weights; the task-aware module is used to dynamically adjust the activation function so that the model can adaptively adjust parameters according to different input features.

[0100] Improving the performance of the detection head is a key issue in current target detection. DyHead adopts a multi-head architecture, that is, the target detection model contains multiple detection heads of different designs. These heads have different structures, complexities and parameter settings to adapt to target detection tasks of different types and difficulties. The DyHead module can dynamically select the appropriate detection head for target detection according to the performance of the target at different scales. It can effectively deal with the situation where the target appears at different scales and improve the robustness and accuracy of target detection. The dynamic detection head module superimposes scale-aware attention, space-aware attention and task-aware attention in sequence into a dynamic detection head. The structure of the DyHead module is as follows: Fig.14 In order to achieve scale perception, spatial perception, and task perception in the DyHead module, the feature pyramid output by the backbone network needs to be scaled to the same scale. The scaled feature pyramid can be represented as a four-dimensional tensor F∈R L×H×W×C , where L represents the number of feature layers, H and W represent the height and width of the feature, and C represents the number of channels. Further, using S to represent the width and height of the feature, the four-dimensional tensor will be reshaped into a three-dimensional tensor F∈R L×S×C , where L, S, and C focus on scale perception, space perception, and channel perception, respectively, thus obtaining a scale perception module, a space perception module, and a channel perception module.

[0101] Exemplarily, the scale perception module may include a first average pooling module, a 1×1 convolution module, a first relu function, a hard sigmoid activation function, and a dot product module. The first average pooling module, the 1×1 Conv module, the first relu function, and the hard sigmoid activation function are connected in sequence, and the first feature input to the scale perception module is processed in sequence by the first average pooling module, the 1×1 convolution module, the first relu function, and the hard sigmoid activation function to obtain the second feature, and then the first feature is multiplied by the second feature through the dot product module to obtain the third feature. The scale perception module realizes the dynamic adjustment of the importance of features of different scales. First, the feature map is average pooled, and then the features are extracted through 1×1 convolution and the first relu function. Finally, the calculation result is multiplied by the input feature through the hard sigmoid activation function to obtain the output result.

[0102] The spatial perception module may include an index module, a 3×3 convolution module, a sigmoid activation function, and a polymerize module. The index module, the 3×3 convolution module, and the sigmoid activation function are connected in sequence. The third feature input to the spatial perception module is processed in sequence by the index module, the 3×3 convolution module, and the sigmoid activation function to obtain the fourth feature. The third feature and the fourth feature are then aggregated across layers through the polymerize module to obtain the fifth feature. In the spatial perception module, the offset and the offset weight are mainly obtained through deformable convolution, and then the features are aggregated across layers at the same spatial position. First, the feature map is obtained through the index module, and then the feature is processed through the 3×3 convolution module to obtain the feature offset and the offset weight. The result is obtained through the sigmoid activation function, and finally the obtained features are aggregated across layers.

[0103] The task perception module may include a second average pooling module, a first fully connected layer, a second relu function, a second fully connected layer, a normalization module, a summation module, and a max module. The second average pooling module, the first fully connected layer, the second relu function, the second fully connected layer, the normalization module, and the summation module are connected in sequence. The fifth feature input to the task perception module is processed in sequence by the second average pooling module, the first fully connected layer, the second relu function, the second fully connected layer, the normalization module, and the summation module to obtain the sixth feature, and then the maximum value of the fifth feature and the sixth feature is selected and output through the max module. The task perception module mainly enables the model to adjust parameters adaptively according to different input features by dynamically adjusting the activation function. First, the input feature is average pooled, then passes through a fully connected layer and a second relu activation function, and then passes through a fully connected layer and normalization. Parameters will be obtained in this process. After the parameters are normalized to [-1,1], they are summed with the default value [1,0,0,0]. The summed value is calculated with the input feature to obtain the maximum value to obtain the final result.

[0104] The following is the derivation of DyHead implementation, given a specific tensor F∈R L×S×C , the formula for applying self-attention is W(F)=π(F)×F. Where π(F) is an attention function, which is implemented by a fully connected layer. Decomposing the attention function into three consecutive attentions, each focusing on a different direction, we get W(F)=π C (π S (π L (F)×F)×F)×F.

[0105] Among them, π L , π S and π C are the attention functions of scale perception, space perception and channel perception respectively. The scale perception is expressed as in, is a linear function approximated by a 1×1 convolutional layer, and σ(x)=max(0,min(1,(x+1) / 2)) is a hard sigmoid function.

[0106] Spatial perception is expressed as Where H is the number of sparse sampling positions; used to separate the variables, p h +Δp h Denotes the spatial offset of self-learning, using Δp h To focus on different areas. Δm h Yes h are important scalars for self-learning, both of which are learned from the input features of the intermediate levels of F.

[0107] The formula for task perception is π C (F)×F=max(α 1 (F)×F C +β 1 (F),α 2 (F)×F C +β 2 (F)),

[0108] F C is the feature slice of the Cth channel, [α 1 , β 1 , α 2 , β 2 ] T =θ(·) is a hyperfunction for learning to control the activation threshold. The implementation process of θ(·) is as follows: first, average pooling is performed on the L×S dimension to reduce the dimension, then it passes through a fully connected layer and a relu activation function, then passes through a fully connected layer and a normalization layer, and finally the output parameters are normalized to [-1,1]. Before applying the DyHead detection head, the present invention needs to add standard convolution to scale the feature pyramid output by the feature extraction network to a consistent scale.

[0109] Step 103: divide the training image data in a preset format into a training data set, a verification data set, and a test data set.

[0110] In this step, the training image data in the preset format obtained in step 101 may be divided into a training data set, a verification data set, and a test data set according to a preset ratio.

[0111] Step 104: Use the training data set and the validation data set to train the improved YOLOv8 model, and use the test data set to evaluate the improved YOLOv8 model.

[0112] Step 105: input the image data to be detected into the trained YOLOv8 model to obtain the category and position of the small target.

[0113] Specifically, after the improved YOLOv8 model is trained, the image data to be detected is input into the improved YOLOv8 model to obtain the category and position of the small target contained in the image data to be detected.

[0114] It should be noted that the execution order between step 101 and step 102 is not limited to Figure 1 In the manner shown, step 101 may be executed before step 102, may be executed after step 102, or may be executed synchronously with step 102, which is not limited to this.

[0115] The improved YOLOv8 model in the present invention can be applied to the detection of small targets in complex scenarios, for example, to assist autonomous driving systems in identifying small obstacles on the road, or to detect small vehicles, bicycles or pedestrians in traffic monitoring systems for traffic flow analysis and accident prevention.

[0116] The above-mentioned small target detection method, after annotating the training image data and converting it into a preset format, trains the constructed improved YOLOv8 model with the training image data in the preset format, and after the training is completed, inputs the image data to be detected into the trained YOLOv8 model, so that the category and position of the small target can be obtained. The improved YOLOv8 model adds a small target detection layer in the detection head network and the feature enhancement extraction network. The small target detection layer can output predictions from the shallower layers in the network, thereby extracting more detailed features of the small target; the GAM attention mechanism is added to the first C2f module of the backbone feature extraction network to form a C2fG module, which can reduce information loss, better retain input information, and improve the model's detection ability for small targets; the first Conv module in the Bottleneck module in the last two C2f modules in the backbone feature extraction network is adjusted to a DSConv module to form a C2fD module, which enhances the model's detection ability for complex targets. Each detection head of the detection head network uses a structure connected by Conv and Dyhead to replace the original Detect detection head of YOLOv8, which can fuse information from different feature layers, so that the detection network can consider features of multiple scales at the same time, thereby improving the accuracy and robustness of detection.

[0117] The following is a simulation description of the small target detection method according to the embodiment of the present invention.

[0118] In order to ensure the authenticity and accuracy of the experiment, the training and prediction experiments of all small target models in the present invention are carried out under the same operating environment of the same computer, and the specific hardware and software environment is shown in Table 1.

[0119] Table 1 Experimental software and hardware environment

[0120]

[0121] (1) Data collection and preprocessing

[0122] (1.1) Data collection

[0123] The VisDrone2019 dataset was collected by the AISKYEYE team of the Machine Learning and Data Mining Laboratory of Tianjin University. The dataset contains images of 10 categories, namely pedestrian, people, car, van, bus, truck, motor, bicycle, awning-tricycl and tricycle.

[0124] (1.2) Data preprocessing;

[0125] Write a conversion script to read each line of the VisDrone2019 dataset annotation file and convert the annotation format to the format required by YOLO. In this dataset, 6471 images are used for training, 548 images are used for verification, and 1610 images are used for testing.

[0126] (2) Model experiment

[0127] (2.1) Evaluation indicators

[0128] In the evaluation index, the present invention uses mean average precision (mAP) and parameters as the evaluation index of the experiment. To calculate the mAP of the target, we must first calculate the accuracy (Precision, P), recall (Recall, R), and average precision (average precision, AP).

[0129] P refers to the ratio of correctly detected targets to all detected targets, and the calculation formula is P = TP / (TP+FP). TP (True Positives) is the number of correctly classified as positive examples, and FP (False Positives) is the number of incorrectly classified as positive examples. The higher the accuracy, the fewer false detections.

[0130] R refers to the ratio of correctly detected targets to all existing targets, and is calculated as R = TP / (TP+FN), where FN (False Negatives) is the number of predicted negative examples that are actually positive examples. The higher the recall rate, the fewer missed detections.

[0131] AP is the area enclosed by the precision and recall curve (PR curve) of each category and the coordinate axis. The calculation formula is:

[0132] mAP is the average AP value of all categories. Where n represents the number of categories of detection targets in the dataset, and the calculation formula is: mAP@0.5 represents the accurate prediction rate when the IoU (overlap) between the predicted box and the true box is greater than or equal to 0.5, and mAP@0.5:0.95 represents the case where the IoU between the predicted box and the true box is between 0.5 and 0.95.

[0133] (2.2) Experimental results analysis

[0134] In order to prove the superiority and effectiveness of the improved YOLOv8 model for small target detection, the present invention sets up 5 groups of ablation experiments based on the YOLOv8 model: the first group is the traditional YOLOv8 model, the second group is the traditional YOLOv8 model with the C2fG module added, the third group is the traditional YOLOv8 model with the C2fG module and a small target detection layer added, the fourth group is the traditional YOLOv8 model with the C2fG module, the small target detection layer and the C2fD module added, and the fifth group is the traditional YOLOv8 model with the C2fG module, the small target detection layer, the C2fD module and the detection head connected with Conv and DyHead added. The experimental results are shown in Table 2.

[0135] Table 2 Ablation experiment results

[0136]

[0137] The experimental results show that each added module is effective. The improved YOLOv8 model shows better performance than the traditional YOLOv8 model.

[0138] (2.3) Comparative experiments of different models

[0139] The improved YOLOv8 model of the present invention is compared with the current mainstream algorithms SSD, Faster R-CNN, YOLOV3, YOLOv4, YOLOv5, YOLOv7-tiny and traditional YOLOv8, and the results are shown in Table 3. It can be seen from Table 3 that the improved YOLOv8 model of the present invention completely exceeds other models in terms of detection accuracy.

[0140] Table 3 Comparative experiments of different models

[0141]

[0142] The present invention proposes an improved YOLOv8 model with higher accuracy: by integrating the GAM attention mechanism into the C2f structure in Backbone, information loss is reduced; a small target detection layer is added to extract a large amount of small target information from the shallow network; DSConv replaces the convolution of the Bottleneck structure in the C2f structure in Backbone to further enhance the network's detection ability for tiny targets; and a detection head structure connected with Conv and DyHead is used to dynamically select the detection head for different targets in different situations.

[0143] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0144] Corresponding to the small target detection method described in the above embodiment, Fig.15 A structural block diagram of a small target detection device provided by an embodiment of the present invention is shown. For ease of description, only the parts related to the embodiment of the present invention are shown.

[0145] See also Fig.15 An embodiment of the present invention provides a small target detection device, including a data acquisition module 201, a model construction module 202, a data division module 203, a model training module 204 and a target detection module 205.

[0146] The data acquisition module 201 is used to acquire training image data, annotate the training image data, and convert the annotated training image data into a preset format.

[0147] The model construction module 202 is used to construct an improved YOLOv8 model, which includes a backbone feature extraction network, a feature enhancement extraction network and a detection head network. The detection head network includes four detection heads, a small target detection layer is added to the detection head network and the feature enhancement extraction network, and a GAM attention mechanism is added to the first C2f module of the backbone feature extraction network to form a C2fG module, which is connected to the small target detection layer; the first Conv module in the Bottleneck module in the last two C2f modules in the backbone feature extraction network is adjusted to a DSConv module to form a C2fD module; each detection head of the detection head network uses a structure connected with Conv and Dyhead to replace the original Detect detection head of YOLOv8.

[0148] The data division module 203 is used to divide the training image data in a preset format into a training data set, a verification data set and a test data set.

[0149] The model training module 204 is used to train the improved YOLOv8 model using the training data set and the validation data set, and to evaluate the improved YOLOv8 model using the test data set.

[0150] The target detection module 205 is used to input the image data to be detected into the trained YOLOv8 model to obtain the category and position of the small target.

[0151] Optionally, the structure of the GAM attention mechanism includes a channel attention module and a spatial attention module; the data input to the GAM attention mechanism is feature F1, and the data output by the GAM attention mechanism is feature F3; the channel attention module is used to retain the three-dimensional information of the input features, and the spatial attention module is used to perform spatial information fusion on the input features;

[0152] Feature F1 is divided into two paths, one path is processed by the channel attention module to retain the three-dimensional information of feature F1, and the other path is feature F1 that has not been processed by the channel attention module. The two paths are then processed by a multiplier to obtain feature F2; feature F2 is divided into two paths, one path is processed by the spatial attention module, and the other path is feature F2 that has not been processed by the spatial attention module. The two paths are then processed by a multiplier to obtain feature F3.

[0153] Optionally, the channel attention module includes a three-dimensional combination unit, a multi-layer perceptron, an inverse combination unit and a first Sigmoid activation function connected in sequence, the three-dimensional combination unit is used to retain the three-dimensional information of the feature F1, the multi-layer perceptron is used to calculate the weight of each channel, and weight the different channel information of the feature F1 output by the three-dimensional combination unit according to the weight of each channel; the inverse combination unit is used to adjust the number of channels of the feature F1 output by the multi-layer perceptron to the number of channels before the feature F1 enters the channel attention module; the first Sigmoid activation function is used to perform Sigmoid function processing on the feature F1 output by the inverse combination unit;

[0154] The spatial attention module includes a first 7×7 convolutional layer, a second 7×7 convolutional layer and a second Sigmoid activation function connected in sequence, the first 7×7 convolutional layer is used to reduce the number of channels of feature F2 and perform feature extraction, the second 7×7 convolutional layer is used to restore the number of channels of feature F2 to the number of channels when it is input into the spatial attention module and perform feature extraction, and the second Sigmoid activation function is used to perform Sigmoid function processing on the feature F2 output by the second 7×7 convolutional layer.

[0155] Optionally, the C2fG module is located between the second convolution module and the third convolution module in the backbone feature extraction network, the small target detection layer includes a first Upsample module, a first Concat module, a second C2f module, a sixth Conv module, a second Concat module, a third C2f module and the first detection head in the detection head network, the first Upsample module, the first Concat module, the second C2f module, the sixth Conv module, the second Concat module and the third C2f module are connected in sequence, and the output end of the third C2f module is connected to the second detection head in the detection head network.

[0156] Optionally, the DSConv module uses a convolution kernel K of size 9×9;

[0157] In the x-axis direction, the position of each grid in the convolution kernel K is K i±c , where c = 0, 1, 2, 3, 4 represents the horizontal distance from the center grid. The position K of each grid i±c The selection is an iterative process, starting from the center position K i Initially, the distance of each subsequent position from the center grid depends on the position of the previous grid, K i+1 With K i Compared with the increase of offset Δ = {δ|δ∈[-1,1]},

[0158]

[0159] Δy is the offset Δ in the y-axis direction;

[0160] In the y-axis direction, the position of each grid in the convolution kernel K is K j±c , where c = 0, 1, 2, 3, 4 represents the vertical distance from the center grid. The position K of each grid j±c The selection is an iterative process, starting from the center position K j Initially, the distance of each subsequent position from the center grid depends on the position of the previous grid, K j+1 With K j Compared with the increase of offset Δ = {δ|δ∈[-1,1]},

[0161]

[0162] Δx is the offset Δ in the x-axis direction.

[0163] Optionally, since the offset Δ is usually a fraction, but the coordinates are usually in integer form, bilinear interpolation is used to calculate the coordinate values ​​of DSConv. The bilinear interpolation calculation formula is: K = ∑ K’ B(K,K')×K', K represents Ki±c Formula and K j±c The fractional position in the formula, B is the bilinear interpolation kernel, K' enumerates all the integral space positions, B(K,K') = b(K x ,K' x )×b(K y ,K' y ), b is the single linear difference kernel, K x K i±c The x-axis fractional position in the formula, K' x K i±c The integral space position in the formula, K y K j±c The y-axis fractional position in the formula, K' y K j±c The spatial location of the integral in Eq.

[0164] Optionally, each detection head of the detection head network includes a Conv module and a DyHead module connected in sequence, an input end of the Conv module is connected to the feature enhancement extraction network, and an output end of the Conv module is connected to the DyHead module;

[0165] The DyHead module includes a scale-aware module, a space-aware module and a task-aware module connected in sequence. The scale-aware module is used to dynamically adjust the importance of features of different scales; the space-aware module is used to obtain offsets and offset weights through deformable convolution, and then aggregate features across layers at the same spatial position according to the offsets and offset weights; the task-aware module is used to dynamically adjust the activation function so that the model can adaptively adjust parameters according to different input features.

[0166] Optionally, the scale perception module includes a first average pooling module, a 1×1 convolution module, a first relu function, a hard sigmoid activation function and a dot product module; the first average pooling module, the 1×1 Conv module, the first relu function and the hard sigmoid activation function are connected in sequence, and the first feature input to the scale perception module is processed in sequence by the first average pooling module, the 1×1 convolution module, the first relu function, and the hard sigmoid activation function to obtain a second feature, and then the first feature is multiplied by the second feature through the dot product module to obtain a third feature;

[0167] The spatial perception module includes an index module, a 3×3 convolution module, a sigmoid activation function and a polymerize module. The index module, the 3×3 convolution module and the sigmoid activation function are connected in sequence. The third feature input to the spatial perception module is processed in sequence by the index module, the 3×3 convolution module and the sigmoid activation function to obtain a fourth feature. The third feature and the fourth feature are cross-layer aggregated by the polymerize module to obtain a fifth feature.

[0168] The task perception module includes a second average pooling module, a first fully connected layer, a second relu function, a second fully connected layer, a normalization module, a summation module and a max module. The second average pooling module, the first fully connected layer, the second relu function, the second fully connected layer, the normalization module and the summation module are connected in sequence. The fifth feature input to the task perception module is processed in sequence by the second average pooling module, the first fully connected layer, the second relu function, the second fully connected layer, the normalization module and the summation module to obtain a sixth feature, and then the maximum value of the fifth feature and the sixth feature is selected and output through the max module.

[0169] Fig.16 Schematic diagram of a terminal device provided by an embodiment of the present invention. Fig.16 As shown, the terminal device 300 of this embodiment includes a processor 310 and a memory 320, wherein the memory 320 stores a computer program that can be run on the processor 310, such as a small target detection program. When the processor 310 executes the computer program, the steps in the above-mentioned small target detection method embodiment are implemented, such as Figure 1 Alternatively, when the processor 310 executes the computer program, the functions of each module / unit in the above-mentioned device embodiments are realized, for example, Fig.15 The functions of the data acquisition module 201 to the target detection module 205 are shown.

[0170] The terminal device 300 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor 310 and a memory 320. Those skilled in the art will appreciate that Fig.16 It is only an example of the terminal device 300 and does not constitute a limitation of the terminal device 300. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.

[0171] The processor 310 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0172] The memory 320 may be an internal storage unit of the terminal device 300, such as a hard disk or memory of the terminal device 300. The memory 320 may also be an external storage device of the terminal device 300, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 300. Further, the memory 320 may also include both an internal storage unit of the terminal device 300 and an external storage device. The memory 320 is used to store the computer program and other programs and data required by the terminal device. The memory 320 may also be used to temporarily store data that has been output or is to be output.

[0173] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0174] If the integrated module is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0175] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A small target detection method, characterized in that: include: Acquiring training image data, annotating the training image data, and converting the annotated training image data into a preset format; An improved YOLOv8 model is constructed. The improved YOLOv8 model includes a backbone feature extraction network, a feature enhancement extraction network and a detection head network. The detection head network includes four detection heads. A small target detection layer is added to the detection head network and the feature enhancement extraction network. A GAM attention mechanism is added to the first C2f module of the backbone feature extraction network to form a C2fG module, which is connected to the small target detection layer. The first Conv module in the Bottleneck module in the last two C2f modules in the backbone feature extraction network is adjusted to a DSConv module to form a C2fD module. Each detection head of the detection head network replaces the original Detect detection head of YOLOv8 with a structure connected with Conv and Dyhead. Divide the training image data in a preset format into a training data set, a verification data set, and a test data set; The improved YOLOv8 model is trained using the training data set and the validation data set, and the improved YOLOv8 model is evaluated using the test data set; Input the image data to be detected into the trained YOLOv8 model to obtain the category and position of the small target; Among them, the backbone feature extraction network includes the first Conv, the second Conv, C2fG, the third Conv, the first C2f, the fourth Conv, the first C2fD, the fifth Conv, the second C2fD and SPPF connected in sequence; the C2fG module includes Conv, Split, n Bottlenecks, Concat, GAM and Conv connected in sequence, and a GAM module is added between the Concat module and the Conv module at the end of the C2fG module; The feature enhancement extraction network includes a third Upsample, a fifth Concat, a sixth C2f, a second Upsample, a third Concat, a fourth C2f, a first Upsample, a first Concat, a second C2f, a sixth Conv, a second Concat, a third C2f, a seventh Conv, a fourth Concat, a fifth C2f, an eighth Conv, a sixth Concat and a seventh C2f connected in sequence; the output end of the first C2f is connected to the third Concat, the output end of the first C2fD is also connected to the fifth Concat, the output end of the SPPF is connected to the third Upsample and the sixth Concat, the output end of the sixth C2f is also connected to the fourth Concat, and the output end of the fourth C2f is also connected to the second Concat; The detection head network includes four detection heads, each detection head includes a Conv and a DyHead connected in sequence; the output end of the second C2f is also connected to the first detection head, the output end of the third C2f is also connected to the second detection head, the output end of the fifth C2f is also connected to the third detection head, and the output end of the seventh C2f is also connected to the fourth detection head; The small target detection layer includes the first Upsample module, the first Concat module, the second C2f module, the sixth Conv module, the second Concat module, the third C2f module and the first detection head in the detection head network, which are connected in sequence in the feature enhancement extraction network, and the output end of C2fG is also connected to the first Concat.

2. The small target detection method according to claim 1, characterized in that: The structure of the GAM attention mechanism includes a channel attention module and a spatial attention module; the data input to the GAM attention mechanism is feature F1, and the data output by the GAM attention mechanism is feature F3; the channel attention module is used to retain the three-dimensional information of the input features, and the spatial attention module is used to perform spatial information fusion on the input features; Feature F1 is divided into two paths, one path is processed by the channel attention module to retain the three-dimensional information of feature F1, and the other path is feature F1 that has not been processed by the channel attention module. The two paths are then processed by a multiplier to obtain feature F2; feature F2 is divided into two paths, one path is processed by the spatial attention module, and the other path is feature F2 that has not been processed by the spatial attention module. The two paths are then processed by a multiplier to obtain feature F3.

3. The small target detection method according to claim 2, characterized in that: The channel attention module includes a three-dimensional combination unit, a multi-layer perceptron, an inverse combination unit and a first Sigmoid activation function connected in sequence, wherein the three-dimensional combination unit is used to retain the three-dimensional information of the feature F1; the multi-layer perceptron is used to calculate the weight of each channel, and weight the different channel information of the feature F1 output by the three-dimensional combination unit according to the weight of each channel; the inverse combination unit is used to adjust the number of channels of the feature F1 output by the multi-layer perceptron to the number of channels before the feature F1 enters the channel attention module; the first Sigmoid activation function is used to perform Sigmoid function processing on the feature F1 output by the inverse combination unit; The spatial attention module includes a first 7×7 convolutional layer, a second 7×7 convolutional layer and a second Sigmoid activation function connected in sequence, the first 7×7 convolutional layer is used to reduce the number of channels of feature F2 and perform feature extraction, the second 7×7 convolutional layer is used to restore the number of channels of feature F2 to the number of channels when it is input into the spatial attention module and perform feature extraction, and the second Sigmoid activation function is used to perform Sigmoid function processing on the feature F2 output by the second 7×7 convolutional layer.

4. The small target detection method according to claim 1, characterized in that: The DSConv module uses a convolution kernel K of size 9×9; In the x-axis direction, the position of each grid in the convolution kernel K is K i±c , where c = 0, 1, 2, 3, 4 represents the horizontal distance of the grid from the center grid; the position K of each grid i±c The selection is an iterative process, starting from the center position K i Initially, the distance of each subsequent position from the center grid depends on the position of the previous grid, K i+1 With K i Compared with the increase of offset Δ = {δ|δ∈[-1,1]}, Δy is the offset Δ in the y-axis direction; In the y-axis direction, the position of each grid in the convolution kernel K is K j±c , where c = 0, 1, 2, 3, 4 represents the vertical distance of the grid from the center grid; the position K of each grid j±c The selection is an iterative process, starting from the center position K j Initially, the distance of each subsequent position from the center grid depends on the position of the previous grid, K j+1 With K j Compared with the increase of offset Δ = {δ|δ∈[-1,1]}, Δx is the offset Δ in the x-axis direction.

5. The small target detection method according to claim 4, characterized in that: Use bilinear interpolation to calculate the coordinate value of DSConv. The formula for bilinear interpolation is: K = ∑ K’ B(K,K')×K', K represents K i±c Formula and K j±c The fractional position in the formula, B is the bilinear interpolation kernel, K' enumerates all the integral space positions, B(K,K') = b(K x ,K' x )×b(K y ,K' y ), b is the single linear difference kernel, K x K i±c The x-axis fractional position in the formula, K' x K i±c The integral space position in the formula, K y K j±c The y-axis fractional position in the formula, K' y K j±c The spatial location of the integral in Eq.

6. The small target detection method according to claim 1, characterized in that: Each detection head of the detection head network includes a Conv module and a DyHead module connected in sequence, the input end of the Conv module is connected to the feature enhancement extraction network, and the output end of the Conv module is connected to the DyHead module; The DyHead module includes a scale-aware module, a space-aware module and a task-aware module connected in sequence. The scale-aware module is used to dynamically adjust the importance of features of different scales; the space-aware module is used to obtain offsets and offset weights through deformable convolution, and then aggregate features across layers at the same spatial position according to the offsets and offset weights; the task-aware module is used to dynamically adjust the activation function so that the model can adaptively adjust parameters according to different input features.

7. The small target detection method according to claim 6, characterized in that: The scale perception module includes a first average pooling module, a 1×1 convolution module, a first relu function, a hard sigmoid activation function and a dot product module; the first average pooling module, the 1×1 Conv module, the first relu function and the hard sigmoid activation function are connected in sequence, and the first feature input to the scale perception module is processed in sequence by the first average pooling module, the 1×1 convolution module, the first relu function and the hard sigmoid activation function to obtain a second feature, and then the first feature is multiplied by the second feature by the dot product module to obtain a third feature; The spatial perception module includes an index module, a 3×3 convolution module, a sigmoid activation function and a polymerize module. The index module, the 3×3 convolution module and the sigmoid activation function are connected in sequence. The third feature input to the spatial perception module is processed in sequence by the index module, the 3×3 convolution module and the sigmoid activation function to obtain a fourth feature. The third feature and the fourth feature are cross-layer aggregated by the polymerize module to obtain a fifth feature. The task perception module includes a second average pooling module, a first fully connected layer, a second relu function, a second fully connected layer, a normalization module, a summation module and a max module. The second average pooling module, the first fully connected layer, the second relu function, the second fully connected layer, the normalization module and the summation module are connected in sequence. The fifth feature input to the task perception module is processed in sequence by the second average pooling module, the first fully connected layer, the second relu function, the second fully connected layer, the normalization module and the summation module to obtain a sixth feature, and then the maximum value of the fifth feature and the sixth feature is selected and output through the max module.

8. A small target detection device, characterized in that: include: A data acquisition module, used to acquire training image data, annotate the training image data, and convert the annotated training image data into a preset format; A model building module is used to build an improved YOLOv8 model, which includes a backbone feature extraction network, a feature enhancement extraction network and a detection head network. The detection head network includes four detection heads, a small target detection layer is added to the detection head network and the feature enhancement extraction network, a GAM attention mechanism is added to the first C2f module of the backbone feature extraction network to form a C2fG module, and the C2fG module is connected to the small target detection layer; the first Conv module in the Bottleneck module in the last two C2f modules in the backbone feature extraction network is adjusted to a DSConv module to form a C2fD module; each detection head of the detection head network uses a structure connected with Conv and Dyhead to replace the original Detect detection head of YOLOv8; A data partitioning module, used to partition the training image data in a preset format into a training data set, a verification data set, and a test data set; A model training module, using the training data set and the validation data set to train the improved YOLOv8 model; using the test data set to evaluate the improved YOLOv8 model; The target detection module is used to input the image data to be detected into the trained YOLOv8 model to obtain the category and position of small targets; Among them, the backbone feature extraction network includes the first Conv, the second Conv, C2fG, the third Conv, the first C2f, the fourth Conv, the first C2fD, the fifth Conv, the second C2fD and SPPF connected in sequence; the C2fG module includes Conv, Split, n Bottlenecks, Concat, GAM and Conv connected in sequence, and a GAM module is added between the Concat module and the Conv module at the end of the C2fG module; The feature enhancement extraction network includes a third Upsample, a fifth Concat, a sixth C2f, a second Upsample, a third Concat, a fourth C2f, a first Upsample, a first Concat, a second C2f, a sixth Conv, a second Concat, a third C2f, a seventh Conv, a fourth Concat, a fifth C2f, an eighth Conv, a sixth Concat and a seventh C2f connected in sequence; the output end of the first C2f is connected to the third Concat, the output end of the first C2fD is also connected to the fifth Concat, the output end of the SPPF is connected to the third Upsample and the sixth Concat, the output end of the sixth C2f is also connected to the fourth Concat, and the output end of the fourth C2f is also connected to the second Concat; The detection head network includes four detection heads, each detection head includes a Conv and a DyHead connected in sequence; the output end of the second C2f is also connected to the first detection head, the output end of the third C2f is also connected to the second detection head, the output end of the fifth C2f is also connected to the third detection head, and the output end of the seventh C2f is also connected to the fourth detection head; The small target detection layer includes the first Upsample module, the first Concat module, the second C2f module, the sixth Conv module, the second Concat module, the third C2f module and the first detection head in the detection head network, which are connected in sequence in the feature enhancement extraction network, and the output end of C2fG is also connected to the first Concat.

9. A terminal device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the steps of the small target detection method as described in any one of claims 1 to 7 are implemented.

Images