A lightweight model target detection method based on YOLOv5s improvement

Abstract

The application discloses an improved lightweight model target detection method based on YOLOv5s, wherein the global attention mechanism is integrated in the improved lightweight model based on YOLOv5s, the cross-dimension feature information interaction ability between the channel and spatial attention mechanisms is strengthened, the problem that target features are lost due to different dimensions is reduced, and the feature extraction of the target is improved; meanwhile, the adaptive spatial feature fusion algorithm is combined at the detection head, the spatial weight of each scale during fusion is adaptively adjusted, the spatial scale inconsistency is inhibited, the target features obtain more accurate network processing proportion, the multi-scale feature fusion capability of the target is improved, and the missing detection rate and the false detection rate of the small and medium targets in the remote sensing image are reduced.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and image processing, specifically a lightweight model-based object detection method based on improved YOLOv5s. Background Technology

[0002] Remote sensing image target detection is a crucial component of target detection tasks. With the rapid development of satellite technology, the importance of remote sensing satellites has become increasingly prominent, finding widespread application in numerous fields such as natural resource monitoring, urban development, land use, and satellite navigation. Unlike other radar vehicles, remote sensing satellites are not affected by weather conditions. They primarily capture images of target areas from orbiting satellites, offering advantages such as all-weather, all-time, and all-territorial coverage. However, the wide coverage and high resolution of images captured by orbiting satellites, along with the diverse types and scales of targets, mean that small targets occupy very few pixels in the images. Research shows that small targets with pixels smaller than 40*40 are likely to be lost in complex backgrounds when extracting features. Therefore, improving the detection accuracy of targets in remote sensing images is of great research significance.

[0003] Deep learning algorithms have become one of the most important methods for object detection. In deep learning-based object detection, there are mainly two types of algorithms: one-stage and two-stage algorithms. One-stage algorithms, such as SSD and YOLO, have advantages such as high speed and high robustness, and are widely used in real-time object detection. Two-stage algorithms, such as SPP-Net and Faster R-CNN, have high accuracy, but their detection speed is slower. Therefore, maximizing the accuracy of object detection while ensuring detection speed has become a crucial aspect of object detection algorithms. Summary of the Invention

[0004] The purpose of this invention is to provide a lightweight model-based target detection method based on improved YOLOv5s. By introducing a global attention module, it combines cross-dimensional feature information interaction between spatial attention and channel attention mechanisms, thereby improving the model's feature extraction of targets. At the output end, an adaptive spatial feature fusion detection head module adaptively learns the spatial weights between features at different scales, enabling more accurate learning of the scale characteristics of each target and improving multi-scale feature fusion, thus reducing the false negative and false positive rates of targets in remote sensing images.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] A lightweight object detection method based on an improved YOLOv5s model, the object detection method comprising the following steps:

[0007] Step 1: Select the dataset.

[0008] Step 2: Preprocess the dataset.

[0009] Step 3: Improve the attention mechanism by integrating the preprocessed data into the basic lightweight YOLOv5s with a global attention mechanism.

[0010] Step 4: Improve the feature fusion mechanism. After the processing in step 3, the feature map after feature extraction is fused by adaptive feature fusion.

[0011] Step 5: After steps 3 and 4, the image output by the network is fed into the detection head, and the prediction loss is calculated using the loss function to build a lightweight model based on YOLOv5s improvement.

[0012] Step 6: Test and validate the improved lightweight model using the dataset.

[0013] Step 7: Evaluate the metrics of the improved lightweight model based on the test and verification results of Step 6.

[0014] Furthermore, the improved attention mechanism integrates a global attention mechanism into the basic lightweight YOLOv5s, linking channel attention and spatial attention to enhance feature extraction in both spatial and channel aspects. By strengthening the interaction of feature information between the two attention dimensions, more detailed feature textures are obtained, improving the feature extraction rate of small and medium-sized targets, thereby improving the target detection accuracy.

[0015] Furthermore, the improved attention mechanism operates as follows:

[0016] (1) The 3D arrangement replaces the dimension of the feature map from ΔC×ΔW×ΔH to ΔW×ΔH×ΔC, where ΔC represents the number of channels in the input feature map, ΔW represents the width of the feature map, and ΔH represents the height of the feature map. Two multilayer perceptrons enhance the cross-dimensional joint between channels and space, improving the detector's target extraction. After feature extraction, the feature dimension is restored through a 3D inverse arrangement, as shown in the following formula.

[0017]

[0018] Where δ represents the sigmoid function, and ρ represents the three-dimensional arrangement. This indicates an inverted arrangement; MLP stands for Multilayer Perceptron.

[0019] (2) Then, perform a convolution operation on the input feature map using a 7×7 kernel and padding of 3. This compresses and reduces the dimensionality of the channel domain, highlighting the target features at the edges of the spatial domain and the differences between target features, thus obtaining... r represents the reduction factor of spatial dimension.

[0020]

[0021] in, This represents a convolutional network consisting of a 7×7 kernel, regularization (BN), and the ReLU activation function.

[0022] Furthermore, the improved feature fusion mechanism, after processing in step 3, performs a re-fusion of the extracted target features. Adaptive feature fusion is introduced at the feature fusion end. Through adaptive feature fusion, the feature map after feature extraction is fused, and the spatial weights between features at each scale are adaptively learned to improve the representational ability of the features and enable them to adaptively fuse the target features.

[0023] Furthermore, the improved feature fusion mechanism operates as follows:

[0024] (1) Let the three-layer output feature map be x l Let x be the feature mapping of the non-l layer, l∈{1,2,3}. n Since the feature maps in different feature layers have different resolutions and number of channels, Conv3*3 convolutions with a stride of 2 are used to downsample the feature resolution, and Conv1*1 convolutions are used to compress the number of feature channels. Concatenation between different layers enhances the multi-scale fusion of features, and then adaptive learning of the spatial weights of features at different scales improves the feature representation ability. The feature fusion formula is as follows:

[0025]

[0026] in, This means adjusting the size of the feature vector of point D(i,j) in the nth layer feature map to be the same as that of the lth layer. Indicates the output feature map y l The eigenvector at point D(i,j).

[0027] (2) Adaptively extract the spatial weights from layer n to layer l, which are defined as follows:

[0028]

[0029] in, And all are ∈ (0,1). They represent x respectively 1→l x 2→l x 3→l Weighted scalar mapping.

[0030] Furthermore, step 7 evaluates the improved lightweight model, and the evaluation metrics include average precision (AP) and mean average precision (mAP), with the following formulas:

[0031]

[0032]

[0033] Where P represents precision and R represents recall, the formulas for calculating P and R are as follows:

[0034]

[0035]

[0036] Wherein, TP (True Positive) represents the number of samples that are predicted to be true and are actually true, FP (False Positive) represents the number of samples that are predicted to be true but are actually false, and FN (False Negative) represents the number of samples that are predicted to be false but are actually true.

[0037] The beneficial effects of this invention are:

[0038] 1. The target detection method of this invention selects the DOTA dataset and preprocesses the dataset, which effectively reduces the resolution of high-resolution images and saves computational costs;

[0039] 2. The target detection method of this invention utilizes the modular programming concept of YOLOv5s, which can be applied to remote edge devices, saving the deployment of remote servers and has certain market prospects.

[0040] 3. The target detection method of this invention incorporates a global attention mechanism into the basic lightweight YOLOv5s, which strengthens feature extraction in both spatial and channel aspects, while enhancing the interaction of feature information between the two attention dimensions, obtaining more detailed feature textures, and improving the feature extraction rate of small and medium-sized targets, thereby improving the target detection accuracy.

[0041] 4. The target detection method of the present invention performs feature fusion on the feature map after feature extraction through adaptive feature fusion, and adaptively learns the spatial weights between features at different scales to improve the representation ability of features.

[0042] 5. The target detection method of this invention sends the image output by the network into the detection head, calculates the loss through the loss function, and continuously optimizes the model in subsequent training epochs to finally obtain an excellent model. Then, the improved lightweight model is tested and validated through a validation dataset to verify whether it can effectively detect targets in complex situations and small targets with low pixel counts, thereby reducing the false negative rate and false positive rate of targets. Attached Figure Description

[0043] The invention will now be further described with reference to the accompanying drawings.

[0044] Figure 1 This is a flowchart of the target detection method of the present invention;

[0045] Figure 2 This is the network diagram of the lightweight target detection model of the present invention;

[0046] Figure 3 This is a schematic diagram of the global attention mechanism of the present invention;

[0047] Figure 4 This is a schematic diagram of the channel attention mechanism of the present invention;

[0048] Figure 5 This is a schematic diagram of the spatial attention mechanism of the present invention;

[0049] Figure 6 This is a schematic diagram of the adaptive feature fusion of the present invention;

[0050] Figure 7 This is a comparison chart of the improved YOLOv5s algorithm of the lightweight target detection model of this invention and the basic YOLOv5s algorithm. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] A lightweight model-based object detection method based on improved YOLOv5s, such as... Figure 1 , Figure 2 As shown, the target detection method includes the following steps:

[0053] Step 1: Select Dataset

[0054] Download the DOTAv1.0 public dataset from the official DOTA website. The training set of this dataset consists of 2806 images, covering three types of targets: small, medium, and large. There are 15 target categories, with a total of 188,282 instances.

[0055] Step 2: Preprocess the dataset

[0056] By setting an overlap region (gap) of 300, the dataset is divided into 15,749 images through preprocessing. The validation set contains 5,297 images, and the test set contains 937 images. Preprocessing effectively reduces the number of high-resolution images and saves computational costs.

[0057] Step 3: Improve attention mechanisms

[0058] like Figures 3-5 As shown, by incorporating a global attention mechanism into the basic lightweight YOLOv5s, such as Figure 3 As shown, Figure 3 This diagram illustrates the global attention mechanism module, which connects channel attention and spatial attention to enhance feature extraction from both spatial and channel perspectives. It also strengthens the interaction of feature information between the two attention dimensions, resulting in more detailed feature textures and improving the feature extraction rate for small and medium-sized targets. This, in turn, improves the target detection accuracy. Figure 4 For channel attention, convolution operation on the input image. Figure 5 The pooling effect of spatial attention on the input image is as follows:

[0059] (1) The three-dimensional arrangement replaces the dimension of the feature map from ΔC×ΔW×ΔH to ΔW×ΔH×ΔC, where ΔC represents the number of channels in the input feature map, ΔW represents the width of the feature map, and ΔH represents the height of the feature map. The cross-dimensional joint between channels and space is enhanced by two multilayer perceptrons, which improves the detector's ability to extract targets. After feature extraction, the feature dimension is restored by three-dimensional inverse arrangement, as shown in the following formula.

[0060]

[0061] Where δ represents the sigmoid function, and ρ represents the three-dimensional arrangement. This indicates an inverted arrangement; MLP stands for Multilayer Perceptron.

[0062] (2) Then, perform a convolution operation on the input feature map using a 7×7 kernel and padding of 3. This compresses and reduces the dimensionality of the channel domain, highlighting the target features at the edges of the spatial domain and the differences between target features, thus obtaining... r represents the reduction factor of spatial dimension.

[0063]

[0064] in, This represents a convolutional network consisting of a 7×7 kernel, regularization (BN), and the ReLU activation function.

[0065] Step 4: After processing in Step 3, an improved feature fusion mechanism is applied to the extracted target.

[0066] like Figure 6 As shown, Figure 6 To jointly perform adaptive feature fusion and feature pyramid for feature fusion operations on targets at different scales, after processing in step 3, the extracted target features are re-fused. Adaptive feature fusion is introduced at the feature fusion stage. This adaptive feature fusion fuses the feature maps after feature extraction, adaptively learning the spatial weights between features at different scales to improve the representational ability of the features and enable adaptive fusion of target features. Specifically:

[0067] (1) Let the three-layer output feature map be x l Let x be the feature mapping of the non-l layer, l∈{1,2,3}. n Since the feature maps in different feature layers have different resolutions and number of channels, Conv3*3 convolutions with a stride of 2 are used to downsample the feature resolution, and Conv1*1 convolutions are used to compress the number of feature channels. Concatenation between different layers enhances the multi-scale fusion of features. Then, the spatial weights of features at different scales are adaptively learned to improve the feature representation ability. The feature fusion formula is as follows:

[0068]

[0069] in, This means adjusting the size of the feature vector of point D(i,j) in the nth layer feature map to be the same as that of the lth layer. Indicates the output feature map y l The eigenvector at point D(i,j).

[0070] (2) Adaptively extract the spatial weights from layer n to layer l, which are defined as follows:

[0071]

[0072] in, And all are ∈ (0,1). They represent x respectively 1→l x 2→l x 3→l Weighted scalar mapping.

[0073] Step 5: After steps 3 and 4, the image output by the network is fed into the detection head, the prediction loss is calculated using the loss function, and continuous optimization is performed in subsequent training epochs to build a lightweight model based on YOLOv5s improvement.

[0074] Step 6: Test and validate the improved lightweight model using a dataset to verify whether it can effectively detect targets in complex situations and small targets with low pixel counts, thereby reducing the false negative and false positive rates.

[0075] like Figure 7 As shown in Table 1, the validation results of the improved lightweight model on the validation set are as follows:

[0076]

[0077] In this table, detection rate 1 represents the number of samples detected by the group under the YOLOv5s model, and detection rate 2 represents the number of samples detected by the group under the improved YOLOv5s model. Table 1 intuitively demonstrates the detection performance of the improved lightweight model through data comparison.

[0078] Step 7: Based on the testing and validation results of Step 6, evaluate the improved lightweight model using the average precision (AP) and mean average precision (mAP) metrics, as shown in the following formulas:

[0079]

[0080]

[0081] Where P represents precision and R represents recall, the formulas for calculating P and R are as follows:

[0082]

[0083]

[0084] Wherein, TP (True Positive) represents the number of samples that are predicted to be true and are actually true, FP (False Positive) represents the number of samples that are predicted to be true but are actually false, and FN (False Negative) represents the number of samples that are predicted to be false but are actually true.

[0085] By incorporating a global attention mechanism into a lightweight model, the cross-dimensional feature information interaction capability between the channel and spatial attention mechanisms is enhanced, reducing the problem of target feature loss due to different dimensions and improving target feature extraction. At the same time, an adaptive spatial feature fusion algorithm is combined at the detection head. By adaptively adjusting the spatial weights of each scale during fusion, spatial scale inconsistency is suppressed, and the target features obtain a more accurate network processing weight, thereby improving the multi-scale feature fusion capability of the target and reducing the false negative and false positive rates of small targets in remote sensing images.

[0086] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0087] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A lightweight model-based target detection method based on improved YOLOv5s, characterized in that, The target detection method includes the following steps: Step 1: Select the dataset; Step 2: Preprocess the dataset; Step 3: Improve the attention mechanism by integrating the preprocessed data into the basic lightweight YOLOv5s with a global attention mechanism; Step 4: Improve the feature fusion mechanism. After the processing in Step 3, perform feature fusion on the feature map after feature extraction through adaptive feature fusion. Step 5: After steps 3 and 4, the image output by the network is fed into the detection head, and the prediction loss is calculated using the loss function to build a lightweight model based on YOLOv5s improvement. Step 6: Test and validate the improved lightweight model using the dataset; Step 7: Evaluate the metrics of the improved lightweight model based on the test and verification results of Step 6. The improved attention mechanism operates as follows: (1) The three-dimensional arrangement replaces the dimension of the feature map from ΔC×ΔW×ΔH to ΔW×ΔH×ΔC, where ΔC represents the number of channels in the input feature map, ΔW represents the width of the feature map, and ΔH represents the height of the feature map. The cross-dimensional joint between channels and space is enhanced by two multilayer perceptrons, which enhances the detector's extraction of the target. After feature extraction, the feature dimension is restored by three-dimensional inverse arrangement, and the formula is as follows: Where δ represents the sigmoid function. Represents a three-dimensional arrangement. This indicates an inverted arrangement; MLP stands for Multilayer Perceptron. (2) Then, perform a convolution operation on the input feature map using a 7×7 kernel and padding of 3. This compresses and reduces the dimensionality of the channel domain, highlighting the target features at the edges of the spatial domain and the differences between target features, thus obtaining... r represents the reduction factor of spatial dimension. in, This represents a convolutional network consisting of a 7×7 kernel, regularization, and the ReLU activation function. The improved feature fusion mechanism operates as follows: (1) Let the three-layer output feature map be x l Let x be the feature mapping of the non-l layer, l∈{1,2,3}. n Because the feature maps in different feature layers have different resolutions and number of channels, Conv3... 3 convolutions with a stride of 2 are used to downsample the feature resolution, Conv1 1. Convolutional compression reduces the number of channels in the features; concatenation between different layers enhances multi-scale feature fusion, and adaptive learning of spatial weights for features at different scales improves feature representation capabilities; the feature fusion formula is as follows: in, This means adjusting the size of the feature vector of point D(i,j) in the nth layer feature map to be the same as that of the lth layer. Indicates the output feature map y l The eigenvector at point D(i,j); (2) Adaptively extract the spatial weights from layer n to layer l, which are defined as follows: in, , , , And all are ∈ (0,1); , , They represent , , Weighted scalar mapping; In the lightweight model based on YOLOv5s, a GAM-ATTENTION is embedded between the last SPPF and C3 in the backbone network; a GAM-ATTENTION is embedded between C3 and ASFF in the neck network.

2. The lightweight model-based target detection method based on YOLOv5s improvement according to claim 1, characterized in that, The improved attention mechanism integrates a global attention mechanism into the basic lightweight YOLOv5s, linking channel attention and spatial attention to enhance feature extraction from both spatial and channel perspectives. By strengthening the interaction of feature information between the two attention dimensions, it obtains more detailed feature textures, improves the feature extraction rate of small and medium-sized targets, and thus improves the target detection accuracy.

3. The lightweight model-based target detection method based on YOLOv5s improvement according to claim 1, characterized in that, The improved feature fusion mechanism, after processing in step 3, re-fuses the extracted target features. Adaptive feature fusion is introduced at the feature fusion end. Through adaptive feature fusion, the feature map after feature extraction is fused. The spatial weights between features at different scales are adaptively learned to improve the representation ability of the features and enable them to adaptively fuse the target features.

4. The lightweight model-based target detection method based on YOLOv5s improvement according to claim 1, characterized in that, Step 7 evaluates the improved lightweight model, and the evaluation metrics include average accuracy and mean average accuracy, the formulas of which are as follows: Where AP represents average precision, mAP represents mean average precision, P represents precision, and R represents recall. The formulas for calculating P and R are as follows: Where TP represents the number of samples that are predicted to be true and are actually true, FP represents the number of samples that are predicted to be true but are actually false, and FN represents the number of samples that are predicted to be false but are actually true.

Images