Target detection method based on lightweight deep learning

Abstract

The invention relates to a target detection method based on lightweight deep learning, and belongs to the technical field of image recognition. The method comprises the following steps: preprocessing an image through a multi-scale Retinex image enhancement algorithm with color protection to obtain an enhanced image, and merging the enhanced image into a training set; in order to solve the problem of missing detection and error detection of small targets, an original 20 * 20 detection head is replaced by a 160 * 160 small-scale detection head, two transverse connection paths are added on the basis of an original neck network, cross-scale fusion is carried out on shallow-layer features and deep-layer features, and the feature fusion capability is further enhanced; the improved model is lightened, a lightweight network MobileNetv3-ECA is constructed to serve as a backbone network of YOLOv5s, a channel pruning method is adopted to compress the network model, and the parameter quantity is further reduced; according to the invention, target detection can be deployed on the edge computing device with low computing power.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition and relates to a target detection method based on lightweight deep learning. Background Art

[0002] With the rapid development of deep learning technology, significant progress has been made in the field of object detection. Traditional object detection algorithms often rely on manual features and simple classifiers, while deep learning methods achieve higher accuracy and robustness through architectures such as convolutional neural networks. These technologies have enabled object detection to be widely adopted in many application scenarios, such as autonomous driving, security monitoring, and smart manufacturing.

[0003] However, as the complexity of object detection algorithms continues to increase, the computational requirements of the models also increase. This poses a major challenge for edge computing devices with limited computing resources, such as mobile devices, IoT devices, and edge servers. These devices are often limited in computing power, storage space, and energy consumption, so effectively deploying deep learning models on these platforms requires a balance between accuracy and lightweight.

[0004] This algorithm aims to propose a target detection method based on lightweight deep learning to meet this demand. By designing an efficient network architecture and optimization strategy, our goal is to significantly reduce the number of model parameters and computational complexity while ensuring detection accuracy, so that it can run smoothly on edge computing devices. This innovation will promote the application of deep learning technology in resource-constrained environments, meet the needs of intelligence, and improve the universality and practicality of target detection technology. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a target detection method based on lightweight deep learning, the purpose of which is to reduce the complexity of the detection algorithm model while improving the accuracy of target detection and optimizing the portability of the model to ensure that it can run efficiently on edge computing devices with limited computing resources. By building an efficient network architecture design and optimization strategy, the present invention can not only significantly reduce the number of model parameters and computing requirements, but also maintain excellent detection performance, thereby promoting the practical application of deep learning technology on various smart devices.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A target detection method based on lightweight deep learning includes the following steps:

[0008] It is characterized by comprising the following steps:

[0009] S1: Obtain the target dataset, use the Multi-Scale Retinex with Color Preservation (MSRCP) algorithm for image enhancement, and build the YOLOv5s network framework;

[0010] S2: In the head network, the original 20×20 detection head is replaced with a 160×160 small-scale detection head, and two lateral connection paths are added on the basis of the original neck network to fuse shallow features with deep features across scales.

[0011] S3: Use the lightweight network MobileNetv3-ECA as the backbone network of YOLOv5s to reduce the number of model parameters.

[0012] S4: Use the data set for training and verification to obtain an improved model, and finally perform structured pruning on the model to reduce the number of parameters.

[0013] Furthermore, in S1, in order to improve the accuracy of target detection, a multi-scale Retinex image enhancement algorithm with color protection is used for image enhancement. The original image and the enhanced image are incorporated into the training set, so that the model has stronger recognition ability when dealing with complex scenes such as severe weather. The steps of the image enhancement algorithm are:

[0014] Define the sum of the RGB channels at each pixel position as S i , where I R (i) and (I) G (i) and (I) B (i) represents the pixel value of the red, green, and blue channels at position i respectively:

[0015] S i =I R (i)+I G (i)+I B (i)

[0016] Calculate the chromaticity C of the entire image and take S at all pixel positions i The maximum value of:

[0017] C=max(S i )

[0018] Calculate the magnification factor F, where C is the chromaticity value obtained in the previous step, is the intensity image, For a single-channel MSR enhanced image, this step calculates the appropriate magnification factor by taking the minimum value:

[0019]

[0020] Calculate the enhanced image R'(x,y), where F is the magnification factor, I (R,G,B) (x, y) is the RGB feature map of the image at the coordinate (x, y). This step enlarges the RGB feature map of the original image by the enlargement factor F to obtain the enhanced image R'(x, y).

[0021] R′(x,y)=F·I (R,G,B) (x,y)

[0022] Furthermore, in S2, the targets in many scenes of actual applications are often small and medium scale, so the 20×20 detection head used to detect large-scale targets in the original head network is replaced with a 160×160 small-scale detection head to improve the detection performance of small targets. On the basis of the original neck network PANet, two new lateral connection paths are added to fuse shallow features with deep features across scales, further enhancing the feature fusion capability.

[0023] Furthermore, in S3, a lightweight network MobileNetv3-ECA is constructed as the backbone network of YOLOv5s, in which the Bneck structure is an optimized convolution module. The Bneck structure first uses 1×1 convolution to increase the dimension, then uses 3×3 depth-separable convolution and ECA-Net attention mechanism, and finally uses 1×1 convolution to reduce the dimension. By first increasing the dimension and then reducing the dimension, the extraction of small target features is strengthened to improve the detection accuracy.

[0024] The ECA-Net attention mechanism uses global average pooling to extract the global information of the channel, and then calculates the weight of each channel through a one-dimensional convolution operation instead of a fully connected layer, thereby avoiding explicit channel interactions. This mechanism effectively reduces parameters and computational complexity while maintaining effective use of channel information. It mainly involves the following three computational steps:

[0025] After performing channel global average pooling (GAP) on the input feature map H×W×C, a 1×1×C feature map is obtained. The calculation process of GAP is as follows, where W and H represent the width and height of the input feature map, respectively, and X ij Represents the pixel value at position (i, j) in the feature map:

[0026]

[0027] The weights ω of each channel in the feature map are generated by fast 1D convolution of the adaptive convolution kernel to achieve information interaction between each layer and adjacent channels. The convolution kernel size k is related to the dimension C of the feature channel, as shown below, where γ and b are nonlinear adjustment parameters. It means rounding up. By rounding up, multiplying by 2 and adding 1, we can ensure that k is an odd number:

[0028]

[0029] The weight ω of each channel in the feature map can be calculated:

[0030]

[0031] Among them, y is the channel feature, represents a 1D convolution with a convolution kernel size of k, and σ is an activation function used to introduce nonlinearity to improve the expressiveness of the model.

[0032] The weights obtained after the normalization operation are multiplied channel by channel with the original input feature map to obtain the final weighted feature map.

[0033] Furthermore, in S4, the improved model needs to be easy to deploy on edge computing devices with limited computing power. In order to further reduce the number of model parameters, the model is structured pruned. First, the model is sparsely trained to make γ approach 0, then the channels with smaller γ are removed, and the pruned network is fine-tuned. Sparse training is to introduce an Ll regularization penalty term to the γ parameter of the BN layer. The sparse training loss function formula is as follows:

[0034]

[0035] Where W represents the model weight, (x, y) represents the input sample and label. The second term is the regularization penalty term used to constrain γ, where λ is used to balance the two losses, that is, the sparsity rate. The g(*) function is defined as g(γ) = |γ|, that is, L1 regularization. The loss function with the L1 regularization term pushes γ to move toward 0, achieving the purpose of γ sparsification. Y represents the number of labels, which means the total number of different labels in the training set. This term is used to scale the regularization term so that it is associated with the number of labels.

[0036] The beneficial effects of the present invention are:

[0037] 1. The present invention uses the MSRCP algorithm to enhance the original image, incorporates the original image and the enhanced image into the training set, and improves the detection capability of the detection model in complex environments.

[0038] 2. The present invention uses a 160×160 small target detection head to replace a 20×20 large target detection head, and adds two lateral connection paths on the basis of the original neck network to fuse shallow features with deep features across scales, further improving detection accuracy.

[0039] 3. Build a lightweight network MobileNetv3-ECA as the backbone network of YOLOv5s, which effectively reduces the parameters and calculation amount, while maintaining the effective use of channel information, making it easy to transplant edge computing devices and improving the detection speed.

[0040] 4. The present invention further reduces the number of model parameters and improves real-time performance by performing channel pruning and fine-tuning on the improved model, so that it can meet the requirements of deployment on edge computing devices with weak computing power.

[0041] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0043] Figure 1 A schematic diagram of the target detection algorithm provided in the present invention;

[0044] Figure 2 A schematic diagram of a target detection network model provided in the present invention;

[0045] Figure 3 It is a schematic diagram of the Bneck structure of the MobileNet-ECA network model in the present invention;

[0046] Figure 4 Schematic diagram of the structured pruning method used for target detection in the present invention. DETAILED DESCRIPTION

[0047] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0048] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0049] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0050] See also Figure 1 , is a target detection method based on lightweight deep learning, which includes the following steps:

[0051] S1: Use the MSRCP algorithm to preprocess the target detection image, merge the enhanced image and the original image as a training set, generate a txt format dataset file and standardize it to match the input layer format of the YOLOv5s model.

[0052] S2: Use the enhanced dataset to design and pre-train the YOLOv5s model, detect the objects contained in the input image and predict the object bounding box position, size, positioning confidence and category probability; predict the category of each bounding box and calculate the intersection and union ratio of each predicted bounding box model and its true value.

[0053] The input layer implements online image enhancement. It inputs multiple images at the same time and performs random scaling, cropping, and arrangement, and then splices them into one image to enrich the detection data set and enhance robustness.

[0054] The Backbone layer includes the Conv module, C3 module, and SPPF module. The Conv module extracts features from the model, followed by batch normalization and activation functions to accelerate training and improve model stability. The C3 module further improves the feature fusion effect by introducing residual connections. The SPPF module fuses feature maps of different receptive fields to enrich the expressive power of the map.

[0055] The Neck layer adopts the FPN+PAN structure, and on this basis adds two lateral connection paths with 8x downsampling and 16x downsampling to fuse shallow features with deep features across scales, further enhance the feature fusion capability, and reduce the loss of detail information.

[0056] The output layer selects three-scale outputs and replaces the 20×20 detection head with a 160×160 small-scale detection head, so that the network can detect small targets with a resolution lower than 8×8, thereby increasing the number of small targets detected by the network.

[0057] S3: Build a lightweight network MobileNetv3-ECA as the backbone network of YOLOv5s. The inverted residual module (Bneck) increases the number of feature channels by using dilated convolution, then performs deep separable convolution to extract spatial features, and finally reduces the number of channels, thereby reducing the amount of honor calculation and improving the efficiency of information transmission.

[0058] The model is trained using Adam gradient descent and adaptive learning rate to obtain network model parameters, obtain the target contained in the input image and predict the target bounding box position, size, positioning confidence and category probability; predict the category of each border, and calculate the intersection and union ratio of each predicted boundary model and its true value as the basic parameter for network lightweighting.

[0059] S4: The Ll regular penalty term is introduced into the γ parameter of the BN layer to perform sparse training on the channels, so that γ approaches 0, and then the channels with smaller γ are removed, and the pruned network is fine-tuned to obtain the best model after pruning.

[0060] See also Figure 2 , in order to build a YOLOv5 network model with MobileNetv3-ECA backbone, its structure includes input layer, backbone network, neck network, and head network. Its network structure includes the following steps:

[0061] The target detection images are preprocessed using the MSRCP algorithm, and the enhanced images are merged with the original images as training sets. A dataset file in txt format is generated and standardized to match the input layer format of the YOLOv5s model.

[0062] The backbone network uses the Bneck structure of MobileNetv3-ECA for feature extraction. First, the input image is convolved (k=3, s=2, p=1) to quickly extract low-level features and expand the feature dimension, providing richer input for the subsequent Bneck layer. Then, the Bneck structure of MobileNetv3-ECA continuously extracts features from the input image to obtain feature maps of 4x, 8x, 16x, and 32x downsampling.

[0063] Feature fusion is performed in the Neck layer. First, the number of channels is increased through convolution to enhance the feature expression capability. Next, the upsampling operation is used to restore the spatial size of the feature map. Then, the upsampled feature map is concatenated with feature maps from other scales through the Concat module to further enhance the expression of multi-scale feature information. Finally, the C3 module is used for residual fusion, allowing feature information at different levels to be retained and fused, thereby improving the overall performance of the network.

[0064] First, the residual fusion of the C3 module is performed on the upsampled feature map to effectively integrate feature information from different levels. Then, the feature map is downsampled through convolution to reduce the spatial size of the feature map while maintaining the richness of the number of channels. The Concat module is used to concatenate the downsampled feature map with feature maps from other scales to enhance the expression ability of multi-scale features.

[0065] Finally, the Head layer generates three outputs to adapt to object detection at different scales. Each convolution is responsible for processing the fused feature map. These convolution layers output the bounding box, category probability, and confidence of the object respectively. With this design, the model can accurately detect objects at different scales, improving the overall detection performance and robustness.

[0066] See also Figure 3 , the Bneck structure of the MobileNetv3-ECA network model is as follows:

[0067] The dimension increase operation uses a 1×1 convolutional layer to increase the number of channels of the input feature map from C to k·C (usually k=4, C represents the number of channels of the feature map) to capture more feature information.

[0068] Depthwise separable convolution decomposes convolution into two steps: depthwise convolution and pointwise convolution. First, perform channel-by-channel convolution on the multi-channel feature map of the previous layer, that is, perform convolution on each channel separately, so that the feature information in each channel can be extracted without considering the relationship between channels. Then use a 1×1 convolution kernel to perform point-by-point convolution to fuse the information of different channels, and finally stack the results of the point-by-point convolution to obtain the output of depthwise separable convolution. Depthwise separable convolution greatly reduces the number of parameters and the amount of calculation of the model, making the model run more efficiently on edge computing devices with limited computing resources.

[0069] Between the depth convolution layer and the point-by-point convolution layer, the ReLU activation function is used to introduce nonlinearity, so that the model can better fit complex features.

[0070] An ECA module branch is added at the output of the Bneck layer, and its output is then fused with the backbone output. ECA-Net is a method that captures local cross-channel interactions and aims to ensure the efficiency and effectiveness of information.

[0071] After performing channel global average pooling (GAP) on the input feature map H×W×C, a 1×1×C feature map is obtained. The calculation process of GAP is as follows, where W and H represent the width and height of the input feature map, respectively, and X ij Represents the pixel value at position (i, j) in the feature map:

[0072]

[0073] The weights ω of each channel in the feature map are generated by fast 1D convolution of the adaptive convolution kernel to achieve information interaction between each layer and adjacent channels. The convolution kernel size k is related to the dimension C of the feature channel, as shown below, where γ and b are nonlinear adjustment parameters. It means rounding up. By rounding up, multiplying by 2 and adding 1, we can ensure that k is an odd number:

[0074]

[0075] The weight ω of each channel in the feature map can be calculated:

[0076]

[0077] Among them, y is the channel feature, represents a 1D convolution with a convolution kernel size of k, and σ is an activation function used to introduce nonlinearity to improve the expressiveness of the model.

[0078] The weights obtained after the normalization operation are multiplied channel by channel with the original input feature map to obtain the final weighted feature map.

[0079] See also Figure 4 The present invention provides a structured pruning method for target detection, the steps of which include:

[0080] S4.1: First, perform sparse training on the original network to make the scaling factor γ approach 0. Sparse training introduces the Ll regularization penalty term to the γ parameter of the BN layer. The sparse training loss function formula is as follows:

[0081]

[0082] Where W represents the model weight, (x, y) represents the input sample and label. The second term is the regularization penalty term used to constrain γ, where λ is used to balance the two losses, that is, the sparsity rate. The g(*) function is defined as g(γ) = |γ|, that is, L1 regularization. The loss function with the L1 regularization term pushes γ to move toward 0, achieving the purpose of γ sparsification. Y represents the number of labels, which means the total number of different labels in the training set. This term is used to scale the regularization term so that it is associated with the number of labels.

[0083] S4.2: Set a reasonable scaling factor threshold, prune channels whose scaling factor γ is less than the threshold, and obtain the pruned model.

[0084] S4.3: Fine-tune the pruned model. Fine-tune the model is to use the same dataset and load the pruned model for training again. The pruned model usually has a temporary loss of accuracy. Fine-tune the model parameters. After fine-tuning, the pruned model can usually restore the detection accuracy before pruning, or even higher.

[0085] S4.4: Evaluate the model. If the difference between the model and the expected result is too large, perform multiple pruning until the best pruned model is obtained.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A target detection method based on lightweight deep learning, characterized in that: The following steps are involved: S1: Obtain the target dataset, use the Multi-Scale Retinex with Color Preservation (MSRCP) algorithm for image enhancement, and build the YOLOv5s network framework; S2: In the head network, the original 20×20 detection head is replaced with a 160×160 small-scale detection head, and two lateral connection paths are added on the basis of the original neck network Neck to fuse shallow features with deep features across scales; S3: Build a lightweight network MobileNetv3-ECA as the backbone network of YOLOv5s to reduce the number of model parameters; S4: Use the data set for training and verification to obtain an improved model, and finally perform structured pruning on the model to reduce the number of parameters.

2. The target detection method based on lightweight deep learning according to claim 1, characterized in that: In S1, in order to improve the accuracy of target detection, a multi-scale Retinex image enhancement algorithm with color protection is used for image enhancement; and the original image and the enhanced image are incorporated into the training set so that the model has stronger recognition ability when dealing with complex scenes in severe climates; the steps of the image enhancement algorithm are: Define the sum of the RGB channels at each pixel position as S i , where I R (i) and (I) G (i) and (I) B (i) represents the pixel value of the red, green, and blue channels at position i respectively: S i =I R (i)+I G (i)+I B (i) Calculate the chromaticity C of the entire image and take S at all pixel positions i The maximum value of: C=max(S i ) Calculate the magnification factor F, where C is the chromaticity value obtained in the previous step, is the intensity image, For a single-channel MSR enhanced image, the appropriate magnification factor is calculated by taking the minimum value: Calculate the enhanced image R'(x,y), where F is the magnification factor, I (R,G,B) (x, y) is the RGB feature map of the image at the coordinate (x, y); this step enlarges the RGB feature map of the original image by the magnification factor F to obtain the enhanced image R'(x, y); R′(x,y)=F·I (R,G,B) (x,y)。 3. The target detection method based on lightweight deep learning according to claim 1, characterized in that: In S2, the 20×20 detection head used to detect large-scale targets in the original head network is replaced with a 160×160 small-scale detection head to improve the detection performance of small targets; based on the original neck network PANet, two new lateral connection paths are added to fuse shallow features with deep features across scales to enhance the feature fusion capability.

4. The target detection method based on lightweight deep learning according to claim 1, characterized in that: In the S3, a lightweight network MobileNetv3-ECA is constructed as the backbone network of YOLOv5s, wherein the Bneck structure is an optimized convolution module; the Bneck structure first uses 1×1 convolution to increase the dimension, then uses 3×3 depth-separable convolution and ECA-Net attention mechanism, and finally uses 1×1 convolution to reduce the dimension, and strengthens the extraction of small target features by first increasing the dimension and then reducing the dimension, thereby improving the detection accuracy; the ECA-Net attention mechanism uses global average pooling to extract the global information of the channel, and then calculates the weight of each channel through a one-dimensional convolution operation instead of a fully connected layer, thereby avoiding explicit channel-to-channel interaction; It includes the following three operation steps: After performing channel global average pooling GAP on the input feature map H×W×C, a 1×1×C feature map is obtained. The calculation process of GAP is as follows, where W and H represent the width and height of the input feature map, respectively, and X ij Represents the pixel value at position (i, j) in the feature map: The weight ω of each channel in the feature map is generated by fast 1D convolution of the adaptive convolution kernel to achieve information interaction between each layer and adjacent channels. The convolution kernel size k is related to the dimension C of the feature channel, as shown in the following formula: Where γ and b are nonlinear adjustment parameters, It means rounding up, multiplying by 2 and adding 1 to ensure that k is an odd number: Calculate the weight ω of each channel in the feature map: ω=σ(C 1Dk (y)) Among them, y is the channel feature, C 1Dk represents a 1D convolution with a kernel size of k, and σ is an activation function used to introduce nonlinearity to improve the expressiveness of the model; The weights obtained after the normalization operation are multiplied channel by channel with the original input feature map to obtain the final weighted feature map.

5. The target detection method based on lightweight deep learning according to claim 1, characterized in that: In S4, the improved model needs to be easy to deploy on edge computing devices with limited computing power. In order to further reduce the number of model parameters, the model is structured pruned. First, γ is made close to 0 by sparse training of the model, and then channels with smaller γ are removed, and the pruned network is fine-tuned. Sparse training is to construct an L1 regular penalty term for the γ parameter of the BN layer. The formula of the sparse training loss function is as follows: Where W represents the model weight, (x, y) represents the input sample and label; the second term is the regularization penalty term used to constrain γ, where λ is used to balance the two losses, namely the sparsity rate, and the g(*) function is defined as g(γ) = |γ|, i.e. L1 regularization. The loss function with the L1 regularization term pushes γ toward 0, thereby achieving the purpose of sparse γ; |Y| represents the number of labels, which means the total number of different labels in the training set, and is used to scale the regularization term so that it is associated with the number of labels.

Images