Image enhancement and network structure improvement method in target detection YOLO model

Abstract

The invention discloses an image enhancement and network structure improvement method in a target detection YOLO model, relates to a target detection method, and provides a new deep learning model which comprises three modules: a data enhancement module, a feature extraction module and a detection output module. According to the deep learning model, in target detection, the characteristics of a Copy-Past module, a GAM attention module and a decoupling head module are effectively utilized. The model can enrich the number of samples when the data samples are insufficient; feature information of the input image is fully extracted; and the classification and positioning capabilities are enhanced. For problems existing in detection, the model has high detection capability and robustness. In the detection process of the model, the detection accuracy is improved, the positioning capability is more accurate, and the detection of a small target is also improved.

Description

Technical Field

[0001] The present invention relates to a target detection method, and in particular to an image enhancement and network structure improvement method in a target detection YOLO model. Background Art

[0002] With the continuous development of artificial intelligence technology, especially the introduction of deep learning technology, the field of computer vision is also developing rapidly. As one of the important topics in the field of computer vision, target detection mainly aims to identify and locate targets in images or videos. Target detection is also an important part of other complex tasks in the field of computer vision, such as instance segmentation, target tracking, image classification, etc. With the introduction of convolutional neural networks into the field of computer vision, target detection has shifted from traditional algorithms to algorithms based on deep learning, which has improved the accuracy and precision of target detection. At present, target detection has been widely used in daily life, such as in the fields of transportation, agriculture, medicine, and industry. With the continuous improvement of technology, the application fields of target detection technology are constantly expanding.

[0003] Although target detection has developed rapidly driven by deep learning, there are still problems such as missed detection, inaccurate positioning and low detection accuracy. Summary of the invention

[0004] The purpose of the present invention is to provide a method for image enhancement and network structure improvement in the target detection YOLO model. In order to solve the problems existing in the detection process, the present invention proposes to integrate the Copy-Past module, the GAM (Global Attention Mechanism) attention module and the Decoupled Head module into the target detection model to enhance the robustness of the model and improve the detection ability of the model.

[0005] The objective of the present invention is achieved through the following technical solutions:

[0006] A method for image enhancement and network structure improvement in a target detection YOLO model, the method comprising the following components:

[0007] (1) Data enhancement module

[0008] Using the Copy-Paste algorithm, the detection target is copied to another image to increase the number of samples and solve the sample imbalance. Training on the enhanced data set improves the generalization ability of the model and enhances the detection performance of the model.

[0009] (2) Feature extraction module

[0010] Responsible for extracting features from the input image and providing feature maps for subsequent classification and regression tasks; the feature extraction module uses CSPDarknet53 as its backbone network and extracts features from the image through a series of convolution, normalization and activation function operations; it is a feature extraction module combined with the GAM attention module; in the channel direction, for the input feature map F1∈R CxHxW (C is the number of channels, H and W are the height and width of the feature map respectively) The three-dimensional arrangement is (HxW)xC; the channel attention weights are learned through a two-layer MLP (multi-layer perceptron), the input and output dimensions of the MLP are C and C respectively, and the dimension of the middle layer is C / r (r is the compression ratio); finally, the Sigmoid function is used to convert the output of the MLP into the attention weight; the formula is expressed as:

[0011] F c =σ(g(F1))

[0012] Where g is the MLP function and σ is the Sigmoid activation function;

[0013] In the spatial direction, average pooling and maximum pooling are performed on the given feature map F1 to obtain μ(F1) and max(F1) respectively. These two results are concatenated in the channel dimension and the spatial attention weights are learned through a convolutional layer and a Sigmoid activation function. The formula is expressed as:

[0014] F s =σ(conv(μ(F1),max(F1)))

[0015] The input feature map F1 is element-wise multiplied with the channel attention output map Fc and the spatial attention output map Fs to obtain the final feature map F3. Its expression is as follows:

[0016]

[0017] The final feature map can simultaneously consider the importance of channel and spatial dimensions, thereby enhancing feature representation.

[0018] (3) Detection output module

[0019] The detection output module is the output part of the model, which performs multi-scale target detection on the input feature map and outputs the results. This module uses the anchor box mechanism to define a set of bounding boxes with different aspect ratios in advance, which are used to generate candidate boxes on the feature map to adapt to the detection targets of different sizes during the detection process.

[0020] The decoupling head module is integrated into the detection output module to improve the classification and positioning accuracy of the model; the detection output module first reduces the dimension through a 1x1 convolution layer, and then captures the overall details of the target through a fully connected layer in the classification branch to improve the accuracy of the model; a 3x3 convolution layer is used in the positioning branch to retain the spatial feature information, and an iou branch is added to improve the positioning ability of the model.

[0021] The method for image enhancement and network structure improvement in a target detection YOLO model, after the detection output module generates a candidate frame, it is classified and judged whether it is a target through Classification, and Regression is used to regress the detection frame and adjust the position and size of the detection frame in time; and a multi-scale prediction strategy is used, by performing detection on feature maps of different scales, so that the model can detect large targets and small targets at the same time.

[0022] The method for image enhancement and network structure improvement in a target detection YOLO model, wherein the decoupled head is a detection structure that processes classification and positioning tasks separately to improve the performance of the model; before decoupling, the structure fuses features at different levels to obtain richer information; then, the fused features are respectively sent to the classification and regression branches for prediction; and a loss function is designed to guide the training process of the model so that the model can better learn the features and rules of different tasks.

[0023] The advantages and effects of the present invention are:

[0024] 1. The present invention proposes a new deep learning model, which effectively utilizes the characteristics of the Copy-Past module, the GAM attention module and the decoupling head module in target detection. The main features of this model are that it can enrich the number of samples when faced with insufficient data samples; fully extract the feature information of the input image; and further enhance the classification and positioning capabilities. In the face of problems existing in detection, the model has higher detection capabilities and robustness. During the detection process, this model not only improves the accuracy of detection, but also makes its positioning capability more accurate, and also improves the detection of small targets.

[0025] 2. This invention pushes the boundaries of the field in object detection and paves the way for subsequent research. This invention has a far-reaching impact on the application fields of object detection models. DETAILED DESCRIPTION

[0026] The present invention is described in detail below with reference to the embodiments.

[0027] The present invention provides a method for image enhancement and network structure improvement in a target detection YOLO model, comprising the following components:

[0028] Data augmentation module

[0029] Image data enhancement technology is often used in deep learning to perform a series of operations on the input data set, generate new samples, increase the size and diversity of the data set, so as to reduce the overfitting problem of the model and improve the generalization ability and robustness of the model. In many fields such as image processing, natural language processing, and speech recognition, image data enhancement technology has been widely used to solve problems such as insufficient data set samples and sample imbalance, and improve model performance. In the present invention, the Copy-Paste algorithm idea is used to copy the detection target to another image, which not only increases the number of samples and solves problems such as sample imbalance, but also training on the enhanced data set can improve the generalization ability of the model and enhance the detection performance of the model.

[0030] Feature extraction module

[0031] The feature extraction module is the core part of this model, which is mainly responsible for extracting features from the input image and providing feature maps for subsequent classification and regression tasks. The feature extraction module uses CSPDarknet53 as its backbone network and extracts features from the image through a series of operations such as convolution, normalization, and activation functions.

[0032] The GAM (Global Attention Mechanism) attention module is a global attention module. Different from the commonly used attention modules, the GAM attention module can pay attention to the feature information of both space and channel at the same time, and pay attention to the connection between space and channel, retaining the context information, so that the model pays more attention to the area of ​​interest.

[0033] In order to make the model have better feature extraction ability, the present invention designs and implements a feature extraction module combined with the GAM attention module. In the channel direction, for the input feature map F1∈R CxHxW (C is the number of channels, H and W are the height and width of the feature map respectively) The three-dimensional arrangement is in the shape of (HxW)xC. The channel attention weights are learned through a two-layer MLP (multi-layer perceptron). The input and output dimensions of the MLP are C and C respectively, and the dimension of the middle layer is C / r (r is the compression ratio). Finally, the output of the MLP is converted into attention weights using the Sigmoid function. The formula is expressed as:

[0034] F c =σ(g(F1))

[0035] Where g is the MLP function and σ is the Sigmoid activation function.

[0036] In the spatial direction, average pooling and maximum pooling are performed on the given feature map F1 to obtain μ(F1) and max(F1) respectively. These two results are concatenated in the channel dimension and the spatial attention weights are learned through a convolutional layer and a Sigmoid activation function. The formula is expressed as:

[0037] F s =σ(conv(μ(F1),max(F1)))

[0038] The input feature map F1 is element-wise multiplied with the channel attention output map Fc and the spatial attention output map Fs to obtain the final feature map F3. Its expression is as follows:

[0039]

[0040] The final feature map can simultaneously consider the importance of channel and spatial dimensions, thereby enhancing feature representation.

[0041] Detection output module

[0042] The detection output module is the output part of the model, which performs multi-scale target detection on the input feature map and outputs the results. This module uses the anchor box mechanism to define a set of bounding boxes with different aspect ratios in advance, which are used to generate candidate boxes on the feature map to adapt to the detection targets of different sizes during the detection process. After the candidate box is generated, it is classified and judged whether it is a target through Classification, and Regression is used to regress the detection box and adjust the position and size of the detection box in time. A multi-scale prediction strategy is also used to detect on feature maps of different scales, so that the model can detect large and small targets at the same time.

[0043] DecoupledHead is a new detection structure that processes classification and localization tasks separately to improve the performance of the model. Before decoupling, the structure fuses features at different levels to obtain richer information. Then, these fused features are sent to the classification and regression branches for prediction. A loss function is designed to guide the training process of the model so that the model can better learn the characteristics and rules of different tasks.

[0044] The present invention integrates the decoupling head module into the detection output module to improve the classification and positioning accuracy of the model. The detection output module first reduces the dimension through a 1x1 convolution layer, and then captures the overall details of the target through a fully connected layer in the classification branch to improve the accuracy of the model. The use of a 3x3 convolution layer in the positioning branch retains the spatial feature information, and the addition of an iou branch improves the positioning ability of the model.

Claims

1. A method for image enhancement and network structure improvement in a target detection YOLO model, characterized in that: The method It includes the following components: (1) Data enhancement module Using the Copy-Paste algorithm, the detection target is copied to another image to increase the number of samples and solve the sample imbalance. Training on the enhanced data set improves the generalization ability of the model and enhances the detection performance of the model. (2) Feature extraction module Responsible for extracting features from the input image and providing feature maps for subsequent classification and regression tasks; the feature extraction module uses CSPDarknet53 as its backbone network and extracts features from the image through a series of convolution, normalization and activation function operations; it is a feature extraction module combined with the GAM attention module; In the channel direction, for the input feature map F1∈R CxHxW (C is the number of channels, H and W are the height and width of the feature map respectively) The three-dimensional arrangement is (HxW)xC; the channel attention weights are learned through a two-layer MLP (multi-layer perceptron), the input and output dimensions of the MLP are C and C respectively, and the dimension of the middle layer is C / r (r is the compression ratio); finally, the Sigmoid function is used to convert the output of the MLP into the attention weight; the formula is expressed as: F c =σ(g(F1)) Where g is the MLP function and σ is the Sigmoid activation function; In the spatial direction, average pooling and maximum pooling are performed on the given feature map F1 to obtain μ(F1) and max(F1) respectively. These two results are concatenated in the channel dimension, and the spatial attention weights are learned through a convolutional layer and a Sigmoid activation function; the formula is expressed as: F s =σ(conv(μ(F1),max(F1))) The input feature map F1 is element-wise multiplied with the channel attention output map Fc and the spatial attention output map Fs to obtain the final feature map F3; its expression is as follows: The final feature map can consider the importance of both channel and spatial dimensions, thereby enhancing feature representation; (3) Detection output module The detection output module is the output part of the model, which performs multi-scale target detection on the input feature map and outputs the results; This module uses an anchor box mechanism to define a set of bounding boxes with different aspect ratios in advance, which are used to generate candidate boxes on the feature map to adapt to detection targets of different sizes during the detection process. The decoupling head module is integrated into the detection output module to improve the classification and positioning accuracy of the model; the detection output module first reduces the dimension through a 1x1 convolution layer, and then captures the overall details of the target through a fully connected layer in the classification branch to improve the accuracy of the model; a 3x3 convolution layer is used in the positioning branch to retain the spatial feature information, and an iou branch is added to improve the positioning ability of the model.

2. The method for image enhancement and network structure improvement in a target detection YOLO model according to claim 1, characterized in that: After the detection output module generates the candidate box, it is classified and judged whether it is the target through Classification, and Regression is used to regress the detection box and adjust the position and size of the detection box in time; and a multi-scale prediction strategy is used to detect on feature maps of different scales, so that the model can detect large targets and small targets at the same time.

3. The method for image enhancement and network structure improvement in a target detection YOLO model according to claim 1, characterized in that: The decoupled head is a detection structure that processes classification and positioning tasks separately to improve the performance of the model. Before decoupling, the structure fuses features at different levels to obtain richer information. Then, these fused features are sent to the classification and regression branches for prediction; And design a loss function to guide the model training process, so that the model can better learn the characteristics and rules of different tasks.

Images