Target detection method based on gradient equilibrium and Wasserstein distance
By introducing a gradient balance mechanism and normalized Wasserstein distance into the object detection method, the problems of gradient imbalance between categories and degradation of micro-object detection performance are solved, and higher accuracy and faster object detection effects are achieved.
Patent Information
- Application Number
- CN202411527294.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-30
- Publication Date
- 2025-02-14
AI Technical Summary
Existing object detection methods have problems of performance degradation in dealing with gradient imbalances between categories and detecting tiny objects, especially in complex scenarios, which are difficult to achieve high-precision and rapid detection.
Using an object detection method based on gradient equilibrium and Wasserstein distance, a new loss function is designed to improve the detection accuracy and robustness of the model by introducing a gradient equilibrium mechanism and normalizing the Wasserstein distance.
It effectively slows down the negative impact of positive samples of non-current target classes on gradients, improves the model's detection ability of target categories, and improves the detection performance of small targets, achieving higher accuracy and faster target positioning and detection.
Smart Images
Figure CN119445232A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of computer vision and artificial intelligence, and in particular to a target detection method based on gradient balance and Wasserstein distance. Background Art
[0002] Target detection belongs to the field of computer vision and includes two subtasks: positioning (determining the target location) and classification (determining the target category). Target detection can be widely used in various fields, such as autonomous driving navigation, smart agriculture, industrial detection, aerospace, military reconnaissance, smart medical care, etc. In agriculture, through target detection, technicians can quickly and accurately process a large amount of image data in a short time, obtain key information such as the location of stomata, and achieve the purpose of automatically completing the task of plant stoma detection. In medicine, target detection can accurately identify the lesion area and distinguish between normal tissue and abnormal structure, providing doctors with a more reliable and intuitive basis for diagnosis, and helping to achieve early detection and early treatment of diseases. In fact, in actual applications, different targets have their own characteristics, and the process of collecting images cannot completely eliminate interference factors such as illumination and occlusion, so target detection is also challenging.
[0003] The development of object detection can be divided into two stages: the exploration of traditional object detection algorithms and the rise of object detection algorithms based on deep learning. The former adopts a sliding window strategy to obtain image features, while the latter relies on deep learning to automatically extract image features, further improving detection performance.
[0004] In some images, except for the background category, the targets to be detected and other non-target categories are evenly distributed. Not only are the numbers roughly the same, but the characteristics of the two types of targets are also very similar. For example, the shapes, sizes, and edge contours of the stomata and apertures in the microscope images of plants are very similar, making it difficult to accurately distinguish them. For the target category, a large number of non-target positive samples will have a negative impact, resulting in a decrease in the performance of the model.
[0005] On the other hand, tiny objects are ubiquitous in many real-world applications, including driver assistance, large-scale surveillance, and maritime rescue. Although significant progress has been made in object detection due to the development of deep neural networks, most of them are dedicated to detecting objects with normal sizes. Tiny objects usually exhibit extremely limited appearance information, which increases the difficulty of learning discriminative features and leads to a large number of failure cases when detecting tiny objects. Traditional intersection over union itself and its improved versions are very sensitive to the position deviation of small objects. Therefore, when using these metrics in anchor-based object detectors, it may lead to a significant degradation in detection performance.
[0006] In order to reduce the negative impact of non-target categories on the detection task and alleviate the model's sensitivity to small target position deviations, the present invention proposes a target detection method based on gradient balance and Wasserstein distance. Summary of the invention
[0007] In order to address the deficiencies of the prior art, the present invention aims to provide a target detection method based on gradient balance and Wasserstein distance. The method achieves accurate positioning of targets and stable recognition of small targets by combining gradient balance technology and Wasserstein distance. First, in order to improve the model's ability to locate targets, a gradient-based sample balanced classification loss function is designed. By adjusting the gradient contribution of the sample, the negative impact of the gradient caused by positive samples of non-current target classes is mitigated. Furthermore, in order to alleviate the model's sensitivity to small target position deviations, the normalized Wasserstein distance is introduced, and it is fused with the SIoU metric to propose an adjustable regression loss function. Finally, a loss function based on gradient balance and Wasserstein distance is proposed to form a new target detection method. This method is suitable for tasks such as stomatal detection in agriculture and lesion area recognition in medicine, and provides new ideas and solutions for solving target detection problems in complex scenarios.
[0008] In order to achieve the above object, the technical solution adopted by the present invention is as follows:
[0009] A target detection method based on gradient balance and Wasserstein distance comprises the following steps:
[0010] 1) Obtain target detection data: Obtain images to construct training sets, test sets, and validation sets, and annotate the images in the training sets and validation sets respectively;
[0011] 2) Model construction:
[0012] Based on the YOLOv5 target detection model, a parameter-free attention module is introduced to focus on feature information; a gradient balancing mechanism is introduced into the classification loss function of the model to dynamically adjust the gradient weights of samples of different categories to alleviate the impact of gradient imbalance between categories on model training and improve the model's ability to detect target categories; the normalized Wasserstein distance NWD is introduced into the model's bounding box regression loss function;
[0013] 3) Loading model: Load the trained model and use the optimal network to perform intelligent detection and recognition on the detection data;
[0014] 4) Parameter adjustment: Users adjust network parameters according to their own data characteristics to achieve the best detection effect.
[0015] Furthermore, in step 2), the normalized Wasserstein distance is introduced into the model bounding box regression loss function, specifically:
[0016] The bounding box is modeled as a two-dimensional Gaussian distribution, and then NWD is used to calculate the similarity between bounding boxes through their corresponding Gaussian distributions; the traditional IoU is replaced by SIoU which takes into account the angle loss factor to speed up the convergence of the model.
[0017] Furthermore, the construction of the model in step 2) specifically includes:
[0018] 2.1) Read all the images in the training set and adjust the image size to some extent;
[0019] 2.2) Use the CSPDarknet53 feature extraction network to extract features from the training set images, obtain feature maps of three different scales, and integrate the gradient changes into the feature maps;
[0020] 2.3) Use SPPF to perform maximum pooling operations of different scales on the input feature maps to obtain feature maps of multiple scales; then concatenate these feature maps to form a feature vector containing multi-scale information; finally, input this feature vector into the fully connected layer;
[0021] 2.4) The SimAM attention module calculates the local self-similarity of the input feature map and optimizes the energy function, assigns attention weights to each neuron, and recalibrates the feature map accordingly;
[0022] 2.5) Using PANet to aggregate feature maps for object detection of different sizes;
[0023] 2.6) The three feature maps obtained in step 2.4) are respectively used to predict the position of the bounding box through the convolution kernel;
[0024] 2.7) Each bounding box predicts a set of categories, and uses logistic regression as a classifier to predict the confidence score of each category, and sets a threshold. The category higher than the threshold is the true category of the bounding box;
[0025] 2.8) Predict bounding boxes at 3 different sizes;
[0026] 2.9) Through multiple iterative learning, the loss function converges to the minimum and the optimal model parameters are saved.
[0027] Further, step 2.9) specifically includes:
[0028] 2.9.1) Set the model parameters, including the number of images read in at a time and the number of training iterations;
[0029] 2.9.2) Calculate the loss: loss L BL-NWD-IoU It consists of the following two parts:
[0030] L BL-NWD-IoU =L BL +L NWD-SIoU ,
[0031] Among them, L BL is the gradient-based sample balanced classification loss function, L NWD-SIoU It is the regression loss function that integrates NWD and IoU.
[0032] Further, step 2.9.2) includes:
[0033] 2.9.2.1) Gradient-based sample balanced classification loss function:
[0034] In order to solve the problem of gradient imbalance between categories, a weight is introduced on the basis of the binary cross entropy loss function, and a gradient-based sample balanced classification loss function is designed, which is named Balance Loss. It ignores the gradient of non-current detection target categories and balances the positive and negative sample gradients during training.
[0035] Formally, a weight is introduced into the original cross entropy loss function, and the new balanced loss function can be expressed as:
[0036]
[0037] Among them, C is the number of categories, p j Indicates the probability that the detected object belongs to the current detection target category, is the probability of the model predicting correctly, q is the weight, j represents the jth category; y j is the true value label of category j; for a region γ, the weight q is set as follows:
[0038] q=1-E(γ)l(1-y j ),
[0039] In the formula, when γ is the foreground, the exclude function E(γ) value is 1, when γ is the background, the E(γ) value is 0; l is the label label;
[0040] 2.9.2.2) Regression loss function combining NWD and IoU:
[0041] In order to increase the accuracy of detection and improve the convergence speed, NWD and SIoU are fused and an adjustable regression loss function L is proposed. NWD-SIoU , introduce the ratio parameter μ to adjust the ratio of NWD and SIoU:
[0042] L NWD-IoU =μ(1-NWD)+(1-μ)SIoU,
[0043] Among them, μ∈(0,1), NWD refers to the normalized Wasserstein distance, and SIoU refers to SIoU loss;
[0044] Finally, through iterative learning, when the value of the loss function converges to the minimum, the network model at this time is saved as the optimal network model.
[0045] The present invention has the following beneficial effects:
[0046] 1. The present invention uses a gradient balancing algorithm to reduce the importance of positive samples of other categories in the loss function to reduce their negative impact on the gradient of positive samples of the current category, thereby alleviating the problem of gradient imbalance and improving the detection accuracy of the model;
[0047] 2. The present invention introduces the normalized Wasserstein distance into the regression loss, models the bounding box as a two-dimensional Gaussian distribution, and calculates the similarity to improve the model's performance in detecting small targets. On this basis, by setting the proportional hyperparameter, the normalized Wasserstein distance is combined with SIoU to obtain a regression loss function based on Wasserstein distance and intersection-over-union ratio metric. While improving the model's detection performance, it optimizes the model's convergence speed and helps the model achieve higher-precision and faster positioning and detection of targets.
[0048] 3. The target detection method provided by the present invention can effectively process multi-scale targets in data and realize comprehensive detection from small objects to large objects. It can effectively meet the application needs in complex scenes and has good application prospects in many fields. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Figure 1 is a flow chart of the present invention;
[0050] Figure 2 The YOLOv5 model structure diagram of SimAM is introduced in the present invention;
[0051] Figure 3 This is an example diagram of the stomatal detection results of wheat leaves according to the present invention. DETAILED DESCRIPTION
[0052] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not intended to limit the scope of use of the present invention.
[0053] like Figure 1-2As shown, the target detection method based on gradient balance and Wasserstein distance includes the following steps:
[0054] Step 1) Obtain target detection training data: Obtain images to construct training set, test set and validation set, and annotate the images in the training set and validation set respectively;
[0055] Step 2) Model construction: Based on the YOLOv5 target detection model, a parameter-free attention module (Simple parameter-free Attention Module, SimAM) is introduced to focus on feature information.
[0056] A gradient balancing mechanism is introduced into the classification loss function of the model. By dynamically adjusting the gradient weights of samples of different categories, the impact of gradient imbalance between categories on model training is alleviated, thereby improving the model's ability to detect target categories.
[0057] The normalized Wasserstein distance (NWD) is introduced into the model bounding box regression loss function. Specifically, the bounding box is modeled as a two-dimensional Gaussian distribution, and then NWD is used to calculate the similarity between bounding boxes through their corresponding Gaussian distributions; further, the traditional IoU is replaced with SIoU (Smooth Intersection over Union) which takes into account the angle loss factor, which speeds up the convergence of the model.
[0058] The construction of the model specifically includes the following steps:
[0059] 2.1) Read all the images in the training set and adjust the image size.
[0060] 2.2) Use the CSPDarknet53 feature extraction network to extract features from the training set images, obtain feature maps of three different scales, and integrate the gradient changes into the feature maps.
[0061] 2.3) Use SPPF (Spatial Pyramid Pooling-Fast) to perform maximum pooling operations of different scales on the input feature map to obtain feature maps of multiple scales. Then these feature maps are concatenated to form a feature vector containing multi-scale information. Finally, this feature vector is input into the fully connected layer.
[0062] 2.4) The SimAM attention module calculates the local self-similarity of the input feature map and optimizes the energy function, assigns attention weights to each neuron, and recalibrates the feature map accordingly.
[0063] 2.5) PANet (Path Aggregation Network) is used to aggregate feature maps for object detection of different sizes.
[0064] 2.6) The three feature maps obtained in step 2.4) are respectively used to predict the position of the bounding box through the convolution kernel.
[0065] 2.7) Each bounding box predicts a set of categories, and uses logistic regression as a classifier to predict the confidence score of each category, and sets a threshold. The category higher than the threshold is the true category of the bounding box.
[0066] 2.8) Predict bounding boxes at 3 different sizes.
[0067] 2.9) Through multiple iterative learning, the loss function converges to the minimum and the optimal model parameters are saved. Specifically, it includes:
[0068] 2.9.1) Set the model parameters, including the number of images read in at a time, the number of training iterations, etc.
[0069] 2.9.2) Calculate the loss: loss L BL-NWD-IoU It consists of the following two parts:
[0070] L BL-NWD-IoU =L BL +L NWD-SIoU ,
[0071] Among them, L BL is the gradient-based sample balanced classification loss function, L NWD-SIoU It is a regression loss function that integrates the normalized Wasserstein distance and the intersection-over-union ratio. Specifically, it includes:
[0072] 2.9.2.1) Gradient-based sample balanced classification loss function:
[0073] In practical applications, the use of the cross entropy loss function will inhibit the model's learning of the target. In long-tail detection, the problem of gradient imbalance between categories often occurs. In order to solve this problem, inspired by the Equalization Loss (EQL), this paper introduces a weight on the basis of the binary cross entropy loss function and designs a gradient-based sample balanced classification loss function, which is named Balance Loss. It ignores the gradient of non-current detection target categories and balances the positive and negative sample gradients during training.
[0074] Formally, a weight is introduced into the original cross entropy loss function, and the new balanced loss function can be expressed as:
[0075]
[0076] Among them, C is the number of categories, p j Indicates the probability that the detected object belongs to the current detection target category, is the probability of the model predicting correctly, q is the weight, j represents the jth category; y j is the true value label of category j; for a region γ, the weight q is set as follows:
[0077] q=1-E(γ)l(1-y j ),
[0078] Where, when γ is the foreground, the exclude function E(γ) is 1, and when γ is the background, the E(γ) is 0; l is the label label (in the experiment with wheat leaves as the experimental detection object, the stomata in the leaf data set are marked as 0 and the opening is 1).
[0079] 2.9.2.2) Regression loss function integrating normalized Wasserstein distance and intersection-over-union ratio:
[0080] In order to increase the accuracy of detection and improve the convergence speed, this paper combines NWD with SIoU and proposes an adjustable regression loss function L NWD-SIoU , introduce the ratio parameter μ to adjust the ratio of NWD and SIoU:
[0081] L NWD-IoU =μ(1-NWD)+(1-μ)SIoU,
[0082] Among them, μ∈(0,1), NWD refers to the normalized Wasserstein distance, and SIoU refers to SIoU loss.
[0083] Finally, through iterative learning, when the value of the loss function converges to the minimum, the network model at this time is saved as the optimal network model.
[0084] Step 3) Load the model: Load the trained model and use the optimal network to perform intelligent detection and recognition on the test set.
[0085] Step 4) Parameter adjustment: Users adjust network parameters according to their own data characteristics to achieve the best detection effect.
[0086] like Figure 3As shown, the present invention uses wheat leaves as experimental detection objects for simulation experiments. It can be clearly seen from the accompanying drawings that although some stomata of the wheat leaves are deformed due to developmental failure, which increases the difficulty of detection, the directions of the stomata are also randomly rotated, requiring the model to capture the characteristics of the stomata more flexibly and accurately; however, the target detection method provided by the present invention can still successfully and accurately identify the stomata and ignore those stomata with developmental problems, thereby avoiding the occurrence of false detection.
[0087] The present invention innovatively proposes a target detection method based on gradient balance and Wasserstein distance, which is applicable to various data sets and has significantly improved the detection effect. This shows that the method has powerful performance in target detection and meets the application requirements in complex scenes. The target detection method provided by the present invention can effectively process multi-scale targets in data, realize comprehensive detection from small objects to large objects, effectively meet the application requirements in complex scenes, and has good application prospects in multiple fields.
[0088] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.
Claims
1. A target detection method based on gradient balance and Wasserstein distance, characterized in that: The following steps are involved: 1) Obtain target detection data: Obtain images to build training sets, test sets, and validation sets, and annotate the images in the training sets and validation sets respectively; 2) Model construction: Based on the YOLOv5 target detection model, a parameter-free attention module is introduced to focus on feature information; a gradient balancing mechanism is introduced into the classification loss function of the model to dynamically adjust the gradient weights of samples of different categories to alleviate the impact of gradient imbalance between categories on model training and improve the model's ability to detect target categories; the normalized Wasserstein distance NWD is introduced into the model's bounding box regression loss function; 3) Loading model: Load the trained model and use the optimal network to perform intelligent detection and recognition on the detection data; 4) Parameter adjustment: Users adjust network parameters according to their own data characteristics to achieve the best detection effect.
2. The target detection method based on gradient balance and Wasserstein distance according to claim 1, characterized in that: In step 2), the normalized Wasserstein distance is introduced into the model bounding box regression loss function, specifically: The bounding box is modeled as a two-dimensional Gaussian distribution, and then NWD is used to calculate the similarity between bounding boxes through their corresponding Gaussian distributions; the traditional IoU is replaced by SIoU which takes into account the angle loss factor to speed up the convergence of the model.
3. The target detection method based on gradient balance and Wasserstein distance according to claim 2, characterized in that: The construction of the model in step 2) specifically includes: 2.1) Read all the images in the training set and adjust the image size to some extent; 2.2) Use the CSPDarknet53 feature extraction network to extract features from the training set images, obtain feature maps of three different scales, and integrate the gradient changes into the feature maps; 2.3) Use SPPF to perform maximum pooling operations of different scales on the input feature maps to obtain feature maps of multiple scales; then concatenate these feature maps to form a feature vector containing multi-scale information; finally, input this feature vector into the fully connected layer; 2.4) The SimAM attention module calculates the local self-similarity of the input feature map and optimizes the energy function, assigns attention weights to each neuron, and recalibrates the feature map accordingly; 2.5) Using PANet to aggregate feature maps for object detection of different sizes; 2.6) The three feature maps obtained in step 2.4) are respectively used to predict the position of the bounding box through the convolution kernel; 2.7) Each bounding box predicts a set of categories, and uses logistic regression as a classifier to predict the confidence score of each category, and sets a threshold. The category higher than the threshold is the true category of the bounding box; 2.8) Predict bounding boxes at 3 different sizes; 2.9) Through multiple iterative learning, the loss function converges to the minimum and the optimal model parameters are saved.
4. The target detection method based on gradient balance and Wasserstein distance according to claim 3, characterized in that: Step 2.9) specifically includes: 2.9.1) Set the model parameters, including the number of images read in at a time and the number of training iterations; 2.9.2) Calculate the loss: loss L BL-NWD-IoU It consists of the following two parts: L BL-NWD-IoU =L BL +L NWD-SIoU , Among them, L BL is the gradient-based sample balanced classification loss function, L NWD-SIoU It is the regression loss function that integrates NWD and IoU.
5. The target detection method based on gradient balance and Wasserstein distance according to claim 4, characterized in that: Step 2.9.2) includes: 2.9.2.1) Gradient-based sample balanced classification loss function: In order to solve the problem of gradient imbalance between categories, a weight is introduced on the basis of the binary cross entropy loss function, and a gradient-based sample balanced classification loss function is designed, which is named Balance Loss. It ignores the gradient of non-current detection target categories and balances the positive and negative sample gradients during training. Formally, a weight is introduced into the original cross entropy loss function, and the new balanced loss function can be expressed as: Among them, C is the number of categories, p j Indicates the probability that the detected object belongs to the current detection target category, is the probability of the model predicting correctly, q is the weight, j represents the jth category; y j is the true value label of category j; for a region γ, the weight q is set as follows: q=1-E(γ)l(1-y j ), In the formula, when γ is the foreground, the exclude function E(γ) value is 1, when γ is the background, the E(γ) value is 0; l is the label label; 2.9.2.2) Regression loss function combining NWD and IoU: In order to increase the accuracy of detection and improve the convergence speed, NWD and SIoU are fused and an adjustable regression loss function L is proposed. NWD-SIoU , introduce the ratio parameter μ to adjust the ratio of NWD and SIoU: L NWD-IoU =μ(1-NWD)+(1-μ)SIoU, Among them, μ∈(0,1), NWD refers to the normalized Wasserstein distance, and SIoU refers to SIoU loss; Finally, through iterative learning, when the value of the loss function converges to the minimum, the network model at this time is saved as the optimal network model.