Object detection optimization method based on positive and negative sample sampling ratio and model fine-tuning

Abstract

The present invention discloses a target detection optimization method based on the positive-negative sample sampling ratio and model fine-tuning, comprising the following steps: step 1) designing the overall network architecture; step 2) training the model; step 3) verifying the model and taking different optimization measures; step 4) optimizing measures for injecting negative samples with a fixed positive-negative sample ratio; step 5) optimizing measures for detecting targets of different scales. The embodiment of the present invention optimizes both the detection accuracy and the detection speed of the detector constructed with a deep learning solution. The FPN network is introduced into the single-step network SSD architecture with good detection speed performance to achieve a more accurate multi-scale detection effect. Subsequently, more categories of negative samples are injected by fixing the positive-negative sample ratio, and corresponding fixed network parameters and fine-tuning measures are performed for targets of different sizes to optimize the overall detection architecture.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and in particular relates to a target detection optimization method based on positive and negative sample sampling ratios and model fine-tuning. Background Art

[0002] With the increasing maturity of machine learning and deep learning technologies, numerous computer vision-related research topics have been successfully implemented and resolved. Object detection technology is a crucial research branch within this field. Object detection technology classifies and locates regions of interest in images and videos, thereby obtaining the desired target category and location information. Deep learning object detection algorithms based on convolutional neural networks fall into two main categories: a two-step network architecture based on candidate box generation, and a single-step network architecture that does not utilize a separate candidate box generation step.

[0003] The two-step network architecture is represented by the RCNN series, which often uses algorithms such as SS and RPN to extract candidate boxes of the region of interest, and then puts them into the network architecture for scoring and position regression. The single-step network architecture is represented by the SSD and YOLO series. Compared with the two-step network architecture, it omits the additional candidate box generation step and directly uses a deep convolutional neural network to integrate the feature extraction, multi-scale prior box acquisition, classification, and coordinate regression steps. It can be seen that the two-step network architecture often has higher detection accuracy because it uses the additional candidate box generation step to achieve better coverage of more targets, but it also sacrifices detection speed. The single-step network architecture integrates all steps in a unified neural network, so it often has a faster detection speed, but it is lower than the two-step network architecture in terms of detection accuracy.

[0004] Current research on object detection focuses on designing and innovating fast and accurate detection architectures, as well as sample manipulation, which is closely related to network architecture. Regarding network structure innovation, faster single-step network architectures that can meet real-time detection performance requirements often suffer from lower detection accuracy, while two-step network architectures with higher detection accuracy often suffer from slower detection speed. Therefore, research on architectures that combine faster detection speed with higher detection accuracy has become a key focus. A pressing challenge in training-related sample manipulation is the control of positive and negative samples. The training data used in object detection tasks consists of images labeled with object boxes, known as ground truth samples (GTs). During training, images used in each training round are drawn from a training set of the same batch size and are randomly shuffled after each round. Therefore, each minibatch contains both GT-positive and GT-negative samples, known as positive and negative samples. The characteristics of negative samples improve the generalization performance of the network, thereby reducing the occurrence of false positives. When a network architecture experiences false positives, the most effective and direct approach is to train the network on samples corresponding to the falsely detected class. This allows the network to directly learn the characteristics of the falsely detected class, resulting in better detection results. For example, if the network detects a camel in an image as a horse, images containing camels can be trained on the network. However, these images are all negative samples—images without GT annotations. As the number of negative samples increases, the proportion of GT images sampled in each minibatch used in training decreases. This causes the model to update the gradient in a direction that favors negative samples, ultimately reducing the model's detection performance.

[0005] In addition, in deep neural networks, high-level feature maps are generally relatively small in scale, contain relatively less detailed information, and have more semantic information. Low-level feature maps are large in scale and contain more detailed information. In detection tasks, due to the small scale of small targets, after multiple layers of convolution and pooling, target features are lost in high-level feature maps. Therefore, for small target detection, matching detection is generally performed on low-level feature maps. For large targets, due to their large scale, the receptive field of low-level feature maps is relatively small, and it is difficult to obtain the semantic information of the entire target and cannot accurately detect the target. Therefore, for large target detection, matching detection is generally performed on high-level feature maps. Summary of the Invention

[0006] In view of the above technical problems, the present invention is used to provide a target detection optimization method based on positive and negative sample sampling ratios and model fine-tuning.

[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0008] An object detection optimization method based on positive-negative sample sampling ratio and model fine-tuning includes the following steps:

[0009] Step 1) Design the overall network architecture: Using the single-step network architecture (SSD) as the base network, which boasts excellent detection speed and accuracy, we introduce the FPN structure to innovate the detection architecture and achieve high-precision multi-scale detection.

[0010] Step 2) Model training: Use the parameters of the base model trained on ImageNet data as the initial parameters for model training, and then fine-tune the model based on this. Then use the validation set to adjust the relevant hyperparameters of the model to determine the optimal hyperparameters of the final detector;

[0011] Step 3) Validate the model and take different optimization measures: Use test data to verify the performance of the trained model, visualize the model's detection results, and analyze images with missed detections and false detections. Then, take different optimization measures for different missed detections and false detections in different scenarios to achieve targeted optimization of target detection results.

[0012] Step 4) Optimization measures for injecting negative samples with a fixed positive-to-negative sample ratio: Optimize the false positives in step 3) by using the falsely detected target categories as negative samples. Ensure that the ratio of the sampled positive and negative samples remains consistent with the original positive and negative samples without adding negative samples. This increases the number of negative samples while maintaining the same positive-to-negative ratio. This avoids excessive negative samples, which can cause the model's optimization direction to be biased towards negative samples.

[0013] Step 5) Optimization measures for detection of targets of different scales: For the missed detection in step 3), analyze the scale of the missed target and match it with the size of the feature map. If the missed detection is a small target, fix the parameters of the high-level structure in the network, use data enhancement to generate a large number of small targets, and fine-tune the underlying structure of the network to achieve better extraction of feature information of small targets at the bottom layer. If the missed detection is a large target, use data enhancement to generate an image of the large target, fix the underlying network parameters, and no longer train and learn. Fine-tune the parameters in the high-level network structure to achieve better network detection of large targets.

[0014] Preferably, in step 1), scales of 1 / 8, 1 / 16, and 1 / 32 are used as the input layer of the detector. The base network outputs a 1 / 32 feature map as one of the detection input features, and then uses a deconvolution operation to expand the scale of the feature map. The expanded feature map is then fused with the original 1 / 16 feature map, and the fused features are used as the input features of the detector; the fused features are then deconvolved again and fused with the original 1 / 8 feature map, and the fused 1 / 8 feature map is used as the input feature layer of the detector.

[0015] Preferably, in step 2), during model training, the initialization learning rate of the model is 10e-4, and each iteration is 20 epochs, and the learning rate is reduced to 1 / 10 of the original.

[0016] Preferably, in step 4), during the model training process, the number of positive and negative samples in each batch is counted to ensure that the ratio of the number of positive and negative samples is consistent with the ratio of positive and negative samples before optimization.

[0017] Preferably, in step 5), when fine-tuning the model, the learning rate of the fixed layer is set to 0, the parameters of the layer are fixed, and no further learning is performed; the learning rate of the layer that needs to be fine-tuned is set to 10e-5, and the model parameters are fine-tuned.

[0018] The present invention has the following beneficial effects: Embodiments of the present invention optimize both detection accuracy and speed for detectors built using deep learning solutions. By introducing an FPN network into the single-step SSD architecture, which already offers excellent detection speed, higher-precision multi-scale detection is achieved. A fixed positive-negative sample ratio is then used to inject more negative samples, and network parameters are fixed and fine-tuned for targets of varying scales, optimizing the overall detection architecture. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flowchart of the steps of the object detection optimization method based on positive and negative sample sampling ratio and model fine-tuning according to an embodiment of the present invention;

[0020] Figure 2 This is a structural diagram of the FPN model in the target detection optimization method based on the positive and negative sample sampling ratio and model fine-tuning in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0022] Reference Figure 1 The present invention discloses a target detection optimization method based on positive and negative sample sampling ratio and model fine-tuning, and the specific steps are as follows:

[0023] Step 1) Design the overall network architecture: Use the single-step network architecture SSD as the base network, which has excellent detection speed performance and good detection accuracy. On this basis, introduce the FPN structure to innovate the detection architecture and achieve high-precision multi-scale detection effects.

[0024] In order to better detect targets of different scales, the FPN architecture is introduced based on the SSD network, as shown in Figure (2). Scales of 1 / 8, 1 / 16, and 1 / 32 are used as the input layer of the detector. The basic network outputs a 1 / 32 feature map as one of the detection input features, and then uses the deconvolution operation to expand the scale of the feature map. The expanded feature map is then fused with the original 1 / 16 feature map (using the eltwise operation to fuse the features), and the fused features are used as the input features of the detector; the fused features are then deconvolved again and fused with the original 1 / 8 feature map, and the fused 1 / 8 feature map is used as the input feature layer of the detector.

[0025] Step 2) Train the model: Use the parameters of the base model trained on ImageNet data as the initial parameters for model training, and fine-tune them on this basis. Then use the validation set to adjust the relevant hyperparameters of the model to determine the optimal hyperparameters of the final detector.

[0026] For model training, the loss function used for classification is the cross entropy loss function, and the loss function used for coordinate regression is the smooth L1 loss function. The expressions are as follows.

[0027] The overall loss function is calculated as follows:

[0028]

[0029] Among them, N represents the number of positive samples of the prior box; x∈{1,0} is a 0, 1 variable parameter, C is the category confidence prediction value; l represents the position prediction value of the network output of the bounding box corresponding to the prior box; g is the position parameter of the labeled GT.

[0030] Coordinate regression calculation method:

[0031]

[0032]

[0033]

[0034]

[0035] The cross entropy loss function calculation formula is:

[0036]

[0037] Among them, cx, cy, w, and h represent the coordinates of the center point of the coordinate frame and the width and height of the frame respectively. Indicates the matching between the i-th prior frame and the j-th true value frame, and the category corresponding to the label is the k-th category, Represent the coordinates of the center point of the i-th prior box and the width and height corresponding to the box respectively. Represents the position prediction value of the network output of the bounding box corresponding to the i-th prior box; g is the position parameter of the annotated true value box, Represents the position information of the true value frame after encoding, The predicted value representing the confidence level of the class.

[0038] Step 3) Validate the model and take different optimization measures: Use test data to verify the performance of the trained model, visualize the model's detection results, and analyze images with missed detections and false detections. Then, take different optimization measures for missed detections in different situations and false detections in different scenarios to achieve targeted optimization of target detection results. Count the scenarios in which images are prone to false detections, then classify the falsely detected targets in these scenarios and optimize these false detections using the methods in step 4). Statistically analyze missed detections for targets of different sizes, and use feature maps of different scales to optimize for targets of different sizes.

[0039] Step 4) Optimization measures for injecting negative samples with a fixed positive-to-negative sample ratio: Optimize the false positives in step 3) by using the falsely detected target categories as negative samples. Ensure that the ratio of the sampled positive and negative samples remains consistent with the original positive and negative samples without adding negative samples. This increases the number of negative samples while maintaining the same positive-to-negative ratio. This avoids excessive negative samples, which can cause the model's optimization direction to be biased towards negative samples.

[0040] To improve the model's ability to distinguish between them, we inject more negative data. This is data where the detector has falsely detected images of a certain type or category. By adding negative data to the dataset, the network learns the characteristics of these false detections and corrects the model's false detections. Since negative data does not contain actual objects to be detected, it consists of images without labeled boxes.

[0041] Step 5) Optimization measures for detection of targets of different scales: For the missed detection in step 3), analyze the scale of the missed target and match it with the size of the feature map. If the missed detection is a small target, fix the parameters of the high-level structure in the network, use data enhancement to generate a large number of small targets, and fine-tune the underlying structure of the network to achieve better extraction of feature information of small targets at the bottom layer. If the missed detection is a large target, use data enhancement to generate an image of the large target, fix the underlying network parameters, and no longer train and learn. Fine-tune the parameters in the high-level network structure to achieve better network detection of large targets.

[0042] Furthermore, in order to achieve better implementation results, in step 2), during model training, the model initialization learning rate is 10e-4, and each iteration is 20 epochs, and the learning rate is reduced to 1 / 10 of the original.

[0043] Furthermore, in order to achieve better implementation results, in step 4), during the model training process, the number of positive and negative samples in each batch is counted to ensure that the ratio of the number of positive and negative samples is consistent with the ratio of positive and negative samples before optimization.

[0044] Furthermore, in order to achieve better implementation results, in step 5), when fine-tuning the model, the learning rate of the fixed layer is set to 0, the parameters of the layer are fixed, and no further learning is performed. The learning rate of the layer that needs to be fine-tuned is set to 10e-5, and the model parameters are fine-tuned.

[0045] It should be understood that the exemplary embodiments described herein are illustrative and not restrictive. Although one or more embodiments of the present invention have been described in conjunction with the accompanying drawings, it should be understood by those skilled in the art that various changes in form and details may be made without departing from the spirit and scope of the present invention as defined by the appended claims.

Claims

1. A target detection optimization method based on positive and negative sample sampling ratio and model fine-tuning, characterized in that: The following steps are involved: Step 1) Design the overall network architecture: We use the single-step network architecture (SSD) as the base network, which boasts excellent detection speed and accuracy. We then introduce the FPN structure to innovate the detection architecture and achieve high-precision multi-scale detection. The FPN structure is introduced to innovate the detection architecture, including: using 1 / 8, 1 / 16, and 1 / 32 scales as the input layer of the detector, the basic network outputs a 1 / 32 feature map as one of the detection input features, and then uses the deconvolution operation to expand the scale of the feature map. The expanded feature map is then fused with the original 1 / 16 feature map, and the fused feature is used as the input feature of the detector; the fused feature is then deconvolved again and fused with the original 1 / 8 feature map, and the fused 1 / 8 feature map is used as the input feature layer of the detector; Step 2) Model Training: Initial model parameters are trained using the ImageNet dataset. Fine-tune the model parameters based on these parameters and then adjust the model hyperparameters using the validation set to determine the optimal hyperparameters for the detector. Initial learning rate is set to 10e-4 for each training step, and the learning rate is reduced to 1 / 10 of the original value for each 20 epochs. Step 3) Validate the model and take different optimization measures: Use test data to verify the performance of the trained model, visualize the model's detection results, and analyze images with missed detections and false detections. Then, take different optimization measures for different missed detections and false detections in different scenarios to achieve targeted optimization of target detection results. Step 4) Optimization measures for injecting negative samples with a fixed positive-to-negative sample ratio: Optimize the false positives in step 3) by using the falsely detected target categories as negative samples. Ensure that the ratio of the sampled positive and negative samples remains consistent with the original positive and negative samples without adding negative samples. This increases the number of negative samples while maintaining the same positive-to-negative ratio. This avoids excessive negative samples, which could cause the model optimization direction to be updated in a direction that favors negative samples. Step 5) Optimization measures for detecting targets of different scales: For the missed detection in step 3), analyze the scale of the missed target and match it with the size of the feature map. If the missed detection is a small target, fix the parameters of the high-level structure in the network, use data enhancement to generate a large number of small targets, and fine-tune the underlying structure of the network to achieve better extraction of feature information of small targets at the bottom layer. If the missed detection is a large target, use data enhancement to generate an image of the large target, fix the underlying network parameters, and no longer train and learn. Fine-tune the parameters in the high-level network structure to achieve better detection of large targets by the network.

2. The object detection optimization method based on positive and negative sample sampling ratio and model fine-tuning according to claim 1, characterized in that: In step 4), during the model training process, the number of positive and negative samples in each batch is counted to ensure that the ratio of the number of positive and negative samples is consistent with the ratio of positive and negative samples before optimization.

3. The object detection optimization method based on positive and negative sample sampling ratio and model fine-tuning according to claim 1, characterized in that: In step 5), when fine-tuning the model, the learning rate of the fixed layer is set to 0, the parameters of the layer are fixed, and no further learning is performed. The learning rate of the layer that needs to be fine-tuned is set to 10e-5, and the model parameters are fine-tuned.

Images