A dynamic training method for deep learning object detection

Abstract

The present invention discloses a dynamic training method for deep learning target detection, comprising the following steps: S1, extracting target areas and ignored areas in an image; S2, calculating the intersection-over-union (IoU) ratio of a candidate area to each target area, and calculating the proportion a of the intersection area of ​​the candidate area with each ignored area in the candidate area; S3, calculating the intersection of the candidate area with each ignored area according to the IoU, a and threshold T of positive and negative samples. P 、T N 、T a Define candidate negative samples, negative samples, positive samples, candidate positive samples and neutral samples; S4, calculate the difficulty and quality coefficient of non-neutral samples; S5, select positive and negative training samples from non-neutral samples according to the quality of samples; S6, dynamically adjust T P and T N , changing the gap between positive and negative samples, dynamically generating training samples of different difficulty and quality to train the model. The present invention makes full use of the annotations and background images in the dataset, and can train the supervised deep learning target detection model efficiently, with low loss and strong generalization.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular, to a dynamic training method for deep learning target detection. Background Art

[0002] According to the degree of supervision of learning, deep learning is divided into supervised learning, weakly supervised learning and unsupervised learning. In recent years, with the continuous improvement of hardware and software in the field of artificial intelligence, supervised deep learning technology has developed rapidly. Deep learning technology has been applied in various fields, such as computer vision, natural language processing, audio analysis, etc. Image-based object detection technology is an application of deep learning in the field of computer vision. Supervised deep learning object detection requires a large number of training samples to drive, and the difficulty and quality of training samples have largely determined the learning speed and quality. Training samples include positive training samples and negative training samples. Usually in an image, the candidate area (or region of interest, abbreviated as RoI) that just covers the detection target belongs to the positive sample, and all areas that do not overlap with the target area (ground truth) are negative samples. However, the actual situation is often not the case. RoI and ground truth may intersect, but not completely overlap. Therefore, researchers propose to use the intersection over union (IoU) and its threshold to determine positive and negative samples. IoU refers to the ratio of the intersection area and the union area of ​​RoI and ground truth. RoIs with IoU greater than a certain threshold are defined as positive samples, while RoIs with IoU less than a certain threshold are defined as negative samples. After defining positive and negative samples, which samples to use to train the target detection model, that is, sampling training samples, is also crucial to training efficiency and quality.

[0003] For example, Ren Shaoqing et al. adopted the following training method in the second stage network training of Faster-RCNN: the maximum IoU with all ground truth max )RoIs with an IoU of not less than 0.5 are defined as positive samples. max RoIs with ∈(0,0.5) are defined as negative samples, and IoU max = 0 (i.e. pure background) is defined as a neutral sample; if there are enough samples, randomly sample training samples while ensuring that positive and negative training samples are balanced; if there are not enough positive samples, repeat the sampling, and the same is true for negative samples; if there are no positive samples, terminate the training. Ross Girshick adopted the following training method in Fast-RCNN training: IoU max RoIs not less than 0.5 are defined as positive samples; and IoU max ∈[0.1,0.5) is defined as a negative sample, IoU max∈[0,0.1) is defined as a neutral sample and does not participate in training; if there are sufficient training samples, randomly sample training samples while ensuring that the positive and negative training samples are balanced; if there are insufficient positive samples, sampling can be repeated, and the same is true for negative samples; if there are no positive samples, the training is terminated.

[0004] However, the above two training methods are not optimal. First, these two training methods have no interval for the definition of positive and negative samples, that is, they do not clearly separate positive and negative samples, and IoU max Positive and negative samples near 0.5 are easily confused, which is not conducive to network learning. Secondly, they overemphasize the balance between positive and negative samples when sampling, and fail to make full use of the existing target area. Thirdly, they cannot make full use of background images (i.e., images without target areas) to train the model, which narrows the model's field of view and brings more false positives. Fourthly, the training samples they generate are difficult to train target detection models based on deep learning. Summary of the invention

[0005] The purpose of the present invention is to provide a dynamic training method for deep learning target detection to solve the problems raised in the background technology.

[0006] To achieve the above object, the present invention provides a dynamic training method for deep learning target detection, comprising the following steps:

[0007] S1, extract the target area and the ignored area in the image;

[0008] S2, calculate the intersection-over-union (IoU) ratio between the candidate region and each target region, and calculate the proportion a of the intersection area between the candidate region and each ignored region in the candidate region;

[0009] S3, based on all positive sample IoU, negative sample IoU, proportion a and positive sample IoU threshold T P , the threshold T of negative sample IoU N , the threshold T of proportion a a Define candidate negative samples, negative samples, positive samples, candidate positive samples and neutral samples: Maximum intersection over union (IoU) max ) is less than 0.05, and the maximum a(a max ) is less than T a The candidate region is defined as the candidate negative sample; IoU max ∈[0.05, T N ), and a max Less than T a The candidate area is defined as a negative sample; IoU max ∈(T N , T P ), and a max Less than Ta The candidate region is defined as the candidate positive sample of the corresponding category of the target region that matches it; IoU max ∈[T P , 1] is defined as the positive sample of the corresponding category of the target region that matches it; the remaining candidate regions are defined as neutral samples;

[0010] S4. Count the number of candidate negative samples, negative samples, positive samples and candidate positive samples, denoted as N CN 、N N 、N P 、N CP , and calculate the difficulty coefficient c of each non-neutral sample d and quality coefficient c q ; Among them, non-neutral samples refer to candidate negative samples, negative samples, positive samples and candidate positive samples;

[0011] S5. Sampling high-quality training samples: According to the quality coefficient of non-neutral samples from high to low, a small batch N is selected in the order of priority of positive samples, negative samples, and candidate positive samples. B training samples; if the above sample size is less than N B Then randomly select some candidate negative samples as negative training samples to make up N B training samples;

[0012] S6. Gradually change T according to the convergence of training loss N and T P The setting is used to change the gap between positive samples and negative samples, thereby dynamically generating training samples of different difficulty and quality for training deep learning-based target detection models.

[0013] Furthermore, in step S1: the ignored area refers to other annotated areas outside the target area, which have similarities with the target and may interfere with the training.

[0014] Furthermore, in step S3: to avoid interference, neutral samples are not involved in training; the threshold T of the proportion a a =0.5; the threshold T of the positive sample IoU P ∈[0.5, 0.7]; the threshold T of the negative sample IoU N ∈[0.25, 0.5].

[0015] Furthermore, the difficulty coefficient c of the non-neutral sample in step S4 is d and quality coefficient c q The calculation formula is:

[0016]

[0017] cq =1-2|c d -0.25| (2).

[0018] Furthermore, in step S6, T is gradually changed N and T P The specific method of setting is: S6.1, T N Take the minimum value, and T P Take the maximum value, generate the least difficult training sample with the largest positive and negative sample interval, and quickly train the model until the total loss does not decrease for 5 consecutive rounds; S6.2, continue to train the model. If the increase in the training loss of the negative training sample is less than 5% for 3 consecutive rounds, increase T by a small amount of 0.01. N Otherwise, reduce T by 0.01 N ; If the training loss increase of the positive training sample is less than 5% for three consecutive rounds, then reduce T by a small amount of 0.01 P Otherwise, increase T by 0.01 N The dynamic training method of the present invention takes the easy first and then the difficult and ensures quality as its guiding ideology, dynamically generates training samples of different difficulty and quality in stages, and makes full use of the annotations and background images in the data set to make the model training efficient, low-loss and strong in generalization.

[0019] Compared with the prior art, the present invention has the following beneficial effects: making full use of existing data, solving difficult problems first and then easy ones, ensuring quality, speeding up training, reducing training losses, improving model generalization ability, and ultimately reducing missed detection rate and false alarm rate of target detection.

[0020] The dynamic training method for deep learning target detection of the present invention clearly proposes the concept of candidate samples. According to the quality of non-neutral samples, positive and negative training samples are selected from positive samples and negative samples respectively to improve the quality of training samples; when the total amount of training samples is less than the small batch number N, B When the number of training samples is insufficient, the candidate negative samples are randomly selected as negative training samples to make up for the training samples. When there are no positive samples available, only the candidate negative samples are allowed to continue training to broaden the model's vision and reduce false positives. During the training process, the thresholds T of positive sample IoU and negative sample IoU are gradually changed with the guiding ideology of starting with the easy and then the difficult and ensuring quality. N and T P The setting changes the gap between positive and negative training samples, and then dynamically generates training samples of different difficulty and quality, making full use of the annotation area and background images in the dataset to make the model training efficient, low-loss and strong generalization, and ultimately reduce the missed detection rate and false alarm rate of target detection.

[0021] In addition to the above-described purposes, features and advantages, the present invention has other purposes, features and advantages. The present invention will be further described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings constituting a part of this application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0023] Figure 1 It is a schematic diagram of the overall process of a dynamic training method for deep learning target detection of the present invention;

[0024] Figure 2 It is a schematic diagram of a detailed flow chart of step S3 of a dynamic training method for deep learning target detection of the present invention;

[0025] Figure 3 It is a schematic diagram of the detailed process of step S5 of a dynamic training method for deep learning target detection in the present invention. DETAILED DESCRIPTION

[0026] The embodiments of the present invention are described in detail below with reference to the accompanying drawings, but the present invention can be implemented in many different ways as defined and covered by the claims.

[0027] See also Figure 1 , Figure 2 and Figure 3 This embodiment provides a dynamic training method for deep learning target detection. Since training samples of different difficulty and quality are generated in stages during the training process to achieve dynamic training, it is called a dynamic training method. The method includes the following steps:

[0028] S1. Extract the target area and the ignored area in the image; the ignored area refers to other annotated areas outside the target area, which are similar to the target and are used to generate neutral samples to avoid interfering with training.

[0029] S2. Calculate the intersection-over-union (IoU) of the candidate region and each target region respectively, and calculate the proportion a of the intersection area between the candidate region and each ignored region in the candidate region respectively.

[0030] S3, based on all the positive sample IoU, negative sample IoU, proportion a and positive sample IoU threshold T calculated in step S2 P , the threshold T of negative sample IoU N , the threshold T of proportion a a To define candidate negative samples, negative samples, positive samples, candidate positive samples and neutral samples: Maximum intersection over union (IoU max) is less than 0.05, and the maximum a(a max ) is less than T a The candidate region is defined as the candidate negative sample; IoU max ∈[0.05, T N ), and a max Less than T a The candidate area is defined as a negative sample; IoU max ∈(T N , T P ), and a max Less than T a The candidate region is defined as the candidate positive sample of the target region with the largest intersection over union ratio with the candidate region; IoU max ∈[T P , 1] is defined as the positive sample of the target region corresponding to the category with the largest intersection-over-union ratio with the candidate region; the remaining candidate regions are defined as neutral samples. Preferably, T a is the threshold of a, usually T a =0.5, T P is the positive sample IoU threshold, take T P ∈[0.5, 0.7]; T N is the negative sample IoU threshold, take T N ∈[0.25, 0.5].

[0031] S4. Count the number of candidate negative samples, negative samples, positive samples and candidate positive samples, denoted as N CN 、N N 、N P 、N CP , and calculate the difficulty coefficient c of each non-neutral sample d and quality coefficient c q . Preferably, the sample difficulty coefficient c d and quality coefficient c q The calculation formula is:

[0032]

[0033] c q =1-2|c d -0.25| (2).

[0034] S5. Sampling high-quality training samples: According to the quality coefficient of non-neutral samples, select a small batch N in the order of priority of positive samples, negative samples, and candidate positive samples. B training samples; it can be divided into several cases:

[0035] (1) If the number of positive samples is N P Greater than or equal to the number of training samples N B , then select NB positive samples with the highest quality rankings are used as positive training samples;

[0036] (2) If the number of positive samples is N P Less than the number of training samples N B , and the number of training samples N B and the number of positive samples N P The difference is greater than the number of negative samples N N , then first select all positive samples as positive training samples, and then select (N B -N P ) negative samples with high quality ranking are used as negative training samples;

[0037] (3) If the number of positive samples is N P and the number of negative samples N N The sum is less than the number of training samples N B , and the number of training samples N B and the number of positive samples N P And the number of negative samples N N The difference is greater than the number of candidate positive samples N CP , then first select all positive samples as positive training samples, then select all negative samples as negative training samples, and finally select (N B -N P -N N ) candidate positive samples with high quality ranking are used as positive training samples;

[0038] (4) If the number of positive samples is N P , the number of negative samples N N And the number of candidate positive samples N CP The sum of the three is less than the number of training samples N B , that is, the total number of positive samples, negative samples and candidate positive samples is less than N B After selecting all three samples, some candidate negative samples are randomly selected as negative training samples to make up N B The small batch training samples include positive training samples and negative training samples. In a specific implementation, the total number N B Take 32.

[0039] S6. Gradually change T according to the convergence of training loss N and T P The setting of T changes the gap between positive and negative samples, dynamically generates training samples of different difficulty and quality, and is used to train the target detection model based on deep learning. Preferably, the dynamic training process gradually changes T N and T P The setting method (i.e., the threshold of positive sample IoU and the threshold of negative sample IoU) is as follows: (1) TN Take the minimum value, and T P Take the maximum value, generate the least difficult training sample with the largest positive and negative sample interval, and quickly train the model until the total loss does not decrease for 5 consecutive rounds; (2) Continue to train the model. If the increase in the training loss of the negative training sample is less than 5% for 3 consecutive rounds, increase T by a small amount of 0.01. N Otherwise, reduce T by 0.01 N ; If the training loss increase of the positive training sample is less than 5% for three consecutive rounds, then reduce T by a small amount of 0.01 P Otherwise, increase T by 0.01 N The dynamic training method of the present invention takes the easy first and then the difficult and ensures quality as its guiding ideology, dynamically generates training samples of different difficulty and quality in stages, and makes full use of the annotations and background images in the data set to make the model training efficient, low-loss and strong in generalization.

[0040] As shown in Table 1, the training speed and training loss of the method of the present invention are compared with the prior art Fast-RCNN training method and Faster-RCNN training method on the validation set of NightOwls, the largest nighttime pedestrian dataset, and the MR on the validation set of NightOwls. -2 Missed detection rate. Among them, MR -2 The missed detection rate indicator is the mean of the missed detection rate (MR) at 9 FPPIs (average false positives per image), which are uniformly distributed in the logarithmic space [10 -2 ,10 0 ] range. MR -2 It is currently the most popular single target detection performance evaluation indicator because it comprehensively considers the missed detection rate representing safety and the false alarm rate representing efficiency. -2 The lower the value, the better. All three methods provide training samples and methods for the second stage of Faster-RCNN. All three training methods use a small batch stochastic gradient descent algorithm with momentum to train the model from scratch, with a momentum coefficient and decay factor of 0.9 and 0.0005 respectively; the initial value of the learning rate (η) is set to 0.01, and the following strategy is used to update η: if the training loss does not decrease within 5 consecutive rounds, η is reduced by 10 times; if η drops to 1e-10, the training is terminated.

[0041] Table 1. Training speed, training loss and MR of different training samples -2 Missed detection rate comparison

[0042]

[0043] From the data in Table 1, it can be seen that the method of the present invention has a significant improvement in training speed, training loss and model generalization ability (i.e., MR-2 The false detection rate is lower) and is significantly better than the training methods of Faster-RCNN and Fast-RCNN.

[0044] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A dynamic training method for deep learning target detection, characterized in that: The following steps are involved: S1, extracting the target area and the ignored area in the image, wherein the ignored area refers to other annotation areas in the image except the target area; S2. Calculate the intersection and union ratio of the candidate region and each target region respectively , and calculate the proportion of the intersection area between the candidate area and each ignored area in the candidate area ; S3. Based on all positive samples , negative samples , proportion And positive samples Threshold , negative samples Threshold , proportion Threshold Define candidate negative samples, negative samples, positive samples, candidate positive samples and neutral samples: Maximum intersection ratio Less than 0.05, and the maximum proportion Less than The candidate regions are defined as candidate negative samples; ,and Less than The candidate region is defined as a negative sample; ,and Less than The candidate region is defined as the candidate positive sample of the corresponding category of the target region that matches it; The candidate regions are defined as positive samples of the corresponding categories of the target regions that match them; the remaining candidate regions are defined as neutral samples; S4. Count the number of candidate negative samples, negative samples, positive samples and candidate positive samples, expressed as , and calculate the difficulty coefficient of each non-neutral sample and quality coefficient ; Among them, non-neutral samples refer to candidate negative samples, negative samples, positive samples and candidate positive samples; the difficulty coefficient of the non-neutral samples is and quality coefficient The calculation formulas are: (1)； (2)； S5. Sampling high-quality training samples: According to the quality coefficient of non-neutral samples from high to low, select positive samples, negative samples, and candidate positive samples in order of priority. training samples; If the total number of positive samples, negative samples, and candidate positive samples is insufficient Then randomly select some candidate negative samples as negative training samples to make up for training samples; S6. Gradually change according to the convergence of training loss and The setting is used to change the gap between positive samples and negative samples, thereby dynamically generating training samples of different difficulty and quality for training deep learning-based target detection models.

2. The dynamic training method according to claim 1, characterized in that: In step S3: to avoid interference, neutral samples are not involved in training; Threshold ; The positive sample Threshold ; The negative sample Threshold .

3. The dynamic training method according to claim 1, characterized in that: In step S6, the and The specific method of setting is: S6.1, Take the minimum value, and Take the maximum value and generate the least difficult training sample with the largest positive and negative sample intervals. Train the model until the total loss does not decrease for 5 consecutive rounds. S6.

2. Continue training the model. If the training loss increase of negative training samples is less than 5% for three consecutive rounds, increase the training loss by 0.

01. Otherwise, it will be reduced by 0.01 ; If the training loss increase of the positive training sample is less than 5% for three consecutive rounds, then reduce it by a small amount of 0.01 Otherwise, it will increase by 0.01 .

Images