A method and system for training sample allocation of multi-scale target detection

Abstract

The present application relates to a kind of multi-scale target detection training sample distribution method and system, the method includes: obtaining target data set, the multiple feature maps of target data set are extracted by convolutional neural network, and prior frame and predicted frame are determined;The first overlap value of prior frame and real label frame and the second overlap value of predicted frame and real label frame are calculated, the prior frame of first overlap value greater than first set threshold is regarded as first candidate positive sample, the predicted frame of second overlap value greater than first set threshold is regarded as supplementary first candidate positive sample, first positive sample set is constructed, and second set threshold is obtained by calculating mean and variance;According to the relationship of first overlap value and first set threshold and second set threshold, prior frame is divided, and second positive sample and second negative sample are obtained.The imbalance problem of multi-scale target in positive and negative sample distribution under the scene of automatic driving is solved, and the detection precision of multi-scale target in target detection network is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and automatic driving, and particularly relates to a training sample distribution method and system for multi-scale target detection. BACKGROUND

[0002] The core idea of a target detection algorithm based on deep learning is to automatically predict the target class and position information of interest in an input image or video through the detection algorithm, which highly matches the needs of automatic driving. A detection model is trained by inputting a large number of automatic driving scene pictures, so that the detection model obtains the function of identifying target information such as pedestrians, vehicles and traffic signs in an automatic driving scene. However, in an automatic driving scene, the scale difference of target objects such as pedestrians, vehicles and traffic signs is large, and scene changes are numerous, which will affect the accuracy and real-time performance of the detection algorithm, which is also a challenge faced by current target detection algorithms based on computer vision technology in the application of automatic driving scenes.

[0003] Current mainstream object detection algorithms, such as Faster RCNN and YOLO, all consider target detection as a classification problem, that is, first generate a region of interest using a prior box or RPN, and then classify and regress the position of the region.

[0004] The training phase of the target detection network needs to divide positive and negative samples, and then design a loss function according to the task, use an optimization algorithm such as SGD to update the weight and bias of the neuron in an iterative manner, and the optimization goal is to minimize the loss function, so that the trained model can fit the training set.

[0005] In the target detection task, an image is input, and the unit of positive and negative samples is a certain region in an image, so there are multiple positive and negative samples in an image. Then how to obtain these regions (samples) and how to divide so many regions into positive and negative samples? These are two important problems. The former: a common method is to obtain these regions based on the anchor method, and some prior boxes anchors generated on each small block of the image are samples. The latter: the commonly used method is to divide the positive and negative samples based on the IOU of the anchor and the real label box gt, and different algorithm strategies are different. If the anchor is divided into a positive sample, then the regression of the positive sample can obtain a predicted box, and the predicted box can be involved in the calculation of the positioning loss of the loss function.

[0006] Some commonly used training positive and negative sample distribution methods at present include MaxIoUAssigner, which is mainly summarized as follows: the IoU of an anchor and a gt is calculated, and if the IoU of the anchor and the gt is lower than a specified threshold Y, it is a negative sample, and if it is higher than the threshold Y, it is a positive sample. Or a TopK-based distribution method: the Euclidean distance between the center of the anchor and the center of the gt is calculated, and the first K anchors with the shortest distance are selected, and the final positive and negative samples are selected by using the IoU value.

[0007] The existing distribution methods have more or less some defects, especially when the target scale gap is large in the automatic driving scene. Because the sizes of the targets are different, large targets will match more positive anchors than small targets, which will lead to the imbalance of the distribution of positive and negative anchors. The IOU value of a small target and a preset anchor is generally low. If the threshold of the MaxIoUAssigner type distribution method is set to be large, there will be too few positive samples of the small target, and if the threshold is set to be small, there will be more poor positive samples of the large target. The number of positive samples selected by the ToPK type distribution method is limited, and the sample information cannot be fully utilized, and the fitting is slow. SUMMARY

[0008] The purpose of the present application is to provide a multi-scale target detection training sample distribution method and system, which can solve the imbalance problem of multi-scale targets in the automatic driving scene when distributing positive and negative samples, and can effectively improve the detection accuracy of multi-scale targets in the target detection network.

[0009] To achieve the above purpose, the present application provides the following scheme:

[0010] The present application provides a multi-scale target detection training sample distribution method, which comprises:

[0011] Obtaining a target data set in an automatic driving scene; the target data set comprises a target detection object image and corresponding label information; the target detection object image comprises a target detection object, and the target detection object comprises a pedestrian, a vehicle and a traffic sign; the label information comprises the position coordinates and the category of the target detection object in the target detection object image;

[0012] Performing feature extraction on the target data set through forward propagation of a convolutional neural network to obtain a plurality of feature maps, and determining a plurality of prior boxes and prediction boxes based on the feature maps;

[0013] Creating a first sample distribution list, which represents the sample types to which the prior boxes are distributed; the sample types comprise a first positive sample, a first negative sample and a first ignored sample;

[0014] The first overlap value of the prior frame and the real label frame is calculated, and the prior frame index with the first overlap value greater than a first set threshold is taken as a first candidate positive sample; the second overlap value of the predicted frame and the real label frame is calculated, and the predicted frame index with the second overlap value greater than the first set threshold is taken as a supplementary first candidate positive sample;

[0015] A first positive sample set is constructed according to the first candidate positive sample and the supplementary first candidate positive sample, and the mean value and the variance of the first positive sample set are calculated, and a second set threshold is obtained according to the mean value and the variance;

[0016] According to the size relationship between the first overlap value and the first set threshold and the second set threshold, the prior frame is divided to obtain a second positive sample and a second negative sample; the second positive sample and the second negative sample are taken as training data of a convolutional neural network to obtain a target detection network model; the target detection network model is used for multi-scale target detection in an automatic driving scene.

[0017] Optionally, the plurality of prior frames and predicted frames are determined based on the feature map, and specifically include:

[0018] A plurality of prior frames are set at the center of each pixel point on the feature map;

[0019] An output offset value is calculated according to the convolutional neural network, and a predicted frame is determined according to the output offset value and the prior frame.

[0020] Optionally, the second set threshold is the sum of the mean value and the variance.

[0021] Optionally, the prior frame is divided according to the size relationship between the first overlap value and the first set threshold and the second set threshold to obtain a second positive sample and a second negative sample, and specifically includes:

[0022] The prior frame with the first overlap value less than the first set threshold is divided into a second negative sample;

[0023] The prior frame with the first overlap value greater than the second set threshold is divided into a second positive sample.

[0024] To achieve the above object, the application further provides a training sample distribution system for multi-scale target detection, which comprises:

[0025] The target data set acquisition unit is configured to acquire a target data set in an autonomous driving scene; the target data set comprises a target detection object image and corresponding label information; the target detection object image comprises a target detection object, and the target detection object comprises a pedestrian, a vehicle and a traffic sign; and the label information comprises position coordinates and a category of the target detection object in the target detection object image.

[0026] The prior box and prediction box determination unit is configured to perform feature extraction on the target data set through forward propagation of a convolutional neural network to obtain a plurality of feature maps, and determine a plurality of prior boxes and prediction boxes based on the feature maps.

[0027] The first sample assignment list creation unit is configured to create a first sample assignment list, which represents sample types to which the prior boxes are assigned; the sample types comprise a first positive sample, a first negative sample and a first ignored sample.

[0028] The first candidate positive sample and supplementary first candidate positive sample determination unit is configured to calculate a first overlap value between the prior boxes and a real label box, and take an index of a prior box with a first overlap value greater than a first set threshold as a first candidate positive sample; and calculate a second overlap value between the prediction boxes and the real label box, and take an index of a prediction box with a second overlap value greater than the first set threshold as a supplementary first candidate positive sample.

[0029] The second set threshold determination unit is configured to construct a first positive sample set according to the first candidate positive sample and the supplementary first candidate positive sample, calculate a mean value and a variance of the first positive sample set, and obtain a second set threshold according to the mean value and the variance.

[0030] The sample assignment unit is configured to divide the prior boxes according to a size relationship between the first overlap value and the first set threshold and the second set threshold, respectively, to obtain a second positive sample and a second negative sample; the second positive sample and the second negative sample are used as training data of the convolutional neural network to obtain a target detection network model; and the target detection network model is used for multi-scale target detection in the autonomous driving scene.

[0031] Optionally, the determination of the plurality of prior boxes and prediction boxes based on the feature maps specifically comprises:

[0032] a plurality of prior boxes are set at the center of each pixel point on the feature map;

[0033] an output offset value is calculated according to the convolutional neural network, and a prediction box is determined according to the output offset value and the prior box.

[0034] Optionally, the second set threshold is a sum of the mean value and the variance.

[0035] Optionally, the sample distribution unit specifically comprises:

[0036] The second negative sample determination module is configured to divide the prior box with the first overlap value less than the first set threshold into a second negative sample.

[0037] The first positive and negative sample determination module is configured to divide the prior box with the first overlap value greater than the second set threshold into a second positive sample.

[0038] According to the embodiments of the present application, the following technical effects are provided:

[0039] The present application provides a multi-scale target detection training sample distribution method and system, the method comprising: obtaining a target data set in an automatic driving scene, performing feature extraction on the target data set through forward propagation of a convolutional neural network to obtain a plurality of feature maps, and determining a plurality of prior boxes and prediction boxes based on the feature maps; creating a first sample distribution list, calculating a first overlap value of the prior boxes and the real labeled boxes, and taking the index of the prior box with the first overlap value greater than a first set threshold as a first candidate positive sample; calculating a second overlap value of the prediction boxes and the real labeled boxes, and taking the index of the prediction box with the second overlap value greater than the first set threshold as a supplementary first candidate positive sample; constructing a first positive sample set according to the first candidate positive sample and the supplementary first candidate positive sample, calculating the mean and variance of the first positive sample set, and obtaining a second set threshold according to the mean and variance; dividing the prior boxes according to the size relationship of the first overlap value with the first set threshold and the second set threshold to obtain second positive samples and second negative samples; and taking the second positive samples and the second negative samples as training data of the convolutional neural network to obtain a target detection network model. The present application can solve the imbalance problem of multi-scale targets in the automatic driving scene in positive and negative sample distribution, and effectively improve the detection accuracy of multi-scale targets in the target detection network. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0041] Figure 1 The flowchart of the multi-scale target detection training sample distribution method of the present application;

[0042] Figure 2 The schematic diagram of constructing a prior box of the present application;

[0043] Figure 3 A schematic diagram of the relationship between the prediction frame and the real label frame of the present application;

[0044] Figure 4 A module structure schematic diagram of the training sample distribution system of the multi-scale target detection of the present application.

[0045] Symbol explanation:

[0046] The area of the shaded rectangular frame-AC, the area of the prediction frame-A, the area of the real label frame-B, the target data set acquisition unit-1, the prior frame and prediction frame determination unit-2, the first sample distribution list creation unit-3, the first candidate positive sample and the supplemented first candidate positive sample determination unit-4, the second set threshold determination unit-5, the sample distribution unit-6. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0048] The purpose of the present application is to provide a multi-scale target detection training sample distribution method and system, which can solve the imbalance problem of multi-scale targets in positive and negative sample distribution in the automatic driving scene, and can effectively improve the detection accuracy of multi-scale targets in the target detection network.

[0049] In order to make the above-mentioned purposes, characteristics and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0050] Generally speaking, the target detection based on anchor includes the following frames:

[0051] 1. ground-truth, abbreviated as gt, the frame generated by labeling is also called real label frame, which contains the position information and category information of the detected target.

[0052] 2. anchor, also known as prior frame, generally a frame set in advance on the feature map, each pixel point has multiple prior frames, each size and shape is different.

[0053] 3. predict, also known as prediction frame, the output position information of the prediction layer of the network.

[0054] As shown in Figure 1 The present application provides a multi-scale target detection training sample distribution method, which comprises:

[0055] S1: obtaining a target data set in an autonomous driving scene; the target data set includes target detection object images and corresponding label information; the target detection object images include target detection objects, and the target detection objects include pedestrians, vehicles, and traffic signs; the label information includes position coordinates and categories of the target detection objects in the target detection object images; wherein the data set is divided into a training set, a validation set, and a test set, and a plurality of target detection object images in the training set and label information corresponding to each of the images are sequentially input in a network training stage. Multi-scale refers to the existence of different scales in target objects such as pedestrians, vehicles, and traffic signs in the autonomous driving scene, and the scale difference is large.

[0056] S2: performing feature extraction on the target data set through forward propagation of a convolutional neural network to obtain a plurality of feature maps, and determining a plurality of prior boxes anchors and predicted boxes based on the feature maps. The feature map is a low-dimensional feature map, and the size is CxHxW, where C represents the number of channels, H is the height of the feature map, W is the width of the feature map, HxW is calculated from the input image and represents the feature information in the input image.

[0057] S3: creating a first sample assignment list, which represents the sample types to which the prior boxes are assigned; the sample types include first positive samples, first negative samples, and first ignored samples.

[0058] S4: calculating a first overlap value (Intersection over Union, IOU) of the prior boxes and a real annotation box gt, and taking the index of the prior box with the first overlap value greater than a first set threshold Y1 as a first candidate positive sample; calculating a second overlap value of the predicted box and the real annotation box, and taking the index of the predicted box with the second overlap value greater than the first set threshold as a supplementary first candidate positive sample.

[0059] S5: constructing a first positive sample set according to the first candidate positive sample and the supplementary first candidate positive sample, calculating the mean and variance of the first positive sample set, and obtaining a second set threshold according to the mean and the variance.

[0060] S6: dividing the prior boxes according to the size relationship between the first overlap value and the first set threshold and the second set threshold, respectively, to obtain second positive samples and second negative samples; the second positive samples and the second negative samples are used as training data of the convolutional neural network to obtain a target detection network model; and the target detection network model is used for multi-scale target detection in the autonomous driving scene.

[0061] Further, in step S2, the feature map is used to determine a plurality of prior boxes and predicted boxes, specifically including:

[0062] A plurality of prior boxes are set at the center of each pixel point on the feature map, as shown in Figure 2 Figure 2 The entire figure corresponds to the low-dimensional feature map finally output, and each small grid represents a pixel point. Each grid in the figure has generated anchors (more than two anchors shown in the figure).

[0063] The output offset value is calculated according to the convolutional neural network, and the predicted box is determined according to the output offset value and the prior box.

[0064] Specifically, anchors are set under a specified low-dimensional feature size HxW, and the number of anchors set at the center of each pixel point is K. Therefore, the total number of anchors N = HxWxK, and the predicted box predict is calculated according to the output offset value of the convolutional neural network and the coordinate value of the anchor. The number of predicted boxes predict is equal to the number of anchors.

[0065] Further, in step S3, a first sample assignment list is created, all elements of which are 0 and the size is N. This assignment list mainly indicates which category of anchor is assigned as a positive, negative, or ignored sample. It is previously stipulated that the 0 element in the list represents a negative sample, the -1 element represents an ignored sample, and the positive number represents a positive sample and its target class. The output of the target detection network is to frame the target in the image and mark its class. The purpose of dividing the anchor into positive, negative, and ignored samples is to train the network. Finally, the network will regress the anchor frame of the positive sample to mark the target object in the target detection image.

[0066] Further, in step S5, the second set threshold is the sum of the mean and the variance, i.e. Y2 = Mean + Var, Mean represents the mean, Var represents the variance, and Y2 represents the second set threshold. The mean can help to screen out data with large IoU values in the positive sample set, and the variance describes the dispersion degree of the IoU values in the positive sample set. The threshold Y2 calculated for each gt is different, which can adaptively screen out suitable anchors.

[0067] Further, in step S6, the prior boxes are divided according to the size relationship between the first overlap value, the first set threshold, and the second set threshold to obtain second positive samples and second negative samples, specifically including:

[0068] The prior boxes with the first overlap value less than the first set threshold are divided into second negative samples. ​

[0069] divide the prior box whose first overlap value is greater than the second set threshold value into a second positive sample.

[0070] divide the prior box whose first overlap value is between the first set threshold value and the second set threshold value as an ignored sample.

[0071] Further, the second positive sample and the second negative sample participate in the calculation of the subsequent classification loss, and the second positive sample also participates in the calculation of the regression loss, and the ignored sample does not participate in any calculation.

[0072] Further, the method further comprises: calculating a total loss function according to the second positive sample and the second negative sample; the total loss function comprises a classification loss function and a regression loss function; iteratively training the convolutional neural network using the second positive sample and the second negative sample, and optimizing the total loss function, and updating the parameters of the convolutional neural network through back propagation, and finally obtaining a target detection network model with good performance for target detection in an autonomous driving scenario.

[0073] The classification loss function L(P t ) has an expression as follows:

[0074] L(P t )=-α t (1-P t ) γ log(P t );

[0075]

[0076] wherein P t represents the closeness of the model prediction result to the real label box, P t is in the range of 0-1, and represents the closeness of the model prediction to the gt, i.e., the category y, and the greater the pt, the closer to the category y, i.e., the more accurate the classification; γ is an adjustable factor, and is in the range of 0-5, and is used to reduce the loss contribution of simple and easy-to-divide samples (P tThe greater, the smaller γ), to control the number imbalance of simple and difficult-to-distinguish samples; the target detection network generates a confidence parameter when predicting which category the positive sample belongs to, and a high confidence indicates that the model can easily classify it correctly, which is a simple sample, and a low confidence indicates a difficult-to-distinguish sample. For example, when a person observes an image, if he can clearly judge that the target is a certain category, it is a simple sample, but if the image is very blurred and it is difficult to judge that the target belongs to a certain category, it is a difficult-to-distinguish sample. αt is the weight ratio value between positive and negative samples, and the purpose is to balance the weight ratio between positive and negative samples (because the number of positive samples in an image is generally much less than that of negative samples, which will cause the detection model to tend to detect negative samples, resulting in poor detection effect). y is the category, indicating the detection target or background area in the image, and y takes the value of 1 or -1; p is the prediction output value of the target detection model. For example, if the input calculation is a positive sample, then y = 1, and if the prediction output probability of the network is p = 0.8 (good detection effect), Pt = P = 0.8. If the calculation is a negative sample, then y = -1, and if the prediction output is p = 0.8 (poor effect, detecting background as target), Pt = 1 - P = 0.2, Pt represents the closeness of the model prediction to the gt category, and the prediction is correct. Pt is high, and the prediction is wrong. Pt is small. When calculating the classification loss, the positive sample represents the detection target area, and the negative sample represents the background area. For example, if the current calculation is a positive sample, then y = 1, and if the current calculation is a negative sample, then y = -1.

[0077] The regression loss function L GIoU The expression is:

[0078] L GIoU = 1 - GIoU;

[0079]

[0080]

[0081] Wherein, A c represents the closure of the prediction box and the real label box, and the closure represents the smallest rectangle parallel to the coordinate axis that encloses the two rectangular areas. GIoU represents the overlap degree, IoU represents the intersection over union of the prediction box and the real label box, A represents the area of the prediction box, B represents the area of the real label box, and μ represents the area of the union of the prediction box and the real label box. GIoU can be understood as the distance between the two target boxes. The greater the GIOU, the closer the two boxes, and vice versa, the smaller the distance. The significance of the regression loss is to calculate the gap between the prediction box and the real label box, and then try to reduce the gap between the two, so that the final prediction output box can be closer to the real target box.

[0082] As Figure 3As shown, Figure 3 is a case of the predicted box predict and the target real box gt, A is the real target bounding box ( Figure 3 The rectangle marked as A in the figure), B is the predicted box ( Figure 3 The outermost shaded border is the smallest rectangle that encloses rectangles A and B. C The area of ​​the shaded rectangle corresponds to μ, the union area of ​​rectangles A and B, and IoU is the intersection-over-union ratio of rectangles A and B. Figure 3 In the above example, A represents the area of ​​the prediction box, B represents the area of ​​the gt box, and IOU is the intersection and union of the prediction box and the gt box, which is used to measure the distance between the two boxes.

[0083] Furthermore, the total loss function is L = L(P t )+L GIoU , iteratively train the convolutional neural network, use the stochastic gradient descent algorithm to optimize the total loss function, and backpropagate to update the network parameters.

[0084] To achieve the above purpose, if Figure 4 As shown, the present invention also provides a training sample allocation system for multi-scale target detection, which includes: a target data set acquisition unit 1, a priori box and prediction box determination unit 2, a first sample allocation list creation unit 3, a first candidate positive sample and supplementary first candidate positive sample determination unit 4, a second set threshold determination unit 5 and a sample allocation unit 6.

[0085] The target data set acquisition unit 1 is used to acquire a target data set in an autonomous driving scenario; the target data set includes a target detection object image and corresponding label information; the target detection object image includes target detection objects, and the target detection objects include pedestrians, vehicles and traffic signs; the label information includes the position coordinates and category of the target detection objects in the target detection object image.

[0086] The prior frame and prediction frame determination unit 2 is used to extract features of the target data set through the forward propagation of the convolutional neural network to obtain multiple feature maps, and determine multiple prior frames and prediction frames based on the feature maps.

[0087] The first sample allocation list creating unit 3 is configured to create a first sample allocation list, where the first sample allocation list indicates the sample types to which the priori boxes are allocated; the sample types include a first positive sample, a first negative sample, and a first ignored sample.

[0088] The first candidate positive sample and supplementary first candidate positive sample determination unit 4 is configured to calculate a first overlap value of the prior box and the real label box, and take the index of the prior box with the first overlap value greater than a first set threshold as a first candidate positive sample; and calculate a second overlap value of the predicted box and the real label box, and take the index of the predicted box with the second overlap value greater than the first set threshold as a supplementary first candidate positive sample.

[0089] The second set threshold determination unit 5 is configured to construct a first positive sample set according to the first candidate positive sample and the supplementary first candidate positive sample, calculate a mean value and a variance of the first positive sample set, and obtain a second set threshold according to the mean value and the variance.

[0090] The sample allocation unit 6 is configured to divide the prior box according to the size relationship between the first overlap value and the first set threshold and the second set threshold, respectively, to obtain a second positive sample and a second negative sample; the second positive sample and the second negative sample are used as training data of a convolutional neural network to obtain a target detection network model; and the target detection network model is used for multi-scale target detection in an automatic driving scene.

[0091] Further, the plurality of prior boxes and predicted boxes are determined based on the feature map, and specifically include:

[0092] A plurality of prior boxes are set at the center of each pixel point on the feature map.

[0093] An output offset value is calculated according to the convolutional neural network, and a predicted box is determined according to the output offset value and the prior box.

[0094] Further, the second set threshold is the sum of the mean value and the variance.

[0095] Further, the sample allocation unit 6 specifically includes:

[0096] The second negative sample determination module is configured to divide the prior box with the first overlap value less than the first set threshold into a second negative sample.

[0097] The first positive sample determination module is configured to divide the prior box with the first overlap value greater than the second set threshold into a second positive sample.

[0098] The principle of the application: in the distribution stage of the training sample, we use the prediction frame predict as supplementary information, and propose an adaptive matching strategy: calculate the IoU value of the preset anchor and the gt, select a part of the anchor with the highest IoU value as the candidate positive sample, at the same time, we use the IoU value of the prediction frame and the gt to filter out a part of the prediction frame with the highest IoU value as the supplementary candidate positive sample, then calculate a threshold according to the total candidate positive sample, and filter out the final positive and negative sample distribution set.

[0099] The screened positive and negative samples are used for the calculation of the loss function of the target detection network. Generally speaking, the number of negative samples in the image is much larger than that of positive samples, in order to prevent the imbalance of sample distribution from causing poor performance of the detection model on the few-sample class, we use Focal loss to calculate the classification loss. In the regression loss, in order to pay attention to the problem that the gap between the prediction frame predicate and the target real frame gt cannot be evaluated due to no overlap, we use GIoU loss to calculate the regression loss. The total loss function = classification loss + regression loss. Finally, the iterative training network is optimized, and the parameters of the target detection network are updated through back propagation.

[0100] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and the same and similar parts between various embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the related parts can be referred to the method part.

[0101] The principles and implementation modes of the application are described by applying specific examples in this paper, and the above embodiment description is only used to help understand the method and core idea of the application; at the same time, for the general technical personnel in the art, according to the idea of the application, the specific implementation mode and application range will be changed. In conclusion, the content of the specification should not be understood as the limitation of the application.

Claims

1. A training sample allocation method for multi-scale target detection, characterized in that: The method comprises: Obtain a target dataset in an autonomous driving scenario; the target dataset includes a target detection object image and corresponding label information; the target detection object image includes a target detection object, and the target detection object includes a pedestrian, a vehicle, and a traffic sign; the label information includes a position coordinate and a category of the target detection object in the target detection object image; The target data set is subjected to forward propagation of a convolutional neural network to perform feature extraction to obtain multiple feature maps, and multiple prior boxes and prediction boxes are determined based on the feature maps; Creating a first sample allocation list, where the first sample allocation list indicates sample types to which the priori boxes are assigned; the sample types include a first positive sample, a first negative sample, and a first ignored sample; Calculating a first overlap value between the prior frame and the true labeled frame, and taking the prior frame index whose first overlap value is greater than a first set threshold as a first candidate positive sample; calculating a second overlap value between the predicted frame and the true labeled frame, and taking the predicted frame index whose second overlap value is greater than the first set threshold as a supplementary first candidate positive sample; Constructing a first positive sample set based on the first candidate positive sample and the supplemented first candidate positive sample, calculating a mean and a variance of the first positive sample set, and obtaining a second set threshold based on the mean and the variance; According to the relationship between the first overlap value and the first set threshold and the second set threshold, the prior frame is divided to obtain a second positive sample and a second negative sample; the second positive sample and the second negative sample are used as training data for a convolutional neural network to obtain a target detection network model; the target detection network model is used to perform multi-scale target detection in an autonomous driving scenario.

2. The method for allocating training samples for multi-scale target detection according to claim 1, wherein: The determining of a plurality of prior frames and prediction frames based on the feature map specifically includes: Set multiple prior boxes at the center of each pixel on the feature map; An output offset value is calculated according to the convolutional neural network, and a prediction box is determined according to the output offset value and the prior box.

3. The method for allocating training samples for multi-scale target detection according to claim 1, wherein: The second set threshold is the sum of the mean and the variance.

4. The method for allocating training samples for multi-scale target detection according to claim 1, wherein: The step of dividing the priori frame according to the relationship between the first overlap value and the first set threshold and the second set threshold to obtain a second positive sample and a second negative sample specifically includes: Classify the priori boxes whose first overlap value is less than the first set threshold as second negative samples; The priori boxes whose first overlap value is greater than the second set threshold are classified as second positive samples.

5. A training sample allocation system for multi-scale object detection, characterized in that: The system comprises: a target data set acquisition unit, configured to acquire a target data set in an autonomous driving scenario; the target data set comprising a target detection object image and corresponding label information; the target detection object image comprising a target detection object, which may include pedestrians, vehicles, and traffic signs; and the label information comprising the position coordinates and category of the target detection object in the target detection object image; A priori box and prediction box determination unit is used to extract features of the target data set through forward propagation of a convolutional neural network to obtain multiple feature maps, and determine multiple prior boxes and prediction boxes based on the feature maps; A first sample allocation list creating unit, configured to create a first sample allocation list, wherein the first sample allocation list indicates sample types to which the priori boxes are allocated; the sample types include a first positive sample, a first negative sample, and a first ignored sample; A first candidate positive sample and supplementary first candidate positive sample determination unit is configured to calculate a first overlap value between the priori box and the true labeled box, and use the priori box index whose first overlap value is greater than a first set threshold as the first candidate positive sample; and to calculate a second overlap value between the predicted box and the true labeled box, and use the predicted box index whose second overlap value is greater than the first set threshold as the supplementary first candidate positive sample; a second set threshold determination unit, configured to construct a first positive sample set based on the first candidate positive sample and the supplemented first candidate positive sample, calculate a mean and a variance of the first positive sample set, and obtain a second set threshold based on the mean and the variance; A sample allocation unit is used to divide the prior frame according to the size relationship between the first overlap value and the first set threshold and the second set threshold, respectively, to obtain a second positive sample and a second negative sample; the second positive sample and the second negative sample are used as training data for a convolutional neural network to obtain a target detection network model; the target detection network model is used to perform multi-scale target detection in an autonomous driving scenario.

6. The training sample allocation system for multi-scale target detection according to claim 5, characterized in that: The determining of a plurality of prior frames and prediction frames based on the feature map specifically includes: Set multiple prior boxes at the center of each pixel on the feature map; An output offset value is calculated according to the convolutional neural network, and a prediction box is determined according to the output offset value and the prior box.

7. The training sample allocation system for multi-scale target detection according to claim 5, characterized in that: The second set threshold is the sum of the mean and the variance.

8. The training sample allocation system for multi-scale target detection according to claim 5, characterized in that: The sample distribution unit specifically includes: A second negative sample determination module, configured to classify the priori boxes whose first overlap value is less than the first set threshold as second negative samples; The first positive and negative sample determination module is configured to classify the priori boxes whose first overlap value is greater than the second set threshold as second positive samples.

Images