Method, device and storage medium for optimizing training data set of pedestrian detection model

Abstract

The application provides a method for optimizing a training data set for a pedestrian detection model, the method comprising: obtaining original pedestrian data, the original pedestrian data comprising at least image data, pedestrian position data and bounding box size data; obtaining data distribution characteristics of the original pedestrian data; training a pedestrian detection model using the original pedestrian data, thereby obtaining original detection results; training a data generation model using the original pedestrian data, the data generation model being a generative model; generating pedestrian data with specific bounding box size data as additional pedestrian data using the data generation model according to the data distribution characteristics and the original detection results; and constructing a training data set for the pedestrian detection model based on the original pedestrian data and the additional pedestrian data. The application also relates to a corresponding optimization device and a computer readable storage medium. Through the application, the training data set can be optimized to obtain a pedestrian detection model with higher accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to a method for optimizing a training dataset for a pedestrian detection model, an optimization device for optimizing a training dataset for a pedestrian detection model and a computer-readable storage medium. BACKGROUND

[0002] Pedestrian detection technology based on computer vision can detect the position of pedestrians and the like by processing image or video information collected by a camera. Pedestrian detection is of great significance and is a key technology in vehicle assisted driving, vehicle automatic driving, intelligent video monitoring and human behavior analysis and the like. In recent years, pedestrian detection technology has been increasingly valued by the academic and industrial communities.

[0003] Traditional pedestrian detection is mainly realized by HOG+SVM, i.e. extracting the HOG (Histogram of Oriented Gradient) features of an image and using a support vector machine (SVM) for classification. This method uses artificial features to calculate the gradient direction histogram of an image in combination with an SVM classifier to detect pedestrians. Although it achieves good results for upright posture pedestrian images, it is not good for pedestrians occluded with each other and in non-upright states, and has poor generalization ability. In recent years, convolutional neural networks have produced better and better results in the field of image recognition and detection, and the accuracy of a series of target detection models such as RCNN (Regions with CNN features) far exceeds that of traditional methods.

[0004] However, current data-driven deep learning pedestrian detection methods still face some problems in the automatic driving scenario. For example, the pedestrian detection model trained based on original pedestrian data is not accurate enough for the detection of pedestrians in some special postures. The situation may occur that the bounding box as a detection result cannot completely contain the outstretched limbs, resulting in the loss of important posture information of the pedestrian.

[0005] The prior art still has many deficiencies in the accuracy of pedestrian detection models. SUMMARY

[0006] The purpose of the present application is to provide an improved method for optimizing a training dataset for a pedestrian detection model and a corresponding optimization device and computer-readable storage medium, thereby optimizing the training dataset for the pedestrian detection model to obtain a pedestrian detection model with higher accuracy.

[0007] According to a first aspect of the present application, a method for optimizing a training dataset for a pedestrian detection model is provided, wherein the method comprises the following steps:

[0008] obtaining original pedestrian data, the original pedestrian data comprising at least image data, pedestrian position data and bounding box size data;

[0009] obtaining data distribution characteristics of the original pedestrian data;

[0010] training a pedestrian detection model using the original pedestrian data, thereby obtaining original detection results;

[0011] training a data generation model using the original pedestrian data, the data generation model being a generative model;

[0012] generating pedestrian data with specific bounding box size data as additional pedestrian data using the data generation model according to the data distribution characteristics and the original detection results; and

[0013] constructing a training data set for the pedestrian detection model based on the original pedestrian data and the additional pedestrian data.

[0014] According to an embodiment of the present application, the method further comprises the following steps:

[0015] evaluating the original detection results with respect to the bounding box size, and obtaining bounding box sizes with relatively low detection accuracy; and

[0016] determining the specific bounding box size data according to the bounding box sizes with relatively low detection accuracy.

[0017] According to an embodiment of the present application, the bounding box sizes with relatively low detection accuracy are determined as bounding box sizes with detection accuracy lower than an average detection accuracy.

[0018] According to an embodiment of the present application, the method further comprises the following steps:

[0019] analyzing the data distribution characteristics of the original pedestrian data with respect to the bounding box size data, and obtaining bounding box sizes with relatively small data quantity; and

[0020] determining the specific bounding box size data according to the bounding box sizes with relatively small data quantity.

[0021] According to an embodiment of the present application, the bounding box sizes with relatively small data quantity are determined as bounding box sizes with a difference from an expected value of the bounding box size data greater than a standard deviation of the bounding box size data.

[0022] According to an embodiment of the present application, the step of determining the specific bounding box size data comprises determining the specific bounding box size data as the bounding box sizes with relatively low detection accuracy and relatively small data quantity.

[0023] According to an embodiment of the present invention, the step of generating pedestrian data with specific bounding box size data using a data generation model includes: making the pedestrian position data of the generated pedestrian data conform to a normal distribution.

[0024] According to one embodiment of the present invention, the method further includes: training a pedestrian detection model using the training dataset to obtain detection results; and determining the training dataset as an optimized training data set when the detection results are better than the original detection results, wherein:

[0025] Adjust the expected value and / or standard deviation of the normal distribution to obtain different training datasets until a detection result superior to the original detection result is obtained; and / or

[0026] Different subsets of additional pedestrian data are selected to construct different training datasets from the original pedestrian data until detection results that are superior to the original detection results are obtained.

[0027] According to a second aspect of the present invention, an optimization apparatus is provided for optimizing a training dataset for a pedestrian detection model, the optimization apparatus being configured to perform the method according to the present invention, wherein the optimization apparatus comprises:

[0028] The pedestrian data distribution analysis unit is configured to acquire the data distribution characteristics of the raw pedestrian data.

[0029] The detection and evaluation unit is configured to train a pedestrian detection model using raw pedestrian data to obtain raw detection results.

[0030] An extreme pedestrian pose mining unit, configured to determine specific bounding box size data based on data distribution characteristics and raw detection results; and

[0031] The data generation unit is configured to train a data generation model using raw pedestrian data and to generate pedestrian data with specific bounding box dimensions using the data generation model, thereby constructing a training dataset for the pedestrian detection model.

[0032] According to a third aspect of the present invention, a computer-readable storage medium is provided, wherein a calculator program is stored therein, and when the computer program is executed by one or more processors, the processors are capable of performing the method according to the present invention.

[0033] The positive effect of the present application is that, based on the data distribution characteristics of the original pedestrian data and the original detection results, the pedestrian data corresponding to specific postures is generated by using the generation network, so as to achieve the purpose of optimizing the training data set. The present application is particularly based on the following finding: some special postures, such as the posture of pedestrians with limbs stretched out, are less distributed in the naturally collected original pedestrian data, so the pedestrian detection model trained based on such unbalanced original pedestrian data often has inaccurate detection results for such special postures. By using the generation network to generate pedestrian data corresponding to such special postures, the training data set can be optimized in a targeted manner to train a pedestrian detection model with higher accuracy. Further, more complete and accurate information can be provided for subsequent pedestrian posture estimation and intention recognition. In addition, the collection and labeling cost of pedestrian data can be reduced. BRIEF DESCRIPTION OF DRAWINGS

[0034] The principles, features and advantages of the present application can be better understood by referring to the following detailed description of the application in conjunction with the accompanying drawings in which:

[0035] Figure 1 A flowchart of a method for optimizing a training data set for a pedestrian detection model according to an example embodiment of the present application is shown;

[0036] Figure 2 A flowchart of part of the steps in the method according to an example embodiment of the present application is shown;

[0037] Figure 3 A flowchart of part of the steps in the method according to an example embodiment of the present application is shown; and

[0038] Figure 4 A schematic diagram of an optimization device according to an example embodiment of the present application is shown. DETAILED DESCRIPTION

[0039] In order to make the technical problems to be solved by the present application, the technical solutions and the beneficial technical effects more clear, the present application will be further described in detail below in conjunction with the drawings and multiple example embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the protection scope of the present application.

[0040] Figure 1 A flowchart of a method for optimizing a training data set for a pedestrian detection model according to an example embodiment of the present application is shown. The method for optimizing a training data set for a pedestrian detection model can include the following steps:

[0041] S1: obtaining original pedestrian data, the original pedestrian data including at least image data, pedestrian position data and bounding box size data;

[0042] S2: obtaining data distribution characteristics of the original pedestrian data;

[0043] S3: training a pedestrian detection model using the original pedestrian data, thereby obtaining original detection results;

[0044] S4: training a data generation model using the original pedestrian data, the data generation model being a generative model;

[0045] S5: generating pedestrian data with specific bounding box size data as additional pedestrian data using the data generation model according to the data distribution characteristics and the original detection results; and

[0046] S6: constructing a training data set for the pedestrian detection model based on the original pedestrian data and the additional pedestrian data.

[0047] The image data of the original pedestrian data is, for example, real image data collected by a camera device. The pedestrian position data and the bounding box size data can be obtained, for example, by manual labeling.

[0048] It should be understood that the execution order of the above steps is not limited by the order of the description of the steps above, but at least part of the above steps can be executed in other orders.

[0049] According to the present application, specific data is generated using a generative model, which can solve the problem of data imbalance and reduce the cost of data collection and labeling. The training data set constructed based on the original pedestrian data and the additional pedestrian data can improve the accuracy of the pedestrian detection model. Thus, more complete and accurate bounding box information can be provided for subsequent perception tasks such as pedestrian pose estimation and intention recognition of autonomous driving.

[0050] Figure 2 A flowchart showing part of the steps in the method according to an exemplary embodiment of the present application is shown.

[0051] In this embodiment, the method further includes the following steps: after acquiring the original pedestrian data, analyzing the data distribution characteristics of the original pedestrian data based on the bounding box size data to obtain bounding box sizes with a relatively small amount of data. The specific bounding box size data can be determined based on the bounding box sizes with a relatively small amount of data. For example, the original pedestrian data can be segmented according to the aspect ratio of the bounding box (width / height), and data distribution analysis can be performed based on the aspect ratio of the pedestrian bounding boxes. Although pedestrians are only a specific category, due to the diversity of human postures within the pedestrian category, the data distribution based on the bounding box size will cover a certain area, rather than being concentrated in a specific segment. In autonomous driving scenarios, pedestrians are mainly in standing or walking postures, while in the original pedestrian data, data on postures such as outstretched arms are relatively scarce. By analyzing the data distribution characteristics of the original pedestrian data based on the bounding box size data, it is beneficial to more effectively perform data augmentation on pedestrian data representing specific postures.

[0052] For example, a bounding box size with a relatively small amount of data can be defined as a bounding box size that differs from the expected value of the bounding box size data by more than the standard deviation of the bounding box size data. It should be understood that "relatively small amount of data" can also be defined in other ways, such as defining it as the proportion of data in the total amount of data being less than a predetermined percentage.

[0053] like Figure 2 As shown, the method may further include the following steps: after training a pedestrian detection model using the original pedestrian data to obtain the original detection results, evaluating the original detection results based on the bounding box size to obtain a bounding box size with relatively low detection accuracy. The specific bounding box size data can be determined, in particular, based on the bounding box size with relatively low detection accuracy.

[0054] In this embodiment, the pedestrian detection model is a convolutional neural network model composed of convolutional layers and activation layers. For example, the YOLO model can be used as the pedestrian detection model. The input of the pedestrian detection model is image data, and the output of the pedestrian detection model may include pedestrian position and bounding box size. Through the training process, the network parameters are continuously updated and iterated, and the performance of pedestrian detection will reach its optimal level. The original pedestrian data is segmented according to the aspect ratio of the bounding box. Using the trained pedestrian detection model, the detection results of different segments are evaluated, and the accuracy of each segment is obtained and output.

[0055] For example, a bounding box size with relatively low detection accuracy is defined as a bounding box size with detection accuracy below the average detection accuracy. It should be understood that "relatively low detection accuracy" can also be defined in other ways, such as as detection accuracy below a predetermined threshold.

[0056] existFigure 2 In the exemplary embodiment shown, according to the analysis of the data distribution characteristics and the evaluation of the original detection results, the overlapping part of the size segment with relatively less data quantity and the size segment with relatively lower detection accuracy is the data extreme imbalance segment, and the pedestrian posture corresponding to the segment is defined as the extreme pedestrian posture. In this embodiment, it can be concluded that postures such as arms spread are extreme pedestrian postures, which have less data quantity and lower detection accuracy, but play an important role in subsequent tasks such as intent recognition. The step of determining the specific bounding box size data can particularly include determining the specific bounding box size data as the bounding box size with relatively less data quantity and relatively lower detection accuracy. In this way, more data of extreme pedestrian postures can be generated by the generative model guided by the extreme pedestrian postures, so as to more targetedly optimize the training data set for the pedestrian detection model.

[0057] It should be understood that the specific bounding box size data can also be determined according to the bounding box size with relatively less data quantity or the bounding box size with relatively lower detection accuracy alone.

[0058] Figure 3 A flowchart showing part of the steps in the method according to an exemplary embodiment of the application is shown.

[0059] As Figure 3 shown, after determining the data extreme imbalance segment corresponding to the extreme pedestrian posture, a data generation model can be used to generate pedestrian data of such postures guided by the extreme pedestrian postures. The data generation model is, for example, a generative adversarial neural network model. By training the data generation model with the original pedestrian data, the network parameters of the generative model are constantly updated. The optimal generative network model in the training process is used to generate more pedestrian data in the data extreme imbalance segment as additional pedestrian data. In this process, the pedestrian position data of the generated pedestrian data conforms to the normal distribution:

[0060]

[0061] wherein y is the pedestrian position data, μ is the expected value, and σ is the standard deviation. The expected value μ and the standard deviation σ can be set as hyperparameters.

[0062] Based on the original pedestrian data and the additional pedestrian data, a new training data set for the pedestrian detection model can be constructed. For example, a subset of the additional pedestrian data can be selected to enhance the original pedestrian data.

[0063] A new pedestrian detection model is trained using a new training data set, and the detection result based on the pedestrian detection model is evaluated. When the detection result is better than the original detection result, the training data set is determined as an optimized training data set.

[0064] For example, different subsets of additional pedestrian data can be repeatedly selected to build different new training data sets until the detection result is better than the original detection result.

[0065] For example, the mean value μ and the standard deviation σ of the pedestrian position data of the generated pedestrian data can be repeatedly set to build different new training data sets until the detection result is better than the original detection result.

[0066] When the detection result of the pedestrian detection model corresponding to the new training data set is better than the original detection result, the construction of the new training data set is stopped, and the current training data set is output as the optimized training data set.

[0067] Figure 4 A schematic diagram of an optimization device for optimizing a training data set for a pedestrian detection model according to an example embodiment of the present application is shown. The optimization device is configured to be able to perform the method according to the present application. The optimization device comprises:

[0068] a pedestrian data distribution analysis unit 1 configured to be able to obtain data distribution characteristics of original pedestrian data;

[0069] a detection evaluation unit 2 configured to be able to train a pedestrian detection model using the original pedestrian data, thereby obtaining an original detection result;

[0070] an extreme pedestrian posture mining unit 3 configured to be able to determine specific bounding box size data according to the data distribution characteristics and the original detection result; and

[0071] a data generation unit 4 configured to be able to train a data generation model using the original pedestrian data, and to generate pedestrian data with the specific bounding box size data using the data generation model, thereby building a training data set for the pedestrian detection model.

[0072] As Figure 4 shown, the detection evaluation unit 2 can also be configured to train a pedestrian detection model using the training data set, thereby obtaining a detection result. When the detection result is better than the original detection result, the training data set can be determined as an optimized training data set.

[0073] The application also relates to a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the processors can execute the method according to the application. The computer readable storage medium can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory device. The processor can be a central processing unit (CPU), and can also be another general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc. The general-purpose processor can be a microprocessor, or the processor can also be any conventional processor, etc.

[0074] Although specific embodiments of the application are described in detail herein, many modifications are possible (e.g., variations in sizes, dimensions, structures, shapes and proportions of the preferred embodiments, variations in preferred materials, methods of assembly, methods of use, colors, components used, colors, layouts, offsets, orientations, etc.) without materially departing from the scope and nature of the subject application. All such modifications are intended to be part of the various aspects of the subject application. Changes in, or alterations to, selected features where applicable, are contemplated as being within the scope and spirit of the subject application.

Claims

1. A method for optimizing a training dataset for a pedestrian detection model, wherein, The method comprises the following steps: obtaining original pedestrian data, the original pedestrian data comprising at least image data, pedestrian position data and bounding box size data; segmenting the original pedestrian data according to the aspect ratio of the bounding box, and analyzing the data distribution of the aspect ratio of the pedestrian bounding box to obtain the data distribution characteristics; training a pedestrian detection model using the original pedestrian data to obtain original detection results, evaluating the detection results of the segmented data of different segments segmented according to the aspect ratio of the bounding box, and obtaining and outputting the accuracy of each segment; training a data generation model using the original pedestrian data, the data generation model being a generative model; according to the data distribution characteristics and the accuracy of each segment, the overlapping part of the size segment of the bounding box size with relatively less data quantity and the size segment of the bounding box size with relatively low detection accuracy is taken as the segment corresponding to the extreme pedestrian posture, and the data generation model is used to generate pedestrian data with specific bounding box size data as extreme pedestrian posture data, wherein the bounding box size with relatively low detection accuracy is determined as the bounding box size with detection accuracy lower than the average detection accuracy, and the bounding box size with relatively less data quantity is determined as the bounding box size with a difference greater than the standard deviation of the bounding box size data from the expected value of the bounding box size data; and based on the original pedestrian data and the extreme pedestrian posture data, a training data set for the pedestrian detection model is constructed.

2. The method of claim 1, wherein, The step of generating pedestrian data with specific bounding box size data using the data generation model comprises: making the pedestrian position data of the generated pedestrian data conform to a normal distribution.

3. The method of claim 2, wherein, The method further comprises: training a pedestrian detection model using the training data set to obtain detection results, and determining the training data set as an optimized training data set when the detection results are better than the original detection results, wherein: the expected value and / or the standard deviation of the normal distribution are adjusted to obtain different training data sets until detection results better than the original detection results are obtained; and / or different subsets of extreme pedestrian data are selected to construct different training data sets with the original pedestrian data until detection results better than the original detection results are obtained.

4. An optimization device for optimizing a training dataset for a pedestrian detection model, the optimization device configured to perform the method according to any one of claims 1-3, wherein, The optimization device comprises: a pedestrian data distribution analysis unit (1) configured to obtain the data distribution characteristics of the original pedestrian data; a detection evaluation unit (2) configured to train a pedestrian detection model using the original pedestrian data to obtain original detection results; an extreme pedestrian posture mining unit (3) configured to determine specific bounding box size data according to the data distribution characteristics and the original detection results; and a data generation unit (4) configured to train a data generation model using the original pedestrian data, and to generate pedestrian data with specific bounding box size data using the data generation model to construct a training data set for the pedestrian detection model.

5. A computer readable storage medium, wherein, The computer readable storage medium stores a computer program, when the computer program is executed by one or more processors, the processor can execute the method according to any one of claims 1-3. The computer readable storage medium stores a computer program, when the computer program is executed by one or more processors, the processor can execute the method according to any one of claims 1-3.

Images