Intelligent driving image recognition algorithm training data set intelligent production method

Abstract

The application provides a kind of intelligent driving image recognition algorithm training data set intelligent production method, first acquisition multi-source data set, multi-source data set composition factor is decomposed gradually, determine the discrete value of multi-source data set composition factor, generate sample set based on the genetic model constructed in advance, and calculate the similarity of sample set based on the effectiveness evaluation model constructed, and determine the effectiveness of sample set according to sample set similarity index, if the effectiveness of the sample set is reasonable, the genetic model stops iteration, and the data set with known effectiveness is obtained, otherwise, the genetic model will continue to select, cross, mutation operation, until the data set with reasonable effectiveness is obtained. The iteration process of genetic model can eliminate invalid data set and improve the production quality of data set.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving technology, and in particular to an intelligent production method for training datasets for intelligent driving image recognition algorithms. Background Technology

[0002] The dataset consists of images and labels, serving as input for training intelligent driving recognition algorithms. The availability of large-scale labeled datasets is a crucial prerequisite for the successful training of intelligent driving recognition algorithms.

[0003] Currently, most autonomous driving datasets are obtained through real-vehicle data collection or manual image capture. This often requires significant manpower and resources to collect images under varying conditions, including different times of day, weather, roads, and traffic situations. The images are then manually labeled to identify targets. While manually collected and labeled datasets ensure the most realistic scene images are input into the learning model, they are time-consuming and labor-intensive. Furthermore, due to the complexity of the actual operating environment of vehicles, even continuous daily data collection cannot cover all operating conditions, thus this method has significant limitations.

[0004] Given the difficulty of collecting datasets in the real world, some companies and researchers have begun using simulation software to construct datasets. Current software, such as Unity 3D and CARLA, can generate virtual scenes with high visual fidelity, including highways, cities, residences, urban areas, and indoor parking garages. These environments also feature more environmental variations that are relatively difficult and costly to replicate in the real world, such as different times of day and weather conditions, different traffic / obstacles, and varying road surface quality. Using simulation software to generate datasets can effectively reduce the consumption of significant manpower and resources, and can ensure a certain level of coverage through enumeration. However, generating datasets through enumeration is somewhat arbitrary and makes it impossible to determine whether the generated sample set is useful for model learning.

[0005] To increase the amount of data in existing datasets, data augmentation techniques have been proposed. Data augmentation techniques can be broadly categorized into supervised and unsupervised data augmentation. Supervised data augmentation primarily uses methods such as geometric transformations, color transformations, and mosaic enhancement to physically transform images. Unsupervised data augmentation techniques mainly enhance datasets by learning their distribution or the requirements of the current task. Data augmentation techniques can increase the coverage and diversity of datasets, but they also generate a large amount of invalid data, such as unreasonable or duplicate images. Summary of the Invention

[0006] The purpose of this invention is to address the technical deficiencies in the existing technology by providing a method for intelligent production of datasets for training intelligent driving image recognition algorithms.

[0007] The technical solution adopted to achieve the purpose of this invention is:

[0008] A method for intelligently generating datasets for training image recognition algorithms in intelligent driving includes the following steps:

[0009] Step S1: Collect multi-source datasets of intelligent driving images and determine the constituent factors of the multi-source datasets;

[0010] Step S2: Decompose the constituent factors of the multi-source dataset from step S1 step by step to determine the discrete values ​​of the constituent factors of the multi-source dataset.

[0011] Step S3, Dataset Validity Evaluation Model: The dataset generated by the genetic model in step S4 is randomly divided into n sample sets. Based on the constructed validity evaluation model, the feature similarity of each sample set is calculated using the discrete values ​​of the constituent factors of the n sample sets. The validity of the sample set is determined according to the feature similarity index of the sample set. When all sample sets are valid, the genetic model terminates the operation. When any sample set is invalid, the genetic model continues the iterative operation of step S4.

[0012] Step S4, Genetic model automatically generates dataset: The constituent factors of the multi-source dataset obtained in step S2 are randomly combined as the initial sample set. Then, the dataset is generated based on the pre-built genetic model. The iteration continues or terminates according to the iteration termination instruction given in step S3. When the iteration continues, the constituent factors of the obtained dataset are subjected to "selection", "crossover" and "mutation" operations to generate offspring sample images. The dataset with reasonable validity obtained when the iteration terminates is the final dataset S*.

[0013] Step S5, posterior self-learning mechanism: Based on the contribution of the final dataset S* obtained in step S4 to the image recognition model, optimize the probabilities of "selection", "crossover" and "mutation" operations in the genetic model in step S4 and the effectiveness evaluation model in step S3.

[0014] In the above technical solution, in step S2, the constituent factors of the multi-source dataset are decomposed step by step, and after determining the discrete values ​​of the constituent factors of the multi-source dataset, a tree-like analysis model of the constituent factors of the dataset is established.

[0015] In the above technical solution, step S2 describes the relationships between the constituent elements of the multi-source dataset in the form of sets:

[0016] F i,j ={F i+1,k ,k∈S i+1}

[0017] i∈{1,2,3,...,L-1},j∈{1,2,3,...,L i}

[0018] Among them, F i,j S represents the j-th factor in the i-th layer. i+1 It is all belonging to F i,j The set of indices of a node's child nodes, where L is the total number of levels. i Let be the total number of nodes in the i-th layer.

[0019] In the above technical solution, in step S3, the eigenvector variance D(t) of the discrete values ​​of the constituent factors is used. m As a feature similarity index, the feature similarity index is denoted as:

[0020]

[0021] Among them, t m For the feature vector of the sample set, E represents the expected value, where E is the average value of the feature vectors of the sample set. A dataset consisting of a sample set with a larger feature vector variance is more conducive to the image recognition model learning more features. When the feature vector variance is greater than a preset threshold, the sample set is valid; when the feature vector variance is less than the preset threshold, the sample set is invalid.

[0022] In the above technical solution, in step S4, "selection" of the sample means the selection of sample constituent elements and sample transformation, "cross" means the recombination of sample constituent elements, and "mutation" means the transformation of the sample image, including but not limited to geometric transformation, color transformation, noise transformation and combination transformation.

[0023] In the above technical solution, in step S5, based on the contribution of the final dataset S* obtained in S4 to the image recognition model, that is, the performance of the image recognition model trained by the generated final dataset S* in the actual image recognition task, the probability of multi-dimensional feature selection, crossover, mutation and similarity evaluation methods are continuously corrected to realize the "self-growth" of the dataset generation algorithm.

[0024] Compared with the prior art, the beneficial effects of the present invention are:

[0025] This invention first collects multi-source datasets, decomposes the constituent factors of the multi-source datasets level by level, determines the discrete values ​​of the constituent factors, generates a sample set based on a pre-constructed genetic model, and calculates the similarity of the sample sets based on a constructed validity evaluation model. The validity of the sample sets is determined based on the sample set similarity index. If the validity of the sample set is reasonable, the genetic model stops iterating, and a dataset with known validity is obtained. Otherwise, the genetic model continues to perform selection, crossover, and mutation operations until a dataset with reasonable validity is obtained. The iterative process of the genetic model can eliminate invalid datasets, improving the quality of the generated dataset. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating the intelligent production method for datasets according to the present invention;

[0027] Figure 2 This is a schematic diagram of the tree-like mathematical model of the constituent factors of the multi-source dataset of the present invention;

[0028] Figure 3 This is the dataset validity evaluation metric of the present invention;

[0029] Figure 4 This is a schematic diagram illustrating how the effectiveness evaluation model of this invention guides the generation of datasets;

[0030] Figure 5 This is a schematic diagram of the dataset used to construct a genetic model. Detailed Implementation

[0031] The present invention will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0032] Example 1

[0033] Figure 1 This is a flowchart illustrating the intelligent data production method of the present invention, as shown below. Figure 1 As shown, it includes the following steps:

[0034] S1, Collect multi-source datasets of intelligent driving images and determine the constituent factors of the multi-source datasets;

[0035] S2, Tree Analysis Model of Dataset Components: Determine the discrete values ​​of the components of the multi-source dataset, and establish a tree analysis model based on the components and discrete values ​​of the multi-source dataset.

[0036] S3, Dataset Validity Evaluation Model: Based on the constructed validity evaluation model, determine the sample set similarity index generated by the genetic model, and determine the validity of the sample set according to the sample set similarity index. When the sample set is valid, the genetic model terminates the operation. When the sample set is invalid, the genetic model continues the iterative operation of the following step S4.

[0037] S4, Genetic model automatically generates dataset: Based on the pre-built genetic model, the constituent factors of the multi-source dataset are randomly recombined to generate a sample set. The iteration continues or terminates according to the iteration termination instruction given in step S3, and a dataset with reasonable validity is generated as the final dataset S*.

[0038] S5, Post-learning mechanism: Based on the contribution of the final dataset S* image recognition model, optimize the genetic model and the effectiveness evaluation model.

[0039] To reduce the generation of duplicate datasets, the pre-built genetic model can be guided by the constructed dataset validity evaluation model, which can select the constituent factors and transformations of the sample set, recombine the constituent factors of the sample set, and transform the sample set.

[0040] To avoid generating invalid datasets, the validity of the dataset can be determined based on the constructed validity evaluation algorithm. Specifically, based on the constructed validity evaluation algorithm, the feature similarity index corresponding to the sample set is determined. If the feature similarity index corresponding to the sample set is greater than a pre-set threshold, the validity of the sample set is determined to be reasonable; otherwise, the validity of the sample set is determined to be unreasonable. In this case, the genetic model needs to continue to perform selection, crossover, and mutation operations on the constituent factors and transformations of the sample set until a sample set with higher validity is obtained. This can avoid the generation of invalid datasets and improve the quality of dataset production. By observing the contribution of the final dataset S* to the image recognition model, the pre-constructed genetic model and the constructed validity evaluation model are continuously optimized, avoiding the generation of a large number of invalid datasets.

[0041] Example 2

[0042] A method for intelligent production of datasets includes the following steps:

[0043] S1: Decomposition of Dataset Components:

[0044] First, a multi-source dataset of images from intelligent driving systems is collected, and then the constituent factors of the collected multi-source dataset are decomposed hierarchically. Specifically, the constituent factors of the dataset can be decomposed hierarchically from background, weather, lighting, traffic participants, etc., according to a progressive hierarchical structure, from broad to narrow definitions.

[0045] S2: Tree-structured analysis model of dataset components:

[0046] To accurately generate the sample set, the constituent factors are discretized according to their degree. A tree-like mathematical model is then established based on these discrete values. Furthermore, the relationships between the constituent factors of each multi-source dataset can be described in the form of sets, as shown in the following formula.

[0047] F i,j ={F i+1,k ,k∈S i+1}

[0048] i∈{1,2,3,...,L-1},j∈{1,2,3,...,L i}

[0049] Among them, F i,j S represents the j-th factor in the i-th layer. i+1 It is all belonging to F i,j The set of indices of a node's child nodes, where L is the total number of levels. i Let be the total number of nodes in the i-th layer.

[0050] For example, Figure 2 This is a schematic diagram of the tree-like mathematical model of the constituent factors of the multi-source dataset of the present invention, as shown below. Figure 2 As shown, F 1,1 For the sample image, F 2,1 The element representing the composition of this sample image is the scene, and the next level of this element is F. 3,1 The constituent element of this sample image is urban roads, F 3,2 The element representing the composition of this sample image is a country road, F 3,3 The constituent elements of this sample image are highways, etc., F 2,2 This means that the constituent element of the sample image is weather, and the next level of this constituent element is F. 3,4 This indicates that the constituent element of the sample image is open-air, F 3,5 The factor representing the composition of this sample image is rainy weather, F 3,6 The constituent factors of this sample image are cloudy weather, etc., F 2,3 This means that the constituent element of the sample image is weather, and the next level of this constituent element is F. 3,7 The element representing the composition of this sample image is a vehicle, F 3,8 F represents the constituent element of this sample image as pedestrians. 3,9 The constituent elements of this sample image are traffic signs, etc., F 4,1 F 4,2 F 4,3 F 4,4 ···F 4,10This represents the discrete values ​​of the constituent elements of the open-air sample image according to their degree, with the corresponding discrete values ​​of the constituent elements of the sample image being 0.0, 0.1, 0.2, 0.3...1.0.

[0051] S3: Dataset validity evaluation model:

[0052] Dataset validity refers to the effectiveness of a dataset in enabling a learning model (image recognition model) to learn. Dataset validity evaluation models use feature similarity metrics to describe the reasonableness of generated images, and invalid data is removed through similarity metric verification. The variance D(t) of the eigenvectors of the discrete values ​​of the constituent factors is used. m The feature similarity index is calculated as: D(t) m )

[0053]

[0054] Among them, t m For the feature vector of the sample set, Let E be the average of the feature vectors of the sample set, and E represent the expected value.

[0055] Figure 3 This is the dataset validity evaluation metric of the present invention, such as... Figure 3 As shown, datasets composed of sample sets with larger eigenvector variances are more conducive to learning models learning more features; that is, the larger the eigenvector variance, the higher the effectiveness of the sample set.

[0056] Figure 4 This is a schematic diagram illustrating how the effectiveness evaluation model of this invention guides the generation of datasets, as shown below. Figure 4 As shown, the dataset S with unknown validity is randomly divided into n sample sets, denoted as m1, m2, ..., mn. n Each sample set contains K samples. The feature vector of each sample set is extracted by a feature extractor, which is to calculate the average of the feature vectors of the K samples in the m1 sample set. Next, calculate the average value of the eigenvectors of the 2nd to nth sample sets. according to Calculate the variance of the feature vectors for each sample set. The larger the variance of the feature vectors, the higher the feature similarity index of the sample set, and the higher the validity of the sample set. If the feature similarity index of the sample set is greater than a pre-set threshold, the validity of the sample set is determined to be reasonable. Otherwise, the validity of the sample set is determined to be unreasonable. Finally, the dataset S* with known validity is obtained.

[0057] S4: Genetic model automatically generates datasets:

[0058] To reduce the generation of invalid datasets, a sample set can be generated by selecting, crossing over, and mutating the constituent factors and sample sets of multi-source datasets based on a pre-built genetic model.

[0059] Figure 5 This is a diagram illustrating the construction of a dataset for a genetic model, as shown below. Figure 5 As shown, the initial values ​​of the constituent elements are input into a pre-constructed genetic model. This involves inputting the discrete values ​​of the constituent elements from the collected multi-source dataset into the pre-constructed genetic model to generate a sample set. The validity of the generated sample set is then judged according to the validity evaluation model constructed in S3. If the validity does not meet the requirements, selection, crossover, and mutation operations are performed on the constituent elements and the sample set. Selection refers to choosing the constituent elements and transforming the sample set; crossover refers to recombination of the constituent elements; and mutation refers to transformation of the sample set. Transformations of the sample set include geometric transformation, color transformation, noise transformation, and combination transformation. Specifically, the process is as follows: Initially, constituent elements are randomly selected. These randomly selected constituent elements are then randomly recombinated to form the initial sample set. Random transformations are then performed on the resulting sample images to obtain more sample sets. The validity of the generated sample set is judged according to the validity evaluation model constructed in S3. If the validity does not meet the requirements, selection, crossover, and mutation operations are performed on the constituent elements and the sample set until a dataset with higher validity is obtained. This final sample set is retained as the final dataset S*.

[0060] S5: Post-hoc self-learning mechanism

[0061] To generate more accurate and valid datasets, the collected multi-source datasets can be experimentally compared with the final dataset S*. The image recognition model can be trained using the collected multi-source datasets and the final dataset S*. Based on the model performance, the probabilities of selection, crossover, and mutation in the pre-built genetic model and the constructed validity evaluation model can be continuously optimized to avoid the generation of a large number of invalid datasets.

[0062] The above description is only a preferred embodiment of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for intelligently generating datasets for training intelligent driving image recognition algorithms, characterized in that, Includes the following steps: Step S1: Collect multi-source datasets of intelligent driving images and determine the constituent factors of the multi-source datasets; Step S2: Decompose the constituent factors of the multi-source dataset from step S1 step by step to determine the discrete values ​​of the constituent factors of the multi-source dataset. Step S3, Dataset Validity Evaluation Model: The dataset generated by the genetic model in Step S4 is randomly divided into n sample sets. Based on the constructed validity evaluation model, the feature similarity of each sample set is calculated using the discrete values ​​of the constituent factors of the n sample sets. The validity of the sample sets is determined according to the feature similarity index. When all sample sets are valid, the genetic model terminates its operation. When any sample set is invalid, the genetic model continues the iterative operation of Step S4. In Step S3, the eigenvector variance D(t) of the discrete values ​​of the constituent factors is used. m As a feature similarity index, the feature similarity index is denoted as: Among them, t m For the feature vector of the sample set, E represents the expected value, where E is the average value of the feature vectors of the sample set. A dataset consisting of a sample set with a larger feature vector variance is more conducive to the image recognition model learning more features. When the feature vector variance is greater than a preset threshold, the sample set is valid; when the feature vector variance is less than the preset threshold, the sample set is invalid. Step S4, Genetic model automatically generates dataset: The constituent factors of the multi-source dataset obtained in step S2 are randomly combined as the initial sample set. Then, the dataset is generated based on the pre-built genetic model. The iteration continues or terminates according to the iteration termination instruction given in step S3. When it continues to iterate, the constituent factors of the obtained dataset are subjected to "selection", "crossover" and "mutation" operations to generate offspring sample images. The dataset with reasonable validity obtained when the iteration terminates is used as the final dataset S*. Step S5, posterior self-learning mechanism: Based on the contribution of the final dataset S* obtained in step S4 to the image recognition model, optimize the probabilities of "selection", "crossover" and "mutation" operations in the genetic model in step S4 and the effectiveness evaluation model in step S3.

2. The intelligent data generation method as described in claim 1, characterized in that, In step S2, the constituent factors of the multi-source dataset are decomposed step by step, and after determining the discrete values ​​of the constituent factors of the multi-source dataset, a tree-like analysis model of the constituent factors of the dataset is established.

3. The intelligent data generation method as described in claim 1, characterized in that, In step S2, the relationships between the constituent elements of the multi-source dataset are described in the form of sets: F i,j ={F i+1,k ,k∈S i+1 } i∈{1,2,3,...,L-1},j∈{1,2,3,...,L i } Among them, F i,j S represents the j-th factor in the i-th layer. i+1 It is all belonging to F i,j The set of indices of a node's child nodes, where L is the total number of levels. i Let be the total number of nodes in the i-th layer.

4. The intelligent data generation method as described in claim 1, characterized in that, In step S4, "selection" of a sample refers to the selection of sample constituent elements and sample transformation, "cross" refers to the recombination of sample constituent elements, and "mutation" refers to the transformation of the sample image, including but not limited to geometric transformation, color transformation, noise transformation, and combination transformation.

5. The intelligent dataset production method as described in claim 1, characterized in that, In step S5, based on the contribution of the final dataset S* obtained in S4 to the image recognition model, i.e. the performance of the image recognition model trained on the generated final dataset S* in the actual image recognition task, the probability of multi-dimensional feature selection, crossover, mutation and similarity evaluation methods are continuously corrected to achieve the "self-growth" of the dataset generation algorithm.

Images