Object detection method based on multi-objective optimization hyper-parameter

Abstract

The invention discloses an object detection method based on multi-objective optimization hyper-parameter. The method comprises the following steps: S1, collecting road image data including vehicles and pedestrians; s2, performing data preprocessing and labeling, and performing classification labeling on objects in the image; s3, performing hyper-parameter optimization of the object detection model through a multi-target genetic algorithm to obtain an optimal hyper-parameter combination; and S4, using the object detection model formed by the optimal hyper-parameter combination for object detection. Through the method provided by the invention, the phenomenon of data imbalance can be relieved, and the object detection model hyper-parameter combination which can better consider each performance index can be obtained.

Description

Technical Field

[0001] The present application provides an object detection method based on multi-objective optimization of hyper-parameters, which relates to the field of intelligent driving technology. Background Art

[0002] In recent years, the rapid development of intelligent driving technology has led to the continuous improvement of vehicle safety, driving efficiency and comfort. As one of the core tasks of intelligent driving systems, target object detection technology plays a vital role in the automatic identification of objects, pedestrians and other traffic participants in the driving environment. Current target object detection technology not only needs to accurately identify target objects on the road ahead in real time, but also needs to cope with complex and changing road scenes, including different weather conditions, light changes and dense urban traffic. In order to meet these challenges, more and more researchers have begun to introduce deep learning technology, especially convolutional neural networks (CNN), to improve the effect of object detection.

[0003] YOLOv5 is a real-time object detection algorithm currently widely used in the field of intelligent driving. It can learn potential object detection patterns from large-scale driving scene data, and achieve end-to-end object detection by directly regressing bounding boxes and category probabilities in neural networks. In complex driving environments, YOLOv5 can not only detect vehicles, pedestrians, etc. in real time, but also respond quickly to environmental changes, playing a key role in the safety and stability of autonomous driving systems.

[0004] The performance of YOLOv5 depends to a certain extent on the hyperparameter configuration of the model. Currently, most engineers use the default hyperparameters provided by the official based on COCO data tuning. When hyperparameter tuning is needed, empirical methods are often used. YOLOv5 officially provides a hyperparameter tuning method similar to genetic algorithms. In this method, only one individual is mutated in each generation. The fitness function defined is the weighted value of mAP50 and mAP, which belongs to single-objective optimization.

[0005] The default hyperparameters of the YOLOv5 algorithm are not the best match for data in a specific field. It is currently expected to optimize the hyperparameters for a specific dataset (a dataset containing a specific target object to be detected) to obtain a hyperparameter combination that is suitable for such a dataset. The default hyperparameter tuning method of the YOLOv5 algorithm does not have a population-based crossover operation, lacks global optimization content, and the single-objective optimization process cannot obtain a parameter combination that balances multiple optimization objectives. Summary of the invention

[0006] The technical problem to be solved by this application is the hyperparameter optimization problem of the object detection model.

[0007] In order to solve the above technical problems, the technical solution of the present application is to provide an object detection method based on multi-objective optimization hyper-parameters, characterized in that it includes the following steps:

[0008] S1, collecting road image data containing vehicles and pedestrians;

[0009] S2, data preprocessing and labeling, classifying and labeling objects in the image;

[0010] S3. Optimize the hyperparameters of the object detection model through a multi-objective genetic algorithm, including:

[0011] S3.1 select the hyperparameters to be optimized;

[0012] S3.2 determine the search space for each hyperparameter;

[0013] S3.3Set the fitness function;

[0014] S3.4 takes the hyperparameter to be optimized as the optimization target of the multi-objective genetic algorithm, the search space as the variable range of the optimization target, the fitness function as the fitness of the multi-objective genetic algorithm, executes the multi-objective genetic algorithm to search for hyperparameters, and obtains the optimal hyperparameter combination;

[0015] S4. The object detection model constructed by the optimal hyperparameter combination is used for object detection.

[0016] Preferably, in step S1, the road image data covers the following real conditions: lighting conditions include day and night, weather conditions include sunny, cloudy, rainy, foggy, and unknown, and road conditions include urban roads and rural roads.

[0017] Preferably, step S2 specifically includes:

[0018] S2.1 obtain the original image dataset;

[0019] S2.2 screens and cleans the original image dataset to remove damaged and blurred samples;

[0020] S2.3 Use the annotation tool to annotate the processed image data; based on the annotation rules, use a rectangular frame to annotate the target, and the annotation categories are 8 categories. The classification table is shown in Table 1 in the specific implementation method.

[0021] S2.4 extracts part of the data set as a training set.

[0022] Preferably, in step S3, the multi-objective genetic algorithm is set to NSGA-II, and the object detection model is set to a YOLOv5 model.

[0023] Preferably, step S3 specifically includes:

[0024] S3.1 Determine the hyperparameters to be optimized, including the box target box loss gain, cls classification loss gain, cls_pw classification loss positive weight, obj target loss gain, and obj_pw target loss positive weight of the loss function;

[0025] S3.2 Determine the search space of each hyperparameter; box is [0.02, 0.1], cls is [0.2, 1.0], cls_pw is [0.5, 2.0], obj is [0.5, 2.0], and obj_pw is [0.5, 2.0], and each parameter is a real number;

[0026] S3.3 Set the fitness function, including recall rate R and average precision mAP;

[0027] S3.4 uses the hyperparameters to be optimized as the optimization targets of the multi-objective genetic algorithm, the search space as the variable range of the optimization targets, and the fitness function as the fitness of the multi-objective genetic algorithm. The multi-objective genetic algorithm is executed to optimize the hyperparameters and obtain the optimal hyperparameter combination.

[0028] Furthermore, step S3.4 specifically includes:

[0029] S3.4.1 Population initialization P(t);

[0030] S3.4.2 uses recall rate R and average precision mAP as fitness function;

[0031] S3.4.3 Select, crossover and mutate the parent population P(t) to generate the child population Q(t) and merge them into R(t);

[0032] S3.4.4 The YOLOv5 model is trained for 30 epochs, and the validation set val result is output to obtain the fitness value;

[0033] S3.4.5 According to the fitness value, perform fast non-dominated sorting on R(t) to obtain the non-dominated solution set: F = {F(1), F(2), ...}, set layer = 1, and archive the set S(t);

[0034] S3.4.6 Add the non-dominated solution set F to the archive set S(t) hierarchically, and determine the size of the number of individuals in S(t) and the population size N: if it is equal to N, the next generation population P(t+1) = S(t); if it is greater than N, first set P(t+1) = S(t)-F(layer), and select individuals from all individuals in F(layer) according to the crowding distance to enter P(t+1) in turn until the number of individuals in P(t+1) is N;

[0035] S3.4.7 Termination condition judgment: if the number of population iterations is met, the calculation is stopped, and the global approximate optimal parameter combination population is obtained; otherwise, return to step S3.4.3.

[0036] S3.4.8 calculates the fitness value of the population of approximate optimal parameter combinations, performs non-dominated sorting, and selects the parameter combination with the largest mAP in F(1) as the optimal parameter combination.

[0037] This application optimizes the hyperparameters of the detection model for the data set in the field of vehicle-person target object detection, especially for the weight hyperparameters related to the loss function, and can obtain a hyperparameter combination that is suitable for the data in this field to alleviate the data imbalance phenomenon. This application uses the NSGA-II algorithm to perform hyperparameter optimization based on two objectives. In addition to crossover, it also combines mutation operations to balance the recall rate R and the average precision mAP, and obtain a hyperparameter combination that can better take into account various performance indicators. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 Flowchart of the object detection method based on NSGA-II to optimize YOLOv5 hyperparameters provided in this application;

[0039] Figure 2 This is the principle diagram of NSGA-II hyperparameter optimization in this application;

[0040] Figure 3 This is a graph of NSGA-II hyperparameter optimization results in the implementation scheme of this application;

[0041] Figure 4 This is a schematic diagram of the detection effect of the NSGA-II optimization hyperparameter model in the implementation mode of this application;

[0042] Figure 5 This is a schematic diagram of the official optimization hyperparameter model detection effect. DETAILED DESCRIPTION

[0043] To make the application more obvious and easy to understand, various exemplary embodiments will be introduced below. These examples are non-limiting, and it should be understood that they are used for the more general application aspects of the illustrative device, system and method. Without departing from the substantive content and scope of the application, these embodiments can be subjected to multiple changes, and can be replaced by equivalents. In addition, multiple changes can be carried out to adapt to the purpose, content or scope of the application in order to adapt to special situations, materials, material components, treatment types, treatment actions or steps. All these changes will be within the protection scope of the application.

[0044] For any materials, dimensions, quantities introduced in the overview or detailed description, they will be examples only and are not intended to limit the subject matter of the present application. Moreover, the various implementations of the embodiments described herein will complement each other, rather than purely interchangeably, unless otherwise indicated. In other words, implementations from one embodiment can be freely combined with implementations of other embodiments, as those of ordinary skill in the art will readily appreciate, unless it is indicated that these implementations are only for replacement.

[0045] In the description of this application, it should be noted that the terms "inside", "outside", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the application is usually placed when in use, which is only for the convenience of describing this application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on this application. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0046] In the description of this application, it should also be noted that, unless otherwise clearly specified and limited, the terms "disposed" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0047] The embodiments of the present application provide an object detection method based on multi-objective optimization of hyperparameters, which optimizes the model hyperparameters of the object detection model (convolutional neural network) to improve the accuracy, recall rate and average precision of specific object detection.

[0048] Exemplarily, in the implementation scheme provided in the present application, the YOLOv5 model hyperparameters are optimized by the multi-objective genetic algorithm NSGA-II, specifically for a data set containing vehicles and pedestrians as the objects to be detected, and the multi-objective genetic algorithm NSGA-II is used to perform hyperparameter optimization to obtain a hyperparameter combination of the YOLOv5 model adapted to such data set. Wherein, the hyperparameters are parameters whose values ​​are set before the YOLOv5 model starts learning and training, rather than parameters obtained through training. It is understandable that the optimization of the YOLOv5 model hyperparameters by the multi-objective genetic algorithm NSGA-II provided in the implementation scheme of the present application can be for specific objects such as vehicles and pedestrians, or for other specific objects.

[0049] The specific implementation process is as follows Figure 1 As shown, the method comprises the following steps:

[0050] S1. Collect road image data containing vehicles and pedestrians

[0051] S1.1 Data collection: Data is collected through CMBS equipment installed on buses in a certain place, using supporting cameras to collect image data. Such image data is real road images actually collected by bus operating vehicles, which are real and representative. In order to improve the integrity of image data, the coverage and quality of various situations, data collection should cover various real conditions as much as possible, such as lighting conditions including day and night, weather conditions including sunny, cloudy, rainy, foggy, unknown, and road conditions including urban roads and rural roads where buses travel; Collection method and cycle: 10 buses run for one week, and regularly transmit video data to the cloud for data aggregation.

[0052] S2. Data preprocessing and annotation

[0053] S2.1 extracts frames from the collected video data at fixed intervals to form an original image dataset.

[0054] S2.2 screens and cleans the original image dataset to remove damaged and blurred samples.

[0055] S2.3 Use the LabeIimg annotation tool to annotate the processed image data.

[0056] Based on the labeling rules, a rectangular box is used to label the target, and the labeling mode is YOLO.

[0057] There are 8 annotation categories, and the specific categories are shown in Table 1:

[0058] Table 1 Labeling category table

[0059]

[0060]

[0061] After S2.4 is labeled, use the python script to divide the dataset into training set, test set, and validation set in a 6:2:2 ratio. The results are shown in the following table:

[0062] Table 2 Number of dataset labels

[0063]

[0064] Table 3 Dataset division

[0065] Dataset train val test total quantity 6000 2000 2000 10000

[0066] S3. Hyperparameter optimization using the multi-objective genetic algorithm NSGA-II

[0067] NSGA-II is used to optimize the hyperparameters of the YOLOv5 model, and the Pareto optimal solution that weighs various objectives in the parameter search space is obtained as the parameter combination. Among them, a solution that is not dominated by other feasible solutions is called a Pareto optimal solution. For details, please refer to "Pareto Optimization Strategy", which will not be repeated in this implementation.

[0068] The specific steps are:

[0069] S3.1 Determine the optimized hyperparameters:

[0070] In view of the imbalance problem of various targets in the current data set, the box target box loss gain, cls classification loss gain, cls_pw classification loss positive weight, obj target loss gain and obj_pw target loss positive weight involved in the loss function are selected as the hyperparameters to be optimized.

[0071] S3.2 Parameter search space: box is [0.02, 0.1], cls is [0.2, 1.0], cls_pw is [0.5, 2.0], obj is [0.5, 2.0], obj_pw is [0.5, 2.0], and each parameter is a real number.

[0072] S3.3 Set the fitness function:

[0073] There are two fitness functions determined: one is to maximize the recall rate R (recall), and the other is to maximize the average precision mAP (mean average precision).

[0074] The calculation formula is as follows:

[0075]

[0076] Among them, TP (true positives) represents the number of targets detected correctly, FP (false positives) represents the number of targets detected incorrectly, FN (false negatives) represents the number of targets not detected, AP (average precision) represents the average precision of a certain target class, and n represents the number of categories that need to be classified.

[0077] S3.4 Set the multi-objective genetic algorithm parameters and use NSGA-II for hyperparameter optimization

[0078] The parameter values ​​of NSGA-II for multi-objective optimization are as follows: the number of individuals in the population is set to 20, the number of iterations is set to 5, the crossover probability of the simulated binary crossover SBX (crossover operator) is set to 0.9, the distribution index is set to 20, the mutation probability of the polynomial mutation PM is set to 0.1, and the distribution index is set to 20.

[0079] Except for the 6 parameters to be optimized, the other hyperparameters of the YOLOv5 model use the parameters in hyp.scratch.yaml (default model configuration file), epoch = 30, batchsize = 128, imgsz = 512. 20% of the full data is randomly selected as the hyperparameter optimization dataset.

[0080] The specific steps for parameter optimization are as follows:

[0081] S3.4.1 Population initialization P(t);

[0082] S3.4.2 uses recall rate R and average precision mAP as fitness function;

[0083] S3.4.3 Select, crossover and mutate the parent population P(t) to generate the child population Q(t) and merge them into R(t);

[0084] S3.4.4The YOLOv5 model is trained for 30 epochs, and the val result is output to obtain the fitness value.

[0085] S3.4.5 Based on the fitness value, perform fast non-dominated sorting on R(t) to obtain the non-dominated solution set: F = {F(1), F(2), ...}, set the layer layer = 1, and archive the set S(t).

[0086] S3.4.6 Add the non-dominated solution set F to the archive set S(t) hierarchically, and determine the number of individuals in S(t) and the size of the population N: if it is equal to N, the next generation population P(t+1)=S(t); if it is greater than N, then first set P(t+1)=S(t)-F(layer), and for all individuals in F(layer), select individuals according to the "crowding distance" to enter P(t+1) in turn until the number of individuals in P(t+1) is N.

[0087] S3.4.7 Termination condition judgment: if the number of population iterations is met, the calculation is stopped, and the global approximate optimal parameter combination population is obtained; otherwise, return to step S3.4.3.

[0088] S3.4.8 calculates the fitness value of the population of approximate optimal parameter combinations, performs non-dominated sorting, and selects the parameter combination with the largest mAP in F(1) as the optimal parameter combination.

[0089] Exemplary parameter optimization results:

[0090] Figure 2The two-dimensional graph represents the optimization result of NSGA-II. The result is the approximate optimal solution set obtained after 30 generations of evolution. After comparing the two solutions in F(1), the optimal parameter combination individuals are determined to be [0.06792, 0.73866, 1.59867, 1.74409, 1.22942], and their corresponding fitness values ​​are [R=0.21403, mAP=0.07646].

[0091] S4 uses the object detection model composed of the optimal hyperparameter combination for object detection;

[0092] Effect verification: Compare the model based on the multi-objective hyperparameter optimization results with the model with hyperparameters optimized by the official method. Training and testing: Based on the same dataset, train the YOLOv5 vehicle and pedestrian detection models using NSGA-II and hyperparameters optimized by the official method, and perform test evaluation.

[0093] Experimental environment: CPU is Intel(R) Core i7-127002.10GHz, 64.0GB RAM, GPU is NVIDIAGeForce RTX3090, 24GB video memory, Windows 11, 64-bit operating system, integrated development environment is Pycharm, deep learning framework Pytorch1.13.1, program language version is Python3.8, GPU acceleration kit is CUDA11.7 and CUDNN8.4.1.

[0094] Experimental settings: The model uses YOLOv5n configuration, uses the preset anchors obtained by clustering, adopts mosaic data enhancement, the input image size is 512×512, the batch-size is set to 128 during experimental training, the batch-size is set to 1 during testing, the epoch is 200, the optimizer is adam, the regression loss uses CIoUloss, the target loss uses binary cross entropy loss (BCE loss), the classification loss uses binary cross entropy loss (BCE loss), and the dataset is the full data.

[0095] The official optimization hyperparameter combination is [0.02, 0.2, 0.937, 0.2, 2.0], corresponding to R = 0.15955, mAP = 0.06553.

[0096] In order to evaluate the performance of the algorithm, the effect verification experiment of this embodiment adopts several performance indicators such as precision P (precision), recall R (recall) and mAP (mean average precision) for evaluation.

[0097]

[0098] In the formula, TP (true positives) represents the number of targets detected correctly, FP (false positives) represents the number of targets detected incorrectly, FN (false negatives) represents the number of targets not detected, AP (average precision) represents the average precision of a target class, and n represents the number of categories that need to be classified.

[0099] The training steps are as follows:

[0100] Step 1: Generate the configuration file hyp_NSGAII.yaml based on the hyperparameters optimized by NSGA-II, and generate the configuration file hyp_official.yaml based on the hyperparameters optimized by the official method;

[0101] Step 2: Based on a specific dataset, train the YOLOv5n model using different hyperparameter configuration files.

[0102] Step 3: Iterate the network training, continuously adjust the model weight parameters and reduce the loss until the cutoff condition is reached and save the optimal model.

[0103] The test steps are as follows:

[0104] The model trained with NSGA-II optimized hyperparameters is named YOLOv5n-NSGAII, and the model trained with official optimized hyperparameters is named YOLOv5n-Official.

[0105] Based on the test dataset, two models are used for inference, and the models are evaluated using three indicators: P, R, and mAP.

[0106] Based on the test data set, two models are used for inference and the detection result images are saved separately.

[0107] Experimental results analysis

[0108] The following table compares YOLOv5n-NSGAII and YOLOv5n-Official on the full dataset.

[0109] Model P R mAP YOLOv5n-NSGAII 0.631 0.525 0.331 YOLOv5n-Official 0.62536 0.46419 0.30256

[0110] By comparing the experimental results in the table, it can be seen that YOLOv5n-NSGAII, which uses NSGA-II for hyperparameter optimization, has improved accuracy, recall rate, and mAP compared to YOLOv5n-Official, which uses the official hyperparameter optimization method. The overall performance is outstanding, which proves the superiority of the hyperparameter optimization method proposed in this embodiment.

[0111] contrast Figure 4 and Figure 5 In the detection effect diagram of YOLOv5n-NSGAII and YOLOv5n-Official, it is found that the target objects missed by YOLOv5n-Official are correctly detected in YOLOv5n-NSGAII, which shows that it has the effect of reducing the missed detection rate, which is completely consistent with the performance comparison results. While the accuracy has been improved, the recall rate and mAP have been greatly improved.

[0112] The above is only a preferred embodiment of the present application, and is not any formal or substantial limitation to the present application. It should be pointed out that ordinary technicians in this technical field can make several improvements and supplements without departing from the present application, and these improvements and supplements should also be regarded as the protection scope of the present application. Any technician familiar with this profession, without departing from the content and scope of the present application, can make some changes, modifications and evolutions of the technical content disclosed above, which are equivalent embodiments of the present application; at the same time, any changes, modifications and evolutions of any equivalent changes made to the above-mentioned implementation methods based on the substantial technology of the present application are still within the scope of the technical solution of the present application.

Claims

1. An object detection method based on multi-objective optimization hyper-parameters, characterized in that: The following steps are involved: S1, collecting road image data containing vehicles and pedestrians; S2, data preprocessing and labeling, classifying and labeling objects in the image; S3. Hyperparameter optimization of the object detection model using a multi-objective genetic algorithm, including: S3.1 select the hyperparameters to be optimized; S3.2 determine the search space for each hyperparameter; S3.3Set the fitness function; S3.4 takes the hyperparameter to be optimized as the optimization target of the multi-objective genetic algorithm, the search space as the variable range of the optimization target, the fitness function as the fitness of the multi-objective genetic algorithm, executes the multi-objective genetic algorithm to search for hyperparameters, and obtains the optimal hyperparameter combination; S4. The object detection model constructed by the optimal hyperparameter combination is used for object detection.

2. The object detection method based on multi-objective optimization hyper-parameters according to claim 1, characterized in that: In step S1, the road image data covers the following real conditions: lighting conditions include day and night, weather conditions include sunny, cloudy, rainy, foggy, and unknown, and road conditions include urban roads and rural roads.

3. The object detection method based on multi-objective optimization hyper-parameters according to claim 1, characterized in that: Step S2 specifically includes: S2.1 obtain the original image dataset; S2.2 screens and cleans the original image dataset to remove damaged and blurred samples; S2.3 uses the annotation tool to annotate the processed image data; based on the annotation rules, use a rectangular box to annotate the target, and the annotation categories are 8, as shown in the following table: S2.4 extracts part of the data set as a training set.

4. The object detection method based on multi-objective optimization hyper-parameters according to claim 1, characterized in that: In step S3, the multi-objective genetic algorithm is set to NSGA-II, and the object detection model is set to the YOLOv5 model.

5. The object detection method based on multi-objective optimization hyper-parameters according to claim 4, characterized in that: Step S3 specifically includes: S3.1 Determine the hyperparameters to be optimized, including the box target box loss gain, cls classification loss gain, cls_pw classification loss positive weight, obj target loss gain, and obj_pw target loss positive weight of the loss function; S3.2 Determine the search space of each hyperparameter; box is [0.02, 0.1], cls is [0.2, 1.0], cls_pw is [0.5, 2.0], obj is [0.5, 2.0], and obj_pw is [0.5, 2.0], and each parameter is a real number; S3.3 Set the fitness function, including recall rate R and average precision mAP; S3.4 uses the hyperparameters to be optimized as the optimization targets of the multi-objective genetic algorithm, the search space as the variable range of the optimization targets, and the fitness function as the fitness of the multi-objective genetic algorithm. The multi-objective genetic algorithm is executed to optimize the hyperparameters and obtain the optimal hyperparameter combination.

6. The object detection method based on multi-objective optimization hyper-parameters according to claim 5, characterized in that: Step S3.4 specifically includes: S3.4.1 Population initialization P(t); S3.4.2 uses recall rate R and average precision mAP as fitness function; S3.4.3 Select, crossover and mutate the parent population P(t) to generate the child population Q(t) and merge them into R(t); S3.4.4 The YOLOv5 model is trained for 30 epochs, and the validation set val result is output to obtain the fitness value; S3.4.5 According to the fitness value, perform fast non-dominated sorting on R(t) to obtain the non-dominated solution set: F = {F(1), F(2), ...}, set layer = 1, and archive the set S(t); S3.4.6 Add the non-dominated solution set F to the archive set S(t) hierarchically, and determine the size of the number of individuals in S(t) and the population size N: if it is equal to N, the next generation population P(t+1) = S(t); if it is greater than N, first set P(t+1) = S(t)-F(layer), and select individuals from all individuals in F(layer) according to the crowding distance to enter P(t+1) in turn until the number of individuals in P(t+1) is N; S3.4.7 Termination condition judgment: if the population iteration number is satisfied, the calculation is stopped, and the global approximate optimal parameter combination population is obtained; otherwise, return to step S3.4.3; S3.4.8 calculates the fitness value of the population of approximate optimal parameter combinations, performs non-dominated sorting, and selects the parameter combination with the largest mAP in F(1) as the optimal parameter combination.

Images