Methods, devices, electronic equipment and storage media for controlling the accuracy of target detection

Abstract

This application discloses a method, apparatus, electronic device, and storage medium for controlling the accuracy of object detection, belonging to the field of object detection technology. The method includes: acquiring a full image set and training a deep learning algorithm using the full image set to obtain a first deep learning model; calculating the prediction accuracy of the first deep learning model for multiple label categories; generating an optimized image set based on the prediction accuracy; fine-tuning the first deep learning model using the optimized image set to obtain a second deep learning model; and performing object detection using the second deep learning model. This application solves the technical problem of low detection accuracy in related technologies.

Description

Technical Field

[0001] This application relates to the field of target detection technology, and more specifically, to a method, apparatus, electronic device, and storage medium for controlling the accuracy of target detection. Background Technology

[0002] In the field of intelligent driving perception, current deep learning-based perception algorithms are increasingly favored by engineers because their accuracy can grow exponentially with the increase of images. When the number of images reaches a certain level, their accuracy and adaptability to various complex or simple scenarios are far superior to traditional algorithms.

[0003] However, due to the unique characteristics of intelligent driving scenarios, the number of different categories in the images captured by the image acquisition vehicle varies significantly, resulting in an imbalanced image set. For example, the number of cars in the image set may be ten or even dozens of times greater than that of trucks and buses. Models trained using images in this state will exhibit high accuracy for some categories but extremely low accuracy for others, failing to meet the requirements of intelligent driving.

[0004] There are currently no effective solutions to the aforementioned problems in the relevant technologies. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and storage medium for controlling the accuracy of target detection, in order to solve the technical problem of low detection accuracy in related target detection models.

[0006] According to one aspect of the embodiments of this application, a method for controlling the accuracy of target detection is provided, comprising: acquiring a full image set and training a deep learning algorithm using the full image set to obtain a first deep learning model; calculating the prediction accuracy of the first deep learning model for predicting multiple label categories; generating an optimized image set based on the prediction accuracy; fine-tuning the first deep learning model using the optimized image set to obtain a second deep learning model; and performing target detection using the second deep learning model.

[0007] Further, generating an optimized image set based on the prediction accuracy includes: determining a minimum reference value for the number of bounding boxes based on the prediction accuracy; selecting samples containing a first category from the full image set to form a first image set, wherein the first category is a label category with a prediction accuracy less than a preset accuracy and a number of bounding boxes less than the minimum reference value; performing data augmentation on the first category in the first image set to form a second image set; merging the first image set and the second image set to form a third image set, and using the third image set as the optimized image set.

[0008] Further, determining the minimum reference value for the number of bounding boxes based on the prediction accuracy includes: filtering out all second-category image sets in the full image set whose prediction accuracy is greater than or equal to a preset accuracy; counting the total number of bounding boxes for each second category in the image set; selecting a specified second category with the smallest total number of bounding boxes, and using the value of the total number of bounding boxes in the specified second category as the minimum reference value.

[0009] Furthermore, after merging the first image set and the second image set to form the third image set, the method further includes: counting the number of bounding boxes of the first category in the third image set; determining whether the number of bounding boxes of the first category is less than the minimum reference value; if the number of bounding boxes of the first category is less than the minimum reference value, then oversampling the first category to obtain a fourth image set, and using the fourth image set as the optimized image set.

[0010] Further, oversampling the first category to obtain the fourth image set includes: oversampling the first category in the second image set to generate an oversampled image set; and combining the oversampled image set and the third image set to generate the fourth image set.

[0011] Furthermore, after merging the first image set and the second image set to form the third image set, the method further includes: counting the number of bounding boxes for other categories besides the first category in the third image set; determining whether the number of bounding boxes for other categories is less than the minimum reference value; if the number of bounding boxes for other categories is less than the minimum reference value, upsampling samples containing the other categories in the full image set to generate an upsampled image set; combining the upsampled image set and the third image set to generate a fifth image set, and using the fifth image set as the optimized image set.

[0012] Further, performing data augmentation on the first category in the first image set to form a second image set includes: translating, randomly cropping, randomly scaling, adding Gaussian noise, or rotating the current frame point cloud map and the bounding boxes corresponding to the first category in the current frame point cloud map to generate a new image set; and using the new image set as the second image set.

[0013] According to another aspect of the embodiments of this application, an accuracy control device for object detection is also provided, comprising: a training module for acquiring a full image set and training a deep learning algorithm using the full image set to obtain a first deep learning model; a calculation module for calculating the prediction accuracy of the first deep learning model for predicting multiple label categories; a generation module for generating an optimized image set based on the prediction accuracy; a fine-tuning module for fine-tuning the first deep learning model using the optimized image set to obtain a second deep learning model; and a detection module for performing object detection using the second deep learning model.

[0014] Furthermore, the generation module includes a first generation unit, used to determine a minimum reference value for the number of bounding boxes based on the prediction accuracy; to filter samples containing a first category from the full image set to form a first image set, wherein the first category is a label category whose prediction accuracy is less than a preset accuracy and whose number of bounding boxes is less than the minimum reference value; to perform data augmentation on the first category in the first image set to form a second image set; to merge the first image set and the second image set to form a third image set, and to use the third image set as the optimized image set.

[0015] Furthermore, the first generation unit is also configured to filter out all second-category image sets whose prediction accuracy is greater than or equal to a preset accuracy from the full image set; count the total number of bounding boxes for each second category in the image set; select a specified second category with the smallest total number of bounding boxes, and use the value of the total number of bounding boxes of the specified second category as the minimum reference value.

[0016] Furthermore, the generation module includes a second generation unit, used to count the number of bounding boxes of the first category in the third image set; determine whether the number of bounding boxes of the first category is less than the minimum reference value; if the number of bounding boxes of the first category is less than the minimum reference value, then oversample the first category to obtain a fourth image set, and use the fourth image set as an optimized image set.

[0017] Furthermore, the second generation unit is also configured to oversample the first category in the second image set to generate an oversampled image set; and combine the oversampled image set and the third image set to generate a fourth image set.

[0018] Furthermore, the generation module includes a third generation unit, used to count the number of bounding boxes for categories other than the first category in the third image set; determine whether the number of bounding boxes for the other categories is less than the minimum reference value; if the number of bounding boxes for the other categories is less than the minimum reference value, then upsample samples containing the other categories in the full image set to generate an upsampled image set; combine the upsampled image set and the third image set to generate a fifth image set, and use the fifth image set as an optimized image set.

[0019] Furthermore, the first generation unit is also used to translate, randomly crop, randomly scale, add Gaussian noise, or rotate the current frame point cloud map and the annotation box corresponding to the first category in the current frame point cloud map of the first image set to generate a new image set; and use the new image set as the second image set.

[0020] According to another aspect of the embodiments of this application, a storage medium is also provided, the storage medium including a stored program that executes the above steps when the program is run.

[0021] According to another aspect of the embodiments of this application, an electronic device is also provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; wherein: the memory is used to store computer programs; and the processor is used to execute the steps in the above method by running the programs stored in the memory.

[0022] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps in the above-described method.

[0023] The beneficial effects of this invention are:

[0024] 1. This invention creates an optimized image set and further fine-tunes the first deep learning model trained on the full image set to obtain a second deep learning model. The optimized image set improves the balance of the dataset, thereby improving the detection accuracy of the model.

[0025] 2. This invention achieves balanced laser point cloud data samples by performing data augmentation optimization on categories with insufficient preset accuracy and small number in the full image set, thereby improving the detection accuracy of small sample categories in deep learning models. Attached Figure Description

[0026] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0027] Figure 1 This is a hardware structure block diagram of a computer according to an embodiment of this application;

[0028] Figure 2 This is a flowchart of a target detection accuracy control method according to an embodiment of this application;

[0029] Figure 3 This is a schematic diagram of the process of fine-tuning the model according to an embodiment of this application;

[0030] Figure 4 This is a schematic diagram of the optimized training generation process according to an embodiment of this application;

[0031] Figure 5 This is a structural block diagram of a target detection accuracy control device according to an embodiment of this application. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all of them. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present application can be combined with each other.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] Example 1

[0035] The method embodiment provided in Embodiment 1 of this application can be executed in an in-vehicle terminal, a computer, or a similar computing device. Taking its operation on an in-vehicle terminal as an example, Figure 1 This is a hardware structure block diagram of a vehicle-mounted terminal according to an embodiment of this application. Figure 1As shown, the vehicle-mounted terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. Optionally, the vehicle-mounted terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned vehicle-mounted terminal. For example, the vehicle-mounted terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0036] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to a target detection accuracy control method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the aforementioned method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0037] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a computer's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0038] This embodiment provides a method for controlling the accuracy of target detection. Figure 2 This is a flowchart of a target detection accuracy control method according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps:

[0039] Step S10: Obtain the full image set and use the full image set to train a deep learning algorithm to obtain the first deep learning model;

[0040] This embodiment is mainly applied to the field of laser perception in intelligent driving, specifically in laser point cloud target detection based on deep learning. When a vehicle is in motion, the lidar sensor can acquire a large amount of point cloud data, including various vehicles, pedestrians, and road objects in the traffic environment. Acquiring the full image set in this embodiment may include: acquiring a large batch of road test point cloud image data collected by the acquisition vehicle; cleaning the road test data; labeling the cleaned road test data; and using the labeled set of all images as the full dataset. The data labeling involves marking the label category of pedestrians or various vehicles in each frame of the image, as well as the position information of the label box, including the center point, orientation, and size.

[0041] The deep learning algorithm is trained on the full image set to obtain the first deep learning model M1, thus making full use of the original dataset.

[0042] Step S20: Calculate the prediction accuracy of the first deep learning model for predicting multiple label categories;

[0043] Step S30: Generate an optimized image set based on the predicted accuracy;

[0044] Step S40: Fine-tune the first deep learning model using the optimized image set to obtain the second deep learning model;

[0045] Step S50: Target detection is performed using the second deep learning model.

[0046] In this embodiment, the label categories can include different object categories such as pedestrians and vehicles. Vehicles include different types of vehicles, such as large vehicles like trucks, buses, engineering vehicles (including fire trucks, ambulances, tankers, garbage trucks, forklifts, cranes, etc.), and small vehicles like sedans and sports cars. Due to the complexity and uncertainty of road conditions, the number of different categories in the full set of vehicle images collected varies greatly. For example, the number of one category may be tens of times that of another. Using such an imbalanced dataset for model training, even if the dataset reaches hundreds of thousands of frames, the model will still have low detection accuracy for categories with less training data. Therefore, in this embodiment, the prediction accuracy of the first deep learning model for multiple label categories is calculated, an optimized image set is generated based on the prediction accuracy, and the first deep learning model is fine-tuned using the optimized image set to obtain the second deep learning model M2. The second deep learning model is then used for object detection.

[0047] This embodiment creates an optimized image set and further fine-tunes the first deep learning model trained on the full image set to obtain a second deep learning model. The optimized image set improves the balance of the dataset, thereby improving the detection accuracy of the model.

[0048] In one embodiment of this example, generating an optimized image set based on the prediction accuracy includes:

[0049] S32, determine the minimum reference value for the number of annotation boxes based on the prediction accuracy;

[0050] To facilitate the detection of the first category in the full image set where the accuracy is insufficient and the number of samples is small, this embodiment determines a minimum reference value min_num for the number of bounding boxes based on the prediction accuracy. Specifically, determining the minimum reference value for the number of bounding boxes based on the prediction accuracy includes: filtering out all images of the second category in the full image set whose prediction accuracy is greater than or equal to a preset accuracy P; counting the total number of bounding boxes for each second category in the image set; selecting a specific second category with the smallest total number of bounding boxes, and using the value of the total number of bounding boxes in the specified second category as the minimum reference value. The preset accuracy P in this embodiment can be set according to actual conditions.

[0051] Statistical analysis is performed on the entire image set to calculate the proportion and quantity of each label category. Based on the prediction accuracy and quantity of each category in the entire dataset, an optimized training set is created. The quantity of each category is equivalent to the number of bounding boxes for each category. An image set containing a second category (those with a model prediction accuracy greater than or equal to a preset accuracy) is selected from the entire image set. The total number of bounding boxes for each second category in the image set is counted, and the second category with the smallest total number of bounding boxes is selected. This value is used as the minimum reference value. In other words, the category with the fewest bounding boxes among those meeting the minimum accuracy requirement is selected as the minimum reference value. This means that data augmentation is needed for the bounding boxes of the less numerous categories in the entire image set until the value is increased to at least this minimum reference value. In this embodiment, a custom value can also be defined as the minimum reference value.

[0052] S33, Filter out samples containing the first category from the full image set to form the first image set, wherein the first category is a label category whose prediction accuracy is less than the preset accuracy and whose number of bounding boxes is less than the minimum reference value;

[0053] S34, perform data augmentation on the first category in the first image set to form a second image set;

[0054] S35, merge the first image set and the second image set to form a third image set, and use the third image set as the optimized image set.

[0055] Samples containing the first category are selected from the full image set to form a first image set C1. In this embodiment, the first category is a small sample category. Data augmentation is performed on the first category in the first image set C1 to form a new second image set C2. The first image set C1 and the second image set C2 are merged to form a third image set C3. The third image set C3 increases the number of bounding boxes of the first category, so that the sample balance between the less numerous first category and other categories is achieved. Therefore, the third image set C3 can be used as an optimized image set. Specifically, data augmentation of the first category in the first image set to form a second image set includes: translating, randomly cropping, randomly scaling, adding Gaussian noise, or rotating the current frame point cloud map and the bounding boxes corresponding to the first category in the current frame point cloud map of the first image set to generate a new image set; the new image set is used as the second image set.

[0056] In this embodiment, data augmentation can employ translation, random cropping, random scaling, adding Gaussian noise, horizontal or vertical rotation, and rotations of 90, 180, and 270 degrees. Flipping and rotation ensure that the rotation-invariant properties of data features are learned by the model. This embodiment also includes feature transfer. Because the intra-class variance of tail class samples is smaller than that of head class samples, this leads to a bias in the feature space and decision boundary. FTL enhances the features of tail samples by adjusting the intra-class variance of head class samples, thereby giving tail data higher intra-class variance and stronger tail data features.

[0057] This embodiment achieves balanced laser point cloud data samples by performing data augmentation optimization on categories with insufficient preset accuracy and small numbers in the full image set, thereby improving the detection accuracy of small sample categories in the deep learning model.

[0058] In another embodiment of this example, after merging the first image set and the second image set to form a third image set, the method further includes: counting the number of bounding boxes of the first category in the third image set; determining whether the number of bounding boxes of the first category is less than the minimum reference value; if the number of bounding boxes of the first category is less than the minimum reference value, then oversampling the first category to obtain a fourth image set, and using the fourth image set as an optimized image set.

[0059] The number of bounding boxes for label categories in the third image set C3 that have a prediction accuracy less than a preset accuracy P and a bounding box count less than the minimum reference value min_num is counted. For example, if the number of bounding boxes for category A in C3 is still less than the minimum reference value, it indicates that the number of bounding boxes for category A after data augmentation is still relatively low. Therefore, category A is oversampled in the first image set C1 to ensure that the number of bounding boxes for category A is not less than the minimum reference value, thereby directly expanding the third image set C3. Alternatively, a newly generated oversampled image set can be combined with C3 to form a fourth image set with higher sample balance. For label categories in C1 with a bounding box count greater than or equal to the minimum reference value, oversampling is not required.

[0060] Specifically, oversampling the first category to obtain the fourth image set includes: oversampling the first category in the first image set to generate an oversampled image set; and combining the oversampled image set and the third image set to generate the fourth image set.

[0061] In this embodiment, oversampling can be random oversampling, which balances the data by copying minority class examples from the first image set. The first image set belongs to the original dataset, and its image quality is higher than that of the data-augmented second image set. Therefore, in this embodiment, oversampling is performed on the first category from the first image set. Moreover, the first image set is a set of images of the first category selected from the full image set, containing images with prediction accuracy lower than a preset accuracy and the number of bounding boxes lower than the minimum reference value. Compared to the large amount of data in the full dataset, oversampling from the first image set is more efficient.

[0062] In another embodiment of this example, after merging the first image set and the second image set to form a third image set, the method further includes: counting the number of bounding boxes for other categories besides the first category in the third image set; determining whether the number of bounding boxes for other categories is less than the minimum reference value; if the number of bounding boxes for other categories is less than the minimum reference value, upsampling samples containing the other categories in the full image set to generate an upsampled image set; combining the upsampled image set and the third image set to generate a fifth image set, and using the fifth image set as an optimized image set.

[0063] In this embodiment, to maintain the high detection accuracy of the second deep learning model after finetuning for large samples with a data volume greater than the minimum reference value, the number of other categories after removing the categories in the third image set C3 with a prediction accuracy less than P and the number of labeled boxes less than the minimum reference value min_num is counted. Exemplarily, if the number of labeled boxes of category B among these other categories is less than the minimum reference value, it means that the number of corresponding samples of category B with a relatively high prediction accuracy in the original first deep learning model selected into the training data set of C3 is small, which may lead to a decrease in the detection accuracy of the second deep learning model for this category B. Therefore, in this embodiment, samples containing this category B are upsampled from the full data set and cannot be repeated with the samples in C3 to generate an upsampled image set; the upsampled image set and the third image set are combined to generate a fifth image set, and the fifth image set is used as the optimized image set. For other categories with the number of labeled boxes greater than or equal to the minimum reference value, no other operations are required and the current state can be maintained.

[0064] This embodiment improves the accuracy of the deep learning model for small sample categories while maintaining high accuracy for large sample detection.

[0065] An implementation process of this embodiment includes:

[0066] First, the deep learning algorithm is trained using the full data set after noise filtering to obtain the deep learning model M1. Then, the prediction accuracy of each category is obtained using the model M1 on the evaluation set, and at the same time, the proportion and the number of labels of each category in the full data set are counted.

[0067] Set the detection accuracy required for laser perception in intelligent driving to P, record the number of standard boxes of the category with the prediction accuracy closest to P as min_num, record the categories with prediction accuracy < P and the number of standard boxes < min_num as n1,..., nf (f >= 1), and other categories as m1,..., mt (t >= 1). It should be noted that among the categories m1,..., mt (t >= 1), there may be categories with accuracy = min_num. Since the factor restricting the accuracy of this type of category not meeting the conditions is not the number of standard boxes but other factors, they are not considered in this embodiment.

[0068] As Figure 3 shown, the model M1 is finetuned using the optimized training set with sample balance to obtain the model M2. Then, the model M2 is further used to verify on the evaluation set whether the improved accuracy of the small sample categories meets the requirement of >= P.

[0069] The steps to make the optimized training set with sample balance are as Figure 4 shown:

[0070] 1. Select samples containing categories n1, …, nf (f >= 1) from the full set of images that have already participated in training. Among them, for samples with duplicate sample IDs, do not select them repeatedly, only select them once. After screening, form dataset C1;

[0071] 2. Perform data augmentation on categories n1, …, nf (f >= 1) in dataset C1, and keep other categories unchanged to form augmented dataset C2;

[0072] 3. Merge augmented dataset C2 and C1 to form dataset C3.

[0073] 4. Statistically count the annotation quantities num_n1, …, num_nf (f >= 1) of n1, …, nf (f >= 1) in C3 respectively. For category ni where num_ni (i = 1, …, f) (f >= 1) < min_num, perform oversampling in dataset C1 to expand the category ni in dataset C3 so that the number of category ni in C3 reaches min_num.

[0074] 5. Statistically count the annotation quantities num_m1, …, num_mt (t >= 1) of m1, …, mt (t >= 1) in C3 respectively. For category mj where num_mj (j = 1, …, t) (t >= 1) < min_num, upsample samples containing category mj from the full dataset to form dataset P, and denote P + C3 as the optimized dataset Training_Set. The construction of the optimized training dataset with balanced samples is completed.

[0075] Through the description of the above implementation manners, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation manner. Based on such an understanding, the technical solution of this application, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc) and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of this application.

[0076] Embodiment 2

[0077] In this embodiment, a precision control device for object detection is also provided to implement the above embodiments and preferred implementation manners. Those that have been described will not be repeated. As used hereinafter, the term “module” can be a combination of software and / or hardware that realizes a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0078] Figure 5 This is a structural block diagram of a target detection accuracy control device according to an embodiment of this application, such as... Figure 5 As shown, the device includes:

[0079] The training module is used to acquire the full image set and train a deep learning algorithm using the full image set to obtain the first deep learning model.

[0080] The calculation module is used to calculate the prediction accuracy of the first deep learning model for predicting multiple label categories;

[0081] The generation module is used to generate an optimized image set based on the prediction accuracy;

[0082] The fine-tuning module is used to fine-tune the first deep learning model using the optimized image set to obtain the second deep learning model;

[0083] The detection module is used to perform target detection using the second deep learning model.

[0084] Optionally, the generation module includes a first generation unit, configured to determine a minimum reference value for the number of bounding boxes based on the prediction accuracy; select samples containing a first category from the full image set to form a first image set, wherein the first category is a label category whose prediction accuracy is less than a preset accuracy and whose number of bounding boxes is less than the minimum reference value; perform data augmentation on the first category in the first image set to form a second image set; merge the first image set and the second image set to form a third image set, and use the third image set as the optimized image set.

[0085] Optionally, the first generation unit is further configured to filter out all second-category image sets whose prediction accuracy is greater than or equal to a preset accuracy from the full image set; count the total number of bounding boxes for each second category in the image set; select a specified second category with the smallest total number of bounding boxes, and use the value of the total number of bounding boxes of the specified second category as the minimum reference value.

[0086] Optionally, the generation module includes a second generation unit, used to count the number of bounding boxes of the first category in the third image set; determine whether the number of bounding boxes of the first category is less than the minimum reference value; if the number of bounding boxes of the first category is less than the minimum reference value, then oversample the first category to obtain a fourth image set, and use the fourth image set as an optimized image set.

[0087] Optionally, the second generation unit is further configured to oversample the first category in the second image set to generate an oversampled image set; and combine the oversampled image set and the third image set to generate a fourth image set.

[0088] Optionally, the generation module includes a third generation unit, used to count the number of bounding boxes for categories other than the first category in the third image set; determine whether the number of bounding boxes for the other categories is less than the minimum reference value; if the number of bounding boxes for the other categories is less than the minimum reference value, then upsample samples containing the other categories in the full image set to generate an upsampled image set; combine the upsampled image set and the third image set to generate a fifth image set, and use the fifth image set as an optimized image set.

[0089] Optionally, the first generation unit is further configured to translate, randomly crop, randomly scale, add Gaussian noise, or rotate the current frame point cloud map and the annotation box corresponding to the first category in the current frame point cloud map of the first image set to generate a new image set; and use the new image set as the second image set.

[0090] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0091] Example 3

[0092] Embodiments of this application also provide a storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

[0093] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0094] S1. Obtain the full image set and use the full image set to train a deep learning algorithm to obtain the first deep learning model;

[0095] S2, calculate the prediction accuracy of the first deep learning model for multiple label categories;

[0096] S3, Generate an optimized image set based on the predicted accuracy;

[0097] S4, the first deep learning model is fine-tuned using the optimized image set to obtain the second deep learning model;

[0098] S5, the second deep learning model is used for object detection.

[0099] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0100] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0101] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0102] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0103] S1. Obtain the full image set and use the full image set to train a deep learning algorithm to obtain the first deep learning model;

[0104] S2, calculate the prediction accuracy of the first deep learning model for multiple label categories;

[0105] S3, Generate an optimized image set based on the predicted accuracy;

[0106] S4, the first deep learning model is fine-tuned using the optimized image set to obtain the second deep learning model;

[0107] S5, the second deep learning model is used for object detection.

[0108] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0109] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0110] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0111] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0112] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0113] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0114] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0115] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for controlling the accuracy of target detection, characterized in that, The method includes: Obtain the full image set and use the full image set to train a deep learning algorithm to obtain the first deep learning model; Calculate the prediction accuracy of the first deep learning model for multiple label categories; An optimized image set is generated based on the predicted accuracy. The first deep learning model is fine-tuned using the optimized image set to obtain the second deep learning model; The second deep learning model is used for object detection; The method of generating an optimized image set based on the prediction accuracy includes: determining a minimum reference value for the number of bounding boxes based on the prediction accuracy; selecting samples containing a first category from the full image set to form a first image set, wherein the first category is a label category whose prediction accuracy is less than a preset accuracy and whose number of bounding boxes is less than the minimum reference value; performing data augmentation on the first category in the first image set to form a second image set; merging the first image set and the second image set to form a third image set, and using the third image set as the optimized image set; wherein, after merging the first image set and the second image set to form the third image set, the method further includes: counting the number of bounding boxes of other categories in the third image set besides the first category; determining whether the number of bounding boxes of other categories is less than the minimum reference value; if the number of bounding boxes of other categories is less than the minimum reference value, upsampling samples containing the other categories from the full image set to generate an upsampled image set; combining the upsampled image set and the third image set to generate a fifth image set, and using the fifth image set as the optimized image set.

2. The method according to claim 1, characterized in that, The minimum reference value for determining the number of bounding boxes based on the prediction accuracy includes: Select all second-category images from the full image set whose prediction accuracy is greater than or equal to a preset accuracy. Count the total number of bounding boxes for each second category in the image set; Select the category with the smallest total number of annotation boxes and use the total number of annotation boxes in the specified second category as the minimum reference value.

3. The method according to claim 1, characterized in that, After merging the first image set and the second image set to form the third image set, the method further includes: Count the number of bounding boxes in the first category in the third image set; Determine whether the number of bounding boxes in the first category is less than the minimum reference value; If the number of bounding boxes in the first category is less than the minimum reference value, then the first category is oversampled to obtain a fourth image set, which is then used as the optimized image set.

4. The method according to claim 3, characterized in that, Oversampling the first category yields a fourth image set, including: The first category is oversampled in the second image set to generate an oversampled image set; The oversampled image set and the third image set are combined to generate a fourth image set.

5. The method according to claim 1, characterized in that, Data augmentation of the first category in the first image set to form the second image set includes: The point cloud map of the current frame in the first image set and the bounding boxes corresponding to the first category in the point cloud map of the current frame are translated, randomly cropped, randomly scaled, Gaussian noise is added or rotated to generate a new image set; The new image set is used as the second image set.

6. A target detection accuracy control device, characterized in that, include: The training module is used to acquire the full image set and train a deep learning algorithm using the full image set to obtain the first deep learning model. The calculation module is used to calculate the prediction accuracy of the first deep learning model for predicting multiple label categories; The generation module is used to generate an optimized image set based on the prediction accuracy; The fine-tuning module is used to fine-tune the first deep learning model using the optimized image set to obtain the second deep learning model; The detection module is used to perform target detection using the second deep learning model; The generation module includes a first generation unit, configured to determine a minimum reference value for the number of bounding boxes based on the prediction accuracy; select samples containing a first category from the full image set to form a first image set, wherein the first category is a label category whose prediction accuracy is less than a preset accuracy and whose number of bounding boxes is less than the minimum reference value; perform data augmentation on the first category in the first image set to form a second image set; merge the first image set and the second image set to form a third image set, and use the third image set as the optimized image set; The generation module includes a third generation unit, which is used to count the number of bounding boxes in the third image set other than the first category; determine whether the number of bounding boxes in the other categories is less than the minimum reference value; if the number of bounding boxes in the other categories is less than the minimum reference value, then upsample samples containing the other categories in the full image set to generate an upsampled image set; combine the upsampled image set and the third image set to generate a fifth image set, and use the fifth image set as the optimized image set.

7. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, communication interface, and memory communicate with each other through the communication bus; wherein: Memory, used to store computer programs; A processor for executing the method steps of any one of claims 1 to 5 by running a program stored in memory.

8. A storage medium, characterized in that, The storage medium includes a stored program, wherein the program, when executed, performs the method steps of any one of claims 1 to 5.

Images