Method for building deployment models based on corner cases

Abstract

The application relates to a corner case-based deployment model construction method, which comprises the following steps: obtaining a first data set according to automatic driving images and preset open source data, obtaining a second data set according to the automatic driving images, the preset open source data and part of image in a corner case scene image, obtaining a third data set according to the remaining images in the corner case scene image, and merging the second data set and the third data set to obtain a fourth data set; training a preset multi-task detection model based on the fourth data set to obtain an automatic driving perception detection pre-training model, training the pre-training model based on the first data set and a preset period of cosine annealing learning strategy to obtain a plurality of perception detection models; and performing transfer training on the plurality of perception detection models based on the third data set, performing average weight and correction BN on the plurality of perception detection models after the transfer training to obtain a final deployment model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving perception and algorithm training, and particularly relates to a method for constructing a deployment model based on corner cases. BACKGROUND

[0002] Small sample learning technology exists in many different fields today, and can be used to learn some error-prone sample scenarios, create personalized custom services, and improve detection accuracy in indoor and outdoor environments. However, most algorithm applications obtain edge detection information of a scene through multi-model determination, such as laser scanners or structured light sensors assisted search, which are rarely used in most common vehicle types used by mainstream users due to high hardware costs. In any scene environment, only using RGB data from ubiquitous cameras is a challenging problem that may bring new possibilities. Few-shot Learning algorithm research is mostly concentrated in the image field, and Few-shot Learning models can be roughly divided into three categories: Mode Based, Metric Based, and Optimization Based. Among them, the Model Based method aims to quickly update parameters on a small number of samples through the design of the model structure, and directly establishes a mapping function of input x and predicted value P; the Metric Based method measures the distance between samples in the batch set and samples in the support set, and completes classification by means of the nearest neighbor idea; the Optimization Based method considers that the ordinary gradient descent method is difficult to fit in the few-shot scene, and the existing Few-shot Learning technology for corner cases has the disadvantages of low training accuracy and recall rate, difficulty in collecting training data in special scenarios, difficulty in implementation, and high requirements for training hardware.

[0003] Corner cases are a very important problem in automatic driving, and detecting these edge cases through visual perception is a very tricky problem, that is, unexpected or unknown situations occur. The task of detecting corner cases is very critical to safety, and the selection of training data for the detection method is very important for the automation of the technology and for the acceptance of the public to automatic driving technology.

[0004] The algorithm used in the related art can accurately identify larger and more common targets in an image, but small and less common targets have fewer and less clear features, which can easily cause misidentification of small targets, and thus can cause the vehicle to have an incorrect perception of the environment. SUMMARY

[0005] The application provides a corner case-based deployment model construction method to solve the problems of low detection accuracy, low recall rate and complex algorithm of current automatic driving scene perception algorithm, and the method is easy to implement, has high accuracy and high recall rate, has low data collection cost, requires less hardware, greatly reduces training time and is efficient, and is easy to promote to various use scenarios.

[0006] The first aspect of the application provides a corner case-based deployment model construction method, including the following steps: obtaining an automatic driving image of a vehicle and a corner case scene image; obtaining a first data set according to the automatic driving image and preset open source data, obtaining a second data set according to the automatic driving image, the preset open source data and part of the corner case scene images, obtaining a third data set according to the remaining images in the corner case scene images, and merging the second data set and the third data set to obtain a fourth data set; training a preset multi-task detection model based on the fourth data set to obtain an automatic driving perception detection pre-training model, and training the pre-training model based on the first data set and a preset period of cosine annealing learning strategy to obtain a plurality of perception detection models; performing transfer training on the plurality of perception detection models based on the third data set, and performing average weight and correction BN on the plurality of perception detection models after transfer training to obtain a final deployment model.

[0007] According to the above technical means, the transfer training can quickly improve the accuracy of all categories. Small network input and lightweight detection model can be used to ensure high throughput and accuracy, the training results are rich, and diversified category training results are provided. Different task models and data in different scenes can be personalized according to actual needs.

[0008] Further, after obtaining the final deployment model, the method further includes: retraining the preset multi-task detection model based on the fourth data set, and obtaining a first number of times when an evaluation judgment index error is less than or equal to a preset threshold in a training process; retraining the plurality of perception detection models based on the third data set, and obtaining a second number of times when an evaluation judgment index error is less than or equal to a preset threshold in a training process; if the first number of times and the second number of times both satisfy a preset number of times, it is determined that the final deployment model satisfies a preset accuracy requirement, and the final deployment model is released.

[0009] According to the above technical means, through continuous model algorithm training, the situation that the object in the background image is blurred and suspected to be a corner case can be avoided, and the method has the advantages of high recall, low misjudgment and high overall accuracy.

[0010] Further, the first data set is obtained according to the automatic driving image and the preset open source data, and the method comprises: labeling a target of interest in the automatic driving image, wherein the labeling comprises classification labeling, detection labeling, segmentation labeling, key point labeling and corner point labeling; and the first data set is formed based on the automatic driving image, the classification labeling, the detection labeling, the segmentation labeling, the key point labeling and the corner point labeling.

[0011] According to the above technical means, by synthesizing the target image of interest, the recall rate in the corner case scenario can be improved.

[0012] Further, the second data set is obtained according to the automatic driving image, the preset open source data and part of the images in the corner case scenario image, and the method comprises: obtaining mis-detection data in part of the images in the corner case scenario image; labeling the mis-detection data, and obtaining the second data set according to the labeled part of the images in the corner case scenario image, the automatic driving image and the preset open source data.

[0013] According to the above technical means, the overall correctness is improved and the misjudgment rate is reduced.

[0014] Further, the third data set is obtained according to the remaining images in the corner case scenario image, and the method comprises: based on a preset adjustment strategy, adjusting the target of interest cut out from the remaining images in the corner case scenario image to obtain a new image; based on the new image, generating an adversarial image by using a preset Gan network algorithm, and obtaining the third data set according to the label in the adversarial image and the adversarial image.

[0015] According to the above technical means, the relative size and relative position information generated by using the cutout and Gan network algorithm can remove unreasonable target labeling.

[0016] The second aspect embodiment of the application provides a corner case-based deployment model construction method, comprising: a first acquisition module configured to acquire an automatic driving image of a vehicle and a corner case scene image; a second acquisition module configured to obtain a first data set according to the automatic driving image and preset open source data, obtain a second data set according to the automatic driving image, the preset open source data and part of the corner case scene images, obtain a third data set according to the remaining corner case scene images, and obtain a fourth data set by merging the second data set and the third data set; a first training module configured to train a preset multi-task detection model based on the fourth data set to obtain an automatic driving perception detection pre-training model, and train the pre-training model based on the first data set and a preset periodic cosine annealing learning strategy to obtain a plurality of perception detection models; and a second training module configured to perform transfer training on the plurality of perception detection models based on the third data set, and perform average weight and correction BN on the plurality of perception detection models after the transfer training to obtain a final deployment model.

[0017] Further, after obtaining the final deployment model, the second training module is further configured to retrain the preset multi-task detection model based on the fourth data set, and obtain a first number of times when an evaluation criterion error is less than or equal to a preset threshold during training; retrain the plurality of perception detection models based on the third data set, and obtain a second number of times when an evaluation criterion error is less than or equal to a preset threshold during training; and if the first number of times and the second number of times both satisfy a preset number of times, it is determined that the final deployment model satisfies a preset accuracy requirement, and the final deployment model is released.

[0018] Further, the second acquisition module is further configured to label a target of interest in the automatic driving image, wherein the labeling comprises classification labeling, detection labeling, segmentation labeling, key point labeling and corner point labeling; and the first data set is constructed based on the classification labeling, the detection labeling, the segmentation labeling, the key point labeling and the corner point labeling.

[0019] Further, the second acquisition module is further configured to obtain mis-detection data in part of the corner case scene images; label the mis-detection data, and obtain the second data set according to part of the labeled corner case scene images, the automatic driving image and the preset open source data.

[0020] Further, the second acquisition module is further configured to: based on a preset adjustment strategy, adjust a remaining image in the corner case scene image to obtain a new image by cutting out a target of interest; and based on the new image, generate an adversarial image by using a preset Gan network algorithm, and obtain the third data set according to a label in the adversarial image and the adversarial image.

[0021] The third aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the corner case-based deployment model construction method as described in the above embodiments.

[0022] The fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the corner case-based deployment model construction method as described in the above embodiments.

[0023] Therefore, according to the first data set, the second data set and the third data set, the second data set and the third data set are merged to obtain a fourth data set, a preset multi-task detection model is trained to obtain an automatic driving perception detection pre-training model, a preset period of cosine annealing learning strategy is used to train the pre-training model to obtain a plurality of perception detection models, and the plurality of perception detection models after migration training are averaged in weight and corrected in BN to obtain a final deployment model. Therefore, the problems of low detection accuracy, low recall rate and complex algorithm of the current automatic driving scene perception algorithm are solved, and the method is easy to implement, has high accuracy and high recall rate, has low data collection cost, requires less hardware, greatly reduces training time and is efficient, and is easy to promote to various use scenarios.

[0024] Additional aspects and advantages of the present application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0025] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the following description, taken in conjunction with the accompanying drawings, in which:

[0026] Figure 1 A flowchart of a corner case-based deployment model construction method according to an embodiment of the present application is provided.

[0027] Figure 2 A flowchart of a corner case-based deployment model construction method according to an embodiment of the present application is provided.

[0028] Figure 3 A block schematic diagram of an apparatus for constructing a corner case based deployment model according to an embodiment of the present application;

[0029] Figure 4 A structural schematic diagram of an electronic device according to an embodiment of the present application.

[0030] Label explanation: 10 - corner case based deployment model construction apparatus, 100 - first acquisition module, 200 - second acquisition module, 300 - first training module, 400 - second training module, 401 - memory, 402 - processor, 403 - communication interface. DETAILED DESCRIPTION

[0031] Embodiments of the present application are described in detail below with reference to examples thereof shown in the attached drawings, wherein the same or similar numerals represent the same or similar elements throughout. The embodiments described below by reference to the drawings are exemplary and are intended to explain the present application, and cannot be understood as limiting the present application.

[0032] A corner case based deployment model construction method of an embodiment of the present application is described below with reference to the drawings. In view of the problems of low detection accuracy, low recall rate and complex algorithm not easy to implement of the current automatic driving scene perception algorithm mentioned in the background art, the present application provides a corner case based deployment model construction method, in which a first data set, a second data set and a third data set are acquired, the second data set and the third data set are merged to obtain a fourth data set, a pre-set multi-task detection model is trained to obtain an automatic driving perception detection pre-training model, a pre-training model is trained using a cosine annealing learning strategy of a pre-set period to obtain a plurality of perception detection models, and the plurality of perception detection models after migration training are subjected to average weight and correction BN to obtain a final deployment model. Thus, the problems of low detection accuracy, low recall rate and complex algorithm not easy to implement of the current automatic driving scene perception algorithm are solved, and the method is easy to implement, has high accuracy and high recall rate, has low data collection cost, requires less hardware, greatly reduces training time and is efficient, and is easy to promote to various use scenarios.

[0033] Specifically, Figure 1 A flowchart of a corner case based deployment model construction method provided by an embodiment of the present application.

[0034] As Figure 1 shown, the corner case based deployment model construction method includes the following steps:

[0035] In step S101, an automatic driving image and a corner case scene image of a vehicle are acquired.

[0036] Specifically, the automatic driving image of the vehicle, including the image of the environment around the automatic driving vehicle, can be acquired by a camera arranged in the automatic driving vehicle, and the driving environment at different time points of the vehicle, including the starting position and the ending position, as well as the road condition and the weather condition during the driving of the vehicle, are photographed.

[0037] And the corner case scene image of the vehicle during the automatic driving process is acquired by the camera of the vehicle, such as the scene image of the vehicle running a red light, the pedestrian crossing the road, or the intersection with a damaged traffic light.

[0038] In step S102, a first data set is obtained according to the automatic driving image and the preset open source data, a second data set is obtained according to the automatic driving image, the preset open source data and part of the corner case scene images, a third data set is obtained according to the remaining corner case scene images, and a fourth data set is obtained by merging the second data set and the third data set.

[0039] Further, in some embodiments, the first data set obtained according to the automatic driving image and the preset open source data comprises: labeling the target of interest in the automatic driving image, wherein the labeling comprises classification labeling, detection labeling, segmentation labeling, key point labeling and corner point labeling; and the first data set is constituted according to the classification labeling, the detection labeling, the segmentation labeling, the key point labeling and the corner point labeling based on the automatic driving image.

[0040] Specifically, as shown in Figure 2 , first, the preset open source data of the vehicle is acquired from the open source data set, and the target of interest is cut out from the automatic driving image, and after data enhancement, random flipping and noise addition, the target of interest is pasted on the image of the target scene, and the target of interest is labeled to generate classification labeling, detection labeling, segmentation labeling, key point labeling and corner point labeling, and the first data set is constituted by the automatic driving image.

[0041] Further, in some embodiments, the second data set obtained according to the automatic driving image, the preset open source data and part of the corner case scene images comprises: acquiring the mis-detection data in part of the corner case scene images; labeling the mis-detection data, and obtaining the second data set according to the labeled part of the corner case scene images, the automatic driving image and the preset open source data.

[0042] It can be understood that the embodiment of the application collects corner case scene images, obtains mis-detection data in the corner case scene images, labels the mis-detection data, generates new classification labels, detection labels, segmentation labels, key point labels and corner point labels, and obtains a second data set according to part of the labeled corner case scene images, autonomous driving images and pre-set open source data.

[0043] Further, in some embodiments, a third data set is obtained according to the remaining images in the corner case scene images, including: based on a pre-set adjustment strategy, adjusting the target of interest cut out from the remaining images in the corner case scene images to obtain new images; based on the new images, generating adversarial images using a pre-set Gan network algorithm, and obtaining the third data set according to the labels in the adversarial images and the adversarial images.

[0044] It should be understood that the embodiment of the application selects a certain number of corner case scene images, cuts out the target of interest, randomly changes the size, direction and color saturation, and pastes it on the background image to synthesize a new image, and uses a pre-set Gan network algorithm to generate an adversarial image, and calculates the label of the region of interest in the new image, and obtains the third data set based on the labels in the adversarial images and the adversarial images.

[0045] Further, after obtaining the first data set, the second data set and the third data set, the second data set and the third data set are merged to obtain a fourth data set.

[0046] In step S103, based on the fourth data set, the pre-set multi-task detection model is trained to obtain an autonomous driving perception detection pre-training model, and based on the first data set and a pre-set period of cosine annealing learning strategy, the pre-training model is trained to obtain a plurality of perception detection models.

[0047] Specifically, the embodiment of the application uses a pre-set multi-task detection model to train on the fourth data set, trains the model to convergence using a contrastive learning method, obtains a first autonomous driving perception detection pre-training model, evaluates error data and uploads record information, and uses the first autonomous driving perception detection pre-training model to train on the first data set, uses a swa algorithm, and uses a pre-set period of cosine annealing learning strategy with a period of 6 to train to convergence, selects 6 perception detection models with minimum loss, evaluates error data and uploads record information to the corner case data set.

[0048] In step S104, based on the third data set, the plurality of perception detection models are trained by migration, and the plurality of perception detection models after migration training are averaged in weight and corrected in BN to obtain a final deployment model.

[0049] Specifically, the plurality of perception detection models are trained on the third data set until convergence, and six perception detection models are obtained. The six perception detection models are averaged and corrected by BN to obtain a final deployment model. Error data are evaluated in real time, and record information is uploaded to the corner case data set.

[0050] Further, in some embodiments, after obtaining the final deployment model, the method further comprises: retraining the preset multi-task detection model based on the fourth data set, and obtaining a first number of times when an evaluation index error is less than or equal to a preset threshold during the training process; retraining the plurality of perception detection models based on the third data set, and obtaining a second number of times when the evaluation index error is less than or equal to the preset threshold during the training process; and if the first number of times and the second number of times both satisfy a preset number of times, determining that the final deployment model satisfies a preset accuracy requirement, and publishing the final deployment model.

[0051] It can be understood that, retraining the preset multi-task detection model based on the fourth data set, if the first number of times when the evaluation index error occurs during the training process is less than or equal to the preset threshold, it is determined that the final deployment model satisfies the preset accuracy requirement, and the final deployment model is published; retraining the plurality of perception detection models based on the third data set, if the number of times when the evaluation index error occurs during the training process is less than or equal to the preset threshold, it is determined that the final deployment model satisfies the preset accuracy requirement, and the final deployment model is published, otherwise it is determined that the final deployment model does not satisfy the preset accuracy requirement.

[0052] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0053] 1. The training data requirement is small. No additional auxiliary data set is needed, and a small amount of corner case scenes can be used to collect the required images.

[0054] 2. The training data is easy to obtain. The automatic driving detection data set is easy to obtain, and the open source data set and the internally collected images are also easy to obtain. The use of artificially synthesized corner case images can greatly increase the number of uncommon targets.

[0055] 3. The trained model has high precision indicators. Both normal and corner case scenarios can achieve high accuracy. By synthesizing the target image of interest, the recall rate in the corner case scenario can be improved, and the relative size and relative position information generated by the matting and Gan network algorithm can remove unreasonable target labeling. Further, according to the multi-stage small sample learning strategy, the precision can be further improved. Further, by using a continuous model algorithm, the object blur in the background image can be avoided. The present application has the advantages of high recall, low misjudgment, and high overall accuracy.

[0056] 4. Low training cost and fast speed. The first stage of the present application is to train large data, and the target in the common scene is easy to learn. Then, the second stage uses all the data for transfer training to quickly improve the accuracy of all categories. Small network input and lightweight detection model can be used to ensure high throughput and accuracy.

[0057] 5. The model training result is rich, and provides diversified category training results. According to actual needs, individualization can be performed, and different task models and data in different scenes can be trained.

[0058] According to the corner case-based deployment model construction method proposed in the embodiments of the present application, the first data set, the second data set and the third data set are obtained, the second data set and the third data set are merged to obtain a fourth data set, a preset multi-task detection model is trained, an automatic driving perception detection pre-training model is obtained, a pre-training model is trained using a preset period of cosine annealing learning strategy, a plurality of perception detection models are obtained, and the plurality of perception detection models after transfer training are averaged and corrected BN to obtain a final deployment model. Thus, the problems of low detection accuracy, low recall rate and complex algorithm of the current automatic driving scene perception algorithm are solved, which is easy to implement, has high accuracy and high recall rate, has low data collection cost, requires less hardware, greatly reduces training time and is efficient, and is easy to promote to various use scenarios.

[0059] Next, the corner case-based deployment model construction device according to the embodiments of the present application is described with reference to the accompanying drawings.

[0060] Figure 3 is a block diagram of the corner case-based deployment model construction device according to the embodiments of the present application.

[0061] As shown in Figure 3 , the corner case-based deployment model construction device 10 includes a first acquisition module 100, a second acquisition module 200, a first training module 300 and a second training module 400.

[0062] The first acquisition module 100 is configured to acquire an automatic driving image and a corner case scene image of a vehicle. The second acquisition module 200 is configured to obtain a first data set according to the automatic driving image and preset open source data, obtain a second data set according to the automatic driving image, the preset open source data and part of the images in the corner case scene image, obtain a third data set according to the remaining images in the corner case scene image, and obtain a fourth data set by merging the second data set and the third data set. The first training module 300 is configured to train a preset multi-task detection model based on the fourth data set to obtain an automatic driving perception detection pre-training model, and train the pre-training model based on the first data set and a preset periodic cosine annealing learning strategy to obtain a plurality of perception detection models. The second training module 400 is configured to perform transfer training on the plurality of perception detection models based on the third data set, and perform average weight and correction BN on the plurality of perception detection models after the transfer training to obtain a final deployment model.

[0063] Further, after obtaining the final deployment model, the second training module 400 is further configured to retrain the preset multi-task detection model based on the fourth data set, and obtain a first number of times when an evaluation judgment index error is less than or equal to a preset threshold value in a training process. The second training module 400 is further configured to retrain the plurality of perception detection models based on the third data set, and obtain a second number of times when the evaluation judgment index error is less than or equal to the preset threshold value in the training process. If the first number of times and the second number of times both satisfy a preset number of times, it is determined that the final deployment model satisfies a preset accuracy requirement, and the final deployment model is released.

[0064] Further, the second acquisition module 200 is further configured to label a target of interest in the automatic driving image, wherein the labeling includes classification labeling, detection labeling, segmentation labeling, key point labeling and corner point labeling. The second acquisition module 200 is further configured to construct the first data set according to the classification labeling, the detection labeling, the segmentation labeling, the key point labeling and the corner point labeling based on the automatic driving image.

[0065] Further, the second acquisition module 200 is further configured to obtain mis-detection data in part of the images in the corner case scene image, label the mis-detection data, and obtain the second data set according to the labeled part of the images in the corner case scene image, the automatic driving image and the preset open source data.

[0066] Further, the second acquisition module 200 is further configured to adjust the target of interest in the remaining images in the corner case scene image based on a preset adjustment strategy to obtain new images, generate adversarial images by using a preset Gan network algorithm based on the new images, and obtain the third data set according to labels in the adversarial images and the adversarial images.

[0067] It should be noted that the foregoing explanation of the embodiment of the method for constructing a deployment model based on corner cases also applies to the embodiment of the device for constructing a deployment model based on corner cases, and will not be repeated here.

[0068] The device for constructing a deployment model based on corner cases according to the embodiment of the present application obtains a first data set, a second data set and a third data set, merges the second data set and the third data set to obtain a fourth data set, trains a preset multi-task detection model to obtain an automatic driving perception detection pre-training model, trains the pre-training model by using a cosine annealing learning strategy with a preset period to obtain a plurality of perception detection models, performs transfer training on the plurality of perception detection models, and performs average weight and correction BN on the plurality of perception detection models after the transfer training to obtain a final deployment model. Thus, the problems of low detection accuracy, low recall rate and complex algorithm of the current automatic driving scene perception algorithm are solved, and the method is easy to implement, has high accuracy and high recall rate, has low data collection cost, requires less hardware, greatly reduces training time and is efficient, and is easy to promote to various use scenarios.

[0069] Figure 4 The electronic device provided in the embodiment of the present application has the structure shown in the structural schematic diagram of the electronic device. The electronic device can include:

[0070] The memory 401, the processor 402 and the computer program stored in the memory 401 and executable on the processor 402.

[0071] The processor 402 implements the method for constructing a deployment model based on corner cases provided in the above embodiments when executing the program.

[0072] Further, the electronic device further includes:

[0073] The communication interface 403 is used for communication between the memory 401 and the processor 402.

[0074] The memory 401 is used to store the computer program executable on the processor 402.

[0075] The memory 401 can include a high-speed RAM (Random Access Memory, random access memory) memory, and can also include a non-volatile memory, such as at least one disk memory.

[0076] If the memory 401, the processor 402 and the communication interface 403 are implemented independently, the communication interface 403, the memory 401 and the processor 402 can be connected with each other through a bus and complete communication between each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 4 Only one thick line is used in the figure to represent the bus, but it does not mean that there is only one bus or only one type of bus.

[0077] Optionally, in a specific implementation, if the memory 401, the processor 402 and the communication interface 403 are integrated on a chip, the memory 401, the processor 402 and the communication interface 403 can complete communication between each other through an internal interface.

[0078] The processor 402 can be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.

[0079] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the corner case-based deployment model construction method.

[0080] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example" or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms is not necessarily for the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples without contradiction.

[0081] In addition, the terms "first", "second", etc. are used only for descriptive purposes and do not connote or imply relative importance or a quantity of the indicated technical features. Thus, a feature defined with "first", "second", etc. can include at least one of the feature, explicitly or implicitly. In the description of the present application, the meaning of "N" is at least two, such as two, three, etc., unless otherwise explicitly and specifically limited.

[0082] Any process or method descriptions or descriptions of the flow diagrams described herein can be understood as representing code modules, segments, or portions of code which include one or more executable instructions for implementing specific logic functions (or steps) of the application, and that the various systems described herein can be implemented with or without the use of hardware components software components, and that the described embodiments of the application can be implemented using combination of both hardware and software components. It should also be understood that the various figures described herein are merely intended to simplify the generic structure and functions of these modules, segments, or portions of code, and are therefore not interpreted to specifically define measurement of limits of the scope of an embodiment of the present application.

[0083] It should be understood that portions of the present application can be implemented in hardware, software, firmware, or combinations thereof. In the above embodiments, the N steps or methods can be implemented with software or firmware stored in memory and executed by a suitable instruction execution system. As in another embodiment, if implemented in hardware, any of the following technologies known in the art or their combinations can be used: discrete logic circuit with logic gates for implementing logical functions on data signals, application specific integrated circuit with appropriate combination logic gates, programmable gate array, field programmable gate array, etc.

[0084] Those skilled in the art of the present technology can understand that all or part of the steps carried out by the above-mentioned embodiment methods can be completed by programs instructing related hardware, and the programs can be stored in a computer readable storage medium, and the programs include one or a combination of steps of the method embodiments when executed.

[0085] Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and cannot be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A method for constructing a deployment model based on corner cases, characterized in that, The method comprises the following steps: obtaining automatic driving images and corner case scene images of a vehicle; obtaining a first data set according to the automatic driving images and preset open source data, obtaining a second data set according to the automatic driving images, the preset open source data and part of the corner case scene images, obtaining a third data set according to the remaining corner case scene images, and obtaining a fourth data set by merging the second data set and the third data set; training a preset multi-task detection model based on the fourth data set to obtain an automatic driving perception detection pre-training model, and training the pre-training model based on the first data set and a preset period of cosine annealing learning strategy to obtain a plurality of perception detection models; and based on the third data set, the plurality of perception detection models are trained by migration, and the plurality of perception detection models after migration training are subjected to average weight and correction BN to obtain a final deployment model; the second data set is obtained according to the automatic driving images, the preset open source data and part of the corner case scene images, comprising: obtaining misjudgment data in part of the corner case scene images; labeling the misjudgment data, and obtaining the second data set according to the labeled part of the corner case scene images, the automatic driving images and the preset open source data; the third data set is obtained according to the remaining corner case scene images, comprising: based on a preset adjustment strategy, adjusting the target of interest cut out from the remaining corner case scene images to obtain new images; based on the new images, generating an adversarial image using a preset Gan network algorithm, and obtaining the third data set according to the label in the adversarial image and the adversarial image.

2. The method of claim 1, wherein, After obtaining the final deployment model, it further comprises: retraining the preset multi-task detection model based on the fourth data set, and obtaining a first number of times when the evaluation judgment index error is less than or equal to a preset threshold in the training process; retraining the plurality of perception detection models based on the third data set by migration, and obtaining a second number of times when the evaluation judgment index error is less than or equal to a preset threshold in the training process; if the first number of times and the second number of times both satisfy a preset number of times, it is determined that the final deployment model satisfies a preset accuracy requirement, and the final deployment model is released.

3. The method of claim 1, wherein, the first data set is obtained according to the automatic driving images and preset open source data, comprising: labeling the target of interest in the automatic driving images, wherein the labeling comprises classification labeling, detection labeling, segmentation labeling, key point labeling and corner point labeling; based on the automatic driving images, the first data set is constituted according to the classification labeling, the detection labeling, the segmentation labeling, the key point labeling and the corner point labeling.

4. A corner case based deployment model building apparatus, characterized by, comprising: The first acquisition module is configured to acquire an automatic driving image and a corner case scene image of a vehicle. The second acquisition module is configured to obtain a first data set according to the automatic driving image and preset open source data, obtain a second data set according to the automatic driving image, the preset open source data and part of the corner case scene images, obtain a third data set according to the remaining corner case scene images, and obtain a fourth data set by merging the second data set and the third data set. The first training module is configured to train a preset multi-task detection model based on the fourth data set to obtain an automatic driving perception detection pre-training model, and train the pre-training model based on the first data set and a preset periodic cosine annealing learning strategy to obtain a plurality of perception detection models. The second training module is configured to perform transfer training on the plurality of perception detection models based on the third data set, and perform average weight and correction BN on the plurality of perception detection models after the transfer training to obtain a final deployment model. The second acquisition module is configured to obtain a first data set according to the automatic driving image and preset open source data, obtain a second data set according to the automatic driving image, the preset open source data and part of the corner case scene images, obtain a third data set according to the remaining corner case scene images, and obtain a fourth data set by merging the second data set and the third data set. The second acquisition module is configured to obtain a first data set according to the automatic driving image and preset open source data, obtain a second data set according to the automatic driving image, the preset open source data and part of the corner case scene images, obtain a third data set according to the remaining corner case scene images, and obtain a fourth data set by merging the second data set and the third data set. The second acquisition module is configured to obtain a first data set according to the automatic driving image and preset open source data, obtain a second data set according to the automatic driving image, the preset open source data and part of the corner case scene images, obtain a third data set according to the remaining corner case scene images, and obtain a fourth data set by merging the second data set and the third data set. After obtaining the final deployment model, the second training module is further configured to: retrain the preset multi-task detection model based on the fourth data set, and obtain a first number of times when an evaluation judgment index error in a training process is less than or equal to a preset threshold value; retrain the plurality of perception detection models based on the third data set, and obtain a second number of times when an evaluation judgment index error in a training process is less than or equal to a preset threshold value; if the first number of times and the second number of times both satisfy a preset number of times, it is determined that the final deployment model satisfies a preset precision requirement, and the final deployment model is released.

5. The apparatus of claim 4, wherein, The second acquisition module is further configured to: label an object of interest in the automatic driving image, wherein the labeling includes classification labeling, detection labeling, segmentation labeling, key point labeling and corner point labeling; based on the automatic driving image, the first data set is constituted by the classification labeling, the detection labeling, the segmentation labeling, the key point labeling and the corner point labeling. ​ 6. The apparatus of claim 4, wherein, ​ ​ ​ 7. An electronic device, comprising: ​ A memory, a processor, and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the corner case based deployment model building method of any of claims 1-3.

8. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor for implementing the corner case based deployment model building method of any of claims 1-3.

Images