A method for training and validating an autonomous driving algorithm based on a virtual simulation scenario

Abstract

The present invention provides a method for training and verifying an autonomous driving algorithm based on a virtual simulation scenario, belonging to the technical field of data processing. Specifically, it includes: determining a deviation recognition target in a target object based on recognition deviation data in different test environments, using recognition images with deviation recognition targets in the recognition data as matching modeling images, determining matching modeling images for amplification processing in the test scenario based on the distribution association data of the deviation recognition targets in the matching modeling images and the recognition deviation data of the deviation recognition targets in the test scenario, combining the matching modeling images to determine modeling amplification images in the test scenario, performing modeling processing on test data in the test scenario based on the modeling amplification images, the matching modeling images, and the recognition deviation images in the test scenario, and using the test data for training and verification processing of the autonomous driving algorithm, thereby improving the efficiency and accuracy of the training process.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and particularly relates to a method for training and verifying an autonomous driving algorithm based on a virtual simulation scenario. Background Art

[0002] In order to implement the training process of the autonomous driving algorithm, existing technical solutions often perform the training and verification of the autonomous driving algorithm in a real environment. However, there are many defects in the construction of the real driving scenario. Specifically, in the invention patent application CN202411162675.6, "A Method, System and Medium for Generating an Autonomous Driving Simulation Test Scenario", the test scenario is imported into the simulation platform for testing, so that a test scenario including snow day characteristics can be automatically generated, comprehensively covering various situations that may be encountered in snow day driving. Using the generated snow day simulation test scenario, the safety of the autonomous driving system can be tested, but there are the following technical problems:

[0003] When performing the construction process of the virtual scenario, since there are large differences in the recognition accuracy of different objects by the vehicle in different driving scenarios, if the virtual scenario cannot be constructed according to the recognition accuracy, the reliability of the training process cannot be guaranteed.

[0004] In view of the above technical problems, the present invention provides a method for training and verifying an autonomous driving algorithm based on a virtual simulation scenario. Summary of the Invention

[0005] To achieve the object of the present invention, the present invention adopts the following technical solutions:

[0006] According to one aspect of the present invention, there is provided a method for training and verifying an autonomous driving algorithm based on a virtual simulation scenario.

[0007] A method for training and verifying an autonomous driving algorithm based on a virtual simulation scenario specifically includes:

[0008] S1 Using the recognition data of the autonomous driving algorithm in different test scenarios, determine the recognition deviation data of the target object in different test scenarios. When it is determined that the recognition deviation state in the test scenario meets the requirements, proceed to the next step;

[0009] S2 Based on the recognition data of different target objects in different test scenarios, determine the recognition deviation data of different target objects in different test scenarios, and use the recognition deviation data in different test environments to determine the deviation recognition target among the target objects;

[0010] S3 uses the recognition image with deviation recognition targets in the recognition data as the matching modeling image, determines the matching modeling image for amplification processing in the test scenario based on the distribution association data of the deviation recognition targets in the matching modeling image and the recognition deviation data of the deviation recognition targets in the test scenario, and determines the modeling amplification image in the test scenario in combination with the matching modeling image;

[0011] S4 performs modeling processing on the test data in the test scenario based on the modeling amplification image, the matching modeling image, and the recognition deviation image in the test scenario, and uses the test data for training and verification processing of the autonomous driving algorithm.

[0012] A further technical solution is that the test scenario includes rain, cloudy, snow, and fog.

[0013] A further technical solution is that the recognition deviation data of the target object includes the number of recognition deviations of the target object.

[0014] A further technical solution is that determining that the recognition deviation state in the test scenario meets the requirements specifically includes:

[0015] Using the recognition deviation data to determine the number of recognition deviations of different target objects in the test scenario, and using the number of recognition deviations to determine the recognition deviation objects among the target objects;

[0016] Determine whether the recognition deviation state in the test scenario meets the requirements according to the number of the recognition deviation objects.

[0017] A further technical solution is that the recognition deviation object is a target object with the number of recognition deviations greater than the preset deviation number.

[0018] A further technical solution is that the amplification processing amplifies and reduces the image size of the recognition deviation targets in the matching modeling image according to a preset level.

[0019] A further technical solution is that performing modeling processing on the test data in the test scenario specifically includes:

[0020] Using the modeling amplification image, the matching modeling image, and the recognition deviation image in the test scenario as basic data, and performing modeling processing on the test data in the test scenario based on the basic data.

[0021] A further technical solution is that using the test data in different test scenarios for training and verification processing of the autonomous driving algorithm specifically includes:

[0022] Import the model built by the autonomous driving algorithm into the virtual simulation scenario constructed by different test scenarios, select the running mode, and the virtual car runs automatically in the scenario to complete the road condition recognition in the road scenario. According to the recognition result, the car can make corresponding actions to complete the training and verification of the autonomous driving algorithm of the car in the virtual simulation scenario.

[0023] The beneficial effects of the present invention are as follows:

[0024] Based on the distribution correlation data of the deviation recognition target in the matching modeling image and the recognition deviation data of the deviation recognition target in the test scenario, determine the matching modeling image with amplification processing in the test scenario, so as to fully consider the distribution of different recognition deviation targets in the real recognition scenario, realize the screening of the matching modeling image with more recognition deviation targets or the recognition deviation not meeting the requirements, and lay a foundation for improving the test reliability in the test scenario.

[0025] Based on the modeling amplified image, the matching modeling image and the recognition deviation image in the test scenario, perform modeling processing on the test data in the test scenario. Not only consider the recognition deviation image in the test scenario, but also further consider the matching modeling image containing the deviation recognition target and the modeling amplified image with a larger recognition deviation probability, realize the modeling processing of the test data in the test scenario from multiple angles, and improve the reliability of the training and verification processing of the autonomous driving algorithm.

[0026] Other features and advantages will be described in the subsequent specification. The objectives and other advantages of the present invention are achieved and obtained by the structures specifically pointed out in the specification and the drawings.

[0027] To make the above objectives, features and advantages of the present invention more obvious and understandable, the following specific preferred embodiments are given, and in conjunction with the accompanying drawings, the detailed description is as follows. Description of the Drawings

[0028] By referring to the drawings and describing its exemplary embodiments in detail, the above and other features and advantages of the present invention will become more obvious.

[0029] Figure 1 is a flowchart of a method for training and verifying an autonomous driving algorithm based on a virtual simulation scenario;

[0030] Figure 2 is a flowchart of a method for determining that the recognition deviation state in the test scenario meets the requirements;

[0031] Figure 3 is a flowchart of a method for determining the deviation recognition target in the target object;

[0032] Figure 4 is a flowchart of a method for determining the matching modeling image with amplification processing in the test scenario. Detailed implementation manners

[0033] When the present invention performs the construction process of the virtual scene, the virtual scene is constructed according to the recognition accuracy rates of different objects in different driving scenarios of the vehicle. On the basis of ensuring the recognition accuracy rate, the difficulty of the construction process of the virtual scene is reduced at the same time.

[0034] Determine that the recognition deviation state in the test scenario meets the requirements: Determine based on the number of recognition deviations of the target object in different test scenarios. When the number of recognition deviations in the test scenario is greater than 100 times, it is determined that the recognition deviation state in the test scenario does not meet the requirements.

[0035] Deviation recognition target: Determine based on the recognition deviation data of the target object in different test environments. Specifically, determine the number of test environments with the number of recognition deviations greater than 50 times. When the number of test environments with the number of recognition deviations greater than 50 times is greater than 5, it is determined that the recognition target is the deviation recognition target.

[0036] The matching modeling image for amplification processing in the test scenario: Use the matching modeling image with the total number of recognition deviations of the deviation recognition target greater than 200 times as the matching modeling image for amplification processing in the test scenario.

[0037] Perform the modeling process of the test data in the test scenario: Use the modeling amplified image, the matching modeling image, and the recognition deviation image in the test scenario as the basic data. Based on the basic data, perform the modeling process of the test data in the test scenario. Import the model constructed by the autonomous driving algorithm into the virtual simulation scenario constructed by different test scenarios, select the operation mode, and the virtual car automatically runs in the scenario to complete the road condition recognition in the road scenario. According to the recognition result, the car can make corresponding actions to complete the training and verification of the autonomous driving algorithm of the car in the virtual simulation scenario.

[0038] Driving data collection

[0039] Video recording:

[0040] Enter the recording mode in the virtual platform to perform the collection work. In this mode, the user will move in the scenario in the first-person perspective and can modify the scenario parameters (weather, traffic lights, road signs, etc.) through the function menu, so as to customize the configuration of the environment and traffic signs of the entire virtual scene and perform multi-angle collection. Click the start recording button to start recording the current screen. Click the stop recording button, and this section of the screen will be automatically saved in the specified directory in the MP4 format.

[0041] Script processing:

[0042] Read multiple segments of MP4 files and save the pictures in the file to the specified folder according to the read frame frequency (if the current picture is the same as the previously saved picture, it will not be saved repeatedly to ensure the diversity of training materials).

[0043] Data annotation. After processing the video into annotatable pictures, perform data annotation.

[0044] Annotation implementation algorithm:

[0045] When the user clicks on the chart list, render the picture to the specified area.

[0046] When the user starts to annotate, first judge by the x-axis distance and y-axis distance of the click area from the upper left corner (origin) of the window and the x-axis distance and y-axis distance of the picture from the origin when clicking: If both the x-axis and y-axis distances of the click position are less than the distance from the upper left corner of the picture to the origin, it is judged that the click is outside, and the annotation box starts to be annotated from the upper left corner of the picture; If the current x-axis or y-axis distance from the origin exceeds the x-axis plus the picture width or the y-axis plus the picture length of the picture from the upper left corner of the window, it is judged that it exceeds, and the position of the annotation box is specified as the maximum position corresponding to the distance of the picture from the origin to prevent the annotation box from exceeding the picture.

[0047] When the user starts to drag the mouse for annotation, perform the above judgment and corresponding operations in real time during dragging, and calculate and store the data in real time. Four data need to be stored (the ratio of the x-axis distance of the center of the annotation box in the picture to the picture width, the ratio of the y-axis distance of the center of the annotation box in the picture to the picture height, the ratio of the width of the annotation box to the picture width, the ratio of the height of the annotation box to the picture height). The width and height of the annotation box are the absolute values of the mouse movement distance after the initial click position (the mouse moves to the right as a positive number, the mouse moves to the left as a negative number, the mouse moves down as a positive number, and the mouse moves up as a negative number). The x-axis position of the center point of the annotation box in the picture is the x-axis position of the initial click position in the picture plus half of the positive or negative number, and the y-axis position of the center point of the annotation box in the picture is the y-axis position of the initial click position in the picture plus half of the positive or negative number.

[0048] Dividing the above four values by the width and height of the picture can obtain the above four values and store them.

[0049] When the user releases the mouse, it is regarded as the end of annotation;

[0050] Save the annotation information to a file with the same name as the picture and the suffix.txt.

[0051] Model training

[0052] Organize the labeled images and the labeled information documents into corresponding pairs, randomly divide them into a training set, a test set, and a validation set according to the specified division ratio, and automatically generate the configuration file required for YOLO training. Use the YOLO training script, and start training after specifying the path of the configuration file.

[0053] Simulation verification

[0054] Use the detect.py of yolo to convert the above-trained model into the general model in onnx format and save it to the specified path on the virtual platform.

[0055] Start the virtual platform to enter the running mode. In this mode, the virtual platform will load the scene (weather, road signs, etc. of different road sections) through the pre-set scene configuration, and perform local inference on the model loaded from the specified path.

[0056] After successfully loading the model, the vehicle drives forward naturally, transmits the images captured by the camera (from the vehicle's perspective, located at the front of the vehicle) to the model for inference, and marks the frame number of the frame and the information of the camera (position, rotation, FOV, etc.).

[0057] Result display and scoring: Control the driving behavior of the vehicle in the scene (steering, accelerating / decelerating, turning on the lights, honking, etc.) according to the model results. At the same time, according to the frame number corresponding to the result, obtain the camera information corresponding to the frame, and try to find the object recognized by the model (road signs, traffic lights, pedestrians, etc.) by emitting rays, and compare it with the correct result corresponding to the object for scoring and recording. If the model inference result is incorrect, the vehicle drives into the wrong road or reaches the wrong location, the position of the vehicle will be retraced to help the vehicle return to the correct road (if the inference is incorrect or the comparison result is incorrect, the scoring system will not give points).

[0058] To solve the above problems, according to one aspect of the present invention, as Figure 1 shown, a method for training and validating an autonomous driving algorithm based on a virtual simulation scene is provided, specifically including:

[0059] S1 Use the recognition data of the autonomous driving algorithm in different test scenarios to determine the recognition deviation data of the target objects in different test scenarios. When it is determined that the recognition deviation status in the test scenario meets the requirements, proceed to the next step;

[0060] S2 Based on the recognition data of different target objects in different test scenarios, determine the recognition deviation data of different target objects in different test scenarios, and use the recognition deviation data in different test environments to determine the deviation recognition targets among the target objects;

[0061] S3 uses the recognition image with deviation recognition targets in the recognition data as the matching modeling image, and based on the distribution association data of the deviation recognition targets in the matching modeling image and the recognition deviation data of the deviation recognition targets in the test scenario, determines the matching modeling image for amplification processing in the test scenario, and determines the modeling amplification image in the test scenario in combination with the matching modeling image;

[0062] S4 performs modeling processing on the test data in the test scenario based on the modeling amplification image, the matching modeling image, and the recognition deviation image in the test scenario, and uses the test data to perform training and verification processing on the autonomous driving algorithm.

[0063] Furthermore, the test scenario includes rain, cloudy, snow, and fog.

[0064] Specifically, the recognition deviation data of the target object includes the number of recognition deviations of the target object.

[0065] It should be noted that as Figure 2 shown, determining that the recognition deviation state in the test scenario meets the requirements specifically includes:

[0066] Using the recognition deviation data to determine the number of recognition deviations of different target objects in the test scenario, and using the number of recognition deviations to determine the recognition deviation objects among the target objects;

[0067] Determine whether the recognition deviation state in the test scenario meets the requirements according to the number of the recognition deviation objects.

[0068] Furthermore, the recognition deviation object is a target object with the number of recognition deviations greater than the preset deviation number.

[0069] Optionally, determining whether the recognition deviation state in the test scenario meets the requirements specifically includes:

[0070] When the number of the recognition deviation objects is greater than the preset object number, it is determined that the recognition deviation state in the test scenario does not meet the requirements.

[0071] In addition, it should be noted that when the recognition deviation state in the test scenario does not meet the requirements, all the recognition images in the test scenario are used for modeling processing of the test data in the test scenario.

[0072] Optionally, determining that the recognition deviation state in the test scenario meets the requirements specifically includes:

[0073] Determine the recognition deviation conditions of different recognition images in the test scenario using the recognition deviation data, and determine the recognition deviation images in the recognition images according to the recognition deviation conditions;

[0074] Determine whether the recognition deviation status in the test scenario meets the requirements according to the number of the recognition deviation images.

[0075] In another embodiment, determining that the recognition deviation status in the test scenario meets the requirements specifically includes:

[0076] S11 Determine the recognition deviation conditions of different recognition images in the test scenario using the recognition deviation data, and determine the recognition deviation images in the recognition images according to the recognition deviation conditions. Determine the basic deviation coefficient using the proportion of the recognition deviation images in the recognition images;

[0077] Optionally, the above step S11 includes the following content:

[0078] S111 Determine the recognition deviation conditions of different recognition images in the test scenario using the recognition deviation data, and determine the recognition deviation images in the recognition images according to the recognition deviation conditions. When the number of the recognition deviation images does not meet the requirements, determine that the recognition deviation status in the test scenario does not meet the requirements. When the number of the recognition deviation images meets the requirements, proceed to step S112;

[0079] S112 Determine the basic deviation coefficient using the proportion of the recognition deviation images in the recognition images. When the basic deviation coefficient does not meet the requirements, proceed to step S113. When the basic deviation coefficient meets the requirements, proceed to step S12;

[0080] S113 Obtain the number of the recognition deviation images. When the number of the recognition deviation images is within the preset image number range, determine that the recognition deviation status in the test scenario does not meet the requirements. When the number of the recognition deviation images is not within the preset image number range, proceed to step S12.

[0081] S12 Determine the recognition deviation times of different target objects in the test scenario using the recognition deviation data, and determine the recognition deviation coefficients of different target objects using the recognition deviation times of different target objects;

[0082] Optionally, the above step S12 includes the following content:

[0083] S121 Determine the number of recognition deviations of different target objects in the test scenario using the recognition deviation data. When the total number of recognition deviations of different target objects does not meet the requirements, it is determined that the recognition deviation status in the test scenario does not meet the requirements. When the total number of recognition deviations of different target objects meets the requirements, proceed to step S122;

[0084] S122 When it is determined that there are target objects with the number of recognition deviations greater than the preset deviation number using the number of recognition deviations of different target objects, proceed to step S123. When there are no target objects with the number of recognition deviations greater than the preset deviation number, proceed to step S124;

[0085] S123 When the number of target objects with the number of recognition deviations greater than the preset deviation number does not meet the requirements, it is determined that the recognition deviation status in the test scenario does not meet the requirements. When the number of target objects with the number of recognition deviations greater than the preset deviation number meets the requirements, proceed to step S124;

[0086] S124 Determine the recognition deviation coefficient of different target objects using the number of recognition deviations of different target objects. When the number of target objects with the recognition deviation coefficient within the preset deviation coefficient range is greater than the preset target object number, it is determined that the recognition deviation status in the test scenario does not meet the requirements. When the number of target objects with the recognition deviation coefficient within the preset deviation coefficient range is not greater than the preset target object number, proceed to step S13.

[0087] S13 Determine the comprehensive deviation coefficient based on the recognition deviation coefficients of different target objects and the basic deviation coefficient, and use the comprehensive deviation coefficient to determine whether the recognition deviation status in the test scenario meets the requirements.

[0088] Specifically, as Figure 3 shown, the method for determining the deviation recognition target among the target objects is:

[0089] Determine the number of recognition deviations of the target objects in different test environments using the recognition deviation data of the target objects in different test environments;

[0090] Based on the number of recognition deviations of the target objects in different test environments, determine the total number of recognition deviations of the target objects;

[0091] Determine whether the target object is a deviation recognition target according to the total number of recognition deviations;

[0092] Further, when the total number of recognition deviations of the target object is greater than the preset deviation number threshold, it is determined that the target object is a deviation recognition target.

[0093] Optionally, the method for determining the deviation identification target in the target object is as follows:

[0094] Based on the recognition deviation data of the target object in different test environments, determine the number of recognition deviations of the target object in different test environments;

[0095] Based on the number of recognition deviations of the target object in different test environments, determine the recognition deviation environment of the target object in different test environments;

[0096] Determine whether the target object is a deviation identification target according to the number of the recognition deviation environments.

[0097] In another embodiment, the method for determining the deviation identification target in the target object is as follows:

[0098] S21 Based on the recognition deviation data of the target object in different test environments, determine the number of recognition deviations of the target object in different test environments;

[0099] S22 Based on the number of recognition deviations of the target object in different test environments and the number of recognition images containing the target object, determine the recognition deviation coefficient of the target object in different test environments;

[0100] S23 Determine the comprehensive recognition deviation coefficient of the target object according to the recognition deviation coefficient of the target object in different test environments, and use the comprehensive recognition deviation coefficient to determine whether the target object is a deviation identification target.

[0101] Specifically, as Figure 4 shown, the method for determining the matching modeling image for amplification processing in the test scenario is as follows:

[0102] Based on the distribution association data of the deviation identification targets in the matching modeling image, determine the deviation identification targets in different matching modeling images. According to the recognition error data of the deviation identification targets in different matching modeling images in the test environment, determine the number of recognition deviations of different recognition deviation targets in the test environment;

[0103] Use the number of recognition deviations of different recognition deviation targets in the test environment to determine the image recognition deviation coefficient of the matching modeling image;

[0104] Determine the matching modeling image for amplification processing in the test scenario according to the image recognition deviation coefficient.

[0105] Furthermore, the matching modeling image for amplification processing in the test scenario is the matching modeling image whose image recognition deviation coefficient does not meet the requirements.

[0106] Optionally, the method for determining the matching modeling image for amplification processing in the test scenario is as follows:

[0107] Use the distribution correlation data of the deviation recognition targets in the matching modeling image to determine the deviation recognition targets in different matching modeling images. Based on the deviation recognition targets in different matching modeling images, determine the number of recognition deviation times of different recognition deviation targets in the recognition error data in the test environment;

[0108] Use the number of recognition deviation times of different recognition deviation targets in the test environment to determine the image recognition deviation coefficient of the matching modeling image;

[0109] Determine the matching modeling image for amplification processing in the test scenario according to the image recognition deviation coefficient.

[0110] It should be noted that the matching modeling image for amplification processing in the test scenario is a matching modeling image whose image recognition deviation coefficient does not meet the requirements.

[0111] Optionally, the method for determining the matching modeling image for amplification processing in the test scenario is as follows:

[0112] Use the distribution correlation data of the deviation recognition targets in the matching modeling image to determine the deviation recognition targets in different matching modeling images. Based on the deviation recognition targets in different matching modeling images, determine the number of recognition deviation times of different recognition deviation targets in the recognition error data in the test environment;

[0113] When there are recognition deviation targets in the matching modeling image whose number of recognition deviation times does not meet the requirements, then the matching modeling image belongs to the matching modeling image for amplification processing in the test scenario;

[0114] When there are no recognition deviation targets in the matching modeling image whose number of recognition deviation times does not meet the requirements:

[0115] Use the number of recognition deviation times of different recognition deviation targets in the test environment to determine that the number of recognition deviation times of different recognition deviation targets is less than the preset deviation times threshold:

[0116] When the number of recognition deviation targets in the matching modeling image is less than the preset deviation target number:

[0117] Then determine that the matching modeling image does not belong to the matching modeling image for amplification processing in the test scenario;

[0118] When the number of recognition deviation times of different recognition deviation targets is not less than the preset deviation times threshold or the number of recognition deviation targets in the matching modeling image is not less than the preset deviation target number:

[0119] Determine the total number of recognition deviations based on the number of recognition deviations of different recognition deviation targets in the test environment. When the total number of recognition deviations does not meet the requirements, it is determined that the paired modeling image does not belong to the matching modeling image for amplification processing in the test scenario;

[0120] When the total number of recognition deviations meets the requirements:

[0121] Take the recognition deviation targets with the number of recognition deviations within the preset deviation number range as the screened deviation targets. When the number of screened deviation targets does not meet the requirements, it is determined that the paired modeling image does not belong to the matching modeling image for amplification processing in the test scenario;

[0122] When the number of screened deviation targets meets the requirements:

[0123] Determine the image recognition deviation coefficient of the matching modeling image based on the number of recognition deviations of different recognition deviation targets in the test environment, and determine the matching modeling image for amplification processing in the test scenario according to the image recognition deviation coefficient.

[0124] Furthermore, the amplification processing amplifies and reduces the image size of the recognition deviation targets in the matching modeling image according to a preset level.

[0125] Specifically, the modeling processing of the test data in the test scenario includes:

[0126] Use the modeling amplified image, the matching modeling image, and the recognition deviation image in the test scenario as the basic data, and based on the basic data, perform the modeling processing of the test data in the test scenario.

[0127] It should be noted that the training and verification processing of the autonomous driving algorithm using the test data in different test scenarios includes:

[0128] Import the model constructed by the autonomous driving algorithm into the virtual simulation scenario constructed by different test scenarios, select the running mode, the virtual car runs automatically in the scenario, complete the road condition recognition in the road scenario, and the car can make corresponding actions according to the recognition result to complete the training and verification of the autonomous driving algorithm of the car in the virtual simulation scenario.

[0129] Optionally, the above step S21 includes the following content:

[0130] S211 determines the number of recognition deviation times of the target object in different test environments based on the recognition deviation data of the target object in different test environments. When the total number of recognition deviation times of the target object in different test environments does not meet the requirements, the target object is determined as the recognition deviation target. When the total number of recognition deviation times of the target object in different test environments meets the requirements, it proceeds to step S212;

[0131] S212 When it is determined that there is a recognition deviation test environment in the test environment based on the number of recognition deviation times of the target object in different test environments, it proceeds to step S213. When there is no recognition deviation test environment in the test environment, it proceeds to step S22;

[0132] S213 When the number of the recognition deviation test environments does not meet the requirements, the target object is determined as the recognition deviation target. When the number of the recognition deviation test environments meets the requirements, it proceeds to step S22.

[0133] Optionally, the following content is included in the above step S22:

[0134] S221 determines the recognition deviation coefficient of the target object in different test environments based on the number of recognition deviation times of the target object in different test environments and the number of recognition images including the target object. When there is a test environment where the recognition deviation coefficient does not meet the requirements, it proceeds to step S222. When there is no test environment where the recognition deviation coefficient does not meet the requirements, it proceeds to step S23;

[0135] S222 takes the test environment where the recognition deviation coefficient does not meet the requirements as the recognition deviation test environment. When the number of the recognition deviation test environments does not meet the requirements, the target object is determined as the recognition deviation target. When the number of the recognition deviation test environments meets the requirements, it proceeds to step S223;

[0136] S223 determines the environment recognition deviation coefficient based on the recognition deviation coefficients of different recognition deviation test environments. When the environment recognition deviation coefficient does not meet the requirements, the target object is determined as the recognition deviation target. When the environment recognition deviation coefficient meets the requirements, it proceeds to step S23.

[0137] Each embodiment in this specification is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other. The key point of each embodiment is to illustrate the differences from other embodiments. In particular, for the embodiments of the device, equipment, and non - volatile computer storage medium, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments.

[0138] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired results. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0139] The foregoing is only one or more embodiments of the present specification and is not intended to limit the present specification. For those skilled in the art, various changes and modifications can be made to one or more embodiments of the present specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of one or more embodiments of the present specification shall be included within the scope of the claims of the present specification.

Claims

1. A method for training and verifying an autonomous driving algorithm based on a virtual simulation scenario, characterized in that: Specifically include: Determine recognition deviation data of target objects in different test scenarios using recognition data of the autonomous driving algorithm in different test scenarios, and proceed to the next step when determining using the recognition deviation data that the recognition deviation state in the test scenario meets the requirements; Determine recognition deviation data of different target objects in different test scenarios based on recognition data of different target objects in different test scenarios, and determine deviation recognition targets in the target objects with the recognition deviation data in different test environments; Taking the recognition image of the deviation recognition target in the recognition data as the matching modeling image, determining the matching modeling image of the augmentation processing in the test scene based on the distribution association data of the deviation recognition target in the matching modeling image and the recognition deviation data of the deviation recognition target in the test scene, and determining the modeling augmented image in the test scene in combination with the matching modeling image; Modeling the test data in the test scene based on the modeling augmented image, the matching modeling image, and the identification deviation image in the test scene, and using the test data to train and verify the autonomous driving algorithm; When the recognition deviation state in the test scene does not meet the requirement, all the recognition images in the test scene are used to perform modeling processing on the test data in the test scene; The method for determining the matching modeling image for augmentation processing in the test scene is: Determine the deviation recognition targets in different matching modeling images by using the distribution association data of the deviation recognition targets in the matching modeling images, and determine the number of recognition deviations of different recognition deviation targets in the test environment according to the recognition error data of the deviation recognition targets in the different matching modeling images in the test environment; Determine the image recognition deviation coefficient of the matching modeling image by using the recognition deviation times of different recognition deviation targets in the test environment; The matching modeling image for augmentation processing in the test scene is determined based on the image recognition deviation coefficient.

2. The method for training and verifying an autonomous driving algorithm based on a virtual simulation scenario according to claim 1, characterized in that: The test scenarios include rain, cloudy days, snow, and heavy fog.

3. The method for training and verifying an autonomous driving algorithm based on a virtual simulation scenario according to claim 1, characterized in that: The recognition deviation data of the target object includes the number of recognition deviations of the target object.

4. The method for training and verifying an autonomous driving algorithm based on a virtual simulation scenario according to claim 1, wherein: Determining that the recognition deviation state in the test scenario meets the requirements, specifically includes: Determine the recognition deviation times of different target objects in the test scene using the recognition deviation data, and determine the recognition deviation objects among the target objects using the recognition deviation times; It is determined whether the recognition deviation state in the test scene meets the requirement according to the number of the recognition deviation objects.

5. The method for training and verifying an autonomous driving algorithm based on a virtual simulation scenario according to claim 4, characterized in that: The recognition deviation object is a target object whose recognition deviation times are greater than a preset deviation times.

6. The method for training and verifying an autonomous driving algorithm based on a virtual simulation scenario according to claim 4, characterized in that: Determining whether the recognition deviation state in the test scene meets the requirements according to the number of the recognition deviation objects specifically includes: When the number of the recognition deviation objects is greater than the preset number of objects, it is determined that the recognition deviation state in the test scene does not meet the requirements.

7. The method for training and verifying an autonomous driving algorithm based on a virtual simulation scenario according to claim 1, characterized in that: The enlargement process enlarges and reduces the image size of the identification deviation target in the matching modeling image according to a preset level.

8. The method for training and verifying an autonomous driving algorithm based on a virtual simulation scenario according to claim 1, characterized in that: The modeling process of the test data in the test scenario specifically includes: The modeling augmented image, the matching modeling image, and the identification deviation image in the test scene are used as basic data, and based on the basic data, modeling processing of the test data in the test scene is performed.

9. The method for training and verifying an autonomous driving algorithm based on a virtual simulation scenario according to claim 1, characterized in that: The autonomous driving algorithm is trained and verified using test data from different test scenarios, specifically including: The model constructed by the autonomous driving algorithm is imported into the virtual simulation scene constructed by different test scenarios. The operation mode is selected, and the virtual car automatically runs in the scene to complete the road condition recognition in the road scene. According to the recognition results, the car can make corresponding actions to complete the autonomous driving algorithm training and verification of the car in the virtual simulation scene.

Images