An intelligent driving perception model optimization method and device and a storage medium

Abstract

The application provides an optimization method and device of an intelligent driving perception model and a storage medium. The method comprises: obtaining scene data corresponding to an extreme scenario corner case; performing scene reconstruction and identification analysis on the scene data to obtain a corner case feature; performing random reconstruction and generalization processing on the corner case feature to obtain a processed corner case feature; combining the processed corner case feature to generate a same-kind different-structure corner case scene library; and performing model training on at least one perception model of a simulation test platform based on scene data in the same-kind different-structure corner case scene library to obtain an optimized at least one perception model. According to the same-kind different-structure scene data generated by random reconstruction and generalization, training and testing can be performed on the simulation test platform, and the same-kind different-structure corner case can be eliminated.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and in particular to an optimization method, device and storage medium for an intelligent driving perception model. Background Technology

[0002] With the development of vehicle technology, simulation testing, track testing, and road testing are becoming increasingly common. Vehicles equipped with autonomous driving / assisted driving capabilities and related algorithm software must undergo rigorous testing and verification before being released to the market. Simulation testing can significantly reduce software bugs in autonomous driving / assisted driving software during real-vehicle testing and before its market launch.

[0003] Currently, various simulation tests are being conducted on vehicles to assess whether their intelligent driving perception models can handle uncontrollable extreme scenarios (corner cases). When a corner case is encountered, its data is stored, and the algorithm is debugged and the intelligent driving perception model is repaired based on this stored data. Solving a single scenario cannot completely eliminate similar problems; targeted solutions are only possible through large-scale data collection and scenario creation. However, many corner case scenarios are difficult to reproduce.

[0004] Therefore, how to solve similar corner case problems is an urgent issue to be addressed. Summary of the Invention

[0005] This application provides an optimization method, device, and storage medium for an intelligent driving perception model, which addresses the problem that existing intelligent driving perception models cannot cover similar heterogeneous extreme scenarios.

[0006] Firstly, this application provides a method for optimizing an intelligent driving perception model, comprising:

[0007] Obtain the scenario data corresponding to the corner case of the extreme scenario;

[0008] The scene data is reconstructed and identified to obtain corner case features;

[0009] The corner case features are randomly reconstructed and generalized to obtain the processed corner case features.

[0010] The processed corner case features are combined to generate a similar but heterogeneous corner case scenario library.

[0011] Based on the aforementioned heterogeneous corner case scenario library, at least one perception model of the simulation test platform is trained to obtain at least one optimized perception model.

[0012] Optionally, the method further includes:

[0013] Based on the optimized at least one perception model, simulation tests are conducted on similar heterogeneous corner case scenarios in the simulation test platform.

[0014] Optionally, the step of randomly reconstructing and generalizing the corner case features to obtain the processed corner case features includes:

[0015] The corner case features are classified to obtain the features of the vehicle itself, the features of the target object, and the scene features;

[0016] Based on a pre-configured generalization element library, the features of the vehicle itself, the features of the target object, and the scene features are randomly reconstructed and generalized to obtain the processed corner case features.

[0017] Optionally, the pre-configured generalized element library includes: multiple weather environment features, multiple speed features, multiple color features, multiple traffic flow features, and multiple working condition scene features;

[0018] Accordingly, based on the pre-configured generalization element library, the features of the vehicle itself, the features of the target object, and the scene features are randomly reconstructed and generalized to obtain the processed corner case features, including:

[0019] The vehicle's own speed characteristics are reconstructed and generalized by selecting the various speed features from the generalized element library.

[0020] Multiple velocity features and multiple color features are selected from the generalized element library to reconstruct and generalize the velocity and color features of the target object;

[0021] From the generalized element library, various weather environment features, various traffic flow features, and various working condition scenarios are selected to reconstruct and generalize the weather features, traffic flow features, and working condition features of the scenarios.

[0022] Optionally, the step of performing scene reconstruction and recognition analysis on the scene data to obtain corner case features includes:

[0023] The scene data corresponding to the corner case is reconstructed and identified to determine the cause of the corner case.

[0024] Based on the cause of the corner case, the corner case features are extracted from the corner case data.

[0025] Optionally, obtaining the scene data corresponding to the extreme scenario corner case includes:

[0026] The system receives scenario data corresponding to the corner case uploaded by the vehicle, wherein the scenario data is data collected by the vehicle when the simulated decision and the driver's actual decision are inconsistent.

[0027] Secondly, this application also provides an optimization device for an intelligent driving perception model, the device comprising:

[0028] The acquisition module is used to acquire scene data corresponding to extreme scenario corner cases;

[0029] The feature acquisition module is used to reconstruct and identify the scene data to obtain corner case features;

[0030] The generalization processing module is used to randomly reconstruct and generalize the corner case features to obtain the processed corner case features.

[0031] The scene library generation module is used to combine the processed corner case features to generate a similar but heterogeneous corner case scene library.

[0032] The model testing module is used to train at least one perception model of the simulation test platform based on the aforementioned heterogeneous corner case scenario library, so as to obtain at least one optimized perception model.

[0033] Optionally, the device further includes:

[0034] The simulation testing module is used to perform simulation tests on similar heterogeneous corner case scenarios in the simulation testing platform based on at least one optimized perception model.

[0035] Thirdly, this application also provides a server, comprising:

[0036] The processor, the memory communicatively connected to the processor, and the communication interface for interacting with other devices;

[0037] The memory stores computer-executed instructions;

[0038] The processor executes computer execution instructions stored in the memory to implement the optimization method for the intelligent driving perception model as described in any of the first aspects.

[0039] Fourthly, this application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the optimization method of the intelligent driving perception model as described in any of the first aspects.

[0040] This application provides an optimization method, apparatus, and storage medium for an intelligent driving perception model. The method involves: acquiring scene data corresponding to extreme scenario corner cases; performing scene reconstruction and identification analysis on the scene data to obtain corner case features; randomly reconstructing and generalizing the corner case features to obtain processed corner case features; combining the processed corner case features to generate a similar heterogeneous corner case scenario library; and training at least one perception model on a simulation test platform based on scene data from the similar heterogeneous corner case scenario library to obtain at least one optimized perception model. When encountering extreme scenario corner cases, training and testing on the simulation test platform using the randomly reconstructed and generalized similar heterogeneous scene data can eliminate similar heterogeneous corner cases. Attached Figure Description

[0041] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0042] Figure 1 A schematic diagram illustrating a possible application scenario for the optimization method of the intelligent driving perception model provided in this application;

[0043] Figure 2 A flowchart illustrating an embodiment of an optimization method for an intelligent driving perception model provided in this application.

[0044] Figure 3 A flowchart illustrating an implementation of an optimization method for an intelligent driving perception model provided in this application embodiment;

[0045] Figure 4 A flowchart illustrating an embodiment of an optimization device for an intelligent driving perception model provided in this application.

[0046] Figure 5This is a schematic diagram of the structure of a server provided in an embodiment of this application.

[0047] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0048] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0049] First, let me explain the terms used in this application:

[0050] Corner cases refer to special situations encountered in autonomous driving that rarely occur during normal driving but, when they do, can have a significant impact on the autonomous driving system. These special situations may involve very extreme or complex road conditions, weather, vehicle conditions, or specific traffic rules and road signs. Corner cases are crucial in autonomous driving because they can lead to unsafe or unstable driving situations. Therefore, to ensure the safety and reliability of autonomous driving systems, a wide variety of corner cases must be considered during development and testing, and these cases must be thoroughly tested and validated to enable correct decisions and actions in real-world use.

[0051] Generalization, in this application, refers to the process of summarizing and abstracting data features collected under specific circumstances, thereby extending their application to datasets under a wider range of conditions. The aim is to improve the model's generalization ability and predict results more accurately. In the field of autonomous driving, generalizing vehicle behavior and environmental characteristics under specific conditions can improve the safety and reliability of autonomous driving systems.

[0052] Heterogeneous in the same category: In this application, it refers to different Corner Cases that have the same influencing factors but also have different characteristics and attributes.

[0053] It should be noted that in this application, "autonomous driving" and "intelligent driving" refer to the same thing.

[0054] Currently, vehicles equipped with autonomous and assisted driving capabilities require various tests before being put on the road. Among these tests, simulation testing can significantly reduce potential problems with the vehicle's planning and control software. Various mainstream planning and control software programs (VTD, CARsim, etc.) already support simulation testing and refactoring testing of planning and control.

[0055] Before a vehicle is put on the road, it needs to undergo various simulation tests to test whether the vehicle's planning and control software has any extreme scenarios that it cannot cover (corner cases), and then optimize the planning and control software.

[0056] Figure 1 This is a schematic diagram illustrating a possible application scenario of the optimization method for the intelligent driving perception model provided in this application, such as... Figure 1 As shown, in this scenario, when the autonomous driving system encounters extreme scenario corner case A during testing, it fails to execute the correct strategy, leading to a collision. The data for corner case A is stored, and the algorithm is debugged based on this data. The planning and control software is then repaired and can make the correct choice for that scenario. However, due to the specific nature of the planning and control algorithm, solving a single scenario cannot solve similar heterogeneous problems. When encountering a similar scenario, corner case A', the planning and control software still cannot cover that scenario and fails to execute the correct strategy, resulting in another collision. Therefore, it is necessary to retrain based on corner case A' data, and this could pose a danger if it occurs during user operation.

[0057] Existing solutions primarily address similar heterogeneous problems through the following methods:

[0058] Increase the database through large-scale data collection. Collect a sufficient number of corner cases, summarize the collected corner cases, discover the triggering mechanisms of the corner cases, and create targeted scenarios to solve similar heterogeneous problems.

[0059] However, there are some dangerous corner case scenarios, the number of which is limited, and it is difficult to recreate real-world scenarios.

[0060] In view of the above problems, the inventors discovered during their research in this technical field that when encountering a corner case, the features of the corner case scenario are specifically generalized and combined to generate multiple corner case scenario data. Based on the generated multiple corner case scenario data, the perception model of at least one version of the planning and control software in the simulation platform is trained to obtain an optimized perception model. The optimized perception model can solve the problem of heterogeneity within the same corner case. Based on this, this application proposes an optimization method, device, and storage medium for an intelligent driving perception model.

[0061] It should be understood that the optimization method for the intelligent driving perception model proposed in this application is applicable to vehicles with autonomous or assisted driving capabilities, and is not limited to vehicles, but can also include airplanes, ships, intelligent food trucks, intelligent sorting vehicles, mobile robots, etc.

[0062] The entity executing this application may be an in-vehicle terminal with simulation capabilities or a cloud server with simulation capabilities.

[0063] The following describes the technical solution of this application and how it solves the above-mentioned technical problems, taking a cloud server as the execution entity as an example, in conjunction with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0064] Figure 2 A flowchart illustrating an embodiment of an optimization method for an intelligent driving perception model provided in this application is shown below. Figure 2 As shown, the method includes the following steps:

[0065] S101. Obtain the scene data corresponding to the extreme scenario corner case.

[0066] In this step, when the vehicle encounters an extreme corner case that cannot be covered, the planning and control software algorithm needs to be simulated and tested to complete algorithm repair (debug). Since simulation testing requires data to simulate a real traffic environment, the first step is to acquire the scene data corresponding to the corner case. Scene data includes image data of the vehicle's exterior within the corner case, the vehicle's speed, control signals from the vehicle's planning and control software, and weather data, etc.

[0067] In one implementation, the cloud server receives scenario data corresponding to the corner case uploaded by the vehicle. The scenario data is data collected when the vehicle's simulated decision and the driver's actual decision are inconsistent.

[0068] In this implementation, the user drives the vehicle, and the vehicle's planning and control software outputs simulated decision-making planning and control signals in real time. However, these signals do not actually control the vehicle. The vehicle compares the simulated decision-making planning and control signals with the driver's actual control signals. If the comparison results are inconsistent or exceed a preset range, the vehicle collects the scene data corresponding to the corner case and uploads it to the cloud server. The cloud server receives this scene data.

[0069] In another implementation, when the vehicle is undergoing actual field and road testing, the cloud server acquires all scene data recorded by the vehicle and control signals from the planning and control software in real time. When the planning and control software exits or the planning and control signals repeat, it is recorded as a corner case, and the scene data corresponding to the corner case is obtained from the recorded scene data.

[0070] In the aforementioned scenario data, the external image data of the vehicle can be images or videos captured by cameras on the vehicle's exterior. The vehicle's speed can be recorded by the vehicle's speed sensor. The control signals from the vehicle's planning and control software are saved in real time by the planning and control software. Weather data is obtained by identifying the image data or by querying the network based on the vehicle's location information. The scenario data may also include radar point cloud images acquired by radar and basic vehicle status data, which includes steering gear status, brake status, battery level, model identification, location (GPS) data, etc.

[0071] It should be noted that the above scenario data is continuous data over a period of time, such as continuous frame images or videos over a period of time, continuous control signals from planning and control software, and continuous change data of vehicle speed.

[0072] S102. Perform scene reconstruction and identification analysis on the scene data to obtain corner case features.

[0073] In this step, the scene is reconstructed based on the acquired image data of the vehicle's exterior and the vehicle's basic state data, creating a simulated traffic scenario. After scene reconstruction, it is necessary to identify corner cases occurring in the 3D scene and analyze their causes.

[0074] In one implementation, the reconstructed 3D scene is observed by a person, or by wearing a virtual reality device, and the reasons for the corner case are recorded using interactive tools (such as labelbox or supervisely).

[0075] In another implementation, automated cause identification and analysis is achieved by pre-training a cause prediction model. This cause prediction model is trained based on corner case scenario data with known causes and can predict the cause of sending a corner case based on the corner case scenario data.

[0076] After analyzing the causes of corner cases, the corner case feature dimensions to be extracted are determined based on the analysis results. These dimensions may include vehicle speed, direction, position, and weather. If the analysis results indicate that the influence of other objects is involved, then the corner case feature dimensions of those objects (i.e., the target object) also need to be extracted, such as speed, direction, outline, and color. Based on the corner case feature dimensions to be extracted, image processing techniques or computer vision techniques are used to extract the corresponding corner case features from the scene data.

[0077] Optionally, after extracting the relevant features, data cleaning is performed to remove outliers and noise, and normalization is carried out to make them have the same scale and distribution.

[0078] Optionally, for high-dimensional features (such as road structure, weather, the trajectory of a target object, etc.), feature dimensionality reduction techniques (such as principal component analysis, PCA) can be used to convert high-dimensional features into low-dimensional feature representations, which facilitates subsequent processing.

[0079] Optionally, feature importance analysis can be performed between the extracted corner case features and manually determined target variables to identify corner case features that meet preset conditions, such as obtaining the top 10 features from the correlation analysis. The target variable can be the accuracy of obstacle recognition, the accuracy of driving decisions, etc. Importance analysis can employ correlation analysis, embedded methods, or filtering methods, whichever is appropriate and not limited in this application.

[0080] S103. Randomly reconstruct and generalize the corner case features to obtain the processed corner case features.

[0081] In this step, each corner case feature is randomly reconstructed and generalized based on a pre-configured generalization element library to obtain the processed corner case features. Random reconstruction increases the diversity and coverage of the scene data, while generalization makes the model trained on the generalized data more applicable. The pre-configured generalization element library includes various weather environment features, traffic flow features, working condition scene features, speed features, obstacle features, and color features, which can be used for random reconstruction and generalization selection.

[0082] For example, the pre-configured generalized element library contains colors such as red, green, white, black, gray, and generalized light and dark colors. Among them, red, green, and white are light colors, while black and gray are dark colors.

[0083] For example, the pre-configured generalized element library includes weather conditions such as sunny, cloudy, partly cloudy, light rain, moderate rain, heavy rain, as well as generalized sunny and rainy weather. Among them, sunny weather includes sunny, cloudy, and partly cloudy weather, and rainy weather includes light rain, moderate rain, and heavy rain.

[0084] For example, based on a pre-configured generalization element library, the color features in the corner case feature are randomly reconstructed and generalized, and the target object with white features is randomly reconstructed as black or red; and generalized as a bright corner case feature.

[0085] For example, based on a pre-configured generalization element library, the vehicle speed features in the corner case features are randomly reconstructed and generalized, and the lateral and longitudinal speeds are randomly increased or decreased; the speed is generalized to low-speed corner case features.

[0086] It should be noted that there is no requirement for the order of generalization and random reconstruction operations.

[0087] Optionally, before random reconstruction and generalization, the corner case features are classified. The classified features include vehicle-specific features, target object features, and environmental features. Vehicle-specific features include all feature data of the current vehicle; target object features include all feature data of objects influencing the corner case; and environmental features include weather, road conditions, traffic flow, and other feature data. Random reconstruction and generalization are then performed on the classified feature data.

[0088] S104. Combine the processed corner case features to generate a heterogeneous corner case scenario library.

[0089] In this step, the randomly reconstructed and generalized corner case features are combined to generate multiple sets of corner case feature data. Each set of processed corner case feature data replaces the previous feature data in the original scene data. Therefore, multiple sets of similar but heterogeneous corner case scenes can be generated, forming a similar but heterogeneous corner case scene library. The combination method can be random or according to preset rules.

[0090] S105. Train at least one perception model of the simulation test platform based on heterogeneous corner case scenario data of the same type to obtain at least one optimized perception model.

[0091] In this step, multiple scenario data sets from a heterogeneous corner case scenario library are selected, with a portion used as the training set and the remainder as the validation set. The simulation test platform has multiple different versions of perception models for intelligent driving that can be used by vehicles, enabling intelligent planning of vehicle driving paths. At least one perception model on the simulation test platform is trained using the training set to obtain at least one optimized perception model. In one possible implementation, the scenario data is converted into matrix or vector form and input into the perception model for training.

[0092] Optionally, at least one optimized perception model can be simulated and tested in a simulation test platform according to a test set to verify whether the perception model can cover similar heterogeneous scenarios.

[0093] This application provides an optimization method for an intelligent driving perception model. The method involves acquiring scene data corresponding to extreme scenario corner cases; performing scene reconstruction and identification analysis on the scene data to obtain corner case features; randomly reconstructing and generalizing the corner case features to obtain processed corner case features; combining the processed corner case features to generate a similar heterogeneous corner case scenario library; and training at least one perception model on a simulation test platform based on scene data from the similar heterogeneous corner case scenario library to obtain at least one optimized perception model. When encountering extreme scenario corner cases, training and testing on the simulation test platform using the randomly reconstructed and generalized similar heterogeneous scene data can eliminate similar heterogeneous corner cases.

[0094] The following section uses a corner case—an autonomous vehicle rear-ending another vehicle—as an example to illustrate the optimization methods for intelligent driving perception models.

[0095] Figure 3 This application provides a flowchart illustrating an implementation of an optimization method for an intelligent driving perception model, as shown in the following example. Figure 3 As shown, the method includes the following steps:

[0096] S201. During autonomous driving, the vehicle collects raw data in real time.

[0097] In this step, the vehicle collects raw data in real time during autonomous driving to reconstruct scenarios in case of extreme situations. This raw data includes real-time video footage of the vehicle's exterior captured by external cameras, real-time recordings of the vehicle's lateral and longitudinal velocities by vehicle sensors, real-time storage of control signals from the planning and control software, and real-time recording of basic vehicle status data, such as vehicle orientation, brake status, and position information. It may also include 3D point cloud data acquired by LiDAR, information on obstacles and other vehicles detected by radar around the vehicle, and environmental data (such as weather conditions and road conditions) obtained through analysis of the video footage captured by cameras.

[0098] S202. The vehicle obtains corner case data from the original data and uploads it to the cloud server.

[0099] In this step, if a rear-end collision occurs during the autonomous driving process, the vehicle will upload the raw data of the preset time period before and after the rear-end collision as corner case data to the cloud server. The preset time period can be 1 minute, 2 minutes, or 3 minutes.

[0100] Optionally, before a rear-end collision occurs, the autonomous driving system disengages and the user takes over, with the raw data from a preset time period before and after the disengagement serving as corner case data.

[0101] S203. The cloud server performs scene reconstruction and identification analysis based on corner case data, and determines that the cause of the corner case is an irregularly shaped vehicle.

[0102] In this step, the cloud server receives the corner case data, cleans it to remove noise, and then performs scene analysis on the cleaned data. This analysis examines features such as the vehicle's and other objects' paths, speeds, accelerations, directions, outlines, and colors. These features are then mapped onto a virtual 3D scene. Using 3D modeling and virtual environment simulation techniques, a virtual scene similar to the actual scene is generated. Finally, the environmental data from the original data is fused with the virtual scene data to generate complete scene data.

[0103] In one implementation, the cleaned video data is analyzed, and the features of the identified objects are mapped onto a virtual 3D scene.

[0104] Automated or manual analysis of scene data is used to determine the cause of corner cases, identifying rear-end collisions caused by irregularly shaped vehicles. Specific reasons may include: failure to correctly identify the outline of the irregularly shaped vehicle, leading to incorrect execution strategies output by the planning and control software; failure to correctly predict the trajectory of the irregularly shaped vehicle; or repeated perception results of the irregularly shaped vehicle, causing the planning and control software to repeatedly output execution strategies.

[0105] S204. Based on the irregular vehicle shape, extract the corner case features from the corner case data.

[0106] In this step, based on the reason for the corner case (i.e., irregularly shaped vehicles), corner case features such as speed, color, relative position, and size of irregularly shaped vehicles are extracted from the corner case data. Additionally, corner case features such as speed, acceleration, and position of the current vehicle are extracted. Weather environment and road conditions are also extracted.

[0107] S205. Classify the extracted corner case features to obtain the features of the vehicle itself, the features of the target object, and the scene features.

[0108] In this step, based on the reason of the irregularly shaped vehicle, the extracted features are classified into features of the target object (i.e., features of the irregularly shaped vehicle), features of the vehicle itself, and scene features. Among these, weather environment, road conditions, operating conditions, and traffic rules are all considered scene features. After classification, each category can be generalized in a targeted manner to comprehensively cover various scenes.

[0109] By classifying corner cases by feature, the training set used for subsequent training can be divided into multiple subsets. This ensures that training data within the same subset have similar features and target variables. During model training, different models or parameter settings can be used for different subsets to better adapt to the characteristics and target variables of the data, thereby improving the model's generalization ability.

[0110] S206. Based on the pre-configured generalization element library, the features of the vehicle itself, the features of the target object, and the scene features are randomly reconstructed and generalized to obtain the processed corner case features.

[0111] In this step, the pre-configured generalization element library includes a variety of options for random reconstruction and generalization, such as multiple speed features, multiple color features, multiple object size features, multiple weather environment features, multiple traffic flow features, and multiple working condition scene features, which can be used for feature selection for random reconstruction and feature selection for generalization.

[0112] For example, for the features of the irregular vehicle, the longitudinal speed feature of 30 km / h is generalized to a low speed range (20-40 km / h) based on a pre-configured generalization element library, and then randomly reconstructed into a variety of possible speed features (10 km / h, 20 km / h, 40 km / h, 60 km / h); the white feature of the irregular vehicle is generalized to a light color, and then randomly reconstructed into a variety of possible color features (black, red, blue); the size feature of the irregular vehicle is generalized to a medium-sized vehicle, and then randomly reconstructed into a variety of possible sizes (changing the length, width and height data of the vehicle body); the relative position feature of the irregular vehicle from the current vehicle is generalized to a medium distance, and then randomly reconstructed into a variety of possible relative positions.

[0113] For scene features, based on a pre-configured generalization element library, features such as weather, road conditions, traffic flow, and obstacles are randomly reconstructed and generalized. For example, the feature of light rain is generalized to rainy weather and randomly reconstructed into various possible weather conditions, such as sunny day and heavy rain; the feature of highway is generalized to urban road and randomly reconstructed into various possible road conditions; the feature of traffic flow is generalized to moderate traffic flow and randomly reconstructed to generate various traffic flow scenarios, such as the number of pedestrians, traffic light signals, and the number of other vehicles in the surrounding area; obstacle features are randomly added to the scene data according to the generalization element library.

[0114] Regarding the characteristics of the vehicle itself, the lateral speed, longitudinal speed, direction, and vehicle type of the current vehicle are randomly reconstructed and generalized based on the pre-configured generalization element library. The process of generalization and random reconstruction is similar to that of non-standard vehicles, and will not be described in detail here.

[0115] S207. Combine the processed features to generate a similar heterogeneous scene library.

[0116] In this step, the different features of the above random reconstruction and generalization are combined, and the previous feature data is replaced by the processed corner case feature data in the original scene data. The replaced scene data is then mapped onto the virtual 3D scene. Using 3D modeling technology and virtual environment simulation technology, multiple similar heterogeneous virtual scenes are generated, forming a similar heterogeneous scene library.

[0117] S208. Based on scene data from a similar heterogeneous scene library, train the perception model on a simulation test platform to generate an optimized perception model.

[0118] In this step, before model training, based on multiple scene data from a similar heterogeneous scene library, the scene data format is transformed into a data format recognizable by the simulation testing platform. The data is then divided into a training set and a dataset. The training set is further divided into multiple subsets based on feature classification. The original scene data also belongs to the training set. Based on the training set, the perception model of at least one version of the planning and control software in the simulation platform is trained to perform perception, prediction, and planning functions.

[0119] The simulation platform can be a 3D simulation platform based on a physics engine, or a metaverse virtual simulation platform. Simulation tests are conducted in the metaverse virtual simulation platform, and more refined physical simulations are achieved through software simulation. Furthermore, the scene can be flexibly modified and edited during the training process, and the position and speed of vehicles can be dynamically adjusted.

[0120] Optionally, different parameter settings can be used for training on scene data within different subsets after classification. For example, in urban scenarios, a more detailed road network and traffic rules can be used, while in highway scenarios, a simpler road network and traffic rules can be used. In this way, we can train the model more accurately for different subsets, improve the model's generalization ability, and thus improve the stability and reliability of the autonomous driving system.

[0121] S208. Simulation test of the optimized perception model.

[0122] In this step, the optimized perception model is tested through a validation set to see if it can identify similar heterogeneous corner cases related to irregularly shaped vehicles and generate the correct control signals to avoid rear-end collisions. After the simulation test is passed, the optimized model is pushed to the vehicle and the user is prompted to update.

[0123] Optionally, if the current perception model cannot eliminate similar heterogeneous corner cases, it can be randomly reconstructed and generalized again based on these uneliminable corner cases. Then, it can be trained again using a larger library of similar heterogeneous scenes.

[0124] This example provides an optimization method for an intelligent driving perception model. A cloud server acquires raw scene data of corner cases where an autonomous vehicle encounters an irregularly shaped vehicle, leading to a rear-end collision. Based on this raw scene data, reconstruction and causal analysis are performed, and corner case features are extracted. These features are then categorized and generalized using a pre-configured generalization element library, resulting in multiple similar but heterogeneous scene data sets. Simulation training and testing are then conducted on a simulation platform using this data, outputting the final optimized model. This method generates multiple similar but heterogeneous scene data sets, completely eliminating the same type of corner case. Furthermore, by employing feature classification and performing separate generalizations for different categories, it provides more comprehensive coverage of various situations and improves the quality of the generalized model.

[0125] Figure 4 A flowchart illustrating an embodiment of an optimization device for an intelligent driving perception model provided in this application is shown below. Figure 4 As shown, the device 400 includes:

[0126] The acquisition module 411 is used to acquire scene data corresponding to the extreme scenario corner case. The scene data includes image data of the vehicle's exterior in the corner case, the vehicle's speed, the control signals of the vehicle's planning and control software, and weather data.

[0127] Feature acquisition module 412 is used to perform scene reconstruction and recognition analysis on the scene data to obtain cornercase features;

[0128] The generalization processing module 413 is used to randomly reconstruct and generalize the corner case features to obtain the processed corner case features;

[0129] The scene library generation module 414 is used to combine the processed corner case features to generate a similar heterogeneous corner case scene library.

[0130] The model testing module 415 is used to train at least one perception model of the simulation test platform based on the aforementioned heterogeneous corner case scenario library, so as to obtain at least one optimized perception model.

[0131] Optionally, the device further includes:

[0132] The simulation test module 416 is used to perform simulation tests on similar heterogeneous corner case scenarios in the simulation test platform based on the optimized at least one perception model.

[0133] Optionally, the generalization processing module 413 is specifically used for:

[0134] The corner case features are classified to obtain the features of the vehicle itself, the features of the target object, and the scene features;

[0135] Based on a pre-configured generalization element library, the features of the vehicle itself, the features of the target object, and the scene features are randomly reconstructed and generalized to obtain the processed corner case features.

[0136] Optionally, the pre-configured generalized element library includes: multiple weather environment features, multiple speed features, multiple color features, multiple traffic flow features, and multiple working condition scene features;

[0137] Accordingly, the generalization processing module 413 is also used for:

[0138] The vehicle's own speed characteristics are reconstructed and generalized by selecting the various speed features from the generalized element library.

[0139] Multiple velocity features and multiple color features are selected from the generalized element library to reconstruct and generalize the velocity and color features of the target object;

[0140] From the generalized element library, various weather environment features, various traffic flow features, and various working condition scenarios are selected to reconstruct and generalize the weather features, traffic flow features, and working condition features of the scenarios.

[0141] Optionally, the feature acquisition module 412 is specifically used for:

[0142] The scene data corresponding to the corner case is reconstructed and identified to determine the cause of the corner case.

[0143] Based on the cause of the corner case, the corner case features are extracted from the corner case data.

[0144] Optionally, the acquisition module 411 is specifically used for:

[0145] The system receives scenario data corresponding to the corner case uploaded by the vehicle, wherein the scenario data is data collected by the vehicle when the simulated decision and the driver's actual decision are inconsistent.

[0146] The intelligent driving perception model optimization device provided in this application embodiment can be used to execute the intelligent driving perception model optimization method described in any of the above method embodiments. Its implementation principle and technical effect are similar, and will not be repeated here.

[0147] Figure 5 This application provides a schematic diagram of the structure of a server, as shown in the embodiment of the present application. Figure 5 As shown, the server 500 includes:

[0148] The processor 511, the memory 512 communicatively connected to the processor, and the communication interface 513 for interacting with other devices;

[0149] The memory 512 stores computer-executed instructions;

[0150] The processor 511 executes the computer execution instructions stored in the memory 512 to implement the optimization method of the intelligent driving perception model as described in any of the foregoing method embodiments.

[0151] Optionally, the 500 devices in this server can be connected via a system bus.

[0152] The memory 512 can be a separate memory unit or a memory unit integrated into the processor 511.

[0153] Optionally, the communication interface 513 can be used to communicate and interact with external devices. These external devices may be vehicles, as described in the preceding embodiments.

[0154] It should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0155] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. Memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0156] The server provided in this application embodiment can be used to execute the optimization method of the intelligent driving perception model described in any of the above method embodiments. Its implementation principle and technical effect are similar, and will not be repeated here.

[0157] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the optimization method of the intelligent driving perception model as described in any of the foregoing method embodiments.

[0158] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0159] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0160] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. An optimization method for an intelligent driving perception model, characterized in that, The method includes: Obtain the scenario data corresponding to the corner case of the extreme scenario; The scene data corresponding to the corner case is reconstructed and identified to determine the cause of the corner case. Based on the reasons that cause the corner case to occur, extract corner case features from the corner case data; The corner case features are classified to obtain the features of the vehicle itself, the features of the target object, and the scene features; the features of the target object include all feature data of other objects that cause the corner case. Based on a pre-configured generalization element library, the features of the vehicle itself, the features of the target object, and the scene features are randomly reconstructed and generalized to obtain the processed corner case features. The processed corner case features are combined to generate multiple sets of corner case feature data; each set of corner case feature data is used to replace the corresponding feature data in the scene data to obtain multiple sets of replaced scene data; based on the multiple sets of replaced scene data, multiple similar heterogeneous virtual scenes are generated, and the multiple similar heterogeneous virtual scenes constitute a similar heterogeneous corner case scene library. Based on the aforementioned heterogeneous corner case scenario library, at least one perception model of the simulation test platform is trained to obtain at least one optimized perception model. Based on the optimized at least one perception model, simulation tests are conducted on similar heterogeneous cornercase scenarios in the simulation test platform.

2. The method according to claim 1, characterized in that, The pre-configured generalized element library includes: multiple weather environment features, multiple speed features, multiple color features, multiple traffic flow features, and multiple working condition scene features; Accordingly, based on the pre-configured generalization element library, the features of the vehicle itself, the features of the target object, and the scene features are randomly reconstructed and generalized to obtain the processed corner case features, including: The vehicle's own speed characteristics are reconstructed and generalized by selecting the various speed features from the generalized element library. Multiple velocity features and multiple color features are selected from the generalized element library to reconstruct and generalize the velocity and color features of the target object; From the generalized element library, various weather environment features, various traffic flow features, and various working condition scenarios are selected to reconstruct and generalize the weather features, traffic flow features, and working condition features of the scenarios.

3. The method according to claim 1 or 2, characterized in that, The acquisition of scenario data corresponding to the extreme scenario corner case includes: The system receives scenario data corresponding to the corner case uploaded by the vehicle, wherein the scenario data is data collected by the vehicle when the simulated decision and the driver's actual decision are inconsistent.

4. An optimization device for an intelligent driving perception model, characterized in that, The device includes: The acquisition module is used to acquire scene data corresponding to extreme scenario corner cases; The feature acquisition module is used to perform scene reconstruction and identification analysis on the scene data corresponding to the corner case, and to determine the cause of the corner case. Based on the cause of the corner case, features are extracted from the corner case data; the corner case features are classified to obtain the vehicle's own features, the target object's features, and the scene features; the target object's features include all feature data of other objects that caused the corner case. The generalization processing module is used to perform random reconstruction and generalization processing on the features of the vehicle itself, the features of the target object and the scene features based on a pre-configured generalization element library, to obtain the processed corner case features. The scene library generation module is used to combine the processed corner case features to generate multiple sets of corner case feature data; replace the corresponding feature data in the scene data with each set of corner case feature data to obtain multiple sets of replaced scene data; and generate multiple similar heterogeneous virtual scenes based on the multiple sets of replaced scene data. The multiple similar heterogeneous virtual scenes constitute a similar heterogeneous corner case scene library. The model testing module is used to train at least one perception model of the simulation test platform based on the aforementioned heterogeneous corner case scenario library, so as to obtain at least one optimized perception model. The device further includes: The simulation testing module is used to perform simulation tests on similar heterogeneous corner case scenarios in the simulation testing platform based on at least one optimized perception model.

5. A server, characterized in that, include: The processor, the memory communicatively connected to the processor, and the communication interface for interacting with other devices; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the optimization method of the intelligent driving perception model as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the optimization method of the intelligent driving perception model as described in any one of claims 1 to 3.

Images