BEV-based OpenX series standard scene generation method

Abstract

The invention relates to the technical field of intelligent network connection automobile testing, and discloses a BEV-based OpenX series standard scene generation method, which comprises the following steps of 1, importing driving data, and preprocessing the driving data; step 2, based on a BEV lane identification technology, identifying lanes around the main vehicle, determining basic information of the lanes, projecting the basic information to a bird's-eye view angle of the main vehicle, and performing element identification to form identification data; according to the time sequence of the identification data, performing fusion to generate static road elements; according to the main vehicle sensor data of the reconstructed scene segment, generating main vehicle and target vehicle tracks, and further forming dynamic traffic participant elements; and step 3, overwriting the static road elements and the dynamic traffic participant elements into a standard xodr scene file and a standard xosc scene file based on an OpenX series standard. According to the method, the test scene with high coverage rate can be quickly constructed and standardized into a data format adaptive to simulation test, and the research and development cost and period of the automatic driving automobile can be greatly reduced and shortened.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent networked vehicle testing, and in particular to a method for generating OpenX series standard scenarios based on BEV. Background Art

[0002] With the rapid development of perception technology, vehicle network communication, and planning and control algorithms, intelligent connected vehicles are undergoing a revolutionary transformation, gradually evolving from assisted driving systems that rely on human intervention to highly automated or even completely unmanned driving. This transformation not only redefines the way of travel, but also has a profound impact on the future of the automotive industry. However, in this wave of technological innovation, safety verification has become a key factor in determining whether self-driving cars can really go on the road. A research report by Rand Corporation pointed out that in order to ensure that the safety performance of unmanned vehicles exceeds that of human drivers, at least 11 billion miles of testing is required, or 100 unmanned vehicles are tested simultaneously in various traffic environments for half a century without interruption, 24 hours a day, 7 days a week. This goal is obviously beyond the capabilities of traditional automotive testing methods, and traditional testing methods are difficult to meet the requirements of long-term, large-scale testing of self-driving cars.

[0003] In the face of this challenge, the introduction of virtual simulation testing technology is particularly urgent. It can simulate almost unlimited driving conditions and traffic scenarios, greatly improving the efficiency and safety of testing, while reducing dependence on actual road testing. Through simulation, not only can rare or dangerous driving situations be reproduced, but also test strategies can be quickly iterated in a virtual environment, accelerating algorithm optimization, and ensuring that the autonomous driving system can make correct decisions when facing complex and changeable real-world traffic conditions.

[0004] However, to achieve accelerated testing of autonomous vehicles, we must face the exponential growth in the number and complexity of test scenarios. This requires us to build a high-coverage scenario library that includes various possible traffic conditions in order to comprehensively evaluate the performance of autonomous driving functions. But the current difficulty is the lack of a systematic approach from data collection, processing, scenario construction to simulation testing. The lack of uniformity in data storage standards and the fact that test scenario construction generally uses cumbersome and time-consuming manual construction methods with the help of simulation software or tools result in low accuracy and real-time performance of scenario generation, and cannot meet the needs of mass automated production.

[0005] In view of this, using massive amounts of autonomous driving data, natural driving data, and traffic accident data to quickly build high-coverage test scenarios and standardize them into a data format suitable for simulation testing, so as to reduce development costs and cycles for users and improve the autonomous driving R&D capabilities of automobile companies, has become a core issue that the industry needs to solve urgently. This solution provides an effective solution path for this. Summary of the Invention

[0006] The present invention aims to provide a method for generating OpenX series standard scenarios based on BEV, which can quickly construct test scenarios with high coverage and standardize them into a data format suitable for simulation testing, significantly reducing the R & D cost and cycle of autonomous vehicles.

[0007] The basic solution provided by the present invention is: a method for generating OpenX series standard scenarios based on BEV, including the following steps:

[0008] Step 1: Import driving data and preprocess the driving data.

[0009] Step 2: Based on BEV lane recognition technology, recognize the lanes around the host vehicle, and determine the lane basic information of the host vehicle's lane and the road lanes; project the lane basic information onto the host vehicle's bird's-eye view, and perform element recognition to form recognition data; according to the time sequence of the recognition data, fuse and generate static road elements.

[0010] Generate the host vehicle trajectory and the target vehicle trajectory based on the host vehicle sensor data of the reconstructed scene segment, and then form dynamic traffic participant elements.

[0011] Step 3: Based on the OpenX series standard, overwrite the static road elements and dynamic traffic participant elements generated in Step 2 into standard xodr scene files and standard xosc scene files.

[0012] The working principle and advantages of the present invention are as follows:

[0013] First, this solution provides a new way of scene generation, adopting a brand-new perspective to build the scene, which can realize the synchronous construction of the dynamic part (vehicle movement) and the static part (map environment) in the test scene, achieve fully automated scene generation, and can convert it into the OpenX series standard format, which can be directly used for simulation testing, and has high scene simulation accuracy and real-time performance, can meet the test needs of users to the greatest extent, provide more accurate and reliable road perception information for the simulation testing of autonomous driving systems, and is expected to promote the development and application of autonomous driving technology.

[0014] The key point is that this solution adopts a different scenario generation method from the existing scenario construction solutions, shifting from the third-party perspective to the "first-person" perspective of the host vehicle. In the existing test scenario construction solutions, it is often based on the description of the scenario, observing the scenario from a macro overhead perspective, and dividing the scenario into two parts. One part is used to analyze the vehicle's trajectory, and the other part is based on CAD to describe the road structure, and then a map environment is generated. This part of the environment is a static three-dimensional space and often needs to be drawn manually based on software such as CAD; the vehicle can confirm its position through GPS positioning, combine the vehicle's position with the static three-dimensional space, and finally generate a complete simulation scenario. This construction method can only be constructed in a partial and sectional manner. Since the test scenario construction is divided into a dynamic part (the movement of the vehicle) and a static part (the static map and environment), the static part of the existing solution needs to be selected based on the dynamic part and then the overall scenario is combined and matched (that is, the existing solution needs to locate the vehicle's GPS positioning in the dynamic part to a certain coordinate point in the global coordinate system, and then match the map based on this coordinate point, that is, move the single frame of the dynamic part into the static part). That is, the process of coordinate conversion and matching needs to be based on one, and then the associated person can perform the matching, so that an automated process cannot be formed.

[0015] In this solution, the single vehicle (host vehicle) is taken as the main perspective, and the data collected by the host vehicle's perception system is processed. Although the scenario construction is also divided into two parts, one part generates the map scenario and the other part generates the movement trajectory. However, this solution does not require coordinate conversion and combination, and the two parts can be carried out synchronously, and the scenario can be generated in real time dynamically; because the map constructed by this solution is continuous and accurately matched with the vehicle. The reason is that in the generation of the static scenario, this solution breakthroughly adopts the method of generating the map based on BEV, and can generate a continuous map. In conventional applications, the BEV lane recognition technology is only used to identify the relationship between the vehicle and the lane line at a single point in time to assist in realizing functions such as adaptive cruise control (ACC), lane keeping assist (LKA), and lane departure warning (LDW) of intelligent driving vehicles. In this solution, however, the lane line recognition ability and environment perception ability of this technology are cleverly utilized. Based on the lane elements extracted by the BEV lane recognition technology, multiple single-frame BEV maps are formed based on this, and then a complete long continuous scenario map is formed in sequence. Moreover, this solution captures the static map elements and calculates the trajectory from the host vehicle's perspective, and can synchronously obtain highly matched static road elements and dynamic traffic participant elements, and the scenario construction accuracy is relatively high.

[0016] Second, the scenario construction method provided by this solution can meet the test requirements of long continuous scenarios for intelligent connected vehicles and achieve high test reliability and efficiency. In existing tests, if one wants to conduct tests on long continuous scenarios, it is necessary to separately construct a virtual continuous test scenario and a virtual vehicle model, and combine with simulation software for simulation to complete the test of long continuous scenarios. In this method, the constructed test scenario can only meet the road length requirements of long continuous scenarios, and the scenarios around the road and the vehicles on the road are randomly generated, unable to simulate real road conditions, lacking test authenticity, and having relatively low test result reliability. In this solution, however, the scenario is generated based on the real perception data of the host vehicle, which can not only ensure the accuracy of the generated road structure and the trajectory of the target vehicle, but also ensure a high degree of matching between the road structure and the trajectory of the target vehicle. Moreover, based on the BEV map generation method, a continuous map can be obtained to meet the test requirements of long continuous scenarios, without the need for additional construction.

[0017] Third, the construction of the dynamic part (vehicle movement) and the static part (map environment) of this solution is completely separated. The constructed dynamic part (vehicle movement) can be further matched with the complete traffic map constructed by other methods according to the GPS positioning information, and more accurate scenario construction can be achieved relying on the high-precision global map, which can be adapted to different application environments and has strong adaptability. Brief Description of the Drawings

[0018] Figure 1 It is a schematic flowchart of the method of Embodiment 1 of the BEV-based OpenX series standard scenario generation method of the present invention;

[0019] Figure 2 It is a schematic flowchart of the generation process of static road elements in Step 2 of Embodiment 1 of the BEV-based OpenX series standard scenario generation method of the present invention;

[0020] Figure 3 It is a schematic flowchart of the generation process of dynamic traffic participant elements in Step 2 of Embodiment 1 of the BEV-based OpenX series standard scenario generation method of the present invention;

[0021] Figure 4 It is a schematic diagram of the multi-view scenario around the vehicle in Embodiment 1 of the BEV-based OpenX series standard scenario generation method of the present invention;

[0022] Figure 5 It is a schematic diagram of the road information obtained by identifying and extracting BEV image features in Embodiment 1 of the BEV-based OpenX series standard scenario generation method of the present invention;

[0023] Figure 6 It is a schematic diagram of pose matching in Embodiment 1 of the BEV-based OpenX series standard scenario generation method of the present invention;

[0024] Figure 7 This is a schematic flowchart of assigning elevation information to a scene map using a point cloud tool in the first embodiment of the BEV-based OpenX series standard scene generation method of the present invention;

[0025] Figure 8 This is a schematic diagram of a semi-automation tool in the first embodiment of the BEV-based OpenX series standard scene generation method of the present invention;

[0026] Figure 9 This is a schematic diagram of OpenDRIVE conversion in the first embodiment of the BEV-based OpenX series standard scene generation method of the present invention;

[0027] Figure 10 This is a schematic diagram of OpenSCENARIO conversion in the first embodiment of the BEV-based OpenX series standard scene generation method of the present invention;

[0028] Figure 11 This is the first schematic diagram of the application effect in the first embodiment of the BEV-based OpenX series standard scene generation method of the present invention;

[0029] Figure 12 This is the second schematic diagram of the application effect in the first embodiment of the BEV-based OpenX series standard scene generation method of the present invention;

[0030] Figure 13 This is the third schematic diagram of the application effect in the first embodiment of the BEV-based OpenX series standard scene generation method of the present invention. Detailed implementation manners

[0031] The following is a further detailed description through specific implementation manners:

[0032] Embodiment 1

[0033] The embodiment is basically as shown in the appendix Figure 1 The BEV-based OpenX series standard scene generation method includes the following steps:

[0034] Step 1: Import driving data and preprocess the driving data.

[0035] Specifically, the driving data is collected by the host vehicle, including the point cloud data, image data (including the panoramic image of the vehicle surrounding environment), etc. collected by the perception devices such as lidar and cameras on the host vehicle during the vehicle driving process; and the driving data (including the speed, acceleration, yaw acceleration information of the host vehicle; the type of the target vehicle and the position information relative to the host vehicle; the pose information of the host vehicle; GPS coordinates, heading angle, and the eastward speed, northward speed, and heading angle of the host vehicle) collected by the GPS, inertial measurement unit, vehicle-mounted sensors, etc. on the host vehicle during the vehicle driving process.

[0036] The preprocessing includes denoising, coordinate transformation, data format conversion, and multi-sensor target-level data fusion. Through preprocessing, the data format can be effectively unified, the data validity can be improved, which helps to ensure the scene validity and reduce meaningless scenes.

[0037] In this step, the preprocessing also includes data verification of the driving data. If the verification passes, the next step is executed; if the verification fails, the driving data is re-imported. Through data verification, the invalid data in the driving data can be effectively screened out.

[0038] The preprocessed driving data includes driving scene data and annotation data. Specifically, in this step, scene annotation is performed by setting a configuration file; for the data that is not easily collected by a single vehicle, such as weather, road grade (highway, national highway / provincial highway, urban expressway, etc.) and road environment (sidewalk, guardrail, roadside objects, vegetation, buildings, etc.), editable annotation is carried out to form annotation data, so as to facilitate subsequent matching of building information around the road, etc., and can improve the matching degree of the generated scene.

[0039] Step 2, based on the BEV lane recognition technology, identify the lanes around the host vehicle, and determine the basic lane information of the lane where the host vehicle is located and the road lanes; project the basic lane information to the bird's-eye view of the host vehicle, and perform feature recognition to form recognition data; according to the time sequence of the recognition data, fuse and generate static road features, such as Figure 2 shown.

[0040] The static road features are complete long continuous static road features; the static road features include road geometric models, lane markings, and traffic signs.

[0041] In this step, the recognition data is a single-frame BEV map (including road information such as lane lines, curbs, sidewalks, etc.).

[0042] Specifically, based on the preprocessed driving data, inverse perspective transformation (IPM) is performed based on the sensor conversion parameters to project the information to the bird's-eye view with the host vehicle lidar as the origin, as Figure 4As shown, road information (lane lines, curbs, sidewalks, etc.) is then recognized and extracted based on BEV image features, such as Figure 5 shown, and a single-frame BEV map is formed.

[0043] When fusing to generate static road elements, first match the vehicle position and attitude information with the BEV map for temporal fusion, as Figure 6 shown, and obtain a complete long continuous scene map as static road elements.

[0044] Optionally, while performing the above steps, the elevation information of the scene map can also be assigned using a point cloud tool, as Figure 7 shown. In addition, a semi-automated tool PCAT can be used to improve the road attribute information in the scene map, as Figure 8 shown; which helps to increase the scene authenticity.

[0045] According to the ego vehicle sensor data of the reconstructed scene segment, including the ego vehicle speed, acceleration, yaw acceleration from different data sources, the GPS coordinates, heading angle, the ego vehicle eastward speed, northward speed, and heading angle; and based on the Gauss-Krüger longitude and latitude coordinate transformation, the speed-heading angle method, and the northeast relative speed integration method, the ego vehicle trajectory is calculated and the cumulative error is eliminated. Based on the ego vehicle trajectory, the target vehicle is screened - by analyzing the target object type, the frequency of continuous target object data appearance, and the target object ID in the annotation data, and then the main target vehicle to be converted is determined. According to the transverse and longitudinal distances of the target object relative to the ego vehicle, the absolute coordinates and heading angle of the target vehicle in the world coordinate system are calculated, and then the target vehicle trajectory is calculated. Wherein, the reconstructed scene segment refers to the required generated scene, that is, the segment for which data collection is required; when importing driving data in step 1, the driving data of the reconstructed scene segment is also imported.

[0046] Write and save the reconstructed ego vehicle trajectory, target vehicle trajectory, target object type, and target vehicle ID in the standard data format, and then form dynamic traffic participant elements, as Figure 3 shown. The dynamic traffic participant elements include acceleration and deceleration behaviors, lane change behaviors, following behaviors, start and stop behaviors.

[0047] Step 3, based on the OpenX series of standards, overwrite the static road elements and dynamic traffic participant elements generated in step 2 into a standard xodr scene file (corresponding to the OpenDRIVE standard format) and a standard xosc scene file (corresponding to the OpenSCENARIO standard format).

[0048] Specifically, read the static road element and dynamic traffic participant elements, check their data specifications (i.e., check whether they conform to the file format standards of the standard xodr scenario file and the standard xosc scenario file), and confirm whether the data matches. When the match is confirmed, based on the GPS information of the host vehicle's initial position, confirm whether the static road element and the dynamic traffic participant elements match.

[0049] Write the road reference line information in the static road element into OpenDRIVE, and combine the road structure and environment information such as the road grade (highway, national highway / provincial highway, urban expressway, etc.), isolation method (lane lines, isolation belts, two-way separation, etc.), road signs (pavement signs and traffic signs, etc.), and road environment (sidewalks, guardrails, street objects, vegetation, buildings, etc.) in the annotation data to convert and generate a static road scenario file in the OpenDRIVE standard format, such as Figure 9 shown.

[0050] Based on the converted static road scenario file, write the host vehicle trajectory and target vehicle trajectory information in the dynamic traffic participant elements into OpenSCENARIO, and combine the traffic environment information (time, weather, lighting, visibility, road surface conditions, traffic flow density, etc.) and supplementary information such as the target vehicle type (sedan, SUV, pickup truck, van, truck, two-wheeler, three-wheeler, special vehicle, etc.) in the annotation data to convert and generate a dynamic scenario file in the OpenSCENARIO standard format, such as Figure 10 shown.

[0051] In addition, as Figure 11 , Figure 12 and Figure 13 shown, it is a schematic diagram of an application example of this method. It can be seen from the figure that this solution can accurately convert OpenX series files based on the data collected by the host vehicle, which can accurately reproduce the scene elements around the host vehicle, and correspondingly reproduce the driving conditions of the host vehicle and different target vehicles, and display them in a dynamic long continuous scene, and the test scene construction effect is good.

[0052] A method for generating an OpenX series standard scenario based on BEV provided by this embodiment can quickly construct a test scenario with high coverage and standardize it into a data format suitable for simulation testing, which can greatly reduce the R & D cost and cycle of autonomous vehicles.

[0053] Embodiment 2

[0054] This embodiment provides an OpenX series standard scenario generation system based on BEV, which is used to execute a method for generating an OpenX series standard scenario based on BEV as described in Embodiment 1; it includes: a data processing module, a BEV lane generation module, a trajectory reconstruction module, and a scenario conversion module.

[0055] The data processing module is used to execute Step 1, that is, to import driving data and preprocess the driving data.

[0056] The BEV lane generation module and the trajectory reconstruction module are used to execute Step 2. Among them, the BEV lane generation module, based on the BEV lane recognition technology, recognizes the lanes around the host vehicle, and determines the lane basic information of the lane where the host vehicle is located and the road lanes; projects the lane basic information to the bird's-eye view of the host vehicle, and performs feature recognition to form recognition data; according to the time sequence of the recognition data, fuses and generates static road elements.

[0057] The trajectory reconstruction module is used to generate the host vehicle trajectory and the target vehicle trajectory based on the host vehicle sensor data of the reconstructed scene segment, and then form dynamic traffic participant elements.

[0058] The scene conversion module is used to overwrite the static road elements and dynamic traffic participant elements as standard xodr scene files and standard xosc scene files based on the OpenX series of standards.

[0059] An OpenX series standard scene generation system based on BEV provided by this embodiment provides a set of full-chain tools for scene generation and conversion, which can combine the open road data structure provided by the OpenX series of standards, and based on the scene data extracted from a large amount of autonomous driving data, natural driving data, and traffic accident data, quickly construct a high-coverage test scene and standardize it into a data format suitable for simulation testing, which can greatly reduce the R & D cost and cycle of autonomous driving vehicles, and can significantly enhance the competitiveness of automobile manufacturers in the R & D of intelligent driving technology.

[0060] The above are only embodiments of the present invention. Specific structures and characteristics and other common knowledge well known in the art are not described in detail here. Those of ordinary skill in the art know all the common technical knowledge in the technical field to which the invention belongs before the application date or the priority date, can know all the existing technologies in this field, and have the ability to apply the conventional experimental means before this date. Those of ordinary skill in the art can, under the inspiration given in this application, combine their own abilities to improve and implement this solution. Some typical well-known structures or well-known methods should not become an obstacle for those of ordinary skill in the art to implement this application. It should be noted that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, and these will not affect the implementation effect of the present invention and the practicality of the patent.

Claims

1. The method for generating a standard scene of the OpenX series based on BEV is characterized in that: The following steps are involved: Step 1: import driving data and preprocess the driving data; Step 2: Based on the BEV lane recognition technology, identify the lanes around the main vehicle and determine the basic lane information of the lane where the main vehicle is located and the road lane; The basic lane information is projected to the main vehicle's bird's-eye view, and elements are identified to form identification data; according to the time sequence of the identification data, static road elements are generated by fusion; Based on the sensor data of the host vehicle in the reconstructed scene fragment, the host vehicle trajectory and the target vehicle trajectory are generated, thereby forming dynamic traffic participant elements; Step 3, based on the OpenX series standards, overwrite the static road elements and dynamic traffic participant elements generated in step 2 into standard xodr scene files and standard xosc scene files.

2. The method for generating a BEV-based OpenX series standard scene according to claim 1, characterized in that: The preprocessed driving data includes driving scene data and labeled data.

3. The method for generating a BEV-based OpenX series standard scene according to claim 1, characterized in that: The preprocessing includes denoising, coordinate conversion, data format conversion, and multi-sensor target-level data fusion.

4. The method for generating a BEV-based OpenX series standard scene according to claim 1, characterized in that: The static road elements are complete long continuous static road elements; the static road elements include road geometry models, lane markings, and traffic signs.

5. The method for generating a BEV-based OpenX series standard scene according to claim 1, characterized in that: The dynamic traffic participant elements include acceleration and deceleration behavior, lane changing behavior, following vehicle behavior, and start and stop behavior.

6. The method for generating a BEV-based OpenX series standard scene according to claim 1, characterized in that: In step 2, the target type, the frequency of occurrence of continuous target data and the target ID in the labeled data are analyzed based on the main vehicle sensor data, and then the main target vehicle to be converted is determined to generate the target vehicle trajectory.

7. The method for generating a BEV-based OpenX series standard scene according to claim 1, characterized in that: In step 2, the recognition data is a single-frame BEV map; when fusing to generate static road elements, the vehicle position and posture information is first matched with the BEV map, time-series fusion is performed, and a complete long continuous scene map is obtained as a static road element.

8. The method for generating a BEV-based OpenX series standard scene according to claim 7, characterized in that: It also includes using point cloud tools to assign elevation information to the scene map.

9. The method for generating a BEV-based OpenX series standard scene according to claim 8, characterized in that: It also includes using semi-automatic tools to improve road attribute information in the scene map.

10. The method for generating a BEV-based OpenX series standard scene according to claim 1, characterized in that: In step 1, the preprocessed driving data is also verified. If the verification passes, the next step is executed; if the verification fails, the driving data is re-imported.

Images