Simulation scene construction method and system based on natural driving data

Abstract

A method and system for building a simulation scene based on natural driving data relates to a construction technology for driving scenes in the field of autonomous driving technology. It solves the problem that the existing autonomous driving scene construction method relies on manual construction, has poor portability, and cannot completely restore all dynamic information. The method collects scene data obtained under natural driving conditions as a basis; extracts csv information from it; converts the coordinates of the vehicle and the target vehicle into the same coordinate system; generates a static map using the open source map OSM and the collected scene data; projects lane information, vehicle information, and target vehicle information into the static map, and at the same time corrects the road network information in the static map; converts the csv information into an xml format file through SCP instructions; and uses time as a trigger condition to import the static map and the xml format file after the road network information is corrected into VTD to generate a dynamic map to complete the construction. The present invention is suitable for the construction of autonomous driving scenes.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to driving scene construction technology. Background Art

[0002] Test scenario libraries effectively support the development, design, and verification of intelligent connected vehicle safety, serving as the fuel that keeps them on the road. Proving that autonomous vehicles are safer than standard driving would require 100 vehicles, continuously tested 24 / 7 at 40 km / h in various real-world scenarios for 225 years! Therefore, virtual simulation testing capabilities must be established, starting with the construction of simulation scenarios that meet testing requirements. These reconstructed scenarios can restore real data, enabling simulation testing, significantly shortening test verification cycles and significantly advancing the development of autonomous driving capabilities.

[0003] Currently, most scenario reconstruction methods rely on manual construction, lack standardization, and are therefore difficult to port and use across software. Furthermore, manual construction varies widely, making it difficult for a single person to replicate their results. Furthermore, logical scenarios are based on extracted scene features and do not fully restore all dynamic information, leading to a series of errors in virtual testing. Summary of the Invention

[0004] The present invention first solves the problems that existing methods for constructing autonomous driving scenarios rely on manual construction, have poor portability, and cannot fully restore all dynamic information.

[0005] The method for constructing a simulation scene based on natural driving data of the present invention comprises the following steps:

[0006] A scene data collection step for collecting scene data obtained under natural driving conditions;

[0007] CSV information extraction step, used to extract CSV information from the scene data, the CSV information including vehicle information, target vehicle information, and lane information;

[0008] The vehicle coordinate conversion step is used to convert the earth coordinate system where the vehicle is located into the geodetic coordinate system;

[0009] The target vehicle coordinate conversion step is used to convert the local coordinate system of the target vehicle into the earth coordinate system;

[0010] The static map acquisition step is used to generate a static map using the open source map OSM and the collected scene data;

[0011] a road condition information projection step, projecting lane information, the own vehicle information after coordinate conversion, and target vehicle information onto the static map, and simultaneously correcting the road network information in the static map;

[0012] A file format conversion step, for converting the CSV information into an XML format file through an SCP instruction;

[0013] The simulation scene construction step is used to use time as the trigger condition, import the static map and XML format file after the road network information is corrected into VTD to generate a dynamic map, and complete the construction of the simulation scene.

[0014] Furthermore, the target vehicle refers to all vehicles around the vehicle from which information is collected that can have an impact on the driving behavior of the vehicle.

[0015] Preferably, the process of converting the target vehicle coordinates is as follows:

[0016] x″=(xm)cosθ+(yn)sinθ

[0017] y"=(yn)cosθ-(xm)sinθ

[0018] To achieve the conversion of coordinate system, in the formula, x and y are the horizontal and vertical coordinates of the vehicle in the local coordinate system, θ is the rotation angle from the local coordinate system to the transition coordinate system, m and n are the horizontal and vertical offsets from the transition coordinate system to the geodetic coordinate system, and x"y" are the horizontal and vertical coordinates of the vehicle in the geodetic coordinate system.

[0019] Preferably, the process of the static image acquisition step is:

[0020] CARLA, an open-source simulator for autonomous driving research, is used to read the contents of open-source map files generated by OsmOpenStreetMap and convert them into the OpenDRIVE format.

[0021] Each road in the open source map osm has two reference lines by default, one for the incoming road and the other for the outgoing road;

[0022] The lowest-level algorithm of the open-source simulator CARLA is used to expand to the right along the reference line to form two lanes.

[0023] Preferably, in the road condition information projection step, the process of correcting the road network information in the static map is:

[0024] a) Calculate the shortest distance between the vehicle coordinates at each sampling point and the reference line in the open source map (OSM). If the shortest distance meets the conditions, the sampling point is considered to belong to the road where the reference line is located;

[0025] b) Based on the lane number of the vehicle, move the reference line horizontally to draw the vehicle's lane and all surrounding lanes;

[0026] c) Load the isolation zone information based on the central isolation zone type;

[0027] d) Identify the reference line closest to the current reference line and define it as the opposite lane reference line;

[0028] e) In addition to the reference line of the opposite lane, the lane on this side is symmetrically extended to the opposite side to form the opposite lane;

[0029] f) Output the xodr map project file based on the OpenDRIVE standard, which is the static map required for the final simulation scene.

[0030] Preferably, the process of the simulation scene building step is:

[0031] Use the SCP command in the VTD simulation software to extract the vehicle's position and movement direction information from the static map after correcting the road network information;

[0032] Departure traffic participants in the simulation environment through commands;

[0033] Use the rapidxml SDK in C++ language to convert the constructed SCP instructions into standard XML format, and then load them into the VTD software together with the static map file after correcting the road network information for preview, thus completing the construction of the simulation scene.

[0034] The above method is implemented by computer software. Therefore, the present invention also protects:

[0035] A computer storage medium stores a computer program, and when the computer program is run, the simulation scene construction method of the present invention is executed.

[0036] An electronic device includes: a processor and a memory, wherein the memory is used to store executable instructions of the processor, and the processor is configured to execute the simulation scene construction method of the present invention by executing the executable instructions.

[0037] The above method is implemented by computer software, and the systems corresponding to the above method are:

[0038] A simulation scenario building system based on natural driving data, which includes the following modules:

[0039] A scene data acquisition module, used to collect scene data obtained under natural driving conditions;

[0040] CSV information extraction module, used to extract CSV information from scene data, the CSV information includes vehicle information, target vehicle information, lane information;

[0041] The vehicle coordinate conversion module is used to convert the earth coordinate system where the vehicle is located into the geodetic coordinate system;

[0042] The target vehicle coordinate conversion module is used to convert the local coordinate system of the target vehicle into the earth coordinate system;

[0043] Static map acquisition module, used to generate static maps using the open source map OSM and collected scene data;

[0044] A road condition information projection module projects lane information, information about the vehicle after coordinate conversion, and information about the target vehicle onto the static map, and simultaneously modifies the road network information in the static map;

[0045] A file format conversion module, used to convert the CSV information into an XML format file through an SCP instruction;

[0046] The simulation scene building module is used to use time as a trigger condition to import the static map and XML format file after the road network information is corrected into VTD to generate a dynamic map, thereby completing the construction of the simulation scene.

[0047] Preferably, the static image acquisition module further includes the following units:

[0048] An open source map osm acquisition unit is used to read the contents of the open source map file generated by OsmOpenStreetMap using CARLA, an open source simulator for autonomous driving research, and convert it into the OpenDRIVE format;

[0049] The road division unit is used to set each road in the open source map OSM to have two reference lines by default, one for the incoming road and the other for the outgoing road;

[0050] The lane expansion unit uses the lowest-level algorithm of the open source simulator CARLA to expand to the right along the reference line to form two lanes.

[0051] Preferably, the road condition information projection module further includes the following units:

[0052] The vehicle lane confirmation unit is used to calculate the shortest distance between the vehicle coordinates at each sampling point and the reference line in the open source map OSM. If the shortest distance meets the conditions, the sampling point is considered to belong to the road where the reference line is located;

[0053] The lane drawing unit is used to translate the reference line according to the lane number of the vehicle and draw the lane of the vehicle and all surrounding lanes;

[0054] Isolation strip loading unit, used to load isolation strip information in combination with the central isolation strip type;

[0055] A lane reference line determination unit is used to identify the reference line closest to the current reference line and define it as the opposite lane reference line;

[0056] The target lane determination unit is used to delete the reference line of the opposite lane and simultaneously move the lane on the own side symmetrically to the opposite side to form an opposite lane;

[0057] The static map output unit is used to output the xodr map project file based on the OpenDRIVE standard, that is, the static map required for the final simulation scene.

[0058] This paper reconstructs simulation scenarios based on natural driving data collection using VTD (Virtual Test Drive) software, guided by the ASAM OpenDRIVE (a descriptive file for static road traffic networks required for autonomous driving simulation applications) and OpenSCENARIO (a standard organization for ADAS and intelligent driving recommended by European automakers and tool providers). OpenDRIVE corresponds to static road networks, while OpenSCENARIO (a file used to describe the dynamic content of driving simulators and traffic simulators) corresponds to vehicle dynamics.

[0059] The present invention proposes a method and system for manually and automatically generating simulation scenarios based on natural driving data, which has the following advantages:

[0060] 1. Based on a set of manually constructed static road network and vehicle dynamic information methods, representative logical scenarios are extracted and restored based on selected scene elements. The generated project files are highly portable and suitable for use on multiple platforms and software.

[0061] 2. Develop a tool chain that automatically generates simulation scenarios, inputting data acquisition information and open source maps to quickly generate a large number of test cases. This reduces labor costs, eliminates errors caused by manual construction, and improves output efficiency.

[0062] 3. Filter out valid fields from numerous collection indicators and classify them for static and dynamic scene construction.

[0063] 4. Through projection and coordinate transformation methods, accurate matching of vehicle position and high-precision map is achieved.

[0064] 5. Based on the data acquisition results, the OSM open source map and CSV data are organically combined to effectively modify the road network structure, change the number of lanes and isolation zones, and make the map information more realistic.

[0065] 6. Use SCP commands to restore vehicle dynamic information in real time without any packet loss, 100% true restoration, and realize the virtualization of the real world.

[0066] The method and system for manually and automatically generating simulation scenarios based on natural driving data described in the present invention are suitable for setting up test scenarios in the field of autonomous driving technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 This is a flowchart of manually building a simulation scene as described in Implementation Method 1.

[0068] Figure 2 These are the four guide line shapes mentioned in the manual construction method.

[0069] Figure 3 This is the road diversion model diagram mentioned in the manual construction method.

[0070] Figure 4 This is the three-intersection model diagram mentioned in the manual construction method.

[0071] Figure 5 This is the preview result of the OpenX project file obtained by the manual construction method.

[0072] Figure 6 This is a flowchart of the method for automatically generating a simulation scene according to the first embodiment.

[0073] Figure 7 It is a schematic diagram of the coordinate conversion process described in the fourth embodiment.

[0074] Figure 8 This is the relative positional relationship between the target vehicle and the host vehicle described in the second embodiment. Implementation Method

[0075] In order to make the technical solutions and advantages of the present invention more clearly described, several specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. However, the various embodiments described below are only part of the specific solutions claimed for protection in this application, rather than all of the solutions.

[0076] Implementation method 1. The method for building a simulation scene based on natural driving data in this implementation method comprises the following steps:

[0077] A scene data collection step for collecting scene data obtained under natural driving conditions;

[0078] CSV information extraction step, used to extract CSV information from the scene data, the CSV information including vehicle information, target vehicle information, and lane information;

[0079] The vehicle coordinate conversion step is used to convert the earth coordinate system where the vehicle is located into the geodetic coordinate system;

[0080] The target vehicle coordinate conversion step is used to convert the local coordinate system of the target vehicle into the earth coordinate system;

[0081] The static map acquisition step is used to generate a static map using the open source map OSM and the collected scene data;

[0082] a road condition information projection step, projecting lane information, the own vehicle information after coordinate conversion, and target vehicle information onto the static map, and simultaneously correcting the road network information in the static map;

[0083] A file format conversion step, for converting the CSV information into an XML format file through an SCP instruction;

[0084] The simulation scene construction step is used to use time as the trigger condition, import the static map and XML format file after the road network information is corrected into VTD to generate a dynamic map, and complete the construction of the simulation scene.

[0085] In this embodiment, the scene data collection step collects scene data collected under natural driving conditions. The natural driving conditions refer to the driving of the test vehicle under natural conditions on public roads nationwide. These scene data can be collected and acquired during the driving process of the test vehicle under natural conditions on public roads nationwide.

[0086] In actual applications, the test vehicle is driven on public roads across the country, and during the driving process, the driving information of the test vehicle itself and all scene information around the test vehicle are collected. Generally, a test vehicle must travel at least 100,000 kilometers on the road to collect scene data in real time.

[0087] The test vehicle is a vehicle equipped with acquisition equipment, which includes 6 4-line laser radars, 2 millimeter-wave radars, 1 Mobileye sensor and 2 cameras, which are used to collect the driving status information of the test vehicle and all image information around the test vehicle. Based on this information, the scene elements required for the construction of the simulation scene can be obtained, for example: the driving speed, position and other dynamic information of the target vehicle can be obtained, and the lane line and curvature information of the current lane can be identified. At the same time, based on the information fed back by the test vehicle's own gyroscope and GPS combined inertial navigation, all the real-time status data of the vehicle required for the construction of the simulation scene can also be obtained, such as: vehicle speed, vehicle driving posture lights. These data are all real scene information. Using this information as the basic data for constructing the simulation scene can obtain a simulation scene that is closer to reality.

[0088] In this embodiment, the CSV information extraction step is to extract the necessary information required for building a simulation scenario, such as:

[0089] The vehicle information is used to construct all the vehicle information felt by the driver when constructing the simulation scene, so as to achieve the effect of realistically reflecting the driving situation in real time. Generally, the extracted information includes at least: vehicle time, vehicle longitude, vehicle latitude, vehicle orientation, vehicle pitch, vehicle roll, and vehicle type.

[0090] The target vehicle information is used to construct the state of the target vehicles around the host vehicle. Therefore, it is necessary to collect the information required to construct each target vehicle so that the dynamic image of the target vehicle can be realistically displayed in the simulation scene. The information generally extracted includes at least: target vehicle time obj_time, target vehicle ID, target vehicle longitudinal position posX relative to the host vehicle, target vehicle lateral position posY relative to the host vehicle, target vehicle orientation angle obj_orientation relative to the host vehicle, and target vehicle type obj_type;

[0091] The target vehicle type obj_type is classified according to vehicle type, for example: passenger car, motorcycle and truck.

[0092] Lane information is required for drawing lanes in a simulation scenario and determining the relative positions of the host vehicle and target vehicle on the road. The extracted information generally includes at least: the number of lanes on the road where the host vehicle is located (lane_amount), the current lane ID (lane_ID), and the type of median strip.

[0093] The types of isolation belts generally include single guardrails, double guardrails, green belts, etc.

[0094] In practice, the time at which the vehicle collects data is used as the sampling point, and the vehicle's information is complete throughout the entire XML sequence. However, issues such as bad weather, close proximity of the target, and obstruction of the target often result in missing point cloud data collected by the LiDAR, making it impossible to effectively identify all target vehicles. Therefore, target vehicle information may be intermittent, and time may not necessarily be exactly the same as obj_time. If time axis desynchronization occurs, data cleaning should be performed, either by reducing the sampling frequency or using linear interpolation to obtain single-axis timestamps.

[0095] In the collected scene data, the coordinate systems of the host vehicle and the target vehicle are not unified: the host vehicle returns longitude and latitude, while the target vehicle returns its distance relative to the host vehicle. The coordinates of the two must be aligned, otherwise they will not appear in the same area. Therefore, in this embodiment, the coordinates of the host vehicle and the target vehicle are unified through the host vehicle coordinate conversion step and the target vehicle coordinate conversion step.

[0096] In this embodiment, the static map acquisition step is to obtain the corresponding open source map OSM according to the geographical location information in the scene data, and generate a static map in combination with the scene data as the background of the simulation scene.

[0097] The file format conversion step is to implement the format conversion of the scene data using the SCP instruction, which is a commonly used instruction in this field and is used to implement the function of remote file copying.

[0098] There are two existing simulation scene construction methods: manual construction and automatic construction, where:

[0099] Manual construction allows for personalized settings for road structures and traffic participants, offering advantages such as high accuracy and clear detail. However, faced with the ever-increasing demands for test cases, manual scenario construction is difficult to keep up with the speed.

[0100] Automated construction is a growing trend in the industry. This involves creating a realistic, 1:1 replica of collected natural driving scene data within simulation software. This provides a factual basis for test case generation, eliminating scenarios that are unlikely to occur in real life but frequently occur in simulation. This reduces repetitive work for engineers and allows for the mass production of a large number of simulation scenarios and test cases.

[0101] Whether it is manual or automatic generation, the same set of generation rules and methods must be followed. This set of rules ensures that the scenes generated by different engineers in different time and space are the same, allowing subsequent users to get started quickly.

[0102] The simulation scene construction method described in this embodiment is an automatic construction method. This method inherits most of the manual construction methods and adds some new methods suitable for machine recognition processing.

[0103] The following is a brief introduction to the existing manual construction method:

[0104] For the existing manual construction method process, see Figure 1 As shown, the specific method is:

[0105] Step 1: Manually construct a static road network based on the OpenDRIVE standard.

[0106] Before placing a vehicle, you need to build its environment—the container that holds it. The relevant standards cover modeling of roads, lanes, intersections, road signs, and more. The following describes how to model each of these aspects.

[0107] Step 1.1. Draw the road and lane.

[0108] A road is a path from one location to another, and a lane is the width of a single row of vehicles. The two are inclusive, and multiple lanes form a road. In the horizontal plane, lanes must be closely connected without gaps. Before constructing lanes, a reference line should be drawn. The geometric elements that express the shape and additional attributes of the road are defined according to the reference line. According to the shape, the reference line can be roughly divided into four categories, namely straight line, circular arc, spiral line, and polynomial curve. Their curvature is expressed as formula (1), and the trajectory is as follows: Figure 2 As shown, there are four trajectories, which are:

[0109] y1=0

[0110] y2=a

[0111] y3=bx

[0112] y4=a+bx+cx 2 +dx 3 (1)

[0113] In the formula, y1, y2, y3 and y4 represent Figure 2 In the four trajectory curves shown, x is the horizontal coordinate of the trajectory, and a, b, c, and d are polynomial parameters.

[0114] Lane width must be greater than 0 and the numbers must be adjacent. According to their functions, they can be divided into the following 9 types:

[0115] ①Shoulder: The soft edge of the road;

[0116] ② Boundary: The hard boundary at the edge of the road. It has the same height as the normal lane for driving;

[0117] ③ Driving lane: a road that is "normally" drivable and does not belong to other types of roads;

[0118] ④ Parking lane: a lane with parking spaces;

[0119] ⑤Divider: Separates traffic in different directions on large roads;

[0120] ⑥ Bicycle lane: a lane reserved for cyclists;

[0121] ⑦ Sidewalk: a road where pedestrians are allowed to walk;

[0122] ⑧Kerb: The curb is different in height from the adjacent lane;

[0123] ⑨Connecting ramp: A ramp connecting two highways.

[0124] Step 1.2: Connect the intersections.

[0125] An intersection is an area where three or more roads meet.

[0126] To navigate a road network, roads must be connected. Lane connectivity presupposes reference line connectivity. Reference lines cannot be broken or overlapped, while lanes can overlap to a small extent. Roads with lanes approaching an intersection are called incoming roads; roads connecting intersections or ramps are called connecting roads. Incoming roads can be considered outgoing roads.

[0127] Figure 3 This diagram shows a road diversion model, where a main road splits into two, illustrating possible road connection scenarios within an intersection. Black arrows indicate reference lines; -1 and -2 represent lane numbers. Positive values ​​are to the left of the reference line, and negative values ​​are to the right. The dashed lines represent connecting roads.

[0128] Figure 4 This is a three-way intersection model diagram, showing the traffic flow relationships between the three main roads. The gray background represents incoming traffic, while the white background represents outgoing traffic. Roads A and B share the same attributes, with the reference line at the center, flanked by incoming and outgoing traffic, respectively. Road C, with the reference line at the far left, has only incoming traffic, no outgoing traffic. The connecting roads at the intersection strictly adhere to the rules for connecting multiple roads.

[0129] Step 1.3. Place objects and signals.

[0130] Objects can affect traffic lanes, most notably parking spaces, crosswalks, and traffic barriers. When placing them, pay attention to their location and direction.

[0131] Signs primarily include traffic signs and traffic lights. Based on placement, traffic signs can be categorized as ground signs and road signs. Ground signs control traffic behavior, such as speed limits and turn restrictions. Road signs come in a variety of types and are typically placed along active roads. Based on their rotational behavior, they can be categorized as static signs and dynamic signs. Dynamic signs, such as traffic lights, change periodically over time and can be configured in the dynamic information section.

[0132] The above is a method for manually constructing a static map, which ultimately forms a project file with XML as the data storage format and the extension .xodr.

[0133] Step 2: Manually build vehicle dynamics based on the OpenSENARIO standard.

[0134] In step 1, we constructed the static road environment for the scene. This step describes how to construct dynamic information. A scene must contain three components: paths and entities, scene content, and conditional triggers. The following sections describe three of these components.

[0135] 2.1 Draw the path and initialize the entity.

[0136] Paths are used to navigate entity instances, and the simulator will use its path strategy to constrain the entity's movement. In the VTD Scenario Editor, there are two ways to draw paths: Create Path and Create Path Shape. The former's shape is nearly identical to the imported xodr map file, and its length can be modified. The latter offers greater freedom and is generally used for complex trajectory motions, such as parking and lateral movement.

[0137] Entities are objects whose positions can change dynamically over time. Other objects are non-pedestrian and non-vehicle instances, such as obstacles, streetlights, and railings. Each instance has the properties and methods of the class that created it. For vehicles, initialization conditions include name, model, color, presence of a driver, driver style, and position relative to the path.

[0138] Step 2.2: Scene content

[0139] Dynamic scene information primarily addresses the question of "which object performs what action" and consists of an initialization element and one or more scene content elements. Each object is composed of actions, presenting a series of meaningful actions over the entire time sequence.

[0140] Initialization elements are mainly used to set the initial conditions of the scene, such as the position and velocity of the entity. Actions include following a line, changing lanes, cutting in, cutting out, etc.

[0141] Step 2.3, event trigger

[0142] Triggers are the result of a combination of conditions. A scenario aggregates a series of meaningful actions, and triggers control these actions. Therefore, triggers play a significant role in how a scenario unfolds. The same set of actions can lead to a variety of different outcomes, depending on how they are triggered.

[0143] Triggers are categorized as start triggers and end triggers. Simply put, they control when an entity's actions occur and when they end. They include delays and conditional edges. Delay refers to the time it takes from when a condition is met to when the condition is reported as satisfied.

[0144] The above is a method for manually constructing dynamic information, which ultimately forms a project file with XML as the data storage format and an .xml extension.

[0145] Import the above xodr and xml project files into the VTD-GUI main interface and preview the results as follows Figure 5As shown in the figure, a represents the constructed static graph, b represents the constructed dynamic graph, and the two are combined to form the simulation result shown in c.

[0146] Implementation method 2. This implementation method is an explanation of the target vehicle in the simulation scene construction method based on natural driving data described in implementation method 1. In this implementation method, the target vehicle refers to: all vehicles around the vehicle that collects information that can affect the driving behavior of the vehicle.

[0147] The target vehicles are all vehicles that can affect the driving of the vehicle in the real scene. These target vehicles will appear in the simulation scene, so it is necessary to obtain all the information of the target vehicles. Figure 8 In the 18-grid diagram shown, the target vehicles are those located in front of, behind, to the left, to the right, in front of the left, in front of the right, in back of the left, in back of the right, second from the left, second from the left, second from the left, second from the left, in front of the right, second from the right, second from the right, second from the left, and second from the right. These vehicles will affect the driving state of the vehicle. Displaying these target vehicles in the simulation scene can make the scene closer to the actual driving environment.

[0148] Implementation method three. This implementation method is an example of the vehicle coordinate conversion step in the simulation scene construction method based on natural driving data described in implementation method one. The vehicle coordinate conversion step adopts the universal transverse Mercator projection as the projection method to realize the coordinate conversion.

[0149] The step of converting the coordinates of this vehicle is a step for converting the earth coordinate system where this vehicle is located into the geodetic coordinate system. Existing coordinate conversion technical means can be used. This embodiment adopts the universal transverse mercator projection UTM (universal transverse mercator) as a projection method to realize the conversion of coordinates. The Mercator projection is also called the conformal cylindrical projection, which belongs to the conformal cylindrical projection. The farther away from the equator, the greater the area deformation. In contrast, the UTM projection artificially divides the world into 60 different longitudinal zones, and one zone can be selected at a time. The area of ​​the selected zone is almost not deformed. Select the corresponding UTM zone number according to the area where the test vehicle is traveling. For example, if the area where the test vehicle is traveling is within Jilin Province, select UTM zone number 51.

[0150] Implementation method 4. This implementation method is an example of the target vehicle coordinate conversion step in the simulation scene construction method based on natural driving data described in implementation method 1. In this implementation method, the target vehicle coordinate conversion step is: first convert the local coordinate system, and then convert it to the geodetic coordinate system after translation.

[0151] The target vehicle coordinate conversion step described in this embodiment has a small amount of data processing and a fast conversion speed.

[0152] See also Figure 7 , the process of the target vehicle coordinate conversion step described in this embodiment is illustrated by way of example:

[0153] By formula (2)

[0154]

[0155] To realize the conversion of the coordinate system, the coordinate system xy is the original local coordinate system in the target vehicle information. In the formula, x and y are the horizontal and vertical coordinates of the target vehicle in the local coordinate system, θ is the rotation angle of the local coordinate system to the transition coordinate system, m and n are the horizontal and vertical offsets of the transition coordinate system to the geodetic coordinate system, and x'' and y'' are the horizontal and vertical coordinates of the target vehicle in the geodetic coordinate system.

[0156] The earth coordinate system can be considered as the global coordinate system. The target vehicle is located in the local coordinate system xy with the vehicle as the origin, see Figure 7 As shown in the figure, xy is the local coordinate system, x'y' is the transition coordinate system, and x'y' is the geodetic coordinate system. The local coordinate system xy is rotated by an angle θ to become the transition coordinate system x'y', and then the transition coordinate system x'y' is moved along the horizontal axis of the geodetic coordinate system by a distance m, and then along the vertical axis of the geodetic coordinate system by a distance n to convert it into the geodetic coordinate system x'y'. The coordinates of the target vehicle are then converted using formula (2) to obtain its coordinates in the geodetic coordinate system.

[0157] The coordinate conversion process described in this embodiment is implemented based on PROJ.4 projection string. PROJ is a format for exchanging coordinate system data, which contains all parameters defining a spatial reference system.

[0158] Implementation 5. This implementation is an example of the static image acquisition step in the method for building a simulation scene based on natural driving data described in Implementation 1. In this implementation, the process of the static image acquisition step is:

[0159] CARLA, an open-source simulator for autonomous driving research, is used to read the contents of open-source map files generated by OsmOpenStreetMap and convert them into the OpenDRIVE format.

[0160] Each road in the open source map osm has two reference lines by default, one for the incoming road and the other for the outgoing road;

[0161] The lowest-level algorithm of the open-source simulator CARLA is used to expand to the right along the reference line to form two lanes.

[0162] Implementation 6. This implementation is an example of the road condition information projection step in the method for building a simulation scene based on natural driving data described in Implementation 1. In this implementation, in the road condition information projection step, the process of correcting the road network information in the static map is as follows:

[0163] a) Calculate the shortest distance between the vehicle's coordinates at each sampling point and the reference line in the open source map (OSM). If this shortest distance meets the requirements, the sampling point is considered to belong to the road where the reference line lies. For example, a typical highway has a maximum of six lanes in one direction. Assuming a lane width of 3.5 meters, the total distance is 21 meters. If the shortest distance is less than 21 meters, the sampling point is considered to belong to the road where the reference line lies.

[0164] b) Based on the lane number of the vehicle, move the reference line horizontally to draw the vehicle's lane and all surrounding lanes;

[0165] c) Load the isolation zone information based on the central isolation zone type;

[0166] d) Identify the reference line closest to the current reference line and define it as the opposite lane reference line;

[0167] e) Delete the reference line of the opposite lane and move the lane on this side symmetrically to the opposite side to form an opposite lane;

[0168] f) Output the xodr map project file based on the OpenDRIVE standard, which is the static map corresponding to the final simulation scene.

[0169] In practice, because virtual static maps automatically generated using open source maps have drawbacks and cannot meet the requirements of virtual simulation, CARLA (an open source simulator for autonomous driving research) is used to read the content of open source map files generated by OsmOpenStreetMap and convert them to the OpenDRIVE format so that they can be extracted as CARLA maps. However, Osm does not include information about the number of lanes. Each road has two reference lines by default, one for the incoming and one for the outgoing. CARLA's bottom-level algorithm extends two lanes to the right based on these reference lines. Therefore, it is not advisable to simply use the map files generated by CARLA as the generated roads will differ significantly from the actual ones.

[0170] In view of the above reasons, the road network information in the obtained static map must be corrected to obtain a simulation scene that is closer to the actual scene.

[0171] Implementation VII. This implementation is an example of the steps of building a simulation scene in the method for building a simulation scene based on natural driving data described in Implementation 1. In this implementation, the process of building the simulation scene is as follows:

[0172] Using the SCP command in the VTD simulation software, extract the position and movement orientation information of the vehicles in the static map after correcting the road network information;

[0173] Start the traffic participants in the simulation environment through commands;

[0174] Use the reference rapidxml SDK in the c++ language to convert the constructed SCP instructions into the standard xml format, and then load them into the VTD software together with the static map file after correcting the road network information for preview to complete the construction of the simulation scenario.

[0175] In actual applications, manually building a simulation scenario only requires setting the initial conditions and turning conditions to achieve the scenario. The information input for automatic reconstruction is all the data of the entire scenario slice, and the sampling frequency of the sensor needs to be considered. For example: when the sampling period is 0.1 s, if the time length of the scenario is 20 s and there are 3 target vehicles in the scenario, then 800 lines of data will be generated. The process of automatically building a simulation scenario cannot simplify these 800 lines of data sets and abstractly consider them as a simple superposition of several actions. All these data need to be poured into the xml file.

[0176] That is: continuous behaviors are cut into discrete points. Connecting each point to create a complete path sequence.

[0177] Using the SCP command in the VTD simulation software, obtain the positions and movement orientations of all vehicles. Among them, the position of the vehicle itself can be directly obtained according to the longitude and latitude, and the position of the target vehicle is indirectly obtained according to the relative distance from the vehicle itself; the movement orientation of the vehicle itself is based on the road forward direction, and the movement orientation of the target vehicle is indirectly obtained according to the vehicle itself. Since the time difference between each point is very short and can be approximated as a continuous curve, speed and acceleration information are not required. Correspondingly, there is no need to set the path because the driving trajectories of the vehicles have been obtained; similarly, behavior information such as acceleration, deceleration, and lane change no longer exists, and instead, a string of dense data sequences appears.

[0178] The format of SCP is similar to xml, and custom message triggers can be used to trigger actions, and the traffic participants in the simulation environment, that is, the movement states (including positions and orientations) of all vehicles, can be triggered through commands. All interfaces and sub-interfaces of SCP must meet the <SCPGUI format requirements>. There is more than one target vehicle. To distinguish different target vehicles, the target vehicle number ID, vehicle model, and color can be set in SCP, and these contents should correspond to the corresponding contents in the previously obtained scenario data.

[0179] To avoid the complex manual construction of SCPs, this solution developed a desktop application using a C++ language compiler. This application allows users to import driving scenario data (in CSV format). Based on the data source, the application constructs SCP instructions (including the location, orientation, ID, and model of all vehicles at each sampling point) and converts them into a standard XML format. The user interface (UI) was designed using PyQt5 (for graphical interface development), eliminating the need to install an integrated development environment (IDE) and SDK packages. Users can automatically generate XML files through simple UI operations. This XML file can be loaded into the VTD software for preview, along with the previously modified static map file for road network information. This completes the automated simulation scenario reconstruction.

[0180] The methods described in the present invention can be implemented using computer software, so the "systems" corresponding to the various methods described above are not described again here.

Claims

1. A method for constructing a simulation scene based on natural driving data, characterized in that: The method comprises: A scene data collection step for collecting scene data obtained under natural driving conditions; CSV information extraction step, used to extract CSV information from the scene data, the CSV information including vehicle information, target vehicle information, and lane information; The vehicle coordinate conversion step is used to convert the earth coordinate system where the vehicle is located into the geodetic coordinate system; The target vehicle coordinate conversion step is used to convert the local coordinate system of the target vehicle into the earth coordinate system; The static map acquisition step is used to generate a static map using the open source map OSM and the collected scene data; a road condition information projection step, projecting lane information, the host vehicle information after coordinate conversion, and the target vehicle information onto the static map, and simultaneously correcting the road network information in the static map; A file format conversion step, for converting the CSV information into an XML format file through an SCP instruction; The simulation scene construction step is used to use time as the trigger condition, import the static map and XML format file after the road network information is corrected into VTD to generate a dynamic map, and complete the construction of the simulation scene.

2. The method for constructing a simulation scene based on natural driving data according to claim 1, characterized in that: The vehicle coordinate conversion step adopts the universal transverse Mercator projection as a projection method to realize the coordinate conversion.

3. The method for constructing a simulation scene based on natural driving data according to claim 1, characterized in that: In the step of converting the target vehicle coordinates, the formula x″=(xm)cosθ+(yn)sinθ y"=(yn)cosθ-(xm)sinθ To achieve the conversion of the coordinate system, in the formula, x and y are the horizontal and vertical coordinates of the vehicle in the local coordinate system, θ is the rotation angle from the local coordinate system to the transition coordinate system, m and n are the horizontal and vertical offsets from the transition coordinate system to the geodetic coordinate system, and the vertical coordinates, x'' and y'' are the horizontal and vertical coordinates of the target vehicle in the geodetic coordinate system.

4. The method for constructing a simulation scene based on natural driving data according to claim 1, characterized in that: The process of the static image acquisition step is: CARLA, an open-source simulator for autonomous driving research, is used to read the contents of open-source map files generated by OsmOpenStreetMap and convert them into the OpenDRIVE format. Each road in the open source map osm has two reference lines by default, one for the incoming road and the other for the outgoing road; The lowest-level algorithm of the open-source simulator CARLA is used to expand to the right along the reference line to form two lanes.

5. The method for constructing a simulation scene based on natural driving data according to claim 1, characterized in that: In the road condition information projection step, the process of correcting the road network information in the static map is as follows: a) Calculate the shortest distance between the vehicle coordinates at each sampling point and the reference line in the open source map (OSM). If the shortest distance meets the conditions, the sampling point is considered to belong to the road where the reference line is located; b) Based on the lane number of the vehicle, move the reference line horizontally to draw the vehicle's lane and all surrounding lanes; c) Load the isolation zone information based on the central isolation zone type; d) Identify the reference line closest to the current reference line and define it as the opposite lane reference line; e) Delete the reference line of the opposite lane and move the lane on this side symmetrically to the opposite side to form an opposite lane; f) Output the xodr map project file based on the OpenDRIVE standard, which is the static map required for the final simulation scene.

6. The method for constructing a simulation scene based on natural driving data according to claim 1, characterized in that: The file format conversion step is to use the rapidxml SDK referenced in the c++ language to convert the csv information into a standard xml format file; The process of the simulation scene construction steps is as follows: Use the SCP command in the VTD simulation software to extract the vehicle's position and movement direction information from the static map after correcting the road network information; Departure traffic participants in the simulation environment through commands; The standard XML format file and the static map file after the road network information is corrected are loaded into the VTD software for preview to complete the construction of the simulation scene.

7. A simulation scene construction system based on natural driving data, characterized in that: The system includes the following modules: A scene data acquisition module, used to collect scene data obtained under natural driving conditions; CSV information extraction module, used to extract CSV information from scene data, the CSV information includes vehicle information, target vehicle information, lane information; The vehicle coordinate conversion module is used to convert the earth coordinate system where the vehicle is located into the geodetic coordinate system; The target vehicle coordinate conversion module is used to convert the local coordinate system of the target vehicle into the earth coordinate system; Static map acquisition module, used to generate static maps using the open source map OSM and collected scene data; A road condition information projection module projects lane information, coordinate-converted vehicle information, and target vehicle information onto the static map, and simultaneously modifies the road network information in the static map; A file format conversion module, used to convert the CSV information into an XML format file through an SCP instruction; The simulation scene building module is used to use time as a trigger condition to import the static map and XML format file after the road network information is corrected into VTD to generate a dynamic map, thereby completing the construction of the simulation scene.

8. The simulation scene construction system based on natural driving data according to claim 7, characterized in that: The static image acquisition module further includes the following units: An open source map osm acquisition unit is used to read the contents of the open source map file generated by OsmOpenStreetMap using CARLA, an open source simulator for autonomous driving research, and convert it into the OpenDRIVE format; The road division unit is used to set each road in the open source map OSM to have two reference lines by default, one for the incoming road and the other for the outgoing road; The lane expansion unit uses the lowest-level algorithm of the open source simulator CARLA to expand to the right along the reference line to form two lanes.

9. A computer storage medium, characterized in that The storage medium stores a computer program, and when the computer program is run, the simulation scene construction method according to any one of claims 1 to 6 is executed.

10. An electronic device comprising: A processor and a memory, characterized in that the memory is used to store executable instructions of the processor, and the processor is configured to execute the simulation scene construction method described in any one of claims 1 to 6 by executing the executable instructions.

Images