Simulation static scene construction method and device, terminal equipment and storage medium

Abstract

The invention discloses a simulation static scene construction method and device, terminal equipment and a storage medium, and the method comprises the steps: obtaining each frame of playback data collected in a drive test process, and taking the Gaussian coordinate of a vehicle in the first frame of playback data as a coordinate origin for constructing a simulation static scene; rendering the road network data in the 2D base map and the high-precision map in the road test process into the simulation static scene; and after target information of each static target in each frame of playback data is converted to a Gaussian coordinate system, each static target is rendered to a simulation static scene, and construction and output of the simulation static scene are completed. The method is applied to the field of scene simulation, the high-precision map and the static target in the road test process are rendered into the simulation static scene, the environment condition of the unmanned vehicle in the road test process can be restored more accurately and efficiently, the generated simulation static scene can be used for testing, and the test efficiency is improved. And the algorithm test cost of unmanned driving is greatly reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of information processing and scene simulation, and in particular to a method, device, terminal equipment and storage medium for constructing a simulated static scene based on drive test playback data. Background Art

[0002] In the field of driverless vehicles, on-site road testing is time-consuming and laborious, and each release requires repeated testing. Therefore, more and more companies are turning to driverless simulation to reduce or replace on-site road testing. In driverless simulation, static scene construction is a crucial link, and R&D personnel are paying more and more attention to how to build more realistic and intuitive static scenes.

[0003] The current static scene construction method is mainly based on high-precision maps to generate scenes, or to manually model roads. Scene generation based on high-precision maps is to render the structured vector graphics of high-precision maps in 3D again. This method establishes a one-to-one correspondence between each vector graphic of the high-precision map and the standard model library, and then calls different resources in the library for rendering according to the semantics of the vector graphics, thereby constructing a road scene. However, this method has several problems: first, when constructing a road network through a standard model library, there may be a certain deviation from the actual road network; second, it takes a lot of manpower to establish a standard model library; third, it can only be close to the actual road network information, but it is impossible to reconstruct some static obstacles in the real environment, resulting in the inability to reproduce and display the real unmanned driving scene, and it is impossible to restore the environment of the actual road test.

[0004] Therefore, how to improve the static scene construction method to create real and realistic unmanned driving scenes more accurately and efficiently has become a key point of research. Summary of the invention

[0005] In view of the deficiencies in the above-mentioned prior art, the present invention provides a method, apparatus, terminal device and storage medium for constructing a simulated static scene based on road test playback data. Starting from the playback data of real road tests, the simulated static scene is constructed using the relevant information available in the road test data, thereby creating a real and realistic unmanned driving scene more accurately and efficiently.

[0006] To achieve the above object, the present invention provides a method for constructing a simulated static scene, comprising the following steps:

[0007] Obtain each frame of playback data collected during the road test, and use the Gaussian coordinates of the vehicle in the first frame of the playback data as the coordinate origin for constructing a simulated static scene, thereby obtaining a simulated static scene;

[0008] Acquire a high-precision map and a 2D base map during the road test, and render the road network data in the 2D base map and the high-precision map into the simulated static scene;

[0009] After converting the target information of each static target in each frame of the playback data into a Gaussian coordinate system, each static target is rendered into the simulated static scene, and the simulated static scene is constructed and output.

[0010] In one embodiment, the target information of the static target includes but is not limited to ID information, size information, location information and orientation information of the static target.

[0011] In one embodiment, rendering each of the static objects to the simulated static scene includes:

[0012] For any of the playback data, the position information and orientation information of each of the static targets contained therein are converted from the vehicle position coordinate system to the vehicle body coordinate system, and then converted from the vehicle body coordinate system to the Gaussian coordinate system;

[0013] In a Gaussian coordinate system, calculating the average value of the size information, position information, and orientation information of each of the static objects in the playback data of all frames;

[0014] Based on the average value of the size information, position information and orientation information of each static target in the Gaussian coordinate system, each static target is rendered into the simulated static scene, and the ID of each static target is marked.

[0015] In one embodiment, the process of rendering the road network data in the high-precision map to the simulated static scene includes:

[0016] Extracting lane line information, zebra crossing information and intersection information from the high-precision map, and translating the lane line information, the zebra crossing information and the intersection information to the simulated static scene with the Gaussian coordinates of the vehicle in the first frame of the playback data as the origin;

[0017] In the simulated static scene, lane lines, zebra crossings and intersection areas are drawn based on the lane line information, the zebra crossing information and the intersection information to be close to the real driving environment.

[0018] In one embodiment, the method for constructing a simulated static scene further includes:

[0019] Adding building objects at corresponding positions in the simulated static scene according to the 2D base map; and / or

[0020] According to the test requirements and / or the distribution form of roadside vegetation in the actual scene, vegetation is added at the corresponding position in the simulated static scene to simulate the driving condition of the vehicle on the real road; and / or

[0021] According to the test requirements and / or the actual traffic lights and / or traffic signs on the road, traffic lights and / or traffic signs are added at corresponding positions in the simulated static scene for subsequent simulation of the traffic light recognition algorithm and the traffic sign recognition algorithm.

[0022] In one embodiment, the method for constructing a simulated static scene further includes:

[0023] The position information, speed information and / or orientation information of the ego vehicle in each frame of the playback data are extracted for subsequent playback of the ego vehicle trajectory.

[0024] In one embodiment, the method for constructing a simulated static scene further includes:

[0025] Add lighting and / or camera scripts to the simulated static scene to meet the rendering requirements of the simulated scene.

[0026] To achieve the above object, the present invention further provides a device for constructing a simulated static scene. Using the above method, the device for constructing a simulated static scene comprises:

[0027] Data acquisition module, used to obtain each frame of playback data collected during the road test as well as high-precision maps and 2D maps;

[0028] A scene construction module, used to use the Gaussian coordinates of the vehicle in the playback data of the first frame as the coordinate origin for constructing a simulated static scene, thereby obtaining a simulated static scene;

[0029] A base map rendering module, used for rendering the 2D base map into the simulated static scene;

[0030] A map rendering module, used to render the road network data in the high-precision map into the simulated static scene;

[0031] The static target rendering module is used to convert the target information of each static target in each frame of the playback data into a Gaussian coordinate system, and then render each static target into the simulated static scene.

[0032] To achieve the above object, the present invention further provides a terminal device, which is an unmanned vehicle, an unmanned aerial vehicle, an unmanned driving device or a mobile robot, and the terminal device is provided with:

[0033] Memory, used to store programs;

[0034] A processor is used to execute the program stored in the memory. When the program is executed, the processor is used to execute part or all of the steps of the above method.

[0035] To achieve the above objectives, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions; when the computer-executable instructions are executed by a processor, they are used to implement part or all of the steps of the above method.

[0036] Compared with the prior art, the present invention has the following beneficial technical effects:

[0037] 1. The present invention starts with the playback data of real road tests. By rendering the high-precision map and static targets in the road test process into a static scene, it can not only more accurately and efficiently restore the environmental conditions during the road test of the unmanned vehicle, but also use the generated simulated static scene for testing, ensuring that different algorithms are running in the same environment to compare the advantages and disadvantages of the algorithm effects, greatly reducing the cost of unmanned driving algorithm testing;

[0038] 2. In the static scene constructed by the present invention, except for the road network information such as lane lines, zebra crossings and intersection elements, other objects such as static obstacle targets and traffic lights can be modified, thereby simulating some extreme test scenarios, effectively helping testers to clearly locate fault problems and greatly reducing the testing cost of unmanned driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0040] Figure 1 Flow chart of the method for constructing a simulated static scene in Embodiment 1 of the present invention;

[0041] Figure 2 It is a structural block diagram of a device for constructing a simulated static scene in Embodiment 2 of the present invention;

[0042] Figure 3 This is a structural block diagram of the terminal device in Example 3 of the present invention.

[0043] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0046] In addition, in the present invention, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0047] In the present invention, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, a physical connection, or a wireless communication connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0048] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0049] Example 1

[0050] This embodiment discloses a method for constructing a simulated static scene based on drive test playback data. By using some key scene information in the playback data collected during the drive test to construct the simulated static scene, not only can the environmental state during the drive test be restored more realistically, but also new test scenes can be constructed by adjusting the positions of static obstacles in the scene, which can greatly improve the algorithm verification efficiency and test work efficiency of algorithm engineers.

[0051] Unmanned vehicles or mobile robots generate a large amount of driving data during real-life vehicle testing. Extracting some information from the driving data can be used to build simulated static scenes. If this data is used properly, it can truly restore the driving conditions of unmanned vehicles on actual roads, so that problem scenarios can be reproduced multiple times in simulation, thereby debugging and improving the performance and stability of planning and decision-making algorithms. Figure 1 The method for constructing a simulated static scene in this embodiment specifically includes the following steps:

[0052] Obtain each frame of playback data collected during the road test, and use the Gaussian coordinates of the vehicle in the first frame of playback data as the coordinate origin for constructing a simulated static scene in the Unity3D engine to obtain a simulated static scene;

[0053] Monitor the Buffer channel of the unmanned vehicle or mobile robot, obtain the high-precision map and 2D base map during the road test, and render the road network data in the 2D base map and high-precision map into the simulated static scene;

[0054] Monitor the Buffer channel of the unmanned vehicle or mobile robot, convert the target information of each static target in each frame of playback data during the road test into a Gaussian coordinate system, and then render each static target into a simulated static scene to complete the construction and output of the simulated static scene. The static targets include motor vehicles and non-motor vehicles parked in parking spaces, cars and non-motor vehicles parked on the road, and pedestrians sitting in lounge chairs.

[0055] In autonomous driving, these static targets such as motor vehicles, non-motor vehicles and pedestrians may become moving targets at any time, thus affecting the planning and decision-making algorithms of autonomous driving and causing accidents. Since the construction of simulated static scenes requires static scenes, only motor vehicles, non-motor vehicles and pedestrians that are not moving throughout the process are stored. The stored content includes the ID information, size information, position information and orientation information of the static targets. Among them, the size information of the static target is the length, width and height value of the detection box corresponding to the static target, and the position information is the Gaussian coordinate representation of the corresponding static target.

[0056] As an optional implementation method, based on the constructed simulated static scene, the position, speed, direction and other information of the dynamic obstacle target in the playback data can also be integrated, which can be used to restore the dynamic target scene construction.

[0057] In this embodiment, the process of rendering each static object into a simulated static scene is as follows:

[0058] First, for any frame of playback data, the position information and orientation information of each static target contained in it are converted from the vehicle position coordinate system to the vehicle body coordinate system, and then from the vehicle body coordinate system to the Gaussian coordinate system. The vehicle position coordinate system refers to the coordinate system with the starting point of the vehicle as the origin in the high-precision map, and the vehicle body coordinate system refers to the coordinate system with the current position of the vehicle as the origin in each frame of playback data. The Gaussian coordinate system is the world coordinate system.

[0059] Secondly, since the shape of the detected static target object will change in each frame of playback data, but in the actual scene, the static target is always still, so in the Gaussian coordinate system, the average value of the size information, position information, and orientation information of each static target in all frames of playback data is calculated, and the average value of its size information, position information, and orientation information can be used to draw the static target;

[0060] Based on the average value of the size information, position information, and orientation information of each static target in the Gaussian coordinate system, in the Unity3D engine, the cuboid drawing interface is used to input the size, position (relative to the initial Gaussian coordinates of the vehicle) and orientation of the static target, draw the shape frame of the static target in the simulated static scene, fill it with color, render each static target into the simulated static scene, and mark the ID of each static target.

[0061] In a specific implementation process, when calculating the average value of the static target position information in the Gaussian coordinate system, the x-axis average value and the y-axis average value of the static target in each frame of playback data in the Gaussian coordinate system can be directly calculated.

[0062] In a specific implementation process, when calculating the average value of the static orientation position information in the Gaussian coordinate system, the average value of the orientation angle of the static target in each frame of playback data in the Gaussian coordinate system can be directly calculated.

[0063] In the specific implementation process, since the size information has three size parameters of length, width and height, and the size information of the same static object in different frame playback data has a large difference, the specific process of calculating the average value of the static object size information in the Gaussian coordinate system is:

[0064] First, store the length, width and height of the objects in all frames into three lists respectively, traverse the three lists, find the maximum and minimum values ​​in the lists, remove them from the lists, and then calculate the average of the three new list data to get the size information of the static target.

[0065] In this embodiment, in addition to static target rendering, the construction of the simulated static scene also includes rendering of road network data of high-precision maps such as lane lines, zebra crossings, intersection information, and 2D base map rendering. Among them, the 2D base map is directly mapped to the simulated static scene, thereby more realistically reflecting the driving environment at the time. The process of rendering the road network data in the high-precision map to the simulated static scene includes:

[0066] After obtaining the high-precision map file used in the road test, the lane line information, zebra crossing information and intersection information in the high-precision map are parsed and extracted through the Unity file parsing interface, and the lane line information, zebra crossing information and intersection information are translated to the simulated static scene with the Gaussian coordinates of the vehicle in the first frame of playback data as the origin;

[0067] In the simulated static scene, the lane line information, zebra crossing information and intersection information extracted are installed through the Unity line drawing interface to draw the lane lines, zebra crossings and intersection areas to be close to the real driving environment.

[0068] At this point, a simulated static scene containing a 2D base map, static obstacle targets, lane lines, crosswalks, and intersections can be obtained. In addition, preferably, since the 2D base map can intuitively display the building area, the building object can be added to the corresponding position in the simulated static scene according to the 2D base map.

[0069] The above simulation static scene is generated by the road test playback data. However, this simulation static scene still lacks some static obstacles such as vegetation, traffic signs, and signal lights. Therefore, in the preferred implementation, the simulation static scene can also be improved as follows:

[0070] According to the test requirements and / or the distribution of roadside vegetation in the actual scene, artificially add vegetation at the corresponding position in the simulated static scene to simulate the driving condition of the vehicle on the real road; and / or

[0071] According to the test requirements and / or the actual traffic lights and / or traffic signs on the road, add traffic lights and / or traffic signs at the corresponding positions in the simulated static scene, that is, add the corresponding resources from the Unity traffic environment resource library to the appropriate positions in the simulated static scene for subsequent simulation of the traffic light recognition algorithm and the traffic sign recognition algorithm; and / or

[0072] Add lighting and / or camera scripts to the simulated static scene to meet the rendering needs of the simulated scene.

[0073] As an optional implementation, while constructing the simulated static scene, the position information, speed information and / or orientation information of the ego vehicle in each frame of playback data can also be extracted, and then combined with the constructed simulated static scene, it can be used for the playback of subsequent ego vehicle trajectories.

[0074] It is worth noting that after the simulation static scene in this embodiment is constructed, except for road network information such as lane lines, zebra crossings and intersection elements, other objects such as static obstacle targets and traffic lights can be modified in the Unity scene editor, thereby simulating some extreme test scenarios, effectively helping testers to clearly locate fault problems and greatly reducing the testing cost of unmanned driving.

[0075] Example 2

[0076] Based on the method for constructing a simulated static scene in Example 1, this embodiment discloses a device for constructing a simulated static scene. Figure 2 The device for constructing a simulated static scene includes a data acquisition module, a scene construction module, a base map rendering module, a map rendering module, and a static target rendering module. The device for constructing a simulated static scene is used to execute some or all steps of the method for constructing a simulated static scene in Example 1, so as to realize the construction of a simulated static scene of an unmanned vehicle or a mobile robot or other unmanned driving device during driving. Specifically:

[0077] The data acquisition module is used to obtain each frame of playback data collected during the road test as well as high-precision maps and 2D maps;

[0078] The scene construction module is used to use the Gaussian coordinates of the vehicle in the playback data of the first frame as the coordinate origin of constructing the simulated static scene to obtain the simulated static scene;

[0079] The base map rendering module is used to render the 2D base map into the simulated static scene;

[0080] The map rendering module is used to render the road network data in the high-precision map into the simulated static scene;

[0081] The static target rendering module is used to convert the target information of each static target in each frame of the playback data into a Gaussian coordinate system, and then render each static target into the simulated static scene.

[0082] In this embodiment, the specific working process and working principle of the data acquisition module, scene construction module, base map rendering module, map rendering module and static target rendering module are the same as the method in Example 1, so they will not be repeated in this embodiment.

[0083] Example 3

[0084] like Figure 3The terminal device disclosed in this embodiment is shown, including a transmitter, a receiver, a memory and a processor. Among them, the transmitter is used to send instructions and data, the receiver is used to receive instructions and data, the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions stored in the memory to implement some or all of the steps executed by the method for constructing a simulated static scene in the above embodiment 1. Its specific implementation process is the same as the method for constructing a simulated static scene in the above embodiment 1.

[0085] It should be noted that the above memory can be independent or integrated with the processor. When the memory is independently provided, the terminal device further includes a bus for connecting the memory and the processor.

[0086] In the specific application process, the terminal device is an unmanned vehicle, a drone, an unmanned driving device or a mobile robot, etc.

[0087] Example 4

[0088] This embodiment discloses a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, part or all of the steps executed by the method for constructing a simulated static scene in the above-mentioned embodiment 1 are implemented.

[0089] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A method for constructing a simulated static scene, characterized in that: The steps include: Obtain each frame of playback data collected during the road test, and use the Gaussian coordinates of the vehicle in the first frame of the playback data as the coordinate origin for constructing a simulated static scene, thereby obtaining a simulated static scene; Acquire a high-precision map and a 2D base map during the road test, and render the road network data in the 2D base map and the high-precision map into the simulated static scene; After converting the target information of each static target in each frame of the playback data into a Gaussian coordinate system, each static target is rendered into the simulated static scene, and the simulated static scene is constructed and output.

2. The method for constructing a simulated static scene according to claim 1, characterized in that: The target information of the static target includes but is not limited to ID information, size information, location information and orientation information of the static target.

3. The method for constructing a simulated static scene according to claim 2, characterized in that: The step of rendering each of the static objects to the simulated static scene comprises: For any of the playback data, the position information and orientation information of each of the static targets contained therein are converted from the vehicle position coordinate system to the vehicle body coordinate system, and then converted from the vehicle body coordinate system to the Gaussian coordinate system; In a Gaussian coordinate system, calculating the average value of the size information, position information, and orientation information of each of the static objects in the playback data of all frames; Based on the average value of the size information, position information and orientation information of each static target in the Gaussian coordinate system, each static target is rendered into the simulated static scene, and the ID of each static target is marked.

4. The method for constructing a simulated static scene according to claim 1, 2 or 3, characterized in that: The process of rendering the road network data in the high-precision map to the simulated static scene includes: Extracting lane line information, zebra crossing information and intersection information from the high-precision map, and translating the lane line information, the zebra crossing information and the intersection information to the simulated static scene with the Gaussian coordinates of the vehicle in the first frame of the playback data as the origin; In the simulated static scene, lane lines, zebra crossings and intersection areas are drawn based on the lane line information, the zebra crossing information and the intersection information to be close to the real driving environment.

5. The method for constructing a simulated static scene according to claim 1, 2 or 3, characterized in that: Also includes: Adding building objects at corresponding positions in the simulated static scene according to the 2D base map; and / or According to the test requirements and / or the distribution form of roadside vegetation in the actual scene, vegetation is added at the corresponding position in the simulated static scene to simulate the driving condition of the vehicle on the real road; and / or According to the test requirements and / or the actual traffic lights and / or traffic signs on the road, traffic lights and / or traffic signs are added at corresponding positions in the simulated static scene for subsequent simulation of the traffic light recognition algorithm and the traffic sign recognition algorithm.

6. The method for constructing a simulated static scene according to claim 1, 2 or 3, characterized in that: Also includes: The position information, speed information and / or orientation information of the ego vehicle in each frame of the playback data are extracted for subsequent playback of the ego vehicle trajectory.

7. The method for constructing a simulated static scene according to claim 1, 2 or 3, characterized in that: Also includes: Add lighting and / or camera scripts to the simulated static scene to meet the rendering requirements of the simulated scene.

8. A device for constructing a simulated static scene, characterized in that: According to any one of claims 1 to 7, the device for constructing a simulated static scene comprises: Data acquisition module, used to obtain each frame of playback data collected during the road test as well as high-precision maps and 2D maps; A scene construction module, used to use the Gaussian coordinates of the vehicle in the playback data of the first frame as the coordinate origin for constructing a simulated static scene, thereby obtaining a simulated static scene; A base map rendering module, used for rendering the 2D base map into the simulated static scene; A map rendering module, used to render the road network data in the high-precision map into the simulated static scene; The static target rendering module is used to convert the target information of each static target in each frame of the playback data into a Gaussian coordinate system, and then render each static target into the simulated static scene.

9. A terminal device, characterized in that: For an unmanned vehicle, unmanned aerial vehicle, unmanned driving equipment or mobile robot, the terminal device is provided with: Memory, used to store programs; A processor, configured to execute the program stored in the memory; when the program is executed, the processor is configured to execute part or all of the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions; when the computer-executable instructions are executed by a processor, they are used to implement part or all of the steps of any one of the methods described in claims 1 to 7.

Images