Autonomous driving simulation system and method, and storage medium

Abstract

The invention provides an autonomous driving simulation system and method and a storage medium, and the system comprises an environment scene simulation module which is used for generating a simulation environment scene according to the assets of scene elements, preset scene parameters and a drawn road network map; the sensor simulation module is used for generating sensor data of the simulation vehicle in the simulation environment scene based on the set sensor simulation parameters; the traffic flow simulation module is used for generating traffic participants on the road network map according to a preset generation rule and a received generation instruction, and controlling the motion states of the traffic participants in the simulation environment scene; and the dynamics simulation module is used for controlling the simulation vehicle to run in the simulation environment scene based on the motion characteristics of the simulation vehicle according to the set various kinematics parameters, the sensor data and the motion states of the traffic participants, and generating a simulation result. According to the technical scheme provided by the embodiment of the invention, the accuracy of a simulation result is improved, and the development efficiency and the optimization efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of scene simulation, and in particular to an autonomous driving simulation system, method and storage medium. Background Art

[0002] The field of autonomous driving technology is one of the key areas in the advanced intelligent technology of contemporary vehicle platforms. With the continuous development of technologies such as artificial intelligence and automatic control, unmanned vehicles have become one of the important participants in transportation and special operations. There are many types of chassis configurations of ground unmanned vehicles, including micro, light, medium, etc., and the driving modes of the vehicles mainly include 4x4, 6x6, 8x8, etc. Autonomous driving algorithms mainly include environmental perception algorithms, decision-making and planning algorithms, and maneuvering control algorithms. In related technologies, in order to verify and evaluate the performance of autonomous driving algorithms, multiple simulation platforms have been built. However, early simulation platforms mostly adopted semi-physical simulation strategies, and simulated and tested algorithms based on data packets actually collected in real scenarios, which had considerable limitations in practical applications. For example, the amount of data required for simulation testing is huge, and it is impossible to freely build the desired test environment and scenario, and it is impossible to freely change the configuration layout of sensors according to actual needs.

[0003] In addition, existing simulation verification platforms have certain limitations in simulating real environments and vehicle dynamics. This may cause the performance of the algorithm on the simulation platform to deviate from that in the actual environment, resulting in low accuracy and credibility of the simulation results, which in turn limits the development and optimization of the algorithm, resulting in low algorithm development efficiency and optimization efficiency. Summary of the invention

[0004] In view of the above problems, the embodiments of the present invention provide an autonomous driving simulation system, method and storage medium to solve the problems that the simulation verification platform in the prior art has limitations, the accuracy and credibility of the simulation results are low, resulting in low algorithm development efficiency and optimization efficiency.

[0005] In a first aspect, an embodiment of the present invention provides an autonomous driving simulation system, the system comprising:

[0006] The environment scene simulation module is used to generate a simulation environment scene based on the assets of the scene elements, the preset scene parameters and the drawn road network map;

[0007] A sensor simulation module, used for generating sensor data of the simulated vehicle in the simulated environment scene based on set sensor simulation parameters;

[0008] A traffic flow simulation module, used to generate traffic participants on the road network map according to preset generation rules and received generation instructions, and control the movement state of the traffic participants in the simulation environment scene;

[0009] The dynamics simulation module is used to control the simulated vehicle to run in the simulation environment scene and generate simulation results based on the motion characteristics of the simulated vehicle, various set kinematic parameters, the sensor data and the motion status of the traffic participants.

[0010] In a possible implementation, the environmental scene simulation module is specifically used to build a static scene through the Unreal Engine according to the assets of the scene elements and preset scene parameters; associate the road network map with the static scene to generate the simulated environmental scene.

[0011] In a possible implementation manner, the sensor simulation module is further configured to add noise and error when generating the sensor data.

[0012] In a possible implementation, the traffic flow simulation module includes a random generation submodule and a custom generation submodule, and the traffic participants include random traffic participants and custom traffic participants;

[0013] The random generation submodule is used to randomly generate random traffic participants on the road network map based on a preset generation rule;

[0014] The custom generation submodule is used to generate a custom traffic participant on the road network map in response to the generation instruction, and the generation instruction carries the type, quantity, initial position, status and target route of the custom traffic participant.

[0015] In a possible implementation, the system further includes an integrated interface module for connecting the environmental scene simulation module, the sensor simulation module, the traffic flow simulation module and the dynamics simulation module.

[0016] In a possible implementation, the integrated interface module is further used to provide a communication interface between the autonomous driving simulation system and the outside.

[0017] In a possible implementation, the communication interface includes a robot operating system ROS, ROS2, Apollo, C++ or Python application programming interface API.

[0018] In a possible implementation, the sensor modalities of the sensor simulation module include laser radar, millimeter wave radar, red, green and blue (RGB) camera, depth camera, inertial navigation device, and global navigation satellite system (GNSS).

[0019] In a second aspect, an embodiment of the present invention provides an autonomous driving simulation method, the method comprising:

[0020] Generate a simulation environment scene based on the assets of scene elements, preset scene parameters and drawn road network map;

[0021] Generate sensor data of the simulated vehicle in the simulated environment scene based on the set sensor simulation parameters;

[0022] Generate traffic participants on the road network map according to preset generation rules and received generation instructions, and control the movement state of the traffic participants in the simulation environment scene;

[0023] Based on the motion characteristics of the simulated vehicle, according to various set kinematic parameters, the sensor data and the motion status of the traffic participants, the simulated vehicle is controlled to run in the simulation environment scene to generate simulation results.

[0024] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the autonomous driving simulation method as described in the first aspect or any possible implementation of the first aspect.

[0025] In the technical solution provided by the embodiment of the present invention, users can complete the non-physical simulation of the entire process of the autonomous driving algorithm through the autonomous driving simulation system, and can more accurately simulate the real environment scenes and the motion status of traffic participants, thereby improving the accuracy and credibility of the simulation results, and improving development efficiency and optimization efficiency.

[0026] In the embodiments of the present invention, users can quickly restore or create various simulation environment scenarios according to development needs through the autonomous driving simulation system, including conventional simulation environment scenarios, simulation environment scenarios with higher complexity, and simulation environment scenarios under extreme conditions, which can effectively save testing costs and shorten the testing cycle, thereby further improving development efficiency and optimization efficiency.

[0027] In the embodiment of the present invention, during the subsequent development and use process, the user can edit and modify the simulation environment scene at any time, which ensures the flexibility and openness of the development process, thereby further improving the development efficiency and optimization efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A schematic diagram of the structure of an autonomous driving simulation system provided by an embodiment of the present invention.

[0029] Figure 2A schematic diagram of an autonomous driving simulation system provided by an embodiment of the present invention.

[0030] Figure 3 The present invention provides a flowchart of an autonomous driving simulation method. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the present invention clearer and more understandable, the present invention is further described below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] Figure 1 A schematic diagram of the structure of an autonomous driving simulation system provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the autonomous driving simulation system includes an environment scene simulation module 11, a sensor simulation module 12, a traffic flow simulation module 13 and a dynamics simulation module 14. The environment scene simulation module 11 is electrically connected to the sensor simulation module 12, the traffic flow simulation module 13 and the dynamics simulation module 14 respectively, the sensor simulation module 12 is electrically connected to the traffic flow simulation module 13 and the dynamics simulation module 14 respectively, and the traffic flow simulation module 13 is electrically connected to the dynamics simulation module 14. Wherein:

[0033] The environmental scene simulation module 11 is used to generate a simulated environmental scene according to the assets of the scene elements, preset scene parameters and the drawn road network map.

[0034] The sensor simulation module 12 is used to generate sensor data of a simulated vehicle in a simulated environment scene based on set sensor simulation parameters.

[0035] The traffic flow simulation module 13 is used to generate traffic participants on the road network map according to preset generation rules and received generation instructions, and control the movement status of traffic participants in the simulation environment scene.

[0036] The dynamics simulation module 14 is used to control the simulated vehicle to run in the simulated environment scene and generate simulation results based on the motion characteristics of the simulated vehicle, various set kinematic parameters, sensor data and the motion status of traffic participants.

[0037] In the embodiment of the present invention, the overall simulation process that can be implemented by the autonomous driving simulation system can be displayed through a display screen, and the main simulation object of the autonomous driving simulation system is a wheeled motor vehicle.

[0038] In the embodiment of the present invention, the simulation environment scene includes multiple scene elements; the environment scene simulation module 11 is specifically used to build a static scene according to the assets of the scene elements and preset scene parameters through the Unreal Engine (UE); the road network map is associated with the static scene to generate a simulation environment scene. The scene elements include three-dimensional models of various buildings, vegetation, infrastructure, road networks, vehicles, pedestrians, obstacles, and physical environment attributes such as atmospheric conditions, illumination, color temperature, environmental occlusion, rain, fog, snow, and clouds. Among them, the road network includes the road structure for vehicle travel, including lanes, intersections, roundabouts, ramps, etc. The scene parameters include the parameters of the above-mentioned scene elements and the parameters of the physical environment attributes.

[0039] Specifically, the environmental scene simulation module 11 expands the interfaces of the sensor simulation module 12 and the dynamics simulation module 14 based on the Unreal Engine, and can fully integrate three-dimensional materials and convert them into UE assets to complete the construction of static scenes through the Unreal Engine.

[0040] Optionally, the road network map can be drawn by RoadRunner software, which supports the OpenDrive1.4 standard format and supports users to customize road surface micro features based on development requirements. Road surface micro features refer to the surface morphology and structural features of the road surface. In actual applications, users can generate road network maps by other means, which are not limited in the embodiments of the present invention.

[0041] In the embodiment of the present invention, the environment scene simulation module can provide a more realistic physical property interaction, support the import of scenes and three-dimensional model materials in common formats, and map the environment scenes in the real world according to actual development needs, or build a purely virtual environment scene, which can meet the different development needs of users. In addition, in the subsequent development and use process, users can edit and modify the simulated environment scene at any time, ensuring the flexibility and openness of the development process, thereby further improving the development efficiency and optimization efficiency.

[0042] Specifically, the sensor simulation module 12 can provide the modes of sensors commonly used in various autonomous driving algorithms, and render and output sensor data in real time according to the simulated environment scene. The format of the sensor data output by the sensor simulation module 12 is consistent with the data format output by the sensor in the real world. The sensor simulation parameters include the type, quantity and installation location of the sensor. In actual applications, users can set the sensor simulation parameters according to actual usage requirements. For example, in order to detect the environmental information around the simulated vehicle, it is necessary to install 4 laser radar sensors on the simulated vehicle, and the 4 laser radar sensors are respectively arranged at the left front, right front, left rear and right rear of the simulated vehicle. The laser radar sensor can move synchronously with the simulated vehicle to improve the accuracy of the detected environmental information.

[0043] In the embodiment of the present invention, the sensor simulation module 12 is also used to add noise and error when generating sensor data. For example, the laser radar and camera will generate appropriate random disturbances with changes in weather and lighting conditions to simulate the noise characteristics of real-world sensors to improve the authenticity and credibility of the sensor data, thereby improving the accuracy of the simulation results.

[0044] In the embodiment of the present invention, the sensor modalities of the sensor simulation module 12 include laser radar, millimeter wave radar, red green blue (RGB) camera, depth camera, inertial navigation device, and global navigation satellite system (GNSS).

[0045] Specifically, the LiDAR and millimeter-wave radar sensor modes simulate the emission and echo of the beam through the ray tracing algorithm to obtain the obstacle information and ground information around the vehicle. The RGB camera and the depth camera obtain the pixel information within the specified viewing angle and the specified range through scene rendering. The infrared camera obtains the temperature distribution and thermal radiation intensity of the objects in the scene by establishing a thermal radiation model of the scene objects. The inertial navigation device and GNSS read the vehicle's speed, acceleration, angular acceleration, angle and absolute position coordinate information through the Unreal Engine. Among them, the absolute position coordinate information is the latitude and longitude coordinate information of the real world generated by the projection of the Geographic Information System (GIS) reference system.

[0046] In an embodiment of the present invention, the traffic flow simulation module 13 includes a random generation submodule and a custom generation submodule. Traffic participants include random traffic participants and custom traffic participants, and traffic participants are used to simulate other road users who interact with autonomous vehicles in the real world, such as pedestrians, cyclists, other vehicle drivers and other road users. The random generation submodule is used to randomly generate random traffic participants on a road network map based on preset generation rules. The density, distribution pattern and movement characteristics of random traffic participants all comply with the rules of the road network map. The custom generation submodule is used to generate custom traffic participants on a road network map in response to a generation instruction, and the generation instruction carries the type, quantity, initial position, state and target route of the custom traffic participants. Among them, the state of the custom traffic participant can be dynamic or static, and the initial position can be the initial position coordinates. Optionally, the generation instruction can be input by a user through a graphical user interface (Graphical User Interface, referred to as GUI).

[0047] Specifically, the traffic flow simulation module 13 controls the motion state of traffic participants in the simulation environment scene, which means that the traffic flow simulation module 13 controls the traffic participants to simulate the behavior of vehicles or pedestrians in the real world in the simulation environment scene. For example, in the embodiment of the present invention, the vehicle's starting, acceleration and deceleration, turning, lane change, avoidance and other behaviors, as well as pedestrians' normal walking and crossing the road in compliance with traffic lights, etc., are all required to comply with the traffic rules of the real world, thereby improving the authenticity and reliability of the simulation test, and then improving the accuracy of the simulation results, which is conducive to the development and optimization of autonomous driving algorithms.

[0048] In the embodiment of the present invention, the dynamics simulation module 14 supports the custom setting of various kinematic parameters of the simulated vehicle, wherein the kinematic parameters include but are not limited to mass, driving torque, braking torque, driving wheel selection, steering wheel selection, maximum steering angle, suspension system damping, suspension travel, tire friction factor and yaw moment.

[0049] In the embodiment of the present invention, the dynamics simulation module 14 supports dynamics simulation of vehicles with different chassis forms. For example, the chassis forms supported by the dynamics simulation module 14 include but are not limited to conventional four-wheel Ackerman chassis, multi-axle Ackerman chassis and three-wheel chassis, thereby expanding the application scenarios of the autonomous driving simulation system.

[0050] Specifically, the dynamics simulation module 14 is also used to control the operation of the simulated vehicle in the simulation environment scene based on the motion characteristics of the simulated vehicle, according to various set kinematic parameters, sensor data and the motion status of traffic participants, and before generating the simulation results, the motion skeleton binding and collision volume setting of the simulated vehicle are implemented through the Unreal Engine, and the motion characteristics of the simulated vehicle in the simulation environment scene are agreed upon.

[0051] Optionally, the dynamics simulation module 14 can realize the dynamics simulation of the vehicle through the Unreal Engine or an external simulator interface. For example, the external simulator interface includes a CarSim / TruckSim interface. In practical applications, the autonomous driving simulation system can obtain kinematic parameters with a higher degree of refinement through an external simulator. For example, the external simulator can calculate the kinematic parameters such as the position and speed of the simulated vehicle, and transmit them back to the autonomous driving simulation system through the interface, so that the autonomous driving simulation system can obtain more accurate simulation results based on the kinematic parameters with a higher degree of refinement, thereby further improving the efficiency of algorithm development and optimization.

[0052] In an embodiment of the present invention, the autonomous driving simulation system further includes an integrated interface module 15, which is used for data connection with the environment scene simulation module 11, the sensor simulation module 12, the traffic flow simulation module 13 and the dynamics simulation module 14. That is, through the integrated interface module 15, the environment scene simulation module 11, the sensor simulation module 12, the traffic flow simulation module 13 and the dynamics simulation module 14 respectively establish data connections. The integrated interface module 15 is also used to provide a communication interface between the autonomous driving simulation system and the outside, so as to realize the data connection between the autonomous driving simulation system and the external autonomous driving algorithm. It should be noted that the integrated interface module 15 realizes the data communication between the various modules inside the autonomous driving simulation system through the C++ programming interface, and realizes the data communication between the autonomous driving simulation system and the outside through the Python programming interface.

[0053] Specifically, the interface forms of the communication interface include the Robot Operating System (ROS), ROS2, Apollo, C++ or Python application programming interface (API). For example, the communication interface between the autonomous driving simulation system and the outside includes a simulation environment scene setting interface, a sensor data acquisition interface and a vehicle motion control interface. The user can edit and modify the scene parameters in the simulation environment scene through the simulation environment scene setting interface; or, the user can obtain the sensor data generated by the sensor simulation module 12 through the sensor data acquisition interface; or, the user can realize the motion control of the simulated vehicle through the vehicle motion control interface.

[0054] In the embodiment of the present invention, the vehicle motion control interface has two control data input modes, namely, the algorithm input mode and the driving kit input mode. The driving kit includes a steering wheel and pedals. In actual applications, users can set external communication interfaces to implement other autonomous driving algorithm functions according to actual development requirements, and the embodiment of the present invention does not limit this.

[0055] Figure 2 A schematic diagram of an autonomous driving simulation system provided by an embodiment of the present invention is shown in FIG. Figure 2As shown in the figure, the external autonomous driving algorithm establishes data connection with the autonomous driving simulation system through the ROS / ROS2 interface and the Python programming interface, and the vehicle motion control interface establishes data connection with the autonomous driving simulation system through the ROS / ROS2 interface and the Python programming interface. Among them, the autonomous driving simulation system is developed based on the Unreal Engine and the CARLA open source platform. On this basis, internal parameter optimization is carried out, a multi-axis chassis dynamics model is added, and an external integration interface is provided, so that the autonomous driving simulation system can efficiently complete simulation testing, thereby improving development efficiency and optimization efficiency.

[0056] like Figure 2 As shown, the autonomous driving simulation system obtains the road network map generated by the Road Runner software and the three-dimensional model generated by the 3D modeling tool to realize the construction of the simulation environment scene.

[0057] In the technical solution provided by the embodiment of the present invention, users can complete the non-physical simulation of the entire process of the autonomous driving algorithm through the autonomous driving simulation system, and can more accurately simulate the real environment scenes and the motion status of traffic participants, thereby improving the accuracy and credibility of the simulation results, and improving development efficiency and optimization efficiency.

[0058] In the embodiments of the present invention, users can quickly restore or create various simulation environment scenarios according to development needs through the autonomous driving simulation system, including conventional simulation environment scenarios, simulation environment scenarios with higher complexity, and simulation environment scenarios under extreme conditions, which can effectively save testing costs and shorten the testing cycle, thereby further improving development efficiency and optimization efficiency.

[0059] Figure 3 A flowchart of an autonomous driving simulation method provided by an embodiment of the present invention is shown in FIG. Figure 3 As shown, the method includes:

[0060] Step 101: Generate a simulation environment scene according to the assets of scene elements, preset scene parameters and the drawn road network map.

[0061] Each step in the embodiment of the present invention is executed by an autonomous driving simulation system. For a detailed description, please refer to the above-mentioned embodiment of the autonomous driving simulation system, which will not be repeated here.

[0062] Step 102: Generate sensor data of a simulated vehicle in a simulated environment scene based on the set sensor simulation parameters.

[0063] Step 103: Generate traffic participants on the road network map according to preset generation rules and received generation instructions, and control the movement status of the traffic participants in the simulation environment scene.

[0064] Step 104 , based on the motion characteristics of the simulated vehicle, according to various set kinematic parameters, sensor data and the motion status of traffic participants, the simulated vehicle is controlled to run in the simulated environment scene to generate simulation results.

[0065] In the technical solution provided by the embodiment of the present invention, users can complete the non-physical simulation of the entire process of the autonomous driving algorithm through the autonomous driving simulation system, and can more accurately simulate the real environment scenes and the motion status of traffic participants, thereby improving the accuracy and credibility of the simulation results, and improving development efficiency and optimization efficiency.

[0066] In the embodiments of the present invention, users can quickly restore or create various simulation environment scenarios according to development needs through the autonomous driving simulation system, including conventional simulation environment scenarios, simulation environment scenarios with higher complexity, and simulation environment scenarios under extreme conditions, which can effectively save testing costs and shorten the testing cycle, thereby further improving development efficiency and optimization efficiency.

[0067] In the embodiment of the present invention, during the subsequent development and use process, the user can edit and modify the simulation environment scene at any time, which ensures the flexibility and openness of the development process, thereby further improving the development efficiency and optimization efficiency.

[0068] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the steps of the embodiment of the above-mentioned autonomous driving simulation method. For a specific description, please refer to the embodiment of the above-mentioned autonomous driving simulation method.

[0069] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An autonomous driving simulation system, characterized in that: The system comprises: The environment scene simulation module is used to generate a simulation environment scene based on the assets of the scene elements, the preset scene parameters and the drawn road network map; A sensor simulation module, used for generating sensor data of the simulated vehicle in the simulated environment scene based on set sensor simulation parameters; A traffic flow simulation module, used to generate traffic participants on the road network map according to preset generation rules and received generation instructions, and control the movement state of the traffic participants in the simulation environment scene; The dynamics simulation module is used to control the simulated vehicle to run in the simulation environment scene and generate simulation results based on the motion characteristics of the simulated vehicle, various set kinematic parameters, the sensor data and the motion status of the traffic participants.

2. The system according to claim 1, characterized in that The environmental scene simulation module is specifically used to build a static scene through the Unreal Engine according to the assets of the scene elements and preset scene parameters; associate the road network map with the static scene to generate the simulated environmental scene.

3. The system according to claim 1, characterized in that The sensor simulation module is also used to add noise and error when generating the sensor data.

4. The system according to claim 1, characterized in that The traffic flow simulation module includes a random generation submodule and a custom generation submodule, and the traffic participants include random traffic participants and custom traffic participants; The random generation submodule is used to randomly generate random traffic participants on the road network map based on a preset generation rule; The custom generation submodule is used to generate a custom traffic participant on the road network map in response to the generation instruction, and the generation instruction carries the type, quantity, initial position, status and target route of the custom traffic participant.

5. The system according to claim 1, characterized in that The system also includes an integrated interface module for connecting the environmental scene simulation module, the sensor simulation module, the traffic flow simulation module and the dynamics simulation module.

6. The method according to claim 5, characterized in that The integrated interface module is also used to provide a communication interface between the autonomous driving simulation system and the outside.

7. The system according to claim 6, characterized in that The communication interface may be in the form of a robot operating system ROS, ROS2, Apollo, C++ or Python application programming interface API.

8. The system according to claim 1, characterized in that The sensor modes of the sensor simulation module include laser radar, millimeter wave radar, red, green and blue (RGB) camera, depth camera, inertial navigation device, and global navigation satellite system (GNSS).

9. An autonomous driving simulation method, characterized in that: The method comprises: Generate a simulation environment scene based on the assets of scene elements, preset scene parameters and drawn road network map; Generate sensor data of the simulated vehicle in the simulated environment scene based on the set sensor simulation parameters; Generate traffic participants on the road network map according to preset generation rules and received generation instructions, and control the movement state of the traffic participants in the simulation environment scene; Based on the motion characteristics of the simulated vehicle, according to various set kinematic parameters, the sensor data and the motion status of the traffic participants, the simulated vehicle is controlled to run in the simulation environment scene to generate simulation results.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the autonomous driving simulation method as claimed in claim 9.

Images