Method, device and equipment for building automatic driving simulation scene, medium and product

Abstract

The invention relates to an automatic driving simulation scene building method and device, equipment, a medium and a product, and particularly relates to the technical field of automatic driving. Comprising the steps of receiving vehicle-mounted sensor data, and performing fusion processing to obtain fusion data; generating map data according to the fusion data and the vehicle positioning information; and building an automatic driving simulation scene according to the map data so as to test in the automatic driving simulation scene. According to the method, accurate map data is generated through multi-sensor fusion, a simulation scene is built, the environmental adaptability is enhanced, the test efficiency is improved, and system integration standardization is promoted.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular, to a method, device, equipment, medium and product for building an autonomous driving simulation scenario. Background Art

[0002] With the continuous evolution of intelligent technologies, autonomous driving technology in mining areas has become a key component of intelligent mine construction, achieving remarkable results in improving production efficiency and reducing labor costs. In the research and development process of autonomous driving technology in mining areas, the construction of simulation scenarios provides an essential environment for algorithm verification. By testing the algorithm in various typical mining area scenarios constructed, the adaptability and accuracy of the algorithm to different working conditions can be effectively verified, and then the algorithm can be optimized and improved.

[0003] Currently, there are mainly two types of technical solutions for the mining area scene simulation model construction system. One is to perform element segmentation and extraction on the point cloud data collected by lidar to further determine map elements. However, in this solution, lidar data is easily affected by weather conditions, and the cost of lidar is high. In addition, the sensor solutions of different vehicle configurations are different, and the data specifications lack a unified standard, resulting in many challenges in data processing and integration. The other is to use the method of fusing and combining real-time kinematic (RTK) data, point cloud map data, and oblique image data. Although the accuracy of mining area map data is improved, due to the large volume of point cloud data and the frequent update of mining area scenes, a large amount of manpower is required for data processing, which severely restricts efficiency and increases costs. Summary of the Invention

[0004] To solve the above technical problems or at least partially solve the above technical problems, this application provides a method, device, equipment, medium and product for building an autonomous driving simulation scenario, which can solve the problems that the existing technology relying on the lidar solution is greatly affected by weather, has high costs and inconsistent sensor data specifications, while the multi-source data fusion solution faces problems such as large amounts of point cloud data, fast scene updates resulting in a large amount of manpower input. By using multi-sensor fusion to generate accurate map data and building a simulation scenario, the environmental adaptability can be enhanced, the test efficiency can be improved, and the standardization of system integration can be promoted.

[0005] To achieve the above object, the technical solutions provided by the embodiments of this application are as follows:

[0006] In a first aspect, this application provides a method for building an autonomous driving simulation scenario, including: receiving vehicle-mounted sensor data and performing fusion processing to obtain fusion data; generating map data according to the fusion data and vehicle positioning information; building an autonomous driving simulation scenario according to the map data for testing in the autonomous driving simulation scenario.

[0007] As an alternative implementation manner of an embodiment of the present application, the vehicle-mounted sensor data includes radar data, camera data, and inertial navigation data; receiving the vehicle-mounted sensor data and performing fusion processing to obtain fusion data, including: receiving the radar data, camera data, and inertial navigation data; and performing fusion processing on the radar data, camera data, and inertial navigation data based on a Kalman filter-based fusion algorithm to obtain fusion data.

[0008] As an alternative implementation manner of an embodiment of the present application, generating map data according to the fusion data and vehicle positioning information, including: adopting a positioning and mapping construction algorithm, using the fusion data and vehicle positioning information as inputs to generate an initial map; and performing format conversion on the initial map to obtain map data in a preset format.

[0009] As an alternative implementation manner of an embodiment of the present application, building an autonomous driving simulation scenario according to the map data for testing in the autonomous driving simulation scenario, including: importing the map data into an autonomous driving simulation platform for the autonomous driving simulation platform to parse the map data; setting scenario parameters for the autonomous driving simulation platform to build an autonomous driving simulation scenario in combination with the parsed map data; the scenario parameters include climate conditions, operation time periods, and traffic flow.

[0010] In a second aspect, the present application provides a device for building an autonomous driving simulation scenario, and the device includes:

[0011] A data processing module, configured to receive vehicle-mounted sensor data and perform fusion processing to obtain fusion data;

[0012] A map generation module, configured to generate map data according to the fusion data and vehicle positioning information;

[0013] A scenario simulation module, configured to build an autonomous driving simulation scenario according to the map data for testing in the autonomous driving simulation scenario.

[0014] As an alternative implementation manner of an embodiment of the present application, the vehicle-mounted sensor data includes radar data, camera data, and inertial navigation data;

[0015] The data processing module is specifically configured to: receive the radar data, camera data, and inertial navigation data;

[0016] perform fusion processing on the radar data, camera data, and inertial navigation data based on a Kalman filter-based fusion algorithm to obtain fusion data.

[0017] As an alternative implementation manner of the embodiment of the present application, the map generation module is specifically configured to: use a positioning and map construction algorithm, take the fusion data and vehicle positioning information as inputs, generate an initial map; and perform format conversion on the initial map to obtain map data in a preset format.

[0018] As an alternative implementation manner of the embodiment of the present application, the scenario simulation module is specifically configured to: import the map data into an autonomous driving simulation platform for the autonomous driving simulation platform to parse the map data; set scenario parameters for the autonomous driving simulation platform to build an autonomous driving simulation scenario in combination with the parsed map data; the scenario parameters include climate conditions, operation periods, and traffic flows.

[0019] In a third aspect, the present application provides an electronic device, including: a processor, a memory, and a computer program stored on the memory and executable on the processor, where when the computer program is executed by the processor, the method for building an autonomous driving simulation scenario as described in the first aspect or any of its alternative implementation manners is implemented.

[0020] In a fourth aspect, the present application provides a computer-readable storage medium, including: a computer program stored on the computer-readable storage medium, where when the computer program is executed by a processor, the method for building an autonomous driving simulation scenario as described in the first aspect or any of its alternative implementation manners is implemented.

[0021] In a fifth aspect, the present application provides a computer program product, including: the computer program product includes a computer program, and when the computer program runs on a computer, the computer is enabled to implement the method for building an autonomous driving simulation scenario as described in the first aspect or any of its alternative implementation manners.

[0022] The technical solution provided by the embodiment of the present application has the following advantages compared with the prior art:

[0023] Embodiments of the present disclosure provide a method, apparatus, device, medium, and product for building an autonomous driving simulation scenario. The method first receives vehicle-mounted sensor data, performs fusion processing to obtain fused data, then generates map data based on the fused data and vehicle positioning information, and further builds an autonomous driving simulation scenario based on the map data for testing in the autonomous driving simulation scenario. In this way, the present application solves the problem of difficult scenario construction in the existing mine area scenario simulation model building system due to sensor limitations, high costs, data specification differences, and complex processing, so as to meet the accurate testing requirements of autonomous driving algorithms in complex environments. The multi-sensor fused data can more comprehensively and accurately reflect the vehicle's surrounding environment information, and the generated map data is also more in line with the actual scenario, which is beneficial to enhancing the environmental adaptability of autonomous driving. Generating map data based on the fused data and vehicle positioning information and building a simulation scenario can provide a richer and more accurate testing environment for autonomous driving algorithms, thereby improving the accuracy and efficiency of testing. Fusing different sensor data, generating map data, and building a simulation scenario are beneficial to the system integration of different sensor solutions and vehicle configurations, promoting the standardized development of autonomous driving systems, and facilitating the collaborative work and data interaction between different components and systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0026] Figure 1 It is a flowchart showing a method for building an autonomous driving simulation scenario provided by an embodiment of the present application;

[0027] Figure 2 It is a schematic structural diagram of an apparatus for building an autonomous driving simulation scenario provided by an embodiment of the present application;

[0028] Figure 3 It is a schematic structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] In order to be able to more clearly understand the above objects, features, and advantages of the present application, the solutions of the present application will be further described below. It should be noted that, without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other.

[0030] In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application may also be implemented in other ways different from those described herein. Obviously, the embodiments in the specification are only a part of the embodiments of the present application, rather than all of the embodiments.

[0031] To solve some or all of the technical problems existing in the related art, embodiments of the present application provide a method, apparatus, device, medium, and product for building an autonomous driving simulation scenario, and the method.

[0032] The method for building an autonomous driving simulation scenario provided in the embodiments of the present application can be implemented through an apparatus or electronic device for building an autonomous driving simulation scenario. The electronic device includes, but is not limited to, a personal computer, a laptop computer, a tablet computer, a smart phone, etc. The operating system of the electronic device may include Android, iOS developed by Apple Inc., Windows developed by Microsoft Corporation of the United States, etc., and the embodiments of the present application do not limit this. The electronic device can run alone to implement the present application, or can be connected to a network and implement the present application through interaction with other computer devices in the network. Among them, the network where the electronic device is located includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0033] It should be noted that the protection scope of the method for building an autonomous driving simulation scenario described in the embodiments of the present application is not limited to the execution order of the steps listed in this embodiment. Any solution achieved by adding or subtracting steps of the prior art and replacing steps according to the principle of the present application is included in the protection scope of the present application.

[0034] As Figure 1 shown, Figure 1 is a schematic flowchart of a method for building an autonomous driving simulation scenario according to an embodiment of the present application. The method can be executed by an apparatus for building an autonomous driving simulation scenario, where the apparatus can be implemented by software and / or hardware and is generally integrated in an electronic device. The method mainly includes the following steps S101 to S103:

[0035] S101. Receive vehicle-mounted sensor data and perform fusion processing to obtain fusion data.

[0036] Among them, the vehicle-mounted sensor data includes radar data, camera data, inertial navigation data, etc. The radar data includes, but is not limited to, the point cloud data of lidar and the speed and distance data of millimeter-wave radar. The camera data includes the target object information recognized by the camera image, such as traffic signs, vehicles, pedestrians, etc. The inertial navigation data includes the acceleration and angular velocity of the vehicle.

[0037] Sensors include radar, cameras, inertial navigation, etc. Radar includes lidar and millimeter-wave radar. Lidar is used to obtain high-precision three-dimensional point cloud data of the surrounding environment, and can accurately measure the distance and azimuth of target objects. Millimeter-wave radar can monitor the speed and relative distance of target objects in real time and has good performance under adverse weather conditions. Cameras provide rich visual image information for identifying traffic signs, lane lines, and other road participants. Inertial navigation sensors are the core components of an Inertial Navigation System (INS), mainly used to measure the motion state of an object, including physical quantities such as acceleration and angular velocity, and then calculate information such as the position, speed, and attitude of the object.

[0038] Data collected by lidar and millimeter-wave radar are transmitted to the vehicle's Central Processing Unit (CPU) via a high-speed Controller Area Network Bus (CAN). Image data captured by cameras first pass through an image preprocessing module for denoising, enhancement, etc., and then are transmitted to the CPU through an Ethernet interface.

[0039] In some embodiments, when performing step S101, a fusion algorithm based on Kalman filtering is used to fuse radar data, camera data, and inertial navigation data to obtain fused data.

[0040] Kalman filtering is an optimal estimation method based on the state space model of a linear system. Its core idea is to use the state equation and observation equation of the system, and through two steps of prediction and update, recursively estimate the state of the system to obtain the optimal estimation result. In multi-source data fusion, the data obtained from different sensors are used as observation values, and the system state to be estimated (such as the position, speed, etc. of the target) is used as the state variable. These multi-source data are fused through the Kalman filtering algorithm to obtain a more accurate and reliable estimation of the system state.

[0041] Specifically, radar data, camera data, and inertial navigation data are respectively input into the Kalman filter as different observation values. According to the measurement accuracy and noise characteristics of each sensor, corresponding weights are assigned to each observation value. Then, these data are fused through the Kalman filtering algorithm to obtain fused data, which combines the advantages of each sensor and can more accurately describe the state of the vehicle's surrounding environment.

[0042] The sensor may further include vehicle state sensors such as a Global Positioning System (GPS), an ultrasonic sensor, a tire pressure sensor, and a temperature sensor. The GPS obtains the longitude and latitude coordinates of the vehicle in real time. The ultrasonic sensor detects obstacles in the vicinity of the vehicle in real time and transmits the detection result to the microcontroller unit (MCU) of the vehicle in the form of a digital signal.

[0043] In some embodiments, when performing step S101, a fusion algorithm based on a Bayesian network is adopted to fuse the radar data and the camera data according to the probability distributions of the radar data and the camera data to obtain environmental perception information; then the environmental perception information is fused with the inertial navigation data again, and an extended Kalman filter algorithm is used to combine with the motion model of the vehicle to obtain fused data. This data not only contains detailed information about the environment around the vehicle but also fuses the position and attitude information of the vehicle itself.

[0044] In some embodiments, vehicle-mounted sensor data is received. First, the point cloud data collected by the lidar is subjected to preliminary point cloud segmentation and target extraction to identify possible obstacles and road boundaries; the camera data is subjected to target detection and recognition through a deep learning algorithm; the millimeter-wave radar data is subjected to target tracking and speed estimation; and the vehicle state sensor data is filtered and anomaly detected. Then, a fusion algorithm based on data association is adopted to associate and fuse the above processing results to obtain fused data. For example, the position of the obstacle identified by the lidar is matched and fused with the category information of the same target object identified by the camera, and at the same time, the speed information of the millimeter-wave radar is combined to obtain a more accurate description of the state of the target object.

[0045] In the above embodiments, by receiving and fusing various vehicle-mounted sensor data, the problem of being greatly affected by weather caused by relying solely on lidar is avoided. Different sensors have their own advantages in different environments, and the fused data can make up for the deficiencies of a single sensor, improving the stability and comprehensiveness of data acquisition. Moreover, by performing fusion processing on the sensor data of different vehicle configurations, the problem of differences in sensor data specifications is solved, and unified data processing is achieved. In addition, the excessive reliance on high-cost lidar is reduced, and the overall hardware cost of the sensors is lowered by using multi-sensor fusion.

[0046] S102. Generate map data according to the fused data and the vehicle positioning information.

[0047] Among them, the vehicle positioning information is obtained by using GPS and an inertial measurement unit (IMU), and includes the longitude and latitude, altitude, and heading angle, pitch angle, and roll angle of the vehicle, etc.

[0048] When performing step S102, a localization and mapping algorithm is adopted. The fused data and vehicle localization information are used as inputs to generate an initial map. The format of the initial map is converted to obtain map data in a preset format.

[0049] Among them, the Simultaneous Localization and Mapping (SLAM) algorithm aims to enable a robot or other mobile device to construct an environmental map in real time while determining its own position in the map when moving in an unknown environment. Its core goal is to solve the two interdependent problems of robot localization and map construction, enabling the robot to autonomously navigate and complete tasks without prior map information.

[0050] The preset format can be the OpenDRIVE format. The map in the OpenDRIVE format is an XML format map used to describe the road network, including header information, road geometry information, lane information, traffic control information, and intersection descriptions. The header information contains metadata such as version information, date, and geographical location, providing basic identification and positioning information for the map data. The road geometry information uses <planview>The element describes complex geometric shapes, such as straight lines, arcs, spirals, etc., and precisely represents the road alignment and shape through a series of coordinate points and geometric parameters. The lane information details the width, type, connection relationship, etc. of the lanes. Multiple lanes can be set for each road, and the number and width of lanes in different areas can be changed by setting different lane segments. The traffic control information mainly describes static traffic objects. Traffic signs can be passed through <objects>described by elements, the marking line can be passed through <lateralmarking>Element description, providing traffic rules and indication information for an autonomous vehicle. The intersection description uses <junction>The element describes the intersection and, via <connection>And <lanelink>The element defines the connection relationship between the road and the lane, clearly expressing the connection situation of the road and the lane at the intersection.

[0051] Specifically, a graph-optimized SLAM algorithm is adopted, taking the fused data and vehicle positioning information as input. In the algorithm initialization stage, an initial map framework is created based on the initial fused data and vehicle position, which contains some key nodes (representing different poses of the vehicle) and edges (representing the constraint relationships between nodes). As the vehicle moves, new fused data and positioning information are continuously obtained. The lidar point cloud data is used to construct an accurate geometric map. Through the scan matching algorithm, the new point cloud data is matched with the existing map to determine the relative movement of the vehicle, thereby updating the node positions and edge constraints in the map. The camera image data is used for loop detection. By identifying similar visual scenes, it is detected whether the vehicle has returned to an area visited before. Once a loop is detected, the nodes and edges in the map are adjusted through an optimization algorithm to eliminate the accumulated error and improve the consistency and accuracy of the map. During the whole process, the vehicle positioning information is used to impose global positioning constraints on the map to ensure the accuracy of the position and pose of the map in the global coordinate system.

[0052] After constructing the map based on the SLAM algorithm, map format conversion is carried out. First, the road geometric information in the map (such as the road centerline, lane boundaries, etc.) is extracted. By analyzing the relationship between the vehicle driving trajectory and the surrounding environment in the map, the orientation and shape of the road are determined. For example, the continuous vehicle pose points are connected to form a preliminary outline of the road centerline, and then combined with the measurement information of the road boundary in the lidar point cloud data to accurately determine the lane boundaries. For traffic sign and marking information, according to the camera image recognition results and the corresponding relationship with the position in the map, the corresponding elements are added to the OpenDRIVE format map. For example, the recognized traffic signs (such as speed limit signs, no-entry signs, etc.) and markings (such as lane lines, zebra crossings, etc.) are described and recorded in the format specified by OpenDRIVE according to their actual positions and directions in the map. Finally, all the extracted and converted information is organized and stored according to the specifications of the OpenDRIVE format to generate an OpenDRIVE format map data file that meets the requirements. This file contains complete road networks, traffic facilities, and vehicle driving trajectories and other information, which can be used in application scenarios such as autonomous driving simulation and testing.

[0053] Exemplarily, during the vehicle's driving process, the incremental SLAM algorithm based on factor graph optimization receives real-time fusion data and vehicle positioning information. According to the new lidar point cloud data, it quickly constructs a local map and determines the vehicle's real-time pose by matching it with the existing global map. It uses camera image data for real-time object recognition and scene classification to add semantic information to the map and assist in loop closure detection to improve the accuracy of map construction. Millimeter-wave radar data is used to monitor the motion states of dynamic objects (such as other vehicles and pedestrians) in real time and update them in the map. The nodes and edges in the factor graph are continuously updated. The nodes represent the vehicle's pose and feature points in the map, and the edges represent the constraint relationships between the nodes (such as distance constraints measured by lidar, constraints of visual feature matching, etc.). By optimizing the factor graph in real time, the positions and postures of the nodes are adjusted, thereby realizing the real-time update and optimization of the map. The map information constructed and optimized by the SLAM algorithm is obtained in real time. For road information, according to the vehicle's driving trajectory and the geometric features of the surrounding environment, parameters such as the center line, lane width, and slope of the road are extracted. Using lidar point cloud data and visual recognition results, the positions of the road boundaries and roadside facilities (such as guardrails, streetlights, etc.) are determined. For traffic elements, such as traffic signs and markings, according to the real-time recognition results of camera images, their positions, types, and directions are accurately marked in the map. These information are organized and encoded according to the requirements of the OpenDRIVE format. For example, the road information is in <road>Describe the label, and the traffic sign and marking information are respectively in <objects>and <lateralmarking>Record using tags such as . Generate OpenDRIVE format map data in real time, and continuously update and improve it. As the vehicle moves, new map information is continuously added, and the existing map information is corrected and optimized according to the latest sensor data to ensure that the generated OpenDRIVE map data can reflect the real environment around the vehicle in real time and accurately, providing reliable map support for the autonomous driving system.

[0054] In some embodiments, obtain the historical vehicle-mounted sensor data stored offline from the local database, which can be used as historical data, and the vehicle-mounted sensors received in step S101 are real-time data. When performing step S102, use a hybrid data-driven SLAM algorithm to fuse the historical data and the real-time data. First, construct an initial map prior model using the historical data. By analyzing and processing a large amount of historical lidar point cloud data, extract common road features and environmental models to provide a basic framework for real-time SLAM. During the vehicle's driving, the real-time sensor data is matched and fused with the historical data. For example, through a feature matching algorithm, compare the features in the real-time lidar point cloud data with the features in the historical data to determine the approximate position of the vehicle in the historical map. Then, combine the real-time GPS and IMU positioning information to accurately estimate the pose of the vehicle. Use the camera image data to further verify and optimize the matching result, and at the same time perform loop detection and map update. Continuously correct and improve the historical map according to the real-time data, and adjust the nodes and edges in the map through an optimization algorithm to make the map better adapt to the changes in the real-time environment. Generate OpenDRIVE format map data according to the fused SLAM map.

[0055] S103. Build an autonomous driving simulation scenario based on the map data for testing in the autonomous driving simulation scenario.

[0056] In some embodiments, when performing step S103, import the map data into the autonomous driving simulation platform for the autonomous driving simulation platform to parse the map data; set the scenario parameters for the autonomous driving simulation platform to build an autonomous driving simulation scenario in combination with the parsed map data; the scenario parameters include climate conditions, operation periods, and traffic flows.

[0057] The autonomous driving simulation platform can be Carla (Center for Autonomous Research in a Learning Automata, CARLA). This platform has high customizability and can be compatible with multiple map data formats.

[0058] Specifically, import the previously generated OpenDRIVE format map data into the CARLA platform. During the import process, the platform will automatically parse the map data to identify elements such as road networks, lane information, traffic signs, and markings. Set the traffic flow according to actual needs. Through the traffic generation tool of the platform, randomly generate different types of vehicles in the road network to simulate the traffic flow and vehicle driving behavior in a real traffic scenario. At the same time, adjust the climate conditions and operation periods, such as setting different weather conditions (sunny, rainy, foggy), lighting conditions (day, night, dusk), and road surface conditions (dry, wet, icy), etc., to test the performance of the autonomous driving system in different environments.

[0059] In the established simulation scenario, start the test of the autonomous driving system. By setting different test scenarios, such as going straight, turning, overtaking, avoiding pedestrians, etc., observe the driving situation of the autonomous driving vehicle. During the test, use the data collection function of the platform to record information such as the driving trajectory, sensor data, and decision output of the vehicle for subsequent analysis and evaluation of the test results.

[0060] In some other embodiments, game engines such as Unity or Unreal Engine can be used to build an autonomous driving simulation scenario to utilize the rich resource libraries of the game engines and add realistic environmental elements such as buildings, trees, and grasslands to make the simulation scenario closer to the real world. First, convert the OpenDRIVE map data into a format that the game engine can recognize. Then, integrate the developed autonomous driving algorithm into the game engine. Next, write scripts to implement the interaction between the algorithm and the vehicle model so that the vehicle can drive in the scenario according to the decisions of the algorithm. Subsequently, write scripts in the game engine to implement traffic rules for autonomous driving tests in the constructed simulation scenario.

[0061] When converting the OpenDRIVE map data into a format that the game engine can recognize, write custom scripts to convert information such as roads and lanes in the map data into terrains and models in the game engine. During the test, use the debugging tools and logging functions of the game engine to collect the running data of the vehicle and the decision-making information of the algorithm to evaluate the performance of the autonomous driving system, such as whether the detection algorithm can accurately identify traffic signs and obstacles and whether it can drive safely according to traffic rules.

[0062] The above embodiments generate map data through an efficient data processing process and build a simulation scenario, reducing the labor input cost and solving the problem of high labor cost caused by complex data processing in the prior art. Testing the autonomous driving system in the built simulation scenario can make it better adapt to complex and changeable environments, including different weather conditions, road conditions, etc., and improve the reliability and stability of the autonomous driving system in various environments. Moreover, by generating OpenDRIVE format map data through the SLAM algorithm and then building a simulation scenario, the data processing process is simplified, the data processing efficiency is improved, and the problems of complex data processing and slow scenario update in traditional methods are overcome.

[0063] In summary, the present application provides a method for building an autonomous driving simulation scenario. The method first receives vehicle-mounted sensor data and performs fusion processing to obtain fusion data, and then generates map data according to the fusion data and vehicle positioning information; furthermore, an autonomous driving simulation scenario is built according to the map data for testing in the autonomous driving simulation scenario. In this way, the present application solves the problem of difficult scenario construction in the existing mine area scene simulation model building system due to sensor limitations, high costs, data specification differences, and complex processing, so as to meet the accurate testing requirements of autonomous driving algorithms in complex environments. The multi-sensor fusion data can more comprehensively and accurately reflect the vehicle surrounding environment information, and the generated map data is also more in line with the actual scenario, which is beneficial to enhancing the environmental adaptability of autonomous driving. Generating map data and building a simulation scenario based on the fusion data and vehicle positioning information can provide a richer and more accurate test environment for the autonomous driving algorithm, thereby improving the accuracy and efficiency of testing. Fusing different sensor data, generating map data and building a simulation scenario is beneficial to the system integration of different sensor solutions and vehicle configurations, promotes the standardized development of the autonomous driving system, and facilitates the collaborative work and data interaction between different components and systems.

[0064] As Figure 2 shown, Figure 2 FIG. 10 is a schematic structural diagram of a device for building an autonomous driving simulation scenario provided by an embodiment of the present application. The device includes:

[0065] A data processing module 201, configured to receive vehicle-mounted sensor data and perform fusion processing to obtain fusion data;

[0066] A map generation module 202, configured to generate map data according to the fusion data and vehicle positioning information;

[0067] A scenario simulation module 203, configured to build an autonomous driving simulation scenario according to the map data for testing in the autonomous driving simulation scenario.

[0068] As an optional implementation manner of an embodiment of the present application, the vehicle-mounted sensor data includes radar data, camera data, and inertial navigation data;

[0069] The data processing module 201 is specifically configured to: receive radar data, camera data, and inertial navigation data; perform fusion processing on the radar data, camera data, and inertial navigation data based on a Kalman filter-based fusion algorithm to obtain fused data.

[0070] As an optional implementation manner of the embodiment of the present application, the map generation module 202 is specifically configured to: adopt a positioning and mapping algorithm, use the fused data and vehicle positioning information as inputs to generate an initial map; perform format conversion on the initial map to obtain map data in a preset format.

[0071] As an optional implementation manner of the embodiment of the present application, the scenario simulation module 203 is specifically configured to: import the map data into an autonomous driving simulation platform for the autonomous driving simulation platform to parse the map data; set scenario parameters for the autonomous driving simulation platform to build an autonomous driving simulation scenario in combination with the parsed map data; the scenario parameters include climate conditions, operation periods, and traffic flow.

[0072] As an optional implementation manner of the embodiment of the present application, the device for building an autonomous driving simulation scenario further includes a communication module, which is used to support the UDP / CAN communication protocol and can perform efficient data transmission and interaction with other systems of the vehicle. At the same time, a wireless communication module is equipped to transmit data to a remote server for analysis and processing.

[0073] As an optional implementation manner of the embodiment of the present application, the device for building an autonomous driving simulation scenario further includes a data storage and playback module, which is used to perform offline storage of multi-sensor data using a high-speed solid-state drive for subsequent analysis and processing. A storage management system is designed to store data classified, which can be used to reproduce the test situation under a specific scenario and facilitate subsequent data playback and analysis.

[0074] For the specific limitations on the device for building an autonomous driving simulation scenario, reference can be made to the limitations on the method for building an autonomous driving simulation scenario in the above text, which will not be elaborated here. Each module in the above device for building an autonomous driving simulation scenario can be implemented in whole or in part through software, hardware, and their combination. The above modules can be embedded in or independent of the processor in a computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so as to facilitate the processor to call and execute the operations corresponding to the above respective modules.

[0075] In one embodiment, the present application provides an electronic device, which can be a terminal, and its internal structure diagram can be as Figure 3 As shown. The electronic device includes a processor, a memory, a communication interface, a display screen, and an input device connected via a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be achieved through WIFI, carrier networks, near-field communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a method for detecting lags. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads provided on the housing of the electronic device, or an external keyboard, touchpad, or mouse, etc.

[0076] Those skilled in the art can understand that Figure 3 the structure shown in is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the electronic device to which the solution of this application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0077] In one embodiment, the device for building an autonomous driving simulation scenario provided by this application can be implemented in the form of a computer program, and the computer program can run on an electronic device such as Figure 3 shown. Each program module that makes up the device for building an autonomous driving simulation scenario can be stored in the memory of the electronic device. For example, Figure 2 the data processing module 201, the map generation module 202, and the scenario simulation module 203 shown. The computer program composed of each program module enables the processor to execute the steps in the method for building an autonomous driving simulation scenario in each embodiment of this application described in this specification.

[0078] For example, Figure 3 the electronic device shown can execute receiving vehicle-mounted sensor data through the data processing module 201 in the device for building an autonomous driving simulation scenario shown in Figure 2 and performing fusion processing to obtain fusion data; the electronic device can execute generating map data according to the fusion data and vehicle positioning information through the map generation module 202; the electronic device can execute building an autonomous driving simulation scenario according to the map data through the scenario simulation module 203 for testing in the autonomous driving simulation scenario.

[0079] In one embodiment, the present application provides an electronic device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0080] Receive vehicle sensor data and perform fusion processing to obtain fusion data; generate map data based on the fusion data and vehicle positioning information; build an autonomous driving simulation scenario according to the map data for testing in the autonomous driving simulation scenario.

[0081] In one embodiment, when the processor executes the computer program, the following steps are further implemented: The vehicle sensor data includes radar data, camera data, and inertial navigation data; receiving vehicle sensor data and performing fusion processing to obtain fusion data includes: receiving radar data, camera data, and inertial navigation data; performing fusion processing on the radar data, camera data, and inertial navigation data based on a Kalman filter-based fusion algorithm to obtain fusion data.

[0082] In one embodiment, when the processor executes the computer program, the following steps are further implemented: generating map data based on the fusion data and vehicle positioning information includes: using a positioning and mapping construction algorithm, taking the fusion data and vehicle positioning information as inputs to generate an initial map; performing format conversion on the initial map to obtain map data in a preset format.

[0083] In one embodiment, when the processor executes the computer program, the following steps are further implemented: building an autonomous driving simulation scenario according to the map data for testing in the autonomous driving simulation scenario includes: importing the map data into an autonomous driving simulation platform for the autonomous driving simulation platform to parse the map data; setting scenario parameters for the autonomous driving simulation platform to build an autonomous driving simulation scenario in combination with the parsed map data; the scenario parameters include climate conditions, operation time periods, and traffic flow.

[0084] When the processor in the electronic device provided by this application executes the computer program, it first receives vehicle-mounted sensor data, performs fusion processing to obtain fusion data, then generates map data based on the fusion data and vehicle positioning information, and further constructs an autonomous driving simulation scenario based on the map data for testing in the autonomous driving simulation scenario. In this way, this application solves the problem of difficult scenario construction in the existing mine area scenario simulation model construction system due to sensor limitations, high costs, data specification differences, and complex processing, so as to meet the accurate testing requirements of autonomous driving algorithms in complex environments. The multi-sensor fusion data can more comprehensively and accurately reflect the vehicle's surrounding environment information, and the generated map data is also more in line with the actual scenario, which is beneficial to enhancing the environmental adaptability of autonomous driving. Generating map data based on the fusion data and vehicle positioning information and constructing a simulation scenario can provide a richer and more accurate testing environment for the autonomous driving algorithm, thereby improving the accuracy and efficiency of testing. Fusing different sensor data, generating map data, and constructing a simulation scenario are beneficial to the system integration of different sensor solutions and vehicle configurations, promoting the standardized development of the autonomous driving system, and facilitating the collaborative work and data interaction between different components and systems.

[0085] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a computer, the following steps are implemented:

[0086] Receive vehicle-mounted sensor data, perform fusion processing to obtain fusion data; generate map data based on the fusion data and vehicle positioning information; construct an autonomous driving simulation scenario based on the map data for testing in the autonomous driving simulation scenario.

[0087] In one embodiment, when the computer program is executed by a computer, the following steps are also implemented: The vehicle-mounted sensor data includes radar data, camera data, and inertial navigation data; receiving the vehicle-mounted sensor data and performing fusion processing to obtain fusion data includes: receiving radar data, camera data, and inertial navigation data; performing fusion processing on the radar data, camera data, and inertial navigation data based on the Kalman filter-based fusion algorithm to obtain fusion data.

[0088] In one embodiment, when the computer program is executed by a computer, the following steps are also implemented: Generating map data based on the fusion data and vehicle positioning information includes: using a positioning and mapping algorithm, taking the fusion data and vehicle positioning information as inputs to generate an initial map; performing format conversion on the initial map to obtain map data in a preset format.

[0089] In one embodiment, when the computer program is executed by a computer, the following steps are further implemented: building an autonomous driving simulation scenario based on map data for testing in the autonomous driving simulation scenario, including: importing the map data into an autonomous driving simulation platform for the autonomous driving simulation platform to parse the map data; setting scenario parameters for the autonomous driving simulation platform to build an autonomous driving simulation scenario in combination with the parsed map data; the scenario parameters including climate conditions, operation time periods, and traffic flow.

[0090] When the computer program in the computer-readable storage medium provided by this application is executed by a computer, it first receives vehicle-mounted sensor data and performs fusion processing to obtain fusion data, and then generates map data based on the fusion data and vehicle positioning information; furthermore, it builds an autonomous driving simulation scenario based on the map data for testing in the autonomous driving simulation scenario. In this way, this application solves the problem of difficult scenario construction in the existing mine area scenario simulation model building system due to sensor limitations, high costs, data specification differences, and complex processing, so as to meet the needs of accurate testing of autonomous driving algorithms in complex environments. The multi-sensor fusion data can more comprehensively and accurately reflect the vehicle's surrounding environment information, and the generated map data is also more in line with the actual scenario, which is beneficial to enhancing the environmental adaptability of autonomous driving. Generating map data based on the fusion data and vehicle positioning information and building a simulation scenario can provide a richer and more accurate testing environment for autonomous driving algorithms, thereby improving the accuracy and efficiency of testing. Fusing different sensor data, generating map data, and building a simulation scenario are beneficial to the system integration of different sensor solutions and vehicle-mounted configurations, promoting the standardized development of autonomous driving systems, and facilitating the collaborative work and data interaction between different components and systems.

[0091] Those skilled in the art should understand that the embodiments of this application can be provided as a method, a system, or a computer program product. Therefore, this application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0092] In several embodiments provided by the present application, it should be understood that the disclosed apparatus and method can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of apparatuses, methods, and computer program products according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0093] In the present application, the processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0094] In the present application, the memory may include non-permanent memory in a computer-readable medium, random access memory (RAM), and / or non-volatile memory in the form of, for example, read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0095] In this application, computer-readable media include both permanent and non-permanent, removable and non-removable storage media. The storage media can implement information storage by any method or technology, and the information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information accessible by a computing device. As defined herein, computer-readable media do not include transitory media such as modulated data signals and carrier waves.

[0096] It should be noted that in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the element.

[0097] The above are only specific embodiments of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application will not be limited to these embodiments herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.< / lateralmarking> < / objects> < / road> < / lanelink> < / connection> < / junction> < / lateralmarking> < / objects> < / planview>

Claims

1. A method for building an autonomous driving simulation scene, characterized in that: include: Receive vehicle-mounted sensor data and perform fusion processing to obtain fused data; Generate map data based on the fused data and vehicle positioning information; An autonomous driving simulation scenario is constructed according to the map data to perform testing in the autonomous driving simulation scenario.

2. The method according to claim 1, characterized in that The vehicle-mounted sensor data includes radar data, camera data and inertial navigation data; The receiving of the vehicle-mounted sensor data and performing fusion processing to obtain fused data includes: Receiving the radar data, the camera data, and the inertial navigation data; The radar data, the camera data and the inertial navigation data are fused using a fusion algorithm based on Kalman filtering to obtain the fused data.

3. The method according to claim 1, characterized in that The generating of map data according to the fused data and the vehicle positioning information comprises: Using a positioning and mapping algorithm, the fused data and the vehicle positioning information are used as input to generate an initial map; The format of the initial map is converted to obtain the map data in a preset format.

4. The method according to claim 1, characterized in that: Building an autonomous driving simulation scenario according to the map data to perform testing in the autonomous driving simulation scenario, including: Importing the map data into an autonomous driving simulation platform so that the map data is parsed by the autonomous driving simulation platform; The scene parameters are set so that the autonomous driving simulation platform can build the autonomous driving simulation scene in combination with the parsed map data; the scene parameters include climate conditions, operating hours and traffic flow.

5. A device for building an autonomous driving simulation scene, characterized in that: include: A data processing module is used to receive the vehicle-mounted sensor data and perform fusion processing to obtain fused data; A map generation module, used to generate map data according to the fused data and vehicle positioning information; The scenario simulation module is used to build an autonomous driving simulation scenario based on the map data so as to perform testing in the autonomous driving simulation scenario.

6. The device according to claim 5, characterized in that The vehicle-mounted sensor data includes radar data, camera data and inertial navigation data; The data processing module is specifically used to: receive the radar data, the camera data and the inertial navigation data; The radar data, the camera data and the inertial navigation data are fused using a fusion algorithm based on Kalman filtering to obtain the fused data.

7. The device according to claim 5, characterized in that The map generation module is specifically used for: Using a positioning and mapping algorithm, the fused data and the vehicle positioning information are used as input to generate an initial map; The format of the initial map is converted to obtain the map data in a preset format.

8. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements a method for constructing an autonomous driving simulation scenario as described in any one of claims 1 to 4.

9. A computer-readable storage medium, characterized in that: include: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for building an autonomous driving simulation scene according to any one of claims 1 to 4 is implemented.

10. A computer program product, characterized in that include: The computer program product includes a computer program, and when the computer program is run on a computer, the computer implements the method for building an autonomous driving simulation scenario as described in any one of claims 1 to 4.

Images