Simulation test method of automatic driving algorithm, cloud platform, equipment, storage medium and program product

Abstract

The embodiment of the invention provides a simulation test method of an automatic driving algorithm, a cloud platform, equipment, a storage medium and a program product. The method comprises the following steps: acquiring multi-frame target simulation data through a data reading module, and sending multi-frame scene dynamic data to message middleware; constructing a simulation scene based on the multi-frame scene static data through a scene construction module; in the simulation scene, determining a control instruction of each frame of the simulation vehicle based on the multi-frame scene dynamic data in the message-oriented middleware through an automatic driving algorithm module, and sending the control instruction of each frame to the message-oriented middleware; determining the simulation state of each frame of the simulation vehicle in the simulation scene according to the control instruction of each frame in the message-oriented middleware through the dynamic model module, and sending the simulation state of each frame to the message-oriented middleware; and rendering the simulation scene and all data in the message-oriented middleware through a visualization module. According to the embodiment of the invention, the effectiveness of the automatic driving algorithm can be verified.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a simulation test method, cloud platform, equipment, storage medium and program product for an autonomous driving algorithm. Background Art

[0002] With the continuous development of the field of autonomous driving, more and more companies are investing more energy in the research of autonomous driving. This phenomenon is promoting the continuous optimization of autonomous driving algorithms. The optimization of autonomous driving algorithms is inseparable from a large number of tests so that the algorithm effects can cover more scenarios or perform more intelligently. The testing methods currently rely on two methods: actual vehicle testing and simulation environment testing.

[0003] However, there are still many problems in the existing technology for simulation environment testing, which makes it impossible to verify the effectiveness of the autonomous driving algorithm. Summary of the invention

[0004] The present invention provides a simulation test method, cloud platform, equipment, storage medium and program product of an autonomous driving algorithm, which can verify the effectiveness of the autonomous driving algorithm and improve the efficiency of the simulation test of the autonomous driving algorithm.

[0005] In a first aspect, an embodiment of the present disclosure provides a simulation test method for an autonomous driving algorithm, comprising: acquiring multiple frames of target simulation data through a data reading module, and sending multiple frames of scene dynamic data to a message middleware; wherein the target simulation data includes scene static data and scene dynamic data; constructing a simulation scene based on the multiple frames of scene static data through a scene construction module; in the simulation scene, determining the control instructions of each frame of a simulated vehicle based on the multiple frames of scene dynamic data in the message middleware through an autonomous driving algorithm module, and sending the control instructions of each frame to the message middleware; determining the simulation state of each frame of the simulated vehicle in the simulation scene through a dynamic model module according to the control instructions of each frame in the message middleware, and sending the simulation state of each frame to the message middleware; rendering the simulation scene and all data in the message middleware through a visualization module.

[0006] In the second aspect, the embodiment of the present disclosure also provides a simulation test cloud platform for an autonomous driving algorithm, including: a data reading module, used to obtain multi-frame target simulation data, and send multi-frame scene dynamic data to a message middleware; wherein the target simulation data includes scene static data and scene dynamic data; a scene construction module, used to construct a simulation scene based on the multi-frame scene static data; an autonomous driving algorithm module, used to determine the control instructions for each frame of a simulated vehicle in the simulation scene based on the multi-frame scene dynamic data in the message middleware, and send the control instructions for each frame to the message middleware; a dynamic model module, used to determine the simulation state of each frame of the simulated vehicle in the simulation scene according to the control instructions for each frame in the message middleware, and send the simulation state of each frame to the message middleware; a visualization module, used to render the simulation scene and all data in the message middleware.

[0007] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising: one or more processors; and a storage cloud platform for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement a simulation test method for an autonomous driving algorithm as described in an embodiment of the present disclosure.

[0008] In a fourth aspect, an embodiment of the present disclosure further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to perform a simulation test method for an autonomous driving algorithm as described in an embodiment of the present disclosure.

[0009] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, including a computer program, which, when executed by a processor, implements a simulation test method for an autonomous driving algorithm as described in an embodiment of the present disclosure.

[0010] The technical solution of the disclosed embodiment is to obtain multiple frames of target simulation data through a data reading module, and send multiple frames of scene dynamic data to the message middleware; wherein the target simulation data includes scene static data and scene dynamic data; construct a simulation scene based on the multiple frames of scene static data through a scene construction module; in the simulation scene, determine the control instructions of each frame of the simulated vehicle based on the multiple frames of scene dynamic data in the message middleware through an autonomous driving algorithm module, and send the control instructions of each frame to the message middleware; determine the simulation state of each frame of the simulated vehicle in the simulation scene through a dynamic model module according to the control instructions of each frame in the message middleware, and send the simulation state of each frame to the message middleware; render the simulation scene and all the data in the message middleware through a visualization module. The disclosed embodiment realizes the dynamic behavior simulation and real-time display of the autonomous driving vehicle in the simulation scene through the collaborative work between the data reading module, the scene construction module, the autonomous driving algorithm module, the dynamic model module and the visualization module, and can verify the effectiveness of the autonomous driving algorithm. At the same time, the modules are decoupled through the message middleware, allowing them to communicate with each other through the message middleware, so that each module can be started or stopped independently. In this way, simulation tests of different autonomous driving algorithms can be achieved by switching the autonomous driving algorithm modules without restarting all modules, thereby improving the efficiency and flexibility of the autonomous driving algorithm simulation test. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0012] Figure 1 A schematic flow chart of a simulation test method for an autonomous driving algorithm provided by an embodiment of the present invention;

[0013] Figure 2 A schematic diagram of the structure of a simulation test cloud platform for an autonomous driving algorithm provided by an embodiment of the present invention;

[0014] Figure 3 The present invention is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0015] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0016] It should be understood that the various steps recorded in the method implementation of the present disclosure may be executed in different orders and / or in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect. The term "including" and its variations used herein are open inclusions, that is, "including but not limited to". It should be noted that the concepts of "first", "second", etc. mentioned in the present disclosure are only used to distinguish different cloud platforms, modules or units, and are not used to limit the order or interdependence of the functions performed by these cloud platforms, modules or units. It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative and not restrictive. Those skilled in the art should understand that unless otherwise clearly stated in the context, it should be understood as "one or more". It is understandable that the data involved in the present technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of the relevant laws, regulations and relevant provisions.

[0017] Figure 1 The following is a flow chart of a simulation test method for an autonomous driving algorithm provided by an embodiment of the present invention. The embodiment of the present disclosure is applicable to the situation of performing simulation test on the autonomous driving algorithm. The method can be executed by a simulation test cloud platform for autonomous driving algorithm (hereinafter referred to as the simulation test cloud platform), which can be implemented in the form of software and / or hardware, and optionally, by an electronic device, which can be a mobile terminal, PC or server, etc. Figure 1 As shown, the method includes:

[0018] S110 . Acquire multiple frames of target simulation data through a data reading module, and send multiple frames of scene dynamic data to a message middleware.

[0019] Wherein, the target simulation data includes scene static data and scene dynamic data.

[0020] The scene static data may include information such as lane lines, traffic lights (position and status), static obstacles (such as the position and shape of buildings and trees, etc.), etc. The scene dynamic data may include dynamic obstacle information, such as the real-time position, speed, direction, etc. of pedestrians and vehicles.

[0021] Among them, the multi-frame target simulation data includes multi-frame scene static data and multi-frame scene dynamic data. The multi-frame scene static data can be obtained through a map. The multi-frame scene dynamic data comes from real data within a set time range recorded by an actual data acquisition vehicle. In this embodiment, there is no specific restriction on the set time range, for example, it can be several hours.

[0022] The message middleware uses the CyberRT (Robotic Technology) development framework. CyberRT provides message subscription and publishing functions, that is, the message middleware communicates messages through the publish-subscribe model. The communication process is to read and send data through memory, so the communication efficiency is very high. Channels are the basis of message middleware communication. When a module sends a data to the message middleware (for example, sending "hello" data to the " / communication" channel), other modules that subscribe to the " / communication" channel can receive the "hello" data immediately.

[0023] Optionally, obtaining multiple frames of target simulation data through a data reading module includes: obtaining multiple frames of initial simulation data by setting a data reading interface; wherein the initial simulation data is data in any data format; converting the multiple frames of initial simulation data into multiple frames of target simulation data; and the target simulation data is data in a format corresponding to the simulation test cloud platform of the autonomous driving algorithm.

[0024] The arbitrary data format may be a json data format, an xml data format, a bag data format, etc. The target simulation data may be understood as a unified data format, and this embodiment does not limit the specific format.

[0025] It should be noted that the data reading module can read data in any data format, but the user needs to provide a set data reading interface when providing data, but the initial simulation data output by the set data reading interface can meet the following requirements: the initial simulation data is in frames; each frame of initial simulation data includes the data obtained by various sensors on the actual data acquisition vehicle at a certain moment, such as data obtained by cameras, radars, chassis, etc. If there is user-sensitive data, it is represented by ciphertext. Among them, the scene dynamic data can be obtained based on the data obtained by each sensor.

[0026] In this embodiment, by setting a data reading interface to obtain multiple frames of initial simulation data, and converting the multiple frames of initial simulation data into multiple frames of target simulation data in a format corresponding to the simulation test cloud platform of the autonomous driving algorithm, the data reading module can process data in different formats and receive multiple data sources, so that the data format used in the development of the autonomous driving algorithm is decoupled from the simulation test cloud platform, thereby improving the versatility and flexibility of the simulation test cloud platform. At the same time, the new autonomous driving algorithm can be quickly connected to the simulation test cloud platform provided by the embodiment of the present invention for testing and verification without the need to redevelop the simulation test cloud platform, thereby reducing the cost of autonomous driving algorithm innovation. Since the data reading module can receive data in multiple formats, and the simulation test cloud platform provided by the embodiment of the present invention separates each module independently, that is, the autonomous driving algorithm module and the scene construction module, the simulation test cloud platform can obtain more multiple frames of initial simulation data to construct the simulation scene, and can allow different autonomous driving algorithm modules to share multiple frames of initial simulation data. Large-scale multiple frames of initial simulation data are more helpful in verifying the effectiveness of the autonomous driving algorithm and improving the efficiency of the simulation test of the autonomous driving algorithm. The simulation test cloud platform provided in the embodiment of the present invention allows the autonomous driving algorithm that the user wants to test to run in a simulation scenario constructed by data in various formats, which is equivalent to expanding the source channel of the initial simulation data, enhancing the scenario coverage of the simulation test cloud platform, and effectively improving the simulation test capabilities.

[0027] The embodiments of the present invention can receive data sources in various formats to perform simulation tests on autonomous driving algorithms, effectively solving the difficulties in reusing traditional simulation systems and avoiding the dilemma of repeated development.

[0028] S120: construct a simulation scene based on the multiple frames of scene static data through a scene construction module.

[0029] In this embodiment, through the scene construction module, multiple frames of static scene data can be converted into realistic three-dimensional simulation scenes. This process can use simulation technologies such as three-dimensional modeling and scene integration to ensure that the constructed simulation scene can accurately reflect the actual road environment and provide a rich virtual test environment for the testing of autonomous driving algorithms.

[0030] Optionally, a simulation scene is constructed based on the multi-frame scene static data through a scene construction module, including: determining the positions of multiple frames of static objects based on the multi-frame scene static data; the static objects include at least one of the following: lane lines, traffic lights, static obstacles; and determining an initial simulation state of the simulated vehicle.

[0031] For example, the positions of static objects in multiple frames of scene static data can be determined, and these static objects can include lane lines, traffic lights, static obstacles, etc. Subsequently, according to the position information of these static objects, an initial simulation state is set for the simulated vehicle, such as the starting position, direction, and speed, so as to simulate the operation of the autonomous driving vehicle in the actual road environment in subsequent simulation tests.

[0032] S130. In the simulation scenario, the control instructions for each frame of the simulated vehicle are determined based on the multi-frame scene dynamic data in the message middleware through the automatic driving algorithm module, and the control instructions for each frame are sent to the message middleware.

[0033] Among them, the autonomous driving algorithm module includes a planning module and a control module.

[0034] Specifically, the planning module can be used to obtain multiple frames of scene dynamic data from the message middleware, determine the action planning information of each frame of the simulated vehicle based on the multiple frames of scene dynamic data (one frame of scene dynamic data corresponds to one frame of action planning information), and send the action planning information of each frame to the message middleware; obtain the action planning information of each frame of the simulated vehicle in the message middleware through the control module, determine the control instructions of each frame of the simulated vehicle based on the action planning information of each frame, and send the control instructions of each frame to the message middleware. Among them, the action planning information can be understood as planning information such as acceleration, braking, and turn signal adjustment. The control instructions can be understood as control instructions that simulate the actual operation of the autonomous driving vehicle.

[0035] Optionally, the control instructions for each frame of the simulated vehicle are determined by the autonomous driving algorithm module based on the multi-frame scene dynamic data in the message middleware, including: determining the action planning information for each frame of the simulated vehicle based on the multi-frame scene dynamic data in the message middleware through the planning module, and sending the action planning information for each frame to the message middleware; determining the control instructions for each frame of the simulated vehicle based on the action planning information for each frame in the message middleware through the control module.

[0036] Among them, the planning module and the control module can be understood as the core algorithm modules of the autonomous driving algorithm, and can also be understood as user-customizable modules. Among them, the planning module and the control module each include a binary executable program corresponding to the autonomous driving algorithm, and the binary executable program is uploaded by the user through a plug-in, which can ensure the customizability of the autonomous driving algorithm. The binary executable program corresponding to the planning module and the binary executable program corresponding to the control module have different functions. The binary executable program corresponding to the planning module is used to determine the action planning information of each frame of the simulated vehicle based on multiple frames of scene dynamic data. The binary executable program corresponding to the control module is used to determine the control instructions of each frame of the simulated vehicle based on the action planning information of each frame.

[0037] For example, the user can compile multiple versions of binary executable programs in advance and switch them at any time during testing. The planning module and the control module simultaneously use a black box method (without knowing the specific code implementation of the algorithm) to call the corresponding binary executable program for simulation testing, avoiding the risk of code leakage and protecting the security of the core code.

[0038] S140, determining the simulation state of each frame of the simulated vehicle in the simulation scenario according to the control instructions of each frame in the message middleware through a dynamic model module, and sending the simulation state of each frame to the message middleware.

[0039] The simulation state may be understood as the driving state of the simulated vehicle, which may include information such as position and speed.

[0040] In this embodiment, the dynamic model module reads the control instructions of each frame of the control module from the message middleware, determines the simulation state of each frame of the simulated vehicle in the simulation scene, simulates the response of the autonomous driving vehicle to the control instructions in the actual environment, and finally controls the simulation state of the simulated vehicle in the simulation scene. Finally, the simulation state of each frame is sent to the message middleware.

[0041] In this embodiment, the dynamic model module uses a single-vehicle model, which can better simulate the driving state of the vehicle in the simulation scenario, so that the simulation state of the simulated vehicle in the simulation scenario and the driving state in the actual environment are basically matched, thereby ensuring the reference of the simulation test, effectively narrowing the distance between the simulation environment and the real environment, and making the simulation test effect more meaningful for reference.

[0042] Among them, there is at least one autonomous driving algorithm module and at least one dynamic model module; it also includes: multiple autonomous driving algorithm modules simultaneously determine the control instructions of each frame of their respective simulated vehicles based on the multi-frame scene dynamic data in the message middleware, and send the control instructions of each frame of their respective simulated vehicles to the message middleware; multiple dynamic model modules determine the simulation status of each frame of their respective simulated vehicles in the simulation scene according to the control instructions of each frame of their respective simulated vehicles in the message middleware, and send the simulation status of each frame of their respective simulated vehicles to the message middleware; there is a one-to-one correspondence between the multiple autonomous driving algorithm modules and the multiple dynamic model modules.

[0043] In this embodiment, there is a one-to-one correspondence between the autonomous driving algorithm module, the dynamic model module, and the simulated vehicle, that is, the number of autonomous driving algorithm modules, the number of dynamic model modules, and the number of simulated vehicles are the same. There can be multiple autonomous driving algorithm modules and dynamic model modules, while there can be only one data reading module, scene construction module, and visualization module.

[0044] It should be noted that the autonomous driving algorithm has uncertainty. For the same test data (equivalent to the same simulation scene), the results of running the same autonomous driving algorithm multiple times may be inconsistent. For example, when there is an obstacle ahead, 90 out of 100 autonomous driving algorithm runs may be able to bypass normally, and the other 10 may cause the simulated vehicle to collide with the obstacle (including dynamic obstacles and static obstacles). Therefore, repeated tests to verify the stability of the autonomous driving algorithm is an important part of the autonomous driving algorithm simulation test.

[0045] In this embodiment, the multi-frame target simulation data read from the disk by the data reading module can be reused, and the scene construction module can be started by using container technology (such as docker container technology, all modules in the embodiment of the present invention use container technology) to build a simulation scene. Multiple autonomous driving algorithm modules are constructed, and the autonomous driving algorithms in each autonomous driving algorithm module are the same. At the same time, multiple dynamic model modules corresponding to the multiple autonomous driving algorithm modules are constructed, thereby realizing the repeated testing of multiple autonomous driving algorithms at the same time using the same test data (equivalent to the same simulation scene).

[0046] In an embodiment of the present invention, while only one simulation scene is constructed, the message middleware can be used to send multiple frames of scene dynamic data to multiple running autonomous driving algorithm modules at the same time, thereby avoiding repeated reading of data from the disk and saving a large amount of disk reading time. At the same time, there is no need to repeatedly construct the simulation scene, thereby reducing data reading time in the algorithm repeated verification scenario, and further improving the efficiency of the simulation test.

[0047] It should be noted that since simulation scenarios can be shared, the autonomous driving algorithms in multiple autonomous driving algorithm modules can also be different, and multiple autonomous driving algorithm modules and multiple dynamic model modules can also run simultaneously to improve the efficiency of simulation testing. Testing of multiple different autonomous driving algorithms can also be achieved through switching. For example, when switching is required, the running autonomous driving algorithm module is stopped, the binary executable program corresponding to the new autonomous driving algorithm is uploaded, and the autonomous driving algorithm module is restarted without restarting the entire simulation test cloud platform.

[0048] The embodiment of the present invention is deployed based on a cloud platform and utilizes container technology to build multiple simulation test processes in parallel, greatly improving the efficiency of simulation testing.

[0049] S150: Render the simulation scenario and all data in the message middleware through a visualization module.

[0050] In this embodiment, the simulation scene is read through the visualization module and an overall three-dimensional rendering is performed. Then, all the data that needs to be rendered in the message middleware is subscribed and real-time rendering is performed. At the same time, the key indicator values ​​can also be displayed so that users can intuitively see the simulation test process and simulation test results.

[0051] Optionally, the simulation scene and all the data in the message middleware are rendered through a visualization module, including: three-dimensional rendering of the simulation scene; rendering the current frame data in the message middleware; displaying key indicator values ​​corresponding to the current frame data; the key indicators include at least one of the following: simulated vehicle position, simulated vehicle deviation angle, dynamic obstacle position and action planning information corresponding to the current frame data; if the test result corresponding to the current frame data is a passed test, the next frame data in the message middleware is used as the current frame data, and the rendering step of the current frame data in the message middleware is returned to continue rendering; if the test result corresponding to the current frame data is a failed test, the simulation test is ended.

[0052] Optionally, the test result corresponding to the current frame data is test failure, including: if after rendering the current frame data, the simulated vehicle has at least one of the following situations: collision, solid line, or running a red light, then the test result corresponding to the current frame data is test failure.

[0053] In this embodiment, the visualization module can first perform a three-dimensional rendering of the simulation scene as a whole, and then perform real-time rendering of the data that needs to be rendered for each frame in the message middleware. During the rendering process, not only can the relevant key indicator values ​​be displayed, but also the test results corresponding to the current frame data need to be judged. If the test passes, the rendering of the next frame data can be continued. If the test fails, the simulation test of the corresponding autonomous driving algorithm module is terminated. Among them, the current frame data includes all the data that needs to be rendered in the current frame, such as data related to key indicators.

[0054] Exemplarily, assuming that three frames of multi-frame data are used as an example, the first frame of data in the message middleware is rendered; the key indicator values ​​corresponding to the first frame of data are displayed; if the test result corresponding to the first frame of data is a passed test, the first frame of data in the message middleware is rendered; the key indicator values ​​corresponding to the second frame of data are displayed; if the test result corresponding to the second frame of data is a failed test, the simulation test of the corresponding autonomous driving algorithm module is terminated. For a failed test, if after rendering the current frame of data, the simulated vehicle has at least one of the following conditions: a collision, a solid line, or a red light run, then the test result corresponding to the current frame of data is a failed test. Of course, in addition to collisions, solid lines, and red light run, other violations may also be included, which will not be given as examples here.

[0055] The technical solution of the disclosed embodiment is to obtain multiple frames of target simulation data through a data reading module, and send multiple frames of scene dynamic data to the message middleware; wherein the target simulation data includes scene static data and scene dynamic data; construct a simulation scene based on the multiple frames of scene static data through a scene construction module; in the simulation scene, determine the control instructions of each frame of the simulated vehicle based on the multiple frames of scene dynamic data in the message middleware through an autonomous driving algorithm module, and send the control instructions of each frame to the message middleware; determine the simulation state of each frame of the simulated vehicle in the simulation scene through a dynamic model module according to the control instructions of each frame in the message middleware, and send the simulation state of each frame to the message middleware; render the simulation scene and all the data in the message middleware through a visualization module. The disclosed embodiment realizes the dynamic behavior simulation and real-time display of the autonomous driving vehicle in the simulation scene through the collaborative work between the data reading module, the scene construction module, the autonomous driving algorithm module, the dynamic model module and the visualization module, and can verify the effectiveness of the autonomous driving algorithm. At the same time, the modules are decoupled through the message middleware, allowing them to communicate with each other through the message middleware, so that each module can be started or stopped independently. In this way, simulation tests of different autonomous driving algorithms can be achieved by switching the autonomous driving algorithm modules without restarting all modules, thereby improving the efficiency and flexibility of the autonomous driving algorithm simulation test.

[0056] Figure 2 A schematic diagram of the structure of a simulation test cloud platform for an autonomous driving algorithm provided by an embodiment of the present invention. A data reading module is used to obtain multiple frames of target simulation data and send multiple frames of scene dynamic data to the message middleware; wherein the target simulation data includes scene static data and scene dynamic data; a scene construction module is used to construct a simulation scene based on the multiple frames of scene static data; an autonomous driving algorithm module is used to determine the control instructions of each frame of the simulated vehicle in the simulation scene based on the multiple frames of scene dynamic data in the message middleware, and send the control instructions of each frame to the message middleware; a dynamic model module is used to determine the simulation state of each frame of the simulated vehicle in the simulation scene according to the control instructions of each frame in the message middleware, and send the simulation state of each frame to the message middleware; a visualization module is used to render the simulation scene and all the data in the message middleware.

[0057] The technical solution of the disclosed embodiment is to obtain multiple frames of target simulation data through a data reading module, and send multiple frames of scene dynamic data to the message middleware; wherein the target simulation data includes scene static data and scene dynamic data; construct a simulation scene based on the multiple frames of scene static data through a scene construction module; in the simulation scene, determine the control instructions of each frame of the simulated vehicle based on the multiple frames of scene dynamic data in the message middleware through an autonomous driving algorithm module, and send the control instructions of each frame to the message middleware; determine the simulation state of each frame of the simulated vehicle in the simulation scene through a dynamic model module according to the control instructions of each frame in the message middleware, and send the simulation state of each frame to the message middleware; render the simulation scene and all the data in the message middleware through a visualization module. The disclosed embodiment realizes the dynamic behavior simulation and real-time display of the autonomous driving vehicle in the simulation scene through the collaborative work between the data reading module, the scene construction module, the autonomous driving algorithm module, the dynamic model module and the visualization module, and can verify the effectiveness of the autonomous driving algorithm. At the same time, the modules are decoupled through the message middleware, allowing them to communicate with each other through the message middleware, so that each module can be started or stopped independently. In this way, simulation tests of different autonomous driving algorithms can be achieved by switching the autonomous driving algorithm modules without restarting all modules, thereby improving the efficiency and flexibility of the autonomous driving algorithm simulation test.

[0058] Optionally, the data reading module is specifically used to: obtain multiple frames of initial simulation data by setting a data reading interface; wherein the initial simulation data is data in any data format; convert the multiple frames of initial simulation data into multiple frames of target simulation data; the target simulation data is data in a format corresponding to the simulation test cloud platform of the autonomous driving algorithm.

[0059] Optionally, the scene construction module is specifically used to: determine the positions of multiple frames of static objects based on the multiple frames of scene static data; the static objects include at least one of the following: lane lines, traffic lights, static obstacles; determine the initial simulation state of the simulated vehicle.

[0060] Among them, the autonomous driving algorithm module includes a planning module and a control module.

[0061] Optionally, the autonomous driving algorithm module is specifically used to: determine the action planning information of each frame of the simulated vehicle based on the multi-frame scene dynamic data in the message middleware through the planning module, and send the action planning information of each frame to the message middleware; wherein the planning module and the control module each include a binary executable program corresponding to the autonomous driving algorithm, and the binary executable program is uploaded by the user through a plug-in; and determine the control instructions of each frame of the simulated vehicle based on the action planning information of each frame in the message middleware through the control module.

[0062] Among them, there is at least one autonomous driving algorithm module; there is at least one dynamic model module. The autonomous driving algorithm module is also used for: multiple autonomous driving algorithm modules simultaneously determine the control instructions for each frame of the respective simulated vehicles based on the multi-frame scene dynamic data in the message middleware, and send the control instructions for each frame of the respective simulated vehicles to the message middleware; the dynamic model module is also used for: multiple dynamic model modules determine the simulation state of each frame of the respective simulated vehicles in the simulation scene according to the control instructions for each frame of the respective simulated vehicles in the message middleware, and send the simulation state of each frame of the respective simulated vehicles to the message middleware; there is a one-to-one correspondence between the multiple autonomous driving algorithm modules and the multiple dynamic model modules.

[0063] Optionally, the visualization module is specifically used to: perform three-dimensional rendering of the simulation scene; render the current frame data in the message middleware; display the key indicator values ​​corresponding to the current frame data; the key indicators include at least one of the following: simulated vehicle position, simulated vehicle deviation angle, dynamic obstacle position and action planning information corresponding to the current frame data; if the test result corresponding to the current frame data is a passed test, then the next frame data in the message middleware is used as the current frame data, and the rendering step of the current frame data in the message middleware is returned to continue rendering; if the test result corresponding to the current frame data is a failed test, then the simulation test is ended.

[0064] The visualization module is also used for: if after rendering the current frame data, the simulated vehicle has at least one of the following conditions: collision, crossing a solid line, or running a red light, then the test result corresponding to the current frame data is a test failure.

[0065] The simulation test cloud platform for the autonomous driving algorithm provided in the embodiments of the present disclosure can execute the simulation test method for the autonomous driving algorithm provided in any embodiment of the present disclosure, and has the functional modules and beneficial effects corresponding to the execution method.

[0066] Figure 3A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0067] like Figure 3 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0068] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0069] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as simulation testing of the method autonomous driving algorithm.

[0070] In some embodiments, the simulation test of the method autonomous driving algorithm may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the simulation test of the method autonomous driving algorithm described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the simulation test of the method autonomous driving algorithm in any other appropriate manner (e.g., by means of firmware).

[0071] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0072] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0073] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0074] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0075] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0076] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0077] An embodiment of the present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements a simulation testing method for an autonomous driving algorithm as provided in any embodiment of the present application.

[0078] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0079] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A simulation test method for an autonomous driving algorithm, characterized in that: include: Acquire multiple frames of target simulation data through a data reading module, and send multiple frames of scene dynamic data to the message middleware; wherein the target simulation data includes scene static data and scene dynamic data; Constructing a simulation scene based on the multiple frames of scene static data through a scene construction module; In the simulation scenario, the control instruction of each frame of the simulated vehicle is determined by the automatic driving algorithm module based on the multi-frame scene dynamic data in the message middleware, and the control instruction of each frame is sent to the message middleware; Determine the simulation state of each frame of the simulated vehicle in the simulation scenario according to the control instructions of each frame in the message middleware through the dynamic model module, and send the simulation state of each frame to the message middleware; The simulation scenario and all the data in the message middleware are rendered through a visualization module.

2. The method according to claim 1, characterized in that Obtain multi-frame target simulation data through the data reading module, including: Acquire multiple frames of initial simulation data by setting a data reading interface; wherein the initial simulation data is data in any data format; The multiple frames of initial simulation data are converted into multiple frames of target simulation data; the target simulation data is data in a format corresponding to the simulation test cloud platform of the autonomous driving algorithm.

3. The method according to claim 1, characterized in that Constructing a simulation scene based on the multi-frame scene static data through a scene construction module, including: Determine the positions of multiple frames of static objects based on the multiple frames of scene static data; the static objects include at least one of the following: lane lines, traffic lights, and static obstacles; An initial simulation state of the simulated vehicle is determined.

4. The method according to claim 1, characterized in that in, The autonomous driving algorithm module includes a planning module and a control module; Determining the control instructions of each frame of the simulated vehicle based on the multi-frame scene dynamic data in the message middleware through the automatic driving algorithm module includes: Determine the action planning information of each frame of the simulated vehicle based on the multi-frame scene dynamic data in the message middleware through the planning module, and send the action planning information of each frame to the message middleware; wherein the planning module and the control module each include a binary executable program corresponding to the automatic driving algorithm, and the binary executable program is uploaded by the user through a plug-in; The control module determines the control instructions of each frame of the simulated vehicle based on the action planning information of each frame in the message middleware.

5. The method according to claim 1, characterized in that in, The number of the autonomous driving algorithm module is at least one; the number of the dynamics model module is at least one; and further includes: Multiple autonomous driving algorithm modules simultaneously determine control instructions for each frame of their respective simulated vehicles based on multiple frames of scene dynamic data in the message middleware, and send the control instructions for each frame of their respective simulated vehicles to the message middleware; Multiple dynamic model modules determine the simulation state of each frame of each simulated vehicle in the simulation scenario according to the control instructions of each frame of each simulated vehicle in the message middleware, and send the simulation state of each frame of each simulated vehicle to the message middleware; the multiple automatic driving algorithm modules correspond one to one to the multiple dynamic model modules.

6. The method according to claim 4, characterized in that Rendering the simulation scene and all data in the message middleware through a visualization module includes: Performing three-dimensional rendering on the simulation scene; Rendering the current frame data in the message middleware; Displaying the key indicator values ​​corresponding to the current frame data; the key indicators include at least one of the following: the simulated vehicle position, the simulated vehicle deflection angle, the dynamic obstacle position and the action planning information corresponding to the current frame data; If the test result corresponding to the current frame data is that the test passes, the next frame data in the message middleware is used as the current frame data, and the rendering step of the current frame data in the message middleware is returned to continue rendering; If the test result corresponding to the current frame data is that the test fails, the simulation test ends.

7. The method according to claim 6, characterized in that The test result corresponding to the current frame data is test failure, including: If, after rendering the current frame data, the simulated vehicle has at least one of the following conditions: a collision, a solid line, or a red light run, then the test result corresponding to the current frame data is a test failure.

8. A simulation test cloud platform for autonomous driving algorithms, characterized in that: include: A data reading module is used to obtain multiple frames of target simulation data and send multiple frames of scene dynamic data to the message middleware; wherein the target simulation data includes scene static data and scene dynamic data; A scene construction module, used to construct a simulation scene based on the multi-frame scene static data; An automatic driving algorithm module, used to determine the control instructions of each frame of the simulated vehicle based on the multi-frame scene dynamic data in the message middleware in the simulation scenario, and send the control instructions of each frame to the message middleware; A dynamic model module, used for determining the simulation state of each frame of the simulated vehicle in the simulation scenario according to the control instruction of each frame in the message middleware, and sending the simulation state of each frame to the message middleware; The visualization module is used to render the simulation scene and all the data in the message middleware.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; Storage cloud platform, used to store one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the simulation testing method of the autonomous driving algorithm as described in any one of claims 1-7.

10. A storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to perform a simulation test method for an autonomous driving algorithm as described in any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the simulation test method of the autonomous driving algorithm as described in any one of claims 1 to 7.

Images