A simulation test method, device, equipment and medium

Abstract

The present invention relates to the field of automobile intelligent driving technology, and specifically to a simulation test method, device, equipment and medium, the method comprising: obtaining vehicle surrounding traffic flow data collected by a real vehicle; importing the vehicle surrounding traffic flow data into simulation software integrated with an autonomous driving algorithm to generate a simulation scene; using the simulation scene to obtain a first simulation result; using reinforcement learning based on the first simulation result to determine a generalization direction; using the generalization direction to change the vehicle surrounding traffic flow data so as to import it into the simulation software integrated with the autonomous driving algorithm to generate a new simulation scene and then perform simulation testing to obtain a second simulation result; using the second simulation result as the first simulation result and returning to the first simulation result based on which the generalization direction is determined using reinforcement learning, and executing the test in a loop until an end condition is met. The present invention provides a method for ensuring that the complexity of the simulation scene test of the autonomous driving algorithm is improved and the intelligence of the autonomous driving algorithm is improved.

Description

Technical Field

[0001] The present invention relates to the field of automobile intelligent driving technology, and in particular to a simulation test method, device, equipment and medium. Background Art

[0002] At present, autonomous driving is booming, and there are more and more corresponding functional modules. Driven by technologies such as automated driving and artificial intelligence, the reform of the automobile industry has become an inevitable trend, and the integration of information communication and automobile industries has become an inevitable move. As an important product that reflects the country's industrial strength, autonomous driving vehicles are a new representative of the potential growth of the national economy and a manifestation of the integration of the country's industrial manufacturing field with new technologies. At present, the development space of intelligent autonomous driving vehicles is getting bigger and bigger, and with the penetration of such vehicles, the proportion of unmanned vehicles in the future automobile market will continue to increase. It is currently a critical stage for the development of the domestic autonomous driving market. Especially after breaking through the technical bottleneck in this field, it will inevitably replace more closed road network areas, unmanned ports and cargo yards will also become the mainstream trend, and scenic area shuttle vehicles will be unmanned as the final form, bringing tourists a more novel experience. In urban public construction, in the early stage, autonomous driving buses, taxis, and private cars will mix on the road, and in the later stage, autonomous driving vehicles may have exclusive road rights, and non-autonomous driving vehicles will not be allowed to drive on the road.

[0003] During the development of autonomous driving, the functions of each module need to be continuously tested and debugged. Actual vehicle testing is the closest to the actual usage of the vehicle, but it will face the problems of high cost, long cycle, affecting the efficiency of development and testing, and it is impossible to reproduce the complex conditions of actual roads. Summary of the invention

[0004] In view of the shortcomings of the prior art mentioned above, the present invention provides a simulation test method, device, equipment and medium, which can ensure the purpose of improving the complexity of the simulation scenario test of the autonomous driving algorithm and reproducing the complex road conditions of the actual road to the greatest extent on the basis of solving the high cost, long cycle and the impact on the development and testing efficiency. At the same time, it can not only utilize the existing data resources, but also save development costs and shorten the development cycle.

[0005] To achieve the above-mentioned purpose and other related purposes, the present invention provides a simulation test method, comprising the following steps: obtaining vehicle surrounding traffic flow data collected by a real vehicle; importing the vehicle surrounding traffic flow data into simulation software integrated with an autonomous driving algorithm to generate a simulation scene; using the simulation scene to simulate and test the autonomous driving algorithm to obtain a first simulation result; using reinforcement learning based on the first simulation result to determine a generalization direction; using the generalization direction to change the vehicle surrounding traffic flow data; importing the changed vehicle surrounding traffic flow data into the simulation software integrated with the autonomous driving algorithm to generate a new simulation scene; using the new simulation scene to simulate and test the autonomous driving algorithm to obtain a second simulation result; using the second simulation result as the first simulation result and returning to using reinforcement learning based on the first simulation result to determine the generalization direction, and looping through subsequent steps until an end condition is met.

[0006] In an optional embodiment of the present invention, the step of importing the traffic flow data around the vehicle into the simulation software integrated with the autonomous driving algorithm to generate a simulation scene also includes: generating a scene file based on the traffic flow data around the vehicle; and importing the scene file into the simulation software integrated with the autonomous driving algorithm to generate a simulation scene.

[0007] In an optional embodiment of the present invention, before the step of using reinforcement learning to determine the generalization direction based on the first simulation result, the step also includes: judging whether the autonomous driving algorithm has a collision or congestion based on the first simulation result; and based on the judgment result, inputting the first simulation result into the reinforcement learning as a penalty or reward to determine the generalization direction.

[0008] In an optional embodiment of the present invention, the first simulation result is input into the reinforcement learning as a penalty or reward to determine the generalization direction, including: when the autonomous driving algorithm collides or is congested, the first simulation result is input into the reinforcement learning as a reward to determine the generalization direction; when the autonomous driving algorithm does not collide or is congested, the first simulation result is input into the reinforcement learning as a penalty to determine the generalization direction.

[0009] In an optional embodiment of the present invention, changing the traffic flow data around the vehicle based on the generalization direction includes: based on the generalization direction, using generalization software to change the traffic flow data around the vehicle; wherein, in the process of changing the traffic flow data around the vehicle using the generalization software, each time the value of one parameter of the traffic flow data around the vehicle is changed, the values ​​of other parameters remain unchanged. In an optional embodiment of the present invention, the step of changing the traffic flow data around the vehicle based on the generalization direction and importing it into the simulation software integrated with the autonomous driving algorithm to generate a new simulation scene includes: changing the traffic flow data around the vehicle based on the generalization direction to generate a new scene file; importing the new scene file into the simulation software integrated with the autonomous driving algorithm to generate the new simulation scene.

[0010] In an optional embodiment of the present invention, the traffic flow data around the vehicle includes: changes in speed of surrounding vehicles, lane changing methods of surrounding vehicles, walking trajectories of surrounding vehicles and people, and traffic light change logic.

[0011] In an optional embodiment of the present invention, the generated simulation scenario includes: attributes of other targets, traffic environment factors, and types of traffic participants.

[0012] To achieve the above-mentioned object and other related objects, the present invention further provides an electronic device, the electronic device comprising:

[0013] one or more processors;

[0014] A storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the electronic device implements the simulation test method of reinforcement learning traffic flow.

[0015] To achieve the above-mentioned purpose and other related purposes, the present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor of a computer, the computer executes the simulation test method of reinforcement learning traffic flow.

[0016] The beneficial effects of the present invention are as follows: based on the reinforcement learning method, software integration is also added to ensure that the simulation scene is as close to the real road conditions as possible, and highly complex simulation scenes can be built quickly and efficiently. At the same time, on the basis of saving costs and improving development and testing efficiency, the intelligence level of the autonomous driving algorithm is greatly improved.

[0017] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present invention, and together with the specification, are used to explain the principles of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0019] Figure 1 It is a schematic diagram of an application scenario of the simulation test method provided by an embodiment of the present invention;

[0020] Figure 2 is a flow chart of a simulation test method provided by an embodiment of the present invention;

[0021] Figure 3 is a flow chart of generating a simulation scenario provided by an embodiment of the present invention;

[0022] Figure 4 is a flow chart for determining a generalization direction provided by an embodiment of the present invention;

[0023] Figure 5 is another flow chart for determining a generalization direction provided by an embodiment of the present invention;

[0024] Figure 6 is a flow chart of generating a new simulation scenario provided by an embodiment of the present invention;

[0025] Figure 7 is a specific flow chart of the simulation test method provided by an embodiment of the present invention;

[0026] Figure 8 is a functional module block diagram of a simulation test device provided by an embodiment of the present invention;

[0027] Fig. 9 It is a structural block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0029] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0030] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0031] Intelligent driving essentially involves cognitive engineering of attention attraction and attention distraction, mainly including Network Guide Air The prerequisite for intelligent driving is that the vehicle we choose meets the driving requirements. move Mechanics Request, in the car sensor can obtain relevant visual and auditory signals and information, and through Cognitive computing Control the corresponding follow-up system. The intelligent driving system is a comprehensive system that integrates advanced information control technology, scenario environment perception, multi-level assisted driving and other functions. As a part of the intelligent transportation system that is being developed by various countries, the intelligent driving system is still under continuous exploration and experimentation.

[0032] Reinforcement Learning (RL), also known as reinforcement learning, evaluation learning or enhanced learning, is Machine Learning of Paradigm and Methodology One of the most important aspects of AgentThe problem of learning strategies to maximize rewards or achieve specific goals in the process of interaction between an agent and the environment. Reinforcement learning is mainly composed of an agent, an environment, a state, an action, and a reward. After the agent performs an action, the environment will switch to a new state, and the environment will give a reward signal (positive reward or negative reward) for the new state. Subsequently, the agent performs new actions according to a certain strategy based on the new state and the reward fed back by the environment. The above process is the way in which the agent and the environment interact through state, action, and reward. Through reinforcement learning, the agent can know what state it is in and what actions it should take to get the maximum reward. Since the way the agent interacts with the environment is similar to the way humans interact with the environment, reinforcement learning can be considered a general learning framework that can be used to solve the problem of general artificial intelligence. Therefore, reinforcement learning is also called the machine learning method of general artificial intelligence.

[0033] Traffic flow refers to the flow of cars continuously driving on the road. In a broad sense, it also includes the flow of other vehicles and people. In a certain period of time, on the road section not affected by lateral crossing, the traffic flow is in a continuous flow state; at the intersection letter Signal light Traffic flow theory is a theory that analyzes and studies the laws of individual or group movements of pedestrians and motor vehicles (mainly cars) on the road, and explores the relationship between traffic volume, flow rate and density, in order to reduce traffic delays, accidents and improve the efficiency of road traffic facilities. Transportation facilities The impact on traffic flow can be divided into: uninterrupted traffic flow or continuous traffic flow (uninterrupted flow) and interrupted traffic flow (interrupted flow); according to the components of traffic flow, it can be divided into: motor vehicle flow, non-motor vehicle flow, mixed traffic flow; according to the intersection form of traffic flow, it can be divided into: intersection, confluence, diversion, and interweaving flow; according to the operating conditions within the traffic flow and the feelings it produces on drivers and passengers, it can be divided into: free flow, stable flow, unstable flow, and forced flow.

[0034] In other application scenarios, the information transmission of data is configured according to actual conditions, and the embodiments of the present invention are not limited thereto.

[0035] Figure 1: is a schematic diagram of an application scenario of a simulation test method provided in an embodiment of the present invention, which collects traffic flow data around a vehicle on a real vehicle, extracts traffic flow data around the vehicle and saves it in a file, imports the file into a simulation software integrated with an autonomous driving algorithm to generate a simulation scenario, uses the simulation scenario to simulate and test the autonomous driving algorithm, and after obtaining a first simulation result, uses reinforcement learning to determine a generalization direction based on the first simulation result, changes the traffic flow data around the vehicle through the generalization direction, imports the changed traffic flow data around the vehicle into a simulation software integrated with an autonomous driving algorithm to generate a new simulation scenario, and after using the new simulation scenario to simulate and test the autonomous driving algorithm, obtains a second simulation result, returns the second simulation result as the first simulation result to the method of using reinforcement learning to determine the direction based on the first simulation result, and executes subsequent steps in a loop until an end condition is met.

[0036] The electronic device may be any electronic product that can perform human-computer interaction with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), an interactive network television (IPTV), a smart wearable device, etc.

[0037] The electronic device may also include a network device and / or a user device, wherein the network device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud consisting of a large number of hosts or network servers based on cloud computing.

[0038] The network where the electronic device is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0039] Figure 2 It is a flow chart of the simulation test method provided by an embodiment of the present invention. It should be noted that the present invention is based on reinforcement learning and existing traffic flow data around the vehicle. Reinforcement learning is the interaction between the vehicle and the environment through state, action, and reward. After the vehicle performs a certain action, the environment will switch to a new state. The environment will give a reward signal (positive reward or negative reward) for the new state. The vehicle will perform new actions according to a certain strategy based on the new state and the reward fed back by the environment. This can ensure that the vehicle experiences a large number of congestion and collision scenes in a simulated environment, thereby making the autonomous driving algorithm more intelligent. This method can be applied to Figure 1The implementation environment shown is a schematic diagram. It should be understood that the method may also be applicable to other exemplary implementation environments and may be specifically executed by devices in other implementation environments. This embodiment does not limit the implementation environment to which the method is applicable.

[0040] like Figure 2 As shown, the simulation test method of this embodiment at least includes:

[0041] Step S21: Obtaining the traffic flow data around the vehicle collected by the real vehicle. It should be noted that it includes the traffic flow data collected by the vehicle in real time and other traffic flow target data.

[0042] Step S22: Import the traffic flow data around the vehicle into the simulation software integrated with the autonomous driving algorithm to generate a simulation scene. It should be noted that the simulation scene can be compatible with a variety of autonomous driving simulation software.

[0043] In a specific embodiment, the simulation scenario includes other target attributes on the road such as speed, travel trajectory, acceleration and deceleration behavior, lane change behavior and traffic light change logic, traffic environment factors such as heavy fog, rainy days, snowy days, etc., and types of traffic participants such as motor vehicles, two-wheeled vehicles, pedestrians, etc.

[0044] Step S23: Using the simulation scenario to perform simulation testing on the autonomous driving algorithm to obtain a first simulation result. It should be noted that the simulation result includes collision and congestion of the autonomous driving algorithm and no collision and congestion of the autonomous driving algorithm.

[0045] Step S24: Determine the generalization direction by using reinforcement learning based on the first simulation result. It should be noted that the learning is to determine the direction by controlling the generalization software. Furthermore, in a specific embodiment of the generalization software, after the module creates a project, the simulation parameter range is set in the GUI interface. After the parameter range is set, the generalization use case is set, and real-time simulation is performed. At the same time, the results that do not meet the expectations are eliminated, thereby determining the generalization direction.

[0046] Step S25: using the generalization direction to change the vehicle surrounding traffic flow data. It should be noted that the generalization direction is used to change the value of one parameter of the vehicle surrounding traffic flow data, and the values ​​of other parameters remain unchanged.

[0047] Step S26: Import the changed traffic flow data around the vehicle into the simulation software integrated with the autonomous driving algorithm to generate a new simulation scene. It should be noted that the format of the scene file corresponding to the simulation scene modification can be adapted to different simulation software.

[0048] Step S27: Use the new simulation scenario to perform simulation test on the autonomous driving algorithm to obtain a second simulation result. It should be noted that the second simulation result includes the autonomous driving algorithm having a collision or congestion and the autonomous driving algorithm not having a collision or congestion.

[0049] Step S28: The second simulation result is used as the first simulation result and returned to the method of determining the generalization direction by using reinforcement learning based on the first simulation result, and the subsequent steps are executed in a loop until the end condition is met. It should be noted that, in a specific embodiment, the end condition can be manually ended, or it can be automatically ended when the efficiency of the automatic generation of the simulation scene reaches a predetermined value.

[0050] like Figure 3 As shown, the steps of generating a simulation scene provided by the embodiment of the present invention include:

[0051] Step S31: Generate a scene file based on the traffic flow data around the vehicle. It should be noted that the scene file can be adjusted and matched according to the simulation software used.

[0052] Step S32: Import the scenario file into the simulation software integrated with the autonomous driving algorithm to generate a simulation scenario. It should be noted that the simulation software includes scenario simulation software, vehicle dynamics simulation software, and unmanned driving simulation software.

[0053] like Figure 4 As shown, the steps of determining the generalization direction by using reinforcement learning based on the first simulation result provided in this embodiment specifically include:

[0054] Step S41 determines whether the autonomous driving algorithm has a collision or congestion according to the first simulation result. It should be noted that before making the determination, the traffic flow data around the vehicle is stored in the generalization software as a generalization basis.

[0055] Step S42: according to the judgment result, the first simulation result is input into the reinforcement learning as a penalty or a reward to determine the generalization direction. It should be noted that the reinforcement learning determines the generalization direction by controlling the generalization software.

[0056] In a specific embodiment, the reinforcement learning is performed in a "trial and error" manner, where the behavior is guided by rewards obtained through interaction with the environment, and the goal is to dynamically adjust parameters to maximize the reinforcement signal.

[0057] Inputting the first simulation result into the reinforcement learning as a penalty or a reward to determine the generalization direction specifically includes:

[0058] Step S43: When the autonomous driving algorithm collides or is congested, the first simulation result is used as a reward input into reinforcement learning to determine the generalization direction.

[0059] like Figure 5 As shown, it also includes:

[0060] Step 51 and step 52 are the same as the above-mentioned step 41 and step 42.

[0061] In step S53, when no collision or congestion occurs in the autonomous driving algorithm, the first simulation result is used as a penalty input for reinforcement learning to determine the generalization direction.

[0062] like Figure 6 As shown, the steps of generating a new simulation scene provided by the embodiment of the present invention include:

[0063] Step S31: Based on the generalization direction, the traffic flow data around the vehicle is changed to generate a new scene file. It should be noted that the value of one parameter of the traffic flow data around the vehicle in the new scene file is changed, and the values ​​of other parameters are not changed, so that the values ​​of the parameters are adjusted to maximize the enhanced signal.

[0064] Step S32: Import the new scenario file into the simulation software integrated with the autonomous driving algorithm to generate the new simulation scenario. It should be noted that only the scenario with changed parameter values ​​in the new simulation scenario will change, and the others will not change.

[0065] The present invention is based on reinforcement learning. In order to realize the simulation test of autonomous driving traffic flow, reinforcement learning is added in the process of simulation test. On the basis of effectively solving the problem of artificially building scenes and building simple traffic flows and static objects in the prior art, the efficiency of development and testing is effectively improved. While ensuring the generalization of simulation results, it can generate simulation scenes with high complexity more quickly.

[0066] like Figure 7 As shown in the specific flow chart of the simulation test method provided in this embodiment, the technical solution of the present invention is described below in conjunction with a specific embodiment:

[0067] The traffic flow data around the vehicle collected by the front data collector may also be the traffic flow data in the driving trajectory of the existing vehicle. The collected traffic flow data around the vehicle is extracted and saved in a file, and the file is imported into the simulation software integrated with the autonomous driving algorithm to generate a simulation scene. The autonomous driving algorithm is simulated and tested in the simulation scene to obtain a first simulation result. After using reinforcement learning to control the generalization direction of the generalization software based on the first simulation result, the generalization software changes the value of one parameter in the traffic flow data around the vehicle, and the values ​​of other parameters remain unchanged. The changed traffic flow data around the vehicle is returned to the file imported into the simulation software integrated with the autonomous driving algorithm to form a new simulation scene, and the subsequent steps are executed in a loop until the end condition is met.

[0068] In summary, the simulation test method provided by the present invention can effectively solve the problems of low scene construction efficiency, low complexity and low frequency of effective events in the process of autonomous driving simulation testing, and can quickly generate simulation scenes with high complexity, so that the autonomous driving algorithm can experience a large number of congestion and collision scenes in a simulation environment, while achieving the purpose of improving the intelligence level of the autonomous driving algorithm, being able to meet the simulation test conditions of most functional scenarios, and also ensuring the purpose of improving development speed.

[0069] Figure 8 It is a functional module block diagram of the intelligent driving software integrated test deployment device provided by an embodiment of the present invention. The vehicle basement relocation device based on semantic information provided by the present invention includes a data acquisition module 81, a simulation scene generation module 82, a first simulation result acquisition module 83, a generalization direction determination module 84, a data change module 85, a new simulation scene acquisition module 86, a second simulation scene acquisition module 87, and a loop execution module 88. The data acquisition module 81 acquires the vehicle surrounding traffic flow data collected by the real vehicle; the simulation scene generation module 82 imports the vehicle surrounding traffic flow data into the simulation software integrated with the automatic driving algorithm to generate a simulation scene; the first simulation result acquisition module 83 uses the simulation scene to simulate and test the automatic driving algorithm to obtain the first simulation result; the generalization direction determination module 88 is used to obtain the vehicle surrounding traffic flow data collected by the real vehicle; the simulation scene generation module 82 imports the vehicle surrounding traffic flow data into the simulation software integrated with the automatic driving algorithm to generate a simulation scene; the first simulation result acquisition module 83 uses the simulation scene to simulate and test the automatic driving algorithm to obtain the first simulation result; the generalization direction determination module 88 is used to obtain the vehicle surrounding traffic flow data collected by the real vehicle; the simulation scene generation module 82 imports the vehicle surrounding traffic flow data into the simulation software integrated with the automatic driving algorithm to generate the ... Module 84, using reinforcement learning to determine the generalization direction based on the first simulation result; changing data module 85, using the generalization direction to change the traffic flow data around the vehicle; obtaining new simulation scene module 86, importing the changed traffic flow data around the vehicle into the simulation software integrated with the autonomous driving algorithm to generate a new simulation scene; obtaining second simulation scene module 87, using the new simulation scene to simulate and test the autonomous driving algorithm to obtain a second simulation result; loop execution module 88, using the second simulation result as the first simulation result and returning to using reinforcement learning to determine the generalization direction based on the first simulation result, and looping the subsequent steps until the end condition is met.

[0070] It should be noted that the above embodiments provide Figure 8 The intelligent simulation test device shown in the figure belongs to the same concept as the simulation test method provided in the above embodiment, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment, and will not be repeated here. In actual application, the vehicle basement relocation device based on semantic information provided in the above embodiment can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0071] An embodiment of the present invention also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the electronic device implements the simulation test method provided in the above-mentioned embodiments.

[0072] Fig. 9 The structure diagram of the computer system of the electronic device suitable for implementing the embodiment of the present invention is shown. It should be noted that: Fig. 9 The computer system 900 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0073] like Fig. 9 As shown, the computer system 900 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 902 or the program loaded from the storage part 906 to the random access memory (RAM) 903, such as executing the method described in the above embodiment. In the RAM 903, various programs and data required for system operation are also stored. The CPU 901, ROM 902 and RAM 903 are connected to each other through a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0074] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 906 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed so that a computer program read therefrom is installed into the storage section 906 as needed.

[0075] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 909, and / or installed from a removable medium 911. When the computer program is executed by a central processing unit (CPU) 901, various functions defined in the system of the present invention are executed.

[0076] It should be noted that the computer-readable medium shown in the embodiment of the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable computer program. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. A computer program contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0077] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. Among them, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0078] The units involved in the embodiments of the present invention may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, limit the units themselves.

[0079] Another aspect of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor of a computer, causes the computer to execute the forward simulation test method. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device.

[0080] Another aspect of the present invention further provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the simulation test method provided in each of the above embodiments.

[0081] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A simulation test method, characterized in that: include: Obtain the traffic flow data around the vehicle collected by the real vehicle; Importing the traffic flow data around the vehicle into the simulation software integrated with the autonomous driving algorithm to generate a simulation scene, including generating a scene file based on the traffic flow data around the vehicle, and importing the scene file into the simulation software integrated with the autonomous driving algorithm to generate a simulation scene; Performing a simulation test on the autonomous driving algorithm using the simulation scenario to obtain a first simulation result, wherein the first simulation result includes that a collision or congestion occurs to the autonomous driving algorithm, and that no collision or congestion occurs to the autonomous driving algorithm; Determine a generalization direction using reinforcement learning based on the first simulation result; Using the generalization direction to change the traffic flow data around the vehicle; Importing the changed traffic flow data around the vehicle into the simulation software integrated with the autonomous driving algorithm to generate a new simulation scene, including generating a new scene file based on the changed traffic flow data around the vehicle, and importing the new scene file into the simulation software integrated with the autonomous driving algorithm to generate the new simulation scene; Performing a simulation test on the autonomous driving algorithm using a new simulation scenario to obtain a second simulation result, wherein the second simulation result includes that a collision or congestion occurs to the autonomous driving algorithm, and that no collision or congestion occurs to the autonomous driving algorithm; The second simulation result is used as the first simulation result and returns to determining the generalization direction by using reinforcement learning based on the first simulation result, and the subsequent steps are executed in a loop until the end condition is met; The step of determining the generalization direction by using reinforcement learning based on the first simulation result includes: judging whether the autonomous driving algorithm has a collision or congestion according to the first simulation result; When the autonomous driving algorithm encounters a collision or congestion, the first simulation result is used as a reward input into reinforcement learning to determine a generalization direction; When the autonomous driving algorithm does not collide or jam, the first simulation result is used as a penalty input for reinforcement learning to determine a generalization direction; Using the generalization direction to change the traffic flow data around the vehicle includes: based on the generalization direction, using generalization software to change the traffic flow data around the vehicle; Wherein, in the process of changing the traffic flow data around the vehicle by using the generalized software, each time the value of one parameter of the traffic flow data around the vehicle is changed, the values ​​of other parameters remain unchanged.

2. The simulation test method according to claim 1, characterized in that: The traffic flow data around the vehicle includes: speed changes of surrounding vehicles, lane change methods of surrounding vehicles, movement trajectories of surrounding vehicles and people, and traffic light change logic.

3. The simulation test method according to claim 1, characterized in that: The generated simulation scenario includes: attributes of other targets, traffic environment factors, and types of traffic participants.

4. A simulation test device, characterized in that: include: A data acquisition module is used to acquire the traffic flow data around the vehicle collected by the real vehicle; Generate a simulation scene module, import the traffic flow data around the vehicle into the simulation software integrated with the autonomous driving algorithm to generate a simulation scene, including generating a scene file based on the traffic flow data around the vehicle, and importing the scene file into the simulation software integrated with the autonomous driving algorithm to generate a simulation scene; A first simulation result module is provided, wherein the autonomous driving algorithm is simulated and tested by using the simulation scenario to obtain a first simulation result, wherein the first simulation result includes that the autonomous driving algorithm has a collision or congestion, and that the autonomous driving algorithm has no collision or congestion; A generalization direction determination module, which uses reinforcement learning to determine the generalization direction based on the first simulation result; A data changing module, using the generalization direction to change the traffic flow data around the vehicle; Obtaining a new simulation scene module, importing the changed traffic flow data around the vehicle into the simulation software integrated with the autonomous driving algorithm to generate a new simulation scene, including generating a new scene file based on the changed traffic flow data around the vehicle, and importing the new scene file into the simulation software integrated with the autonomous driving algorithm to generate the new simulation scene; Obtaining a second simulation scenario module, and using a new simulation scenario to perform a simulation test on the autonomous driving algorithm to obtain a second simulation result, wherein the second simulation result includes that a collision or congestion occurs to the autonomous driving algorithm, and that no collision or congestion occurs to the autonomous driving algorithm; a loop execution module, taking the second simulation result as the first simulation result and returning to determining the generalization direction by using reinforcement learning based on the first simulation result, and looping the subsequent steps until an end condition is met; The step of determining the generalization direction by using reinforcement learning based on the first simulation result includes: judging whether the autonomous driving algorithm has a collision or congestion according to the first simulation result; When the autonomous driving algorithm encounters a collision or congestion, the first simulation result is used as a reward input into reinforcement learning to determine a generalization direction; When the autonomous driving algorithm does not collide or jam, the first simulation result is used as a penalty input for reinforcement learning to determine a generalization direction; Using the generalization direction to change the traffic flow data around the vehicle includes: based on the generalization direction, using generalization software to change the traffic flow data around the vehicle; Wherein, in the process of changing the traffic flow data around the vehicle by using the generalized software, each time the value of one parameter of the traffic flow data around the vehicle is changed, the values ​​of other parameters remain unchanged.

5. An electronic device, characterized in that: The electronic device includes: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the simulation test method as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is enabled to execute the simulation test method according to any one of claims 1 to 3.

Images