Method and device for constructing a simulation environment for autonomous driving

By randomly determining the behavioral patterns and movement trajectories of obstacle objects in the autonomous driving simulation environment, the problem of insufficient response of obstacle objects in the existing simulation environment is solved, the training effect of the autonomous driving model is improved, and more realistic and diverse simulations are provided. Scene.

CN116882122BActive Publication Date: 2024-12-06BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310263891.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-03-17
Publication Date
2024-12-06
Estimated Expiration
2043-03-17

AI Technical Summary

Technical Problem

In the existing autonomous driving simulation environment, obstacle objects can only move in a predefined way, making it difficult to respond to the driving behavior of autonomous driving vehicles, resulting in the simulation training model being unable to fully simulate the game behavior in the real environment, limiting the training of autonomous driving models. Effect.

Method used

By determining the environmental information of obstacle objects in the simulation environment and randomly determining their behavior patterns, determining the movement trajectory of obstacle objects based on the environmental information and behavior patterns, creating a simulation environment that can respond to environmental information, and then creating a diversified driving in the simulation environment Scene.

Benefits of technology

It realizes that obstacle objects in the simulation environment can independently plan the moving trajectory, enhance the authenticity and diversity of the simulation environment, improve the training effect of the autonomous driving model, and enable the model to learn more complex and diverse driving behaviors.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116882122B_ABST
    Figure CN116882122B_ABST
Patent Text Reader

Abstract

The present disclosure provides a method and device for constructing a simulation environment for autonomous driving, which relates to the field of computer technology, and in particular to the field of autonomous driving technology. The implementation scheme is: determining the environmental information of the obstacle object in the simulation environment; randomly determining the behavior pattern of the obstacle object; and determining the movement trajectory of the obstacle object based on the environmental information and the randomly determined behavior pattern. Utilizing the embodiments of the present disclosure, a simulation environment that is more similar to the real road environment can be provided for the training of the autonomous driving model. Not only can the efficient iteration of simulation be used to enable the model to quickly acquire preliminary capabilities in the early stage of autonomous driving model training, but also after the autonomous driving model is deployed on a real car to start real-car road testing, the simulation environment can still be used to generate a variety of real scenes to continue training the model, so that the training effect of the autonomous driving model can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to the field of autonomous driving technology, and specifically to a method and device for constructing a simulation environment for autonomous driving, a method and device for training an autonomous driving model in a simulation environment, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Artificial intelligence is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0003] Autonomous driving technology integrates many aspects of technology such as recognition, decision-making, positioning, communication security and human-computer interaction.

[0004] The methods described in this section are not necessarily methods that have been previously conceived or employed. Unless otherwise indicated, it should not be assumed that any method described in this section is considered to be prior art simply because it is included in this section. Similarly, unless otherwise indicated, the issues mentioned in this section should not be considered to have been recognized in any prior art. Summary of the invention

[0005] The present disclosure provides a method and device for constructing a simulation environment for autonomous driving, a method and device for training an autonomous driving model in a simulation environment, an electronic device, a computer-readable storage medium, and a computer program product.

[0006] According to one aspect of the present disclosure, a method for constructing a simulation environment for autonomous driving is provided, comprising: determining environmental information of an obstacle object in the simulation environment; randomly determining a behavior pattern of the obstacle object; and determining a moving trajectory of the obstacle object based on the environmental information and the randomly determined behavior pattern.

[0007] According to another aspect of the present disclosure, a device for constructing a simulation environment for autonomous driving is provided, comprising: an environmental information determination unit, configured to determine environmental information of an obstacle object in the simulation environment; a sampling unit, configured to randomly determine a behavior pattern of the obstacle object; and a prediction unit, configured to determine a moving trajectory of the obstacle object based on the environmental information and the randomly determined behavior pattern.

[0008] According to another aspect of the present disclosure, a method for training an autonomous driving model in a simulation environment is provided, wherein the simulation environment is constructed based on the aforementioned method, and the method includes: obtaining simulation input information, wherein the simulation input information includes perception information of a simulated target vehicle deployed with the autonomous driving model in the simulation environment; inputting the simulation input information into the autonomous driving model to obtain first simulated autonomous driving strategy information output by the autonomous driving model; determining simulation evaluation feedback information for the first simulated autonomous driving strategy information; and adjusting parameters of the autonomous driving model based on the simulation input information, the first simulated autonomous driving strategy information and the simulation evaluation feedback information.

[0009] According to another aspect of the present disclosure, there is provided a device for training an autonomous driving model in a simulation environment, wherein the simulation environment is constructed based on the aforementioned method, and the device comprises: an acquisition unit, configured to acquire simulation input information, wherein the simulation input information comprises perception information of a simulated target vehicle deployed with the autonomous driving model in the simulation environment; an autonomous driving strategy determination unit, configured to input the simulation input information into the autonomous driving model to acquire first simulated autonomous driving strategy information output by the autonomous driving model; an evaluation feedback unit, configured to determine simulation evaluation feedback information for the first simulated autonomous driving strategy information; and a parameter adjustment unit, configured to adjust parameters of the autonomous driving model based on the simulation input information, the first simulated autonomous driving strategy information and the simulation evaluation feedback information.

[0010] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.

[0011] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the aforementioned method.

[0012] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the aforementioned method when executed by a processor.

[0013] According to one or more embodiments of the present disclosure, a simulation environment can be created, wherein each obstacle object in the simulation environment can plan its future movement trajectory in response to environmental information. By randomly determining the behavior pattern of each obstacle object, a variety of driving scenarios can be conveniently created in the simulation environment.

[0014] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0016] Figure 1 A schematic diagram showing an exemplary system in which the various methods described herein may be implemented according to an embodiment of the present disclosure;

[0017] Figure 2 An exemplary flow chart of a method for constructing a simulation environment for autonomous driving according to an embodiment of the present disclosure is shown;

[0018] Figure 3 An exemplary flow chart of a method for training an obstacle trajectory prediction model according to an embodiment of the present disclosure is shown;

[0019] Figure 4 The figure shows a closed loop training of an autonomous driving model based on a simulation environment according to an embodiment of the present disclosure;

[0020] Figure 5 An exemplary flowchart for training an autonomous driving model in a simulation environment according to an embodiment of the present disclosure is shown;

[0021] Figure 6 A structural block diagram of a device for constructing a simulation environment for autonomous driving according to an embodiment of the present disclosure is shown;

[0022] Figure 7 A structural block diagram of a device for training an autonomous driving model in a simulation environment according to an embodiment of the present disclosure is shown;

[0023] Figure 8 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0024] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0025] In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, temporal relationship, or importance relationship of these elements, and such terms are only used to distinguish one element from another element. In some examples, the first element and the second element may refer to the same instance of the element, and in some cases, based on the description of the context, they may also refer to different instances.

[0026] The terms used in the description of various examples in this disclosure are only for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element can be one or more. In addition, the term "and / or" used in this disclosure covers any one of the listed items and all possible combinations.

[0027] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0028] Figure 1 FIG. 1 is a schematic diagram of an exemplary system 100 in which various methods and apparatuses described herein may be implemented according to an embodiment of the present disclosure. Figure 1 , the system 100 includes a motor vehicle 110 , a server 120 , and one or more communication networks 130 coupling the motor vehicle 110 to the server 120 .

[0029] In an embodiment of the present disclosure, the motor vehicle 110 may include a computing device according to an embodiment of the present disclosure and / or be configured to perform a method according to an embodiment of the present disclosure.

[0030] The server 120 may run one or more services or software applications that enable autonomous driving. In some embodiments, the server 120 may also provide other services or software applications, which may include non-virtualized environments and virtualized environments. Figure 1In the configuration shown, the server 120 may include one or more components that implement the functions performed by the server 120. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. A user of the motor vehicle 110 may in turn utilize one or more client applications to interact with the server 120 to utilize the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from the system 100. Therefore, Figure 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

[0031] Server 120 may include one or more general purpose computers, dedicated server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain a server's virtual storage device). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0032] The computing units in the server 120 may run one or more operating systems including any of the above operating systems and any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0033] In some embodiments, server 120 may include one or more applications to analyze and consolidate data feeds and / or event updates received from motor vehicle 110. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of motor vehicle 110.

[0034] The network 130 may be any type of network known to those skilled in the art that may support data communications using any of a variety of available protocols, including, but not limited to, TCP / IP, SNA, IPX, etc. By way of example only, one or more networks 130 may be a satellite communications network, a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (including, for example, Bluetooth, WiFi), and / or any combination of these and other networks.

[0035] The system 100 may also include one or more databases 150. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 150 may be used to store information such as audio files and video files. The data repository 150 may reside in various locations. For example, the data repository used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. The data repository 150 may be of different types. In some embodiments, the data repository used by the server 120 may be a database, such as a relational database. One or more of these databases may store, update, and retrieve data to and from the database in response to a command.

[0036] In some embodiments, one or more of the databases 150 may also be used by an application to store application data. The databases used by the application may be different types of databases, such as a key-value store, an object store, or a conventional store backed by a file system.

[0037] The motor vehicle 110 may include a sensor 111 for sensing the surrounding environment. The sensor 111 may include one or more of the following sensors: a visual camera, an infrared camera, an ultrasonic sensor, a millimeter wave radar, and a laser radar (LiDAR). Different sensors may provide different detection accuracy and range. The camera may be installed at the front, rear, or other location of the vehicle. The visual camera may capture the situation inside and outside the vehicle in real time and present it to the driver and / or passengers. In addition, by analyzing the images captured by the visual camera, information such as traffic light indications, intersection conditions, and the operating status of other vehicles may be obtained. The infrared camera may capture objects under night vision conditions. Ultrasonic sensors may be installed around the vehicle to measure the distance of objects outside the vehicle from the vehicle using the characteristics of strong ultrasonic directionality. Millimeter wave radars may be installed at the front, rear, or other location of the vehicle to measure the distance of objects outside the vehicle from the vehicle using the characteristics of electromagnetic waves. Laser radars may be installed at the front, rear, or other location of the vehicle to detect object edges and shape information, thereby identifying and tracking objects. Due to the Doppler effect, the radar device may also measure the speed change of the vehicle and the moving object.

[0038] The motor vehicle 110 may also include a communication device 112. The communication device 112 may include a satellite positioning module capable of receiving satellite positioning signals (e.g., Beidou, GPS, GLONASS, and GALILEO) from satellites 141 and generating coordinates based on these signals. The communication device 112 may also include a module for communicating with a mobile communication base station 142, and the mobile communication network may implement any suitable communication technology, such as GSM / GPRS, CDMA, LTE, and other current or evolving wireless communication technologies (e.g., 5G technology). The communication device 112 may also have a vehicle-to-vehicle or vehicle-to-everything (V2X) module, which is configured to implement, for example, vehicle-to-vehicle (V2V) communication with other vehicles 143 and vehicle-to-infrastructure (V2I) communication with infrastructure 144. In addition, the communication device 112 may also have a module configured to communicate with a user terminal 145 (including but not limited to a smartphone, a tablet computer, or a wearable device such as a watch) by, for example, using a wireless LAN or Bluetooth of the IEEE802.11 standard. Using the communication device 112, the motor vehicle 110 may also access the server 120 via the network 130.

[0039] The motor vehicle 110 may also include a control device 113. The control device 113 may include a processor that communicates with various types of computer-readable storage devices or media, such as a central processing unit (CPU) or a graphics processing unit (GPU), or other dedicated processors. The control device 113 may include an automatic driving system for automatically controlling various actuators in the vehicle. The automatic driving system is configured to control the powertrain, steering system, and braking system of the motor vehicle 110 (not shown) via multiple actuators in response to inputs from multiple sensors 111 or other input devices to control acceleration, steering, and braking, respectively, without human intervention or limited human intervention. Some processing functions of the control device 113 can be implemented through cloud computing. For example, some processing can be performed using an on-board processor, while other processing can be performed using computing resources in the cloud. The control device 113 can be configured to perform a method according to the present disclosure. In addition, the control device 113 can be implemented as an example of a computing device on the motor vehicle side (client) according to the present disclosure.

[0040] Figure 1 The system 100 may be configured and operated in various ways to enable the application of various methods and apparatuses described in the present disclosure.

[0041] By running the autonomous driving model in a simulation environment, simulated autonomous driving behavior can be achieved, and the autonomous driving model can be trained according to the results of the autonomous driving behavior in the simulation environment. Various obstacle objects (such as pedestrians, motor vehicles, non-motor vehicles, etc.) in the simulation environment can move in the simulation environment based on a predefined method. For example, the moving direction, speed, trajectory and other parameters of the obstacle objects in the simulation environment can be predefined. When driving a vehicle on a real road, there should be a game behavior between the various objects on the road. For example, in a real environment, when an autonomous driving vehicle goes straight through an intersection, a turning vehicle that intersects with the trajectory of the autonomous driving vehicle can choose to accelerate through the intersection or decelerate according to the distance from the autonomous driving vehicle and continue driving after the autonomous driving vehicle passes. Correspondingly, the autonomous driving vehicle can also change its driving strategy according to the driving behavior (acceleration or deceleration) of the turning vehicle. However, in the simulation environment, the simulated obstacle objects can only move according to a predefined method, and it is difficult to respond to the driving behavior of the autonomous driving vehicle. Due to the difference between the simulation environment and the real environment, the model trained by simulation can only be used as a real environment hot start to conduct preliminary training of the autonomous driving model.

[0042] In order to solve the above problems, the present disclosure provides a new method for constructing a simulation environment for autonomous driving.

[0043] Figure 2 An exemplary flowchart of a method for constructing a simulation environment for autonomous driving according to an embodiment of the present disclosure is shown.

[0044] In step S202, the environment information of the obstacle object in the simulation environment is determined. In step S204, the behavior pattern of the obstacle object is randomly determined. In step S206, the movement trajectory of the obstacle object is determined based on the environment information and the randomly determined behavior pattern.

[0045] By using the method provided by the embodiments of the present disclosure, a simulation environment can be created, wherein each obstacle object in the simulation environment can plan its future movement trajectory in response to environmental information. By randomly determining the behavior pattern of each obstacle object, a variety of driving scenarios can be conveniently created in the simulation environment.

[0046] The principles of the present disclosure will be described in detail below.

[0047] In step S202, environmental information of obstacle objects in the simulation environment may be determined. The obstacle objects may include at least one of pedestrians, non-motor vehicles, and motor vehicles. It is understood that those skilled in the art may configure more obstacle objects in the simulation environment according to actual conditions to make the simulation environment closer to the real road environment.

[0048] In some embodiments, environmental information may include information about the obstacle object's perception of the surrounding environment. In some examples, environmental information may be structured information. The structured information mentioned here includes, but is not limited to, obstacle information, other obstacle object information, lane line information, traffic light information, and static road surface information. In other examples, environmental information may also be an implicit expression of structured information. For example, data including cameras, lidars, and millimeter-wave radars for the surrounding environment of the obstacle object may be input into a multimodal coding layer. Encoding and calculating the above data using a multimodal coding layer may obtain an implicit representation of environmental information, such as an implicit representation in a bird's-eye view (BEV) space. In some implementations, environmental information may be a sequence of information at multiple moments.

[0049] In some examples, the surrounding environment of the obstacle object may include a simulated target vehicle driven based on an autonomous driving model. That is, the simulated target vehicle deployed with the autonomous driving model appears near the obstacle object in the simulated environment. Therefore, the driving trajectory of the simulated target vehicle may affect the moving trajectory of the obstacle object. In this case, the environmental information of the obstacle object may include perception information for the simulated target vehicle.

[0050] In step S204, the behavior pattern of the obstacle object is randomly determined.

[0051] In some embodiments, one item may be randomly selected from a plurality of predefined behavior pattern categories as the behavior pattern of the obstacle object.

[0052] In some examples, the above-mentioned multiple behavior pattern categories are manually labeled. For example, extremely reckless, relatively reckless, relatively conservative, and extremely conservative behavior pattern categories can be manually labeled. It is understandable that without departing from the principles of the present disclosure, those skilled in the art can set a greater or lesser number of behavior pattern categories according to actual conditions, and can also define different behavior pattern categories based on other criteria that affect the movement behavior of obstacle objects on the road. For example, the behavior pattern can be classified based on the age, gender, and other information of the controller of the obstacle object (such as the pedestrian himself, the driver of a motor vehicle or a non-motor vehicle).

[0053] In other examples, the above-mentioned multiple behavior pattern categories are obtained by clustering the drive test data. For example, the CVAE model can be used to perform unlabeled training on the drive test data as a whole to obtain multiple behavior pattern categories. Among them, the number of behavior pattern categories in the obtained results can be predefined, but the meaning of each behavior pattern category is not predefined. The drive test data is automatically clustered into multiple different categories by unlabeled training.

[0054] In some implementations, the behavior pattern of the obstacle object can be determined by sampling from the multiple behavior pattern categories based on the sampling probability predetermined for each behavior pattern category. Taking the behavior pattern categories including reckless and conservative as an example, in the actual road environment, there may be 10% reckless behavior and 90% conservative behavior, that is, most people carry out activities on the road in a relatively safe manner. However, in the simulation environment constructed according to the embodiment of the present disclosure, the sampling probability of the behavior pattern of the reckless category can be set to 90%, and the sampling probability of the behavior pattern of the conservative category can be set to 10%. In other words, a situation is artificially created in the simulation environment where most people carry out activities on the road in a relatively dangerous manner. In this way, a driving environment that is more complex than the real environment can be created for the simulated target vehicle that performs simulated driving based on the autonomous driving model in the simulation environment, and more extreme cases can be created in the simulation environment to help the autonomous driving model trained in the simulation environment learn more diverse information.

[0055] In step S206 , the movement trajectory of the obstacle object may be determined based on the environmental information and the randomly determined behavior pattern.

[0056] In some embodiments, the obstacle trajectory prediction model may be used to process the environmental information and the randomly determined behavior pattern to obtain the movement trajectory of the obstacle object. In other embodiments, the movement trajectory of the obstacle object may be predicted based on pre-set rules.

[0057] The obstacle trajectory prediction model can be determined based on the category of the obstacle object. Different trajectory prediction models can be trained for different types of obstacle objects. For example, a pedestrian trajectory prediction model can be trained for pedestrians, a non-motor vehicle trajectory prediction model can be trained for non-motor vehicles, and a motor vehicle trajectory prediction model can be trained for motor vehicles. The trajectory prediction models corresponding to different obstacle objects can have the same structure but different parameters. In some examples, the network structure of the obstacle trajectory prediction model can be determined using a Transformer-based encoding-decoding structure.

[0058] By determining trajectory prediction models for different categories of obstacle objects in the simulation environment, the simulation environment can be made as close as possible to the real road environment, thereby improving the training effect of the autonomous driving model.

[0059] Figure 3 An exemplary flowchart of a method for training an obstacle trajectory prediction model according to an embodiment of the present disclosure is shown.

[0060] like Figure 3As shown, in step S302, the environmental sample data collected by the autonomous driving vehicle on the real road and the real trajectory corresponding to the environmental sample data can be determined. The data collected when the autonomous driving vehicle deployed with the autonomous driving model is driving on the real road is used as a training sample, and the real trajectory of the vehicle is used as the evaluation criterion for training. The above method can make the trained prediction model closer to the behavior pattern of the real object. In some examples, the environmental sample data can be structured information, including but not limited to obstacle information, other obstacle object information, lane line information, traffic light information, and static road surface information. In other examples, the environmental sample data can also be an implicit expression of structured information, such as an implicit representation in a bird's-eye view (BEV) space. In some implementations, the environmental sample data can be a sequence formed by environmental sample information at multiple moments.

[0061] In step S304, the environmental sample data may be classified to obtain the behavior pattern category corresponding to the environmental sample data. The environmental sample data may be processed using a recognition classification layer to obtain a classification result of the behavior pattern. The recognition classification layer may be implemented using any neural network structure capable of implementing a classification function. The recognition classification layer may output the probability that the environmental sample data belongs to each predefined behavior pattern category.

[0062] In step S306, the environment sample data and the behavior pattern category corresponding to the environment sample data can be processed using the initial parameters of the obstacle trajectory prediction model to obtain a predicted trajectory for the environment sample data. The obstacle trajectory prediction model can be a Transformer-based encoding-decoding structure. By analyzing both the environment sample data and the behavior pattern category, the obstacle trajectory prediction model can learn to predict future trajectories under a predetermined behavior pattern category.

[0063] In step S308, the initial parameters of the obstacle trajectory prediction model can be adjusted based on the difference between the real trajectory and the predicted trajectory to obtain updated parameters of the obstacle trajectory prediction model. A loss function can be designed based on the difference between the real trajectory and the predicted trajectory, and the parameters in the obstacle trajectory prediction model can be updated by back propagation. After a predetermined number of iterations or when the difference between the real trajectory and the predicted trajectory is less than a threshold, a trained trajectory prediction model can be obtained.

[0064] By using the training method provided by the present disclosure, an obstacle trajectory prediction model for obstacle objects in a simulation environment can be trained using real road test data, so that the obstacle objects in the simulation environment can learn the movement strategy in the real environment. By using such a method, a more realistic training environment can be provided for the autonomous driving model trained in the simulation environment, thereby improving the training effect of the model.

[0065] By using the method provided in the present invention to construct a simulation environment for training an autonomous driving model, not only can the efficient iteration of simulation be used to enable the model to quickly acquire preliminary capabilities in the early stages of autonomous driving model training, but also after the autonomous driving model is deployed on a real vehicle and actual road testing begins, the simulation environment can still be used to generate a variety of real scenarios to continue training the model and continuously improve the model effect.

[0066] Figure 4 FIG. 1 shows a closed loop training of an autonomous driving model based on a simulation environment according to an embodiment of the present disclosure. Figure 4 As shown, the simulation system 410 includes a simulated target vehicle that travels based on the driving strategy of the autonomous driving model, obstacle objects (including motor vehicles, pedestrians, and non-motor vehicles), and other environmental modeling information in the simulation environment (including but not limited to lane line information, traffic light information, etc.).

[0067] Among them, the obstacle objects in the simulation system can be controlled based on the obstacle trajectory prediction model trained by the road test data. The diverse control strategy achieved by randomly sampling the behavior patterns of the obstacle objects can generate diverse scenes for the obstacle objects in the simulation environment.

[0068] The simulated target vehicle can use the control signal output by the automatic driving model based on the information collected by sensors (such as cameras, millimeter wave radars, laser radars, etc.) in the simulated scene and / or the navigation information of the simulated target vehicle to drive in the simulated environment. In some embodiments, a safety officer can be used to intervene in the driving process of the simulated target vehicle. The automatic driving model can be trained based on the feedback calculation for the intervention behavior of the safety officer and the data stream generated by the evaluation system for the evaluation and scoring of the driving strategy of the simulated target vehicle in the simulation system 410.

[0069] use Figure 4 According to the process shown in , the autonomous driving model can be trained in a simulation system controlled by a diversity control strategy. The authenticity and diversity of the simulation system can effectively improve the training speed and training effect of the autonomous driving model. After the autonomous driving model completes the initial training in the simulation environment, it can be deployed on a real car for road testing. After a period of road test training, the simulation environment can be reconfigured based on the collected road test data, and further training can be performed in the simulation environment using the road test data. Compared with road test training, the simulation system is more efficient in training the autonomous driving model. When the simulation environment can provide more realistic and diverse simulation scenarios, the training of the autonomous driving model will obtain better results.

[0070] Figure 5An exemplary flow chart for training an autonomous driving model in a simulation environment according to an embodiment of the present disclosure is shown. The simulation environment may be a combination of Figure 2 The method described was constructed.

[0071] In step S502, simulation input information may be obtained. The simulation input information may include perception information of a simulated target vehicle deployed with an autonomous driving model in a simulated environment. In some embodiments, the perception information may include current perception information of the surrounding environment of the simulated target vehicle and historical perception information corresponding to multiple historical moments. There is at least one obstacle object controlled by the obstacle trajectory prediction model in the surrounding environment of the simulated target vehicle.

[0072] In step S504, the simulation input information may be input into the autonomous driving model to obtain first simulated autonomous driving strategy information output by the autonomous driving model.

[0073] In some embodiments, the autonomous driving model may include a multimodal coding layer and a decision control layer. The multimodal coding layer and the decision control layer are connected to form an end-to-end neural network model. The multimodal coding layer is configured to output an implicit representation corresponding to the simulated input information based on the simulated input information, and the decision control layer is configured to obtain the simulated autonomous driving strategy information directly based on the implicit representation output by the multimodal coding layer. The implicit representation can be, for example, an implicit representation in a bird's-eye view (BEV) space. The simulated autonomous driving strategy information may include a target trajectory and / or vehicle control signals (such as throttle, brake, steering amplitude, etc.) of the simulated target vehicle.

[0074] In the example, the autonomous driving model can adopt a Transformer network structure with an encoder and a decoder. It is understandable that the autonomous driving model can also be other neural network models based on the Transformer network structure, which is not limited here. The Transformer architecture can calculate the implicit representation of the model input and output through the self-attention mechanism. In other words, the Transformer architecture can be an Encoder-Decoder model built based on this self-attention mechanism.

[0075] By connecting the multimodal coding layer and the decision control layer to form an end-to-end neural network model, the perception information can be directly responsible for the decision, which can solve the coupling problem between prediction and planning in related technologies. In addition, the input perception information is encoded and calculated to obtain the corresponding implicit representation, which can overcome the problem that the algorithm is prone to failure due to the representation defects of structured information in related technologies. In addition, since the perception information in the simulation input information can be directly responsible for the decision, the perception can be trained to capture the more critical information for the decision, reducing the error accumulation caused by perception errors in model training. Furthermore, since perception is directly responsible for the decision, the autonomous driving technology that emphasizes perception and neglects maps is realized, which can overcome the problem of decision learning failure caused by untimely updates of high-precision maps and restricted areas, and since it gets rid of the dependence on high-precision maps, it can save the cost of updating high-precision maps.

[0076] In some embodiments, the simulation input information may further include a simulation intervention flag. The simulation intervention flag may indicate whether the first simulation autonomous driving strategy information is autonomous driving strategy information with human intervention. By introducing human intervention during simulation training, the model can gradually learn to avoid adverse situations caused by intervention after training.

[0077] In step S506, simulation evaluation feedback information for the first simulated autonomous driving strategy information may be determined.

[0078] Evaluation feedback information can indicate whether the current driving behavior comes from a human driver or a model, whether the current driving is comfortable, whether the current driving violates traffic regulations, and whether the current driving is dangerous driving, thereby improving the user experience.

[0079] In some embodiments, human evaluation of the first simulated autonomous driving strategy information may be obtained as the above-mentioned evaluation feedback information. In other embodiments, the evaluation feedback information for the first simulated autonomous driving strategy information may also be obtained using an evaluation feedback model. In some implementations, the autonomous driving model may include an evaluation feedback layer, and the evaluation feedback layer may be used to process the simulation input information used to generate the first simulated autonomous driving strategy information to obtain the evaluation feedback information for the first simulated autonomous driving strategy information.

[0080] In step S508, the parameters of the autonomous driving model may be adjusted based on the simulation input information, the first simulation autonomous driving strategy information, and the simulation evaluation feedback information.

[0081] In the case where the simulation input information includes a simulation intervention identifier, the parameters of the autonomous driving model can be adjusted based on the simulation intervention identifier, the simulation input information, the first simulation autonomous driving strategy information, and the simulation evaluation feedback information. In some embodiments, the parameters of the autonomous driving model can be adjusted based on a reinforcement learning method (such as a PPO algorithm or a SAC algorithm).

[0082] The automatic driving model for simulating the target vehicle and the obstacle trajectory prediction model for the obstacle object can also be trained in a collaborative manner in the simulation environment provided by the embodiments of the present disclosure. Figure 3 The obstacle trajectory prediction model is pre-trained in the manner described to achieve the configuration of the simulation environment. When the autonomous driving model is trained in the simulation environment, the obstacle trajectory prediction model can also be trained synchronously using the data generated in the simulation environment, thereby further improving the effect of the obstacle trajectory prediction model.

[0083] In some embodiments, the training method 500 may further include: controlling the simulated target vehicle to perform autonomous driving in the simulated environment based on the adjusted autonomous driving model. For obstacle objects in the simulated environment, simulated environment sample information of the obstacle objects may be obtained, wherein the obtained simulated environment sample information may include perception information of the obstacle objects on the simulated target vehicle controlled based on the adjusted autonomous driving model. The simulated environment sample information may be processed using an obstacle trajectory prediction model to obtain a simulated predicted trajectory for the simulated environment sample information. Evaluation feedback information for the simulated predicted trajectory may be determined using an evaluation feedback model or human feedback. The parameters of the obstacle trajectory prediction model may be updated based on the simulated environment sample information, the simulated predicted trajectory, and the evaluation feedback information for the simulated predicted trajectory. In some implementations, the parameters of the obstacle trajectory prediction model may be updated using reinforcement learning.

[0084] Figure 6 FIG. 6 is a block diagram of a device 600 for constructing a simulation environment for autonomous driving according to an embodiment of the present disclosure. Figure 6 As shown, the apparatus 600 includes an environment information determining unit 610 , a sampling unit 620 , and a prediction unit 630 .

[0085] The environment information determination unit 610 may be configured to determine the environment information of the obstacle object in the simulation environment. The sampling unit 620 may be configured to randomly determine the behavior pattern of the obstacle object. The prediction unit 630 may be configured to determine the movement trajectory of the obstacle object based on the environment information and the randomly determined behavior pattern.

[0086] In some embodiments, the environmental information includes information about the obstacle object's perception of the surrounding environment.

[0087] In some embodiments, the surrounding environment of the obstacle object includes a simulated target vehicle driven based on an autonomous driving model, and the environmental information includes perception information of the obstacle object on the simulated target vehicle.

[0088] In some embodiments, the obstacle object includes at least one of a pedestrian, a non-motor vehicle, and a motor vehicle.

[0089] In some embodiments, the sampling unit is configured to randomly select one item from a plurality of predefined behavior pattern categories as the behavior pattern of the obstacle object.

[0090] In some embodiments, randomly selecting one item from a plurality of predefined behavior pattern categories as the behavior pattern of the obstacle object includes: determining the behavior pattern of the obstacle object by sampling from the plurality of behavior pattern categories based on a sampling probability predetermined for each behavior pattern category.

[0091] In some embodiments, the multiple behavior pattern categories are manually labeled or obtained by clustering drive test data.

[0092] In some embodiments, the prediction unit is configured to: process the environmental information and the randomly determined behavior pattern using the obstacle trajectory prediction model to obtain the movement trajectory of the obstacle object.

[0093] In some embodiments, the obstacle trajectory prediction model is determined based on the category of the obstacle object.

[0094] In some embodiments, the obstacle trajectory prediction model is obtained by: determining environmental sample data collected by an autonomous driving vehicle on a real road and a real trajectory corresponding to the environmental sample data; classifying the environmental sample data to obtain the behavior pattern category corresponding to the environmental sample data; using the initial parameters of the obstacle trajectory prediction model to process the environmental sample data and the behavior pattern category corresponding to the environmental sample data to obtain a predicted trajectory for the environmental sample data; adjusting the initial parameters of the obstacle trajectory prediction model based on the difference between the real trajectory and the predicted trajectory to obtain updated parameters of the obstacle trajectory prediction model.

[0095] It should be understood that Figure 6 The various modules or units of the apparatus 600 shown in FIG. 6 can be used in conjunction with the reference Figure 2 The steps in the method 200 described above correspond to each other. Therefore, the operations, features and advantages described above for the method 200 are also applicable to the device 600 and the modules and units included therein. For the sake of brevity, some operations, features and advantages are not repeated here.

[0096] Figure 7FIG. 1 shows a structural block diagram of an apparatus for training an autonomous driving model in a simulation environment according to an embodiment of the present disclosure. Figure 7 As shown, the device 700 includes an acquisition unit 710, an automatic driving strategy determination unit 720, an evaluation feedback unit 730 and a parameter adjustment unit 740.

[0097] The acquisition unit 710 may be configured to acquire simulation input information, wherein the simulation input information includes perception information of a simulated target vehicle deployed with an autonomous driving model in a simulation environment.

[0098] The autonomous driving strategy determination unit 720 may be configured to input simulation input information into the autonomous driving model to obtain first simulated autonomous driving strategy information output by the autonomous driving model.

[0099] The evaluation feedback unit 730 may be configured to determine simulation evaluation feedback information for the first simulated autonomous driving strategy information.

[0100] The parameter adjustment unit 740 may be configured to adjust the parameters of the autonomous driving model based on the simulation input information, the first simulation autonomous driving strategy information, and the simulation evaluation feedback information.

[0101] In some embodiments, the simulation input information also includes a simulation intervention identifier, which indicates whether the first simulated autonomous driving strategy information is autonomous driving strategy information with human intervention. The parameter adjustment unit can be configured to adjust the parameters of the autonomous driving model based on the simulation intervention identifier, the simulation input information, the first simulated autonomous driving strategy information and the simulation evaluation feedback information.

[0102] In some embodiments, adjusting the parameters of the autonomous driving model includes adjusting the parameters of the autonomous driving model based on reinforcement learning.

[0103] In some embodiments, the autonomous driving model includes a multimodal coding layer and a decision control layer, which are connected to form an end-to-end neural network model. The multimodal coding layer is configured to output an implicit representation corresponding to the simulated input information based on the simulated input information, and the decision control layer is configured to obtain the simulated autonomous driving strategy information directly based on the implicit representation output by the multimodal coding layer.

[0104] In some embodiments, the device also includes an obstacle trajectory prediction model adjustment unit, which is configured to: control the simulated target vehicle to perform autonomous driving in a simulated environment based on the adjusted autonomous driving model; obtain simulated environment sample information of the obstacle object in the simulated environment, wherein the simulated environment sample information includes perception information of the obstacle object on the simulated target vehicle controlled based on the adjusted autonomous driving model; process the simulated environment sample information using the obstacle trajectory prediction model to obtain a simulated predicted trajectory for the simulated environment sample information; determine evaluation feedback information for the simulated predicted trajectory; and update the parameters of the obstacle trajectory prediction model based on the simulated environment sample information, the simulated predicted trajectory, and the evaluation feedback information for the simulated predicted trajectory.

[0105] Although specific functionality is discussed above with reference to specific modules, it should be noted that the functionality of the various units discussed herein may be separated into multiple units, and / or at least some functionality of multiple units may be combined into a single unit.

[0106] It should also be understood that various techniques may be described herein in the general context of software hardware elements or program modules. Figure 6 , Figure 7 The various units described can be implemented in hardware or in hardware in combination with software and / or firmware. For example, these units can be implemented as computer program codes / instructions, which are configured to be executed in one or more processors and stored in a computer-readable storage medium. Alternatively, these units can be implemented as hardware logic / circuits. For example, in some embodiments, one or more of units 610 to 630, units 710 to 740 can be implemented together in a system on chip (System on Chip, SoC). SoC may include an integrated circuit chip (which includes a processor (e.g., a central processing unit (CPU), a microcontroller, a microprocessor, a digital signal processor (DSP), etc.), a memory, one or more communication interfaces, and / or one or more components in other circuits), and may optionally execute the received program code and / or include embedded firmware to perform functions.

[0107] According to another aspect of the present disclosure, an electronic device is also provided, including: at least one processor; and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for constructing a simulation environment for autonomous driving according to an embodiment of the present disclosure.

[0108] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is also provided, where the computer instructions are used to enable the computer to execute a method for constructing a simulation environment for autonomous driving according to an embodiment of the present disclosure.

[0109] According to another aspect of the present disclosure, a computer program product is also provided, including a computer program, wherein the computer program, when executed by a processor, implements the method for constructing a simulation environment for autonomous driving according to an embodiment of the present disclosure.

[0110] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0111] According to an embodiment of the present disclosure, an electronic device, a readable storage medium and a computer program product are also provided.

[0112] refer to Figure 8 , a block diagram of an electronic device 800 that can be used as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0113] like Figure 8 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0114] Multiple components in the electronic device 800 are connected to the I / O interface 805, including: an input unit 806, an output unit 807, a storage unit 808, and a communication unit 809. The input unit 806 can be any type of device that can input information to the electronic device 800. The input unit 806 can receive input digital or character information and generate key signal input related to user settings and / or function control of the electronic device, and can include but is not limited to a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone, and / or a remote controller. The output unit 807 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 808 can include but is not limited to a disk, an optical disk. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, an 802.11 device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0115] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as methods 200, 300, and 500. For example, in some embodiments, methods 200, 300, and 500 may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the methods 200, 300, and 500 described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute the methods 200 , 300 , 500 in any other appropriate manner (eg, by means of firmware).

[0116] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0117] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0118] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0119] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0120] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0121] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0122] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0123] Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present invention is not limited by these embodiments or examples, but only by the claims after authorization and their equivalent scope. Various elements in the embodiments or examples can be omitted or replaced by their equivalent elements. In addition, each step can be performed in an order different from that described in the present disclosure. Further, the various elements in the embodiments or examples can be combined in various ways. It is important that with the evolution of technology, many elements described herein can be replaced by equivalent elements that appear after the present disclosure.

Claims

1. A method for constructing a simulation environment for autonomous driving, comprising: Determine environmental information of obstacle objects in the simulation environment; Randomly determining a behavior pattern of the obstacle object; as well as The environment information and the randomly determined behavior pattern are processed using an obstacle trajectory prediction model to obtain a movement trajectory of the obstacle object. Wherein, randomly determining the behavior mode of the obstacle object includes: Randomly select one item from a plurality of predefined behavior pattern categories as the behavior pattern of the obstacle object, Among them, randomly selecting one item from a plurality of predefined behavior pattern categories as the behavior pattern of the obstacle object includes: The behavior pattern of the obstacle object is determined by sampling from the plurality of behavior pattern categories based on a sampling probability predetermined for each behavior pattern category, wherein the sampling probability indicates a safety level of a road in the simulation environment.

2. The method of claim 1, wherein: The environmental information includes the perception information of the obstacle object to the surrounding environment.

3. The method of claim 2, wherein: The surrounding environment of the obstacle object includes a simulated target vehicle driven based on an automatic driving model, and the environmental information includes perception information of the obstacle object on the simulated target vehicle.

4. The method of claim 1, wherein: The obstacle object includes at least one of a pedestrian, a non-motor vehicle, and a motor vehicle.

5. The method of claim 1, wherein: The multiple behavior pattern categories are manually labeled or obtained by clustering drive test data.

6. The method of claim 1, wherein: The obstacle trajectory prediction model is determined based on the category of the obstacle object.

7. The method of claim 1, wherein: The obstacle trajectory prediction model is obtained in the following way: Determine environmental sample data collected by the autonomous driving vehicle on a real road and a real trajectory corresponding to the environmental sample data; Classifying the environmental sample data to obtain a behavior pattern category corresponding to the environmental sample data; Processing the environmental sample data and the behavior pattern category corresponding to the environmental sample data using the initial parameters of the obstacle trajectory prediction model to obtain a predicted trajectory for the environmental sample data; as well as Initial parameters of the obstacle trajectory prediction model are adjusted based on the difference between the actual trajectory and the predicted trajectory to obtain updated parameters of the obstacle trajectory prediction model.

8. A device for constructing a simulation environment for autonomous driving, comprising: An environment information determination unit, configured to determine environment information of an obstacle object in the simulation environment; A sampling unit configured to randomly determine a behavior pattern of the obstacle object; as well as A prediction unit is configured to process the environmental information and the randomly determined behavior pattern using an obstacle trajectory prediction model to obtain a movement trajectory of the obstacle object, Wherein, the sampling unit is configured as follows: Randomly select one item from a plurality of predefined behavior pattern categories as the behavior pattern of the obstacle object, Among them, randomly selecting one item from a plurality of predefined behavior pattern categories as the behavior pattern of the obstacle object includes: The behavior pattern of the obstacle object is determined by sampling from the plurality of behavior pattern categories based on a sampling probability predetermined for each behavior pattern category, wherein the sampling probability indicates a safety level of a road in the simulation environment.

9. The device of claim 8, wherein: The environmental information includes the perception information of the obstacle object to the surrounding environment.

10. The device of claim 9, wherein: The surrounding environment of the obstacle object includes a simulated target vehicle driven based on an automatic driving model, and the environmental information includes perception information of the obstacle object on the simulated target vehicle.

11. The device of claim 8, wherein: The obstacle object includes at least one of a pedestrian, a non-motor vehicle, and a motor vehicle.

12. The device of claim 8, wherein: The multiple behavior pattern categories are manually labeled or obtained by clustering drive test data.

13. The device of claim 12, wherein: The obstacle trajectory prediction model is determined based on the category of the obstacle object.

14. The device of claim 12, wherein: The obstacle trajectory prediction model is obtained in the following way: Determine environmental sample data collected by the autonomous driving vehicle on a real road and a real trajectory corresponding to the environmental sample data; Classifying the environmental sample data to obtain a behavior pattern category corresponding to the environmental sample data; Processing the environmental sample data and the behavior pattern category corresponding to the environmental sample data using the initial parameters of the obstacle trajectory prediction model to obtain a predicted trajectory for the environmental sample data; as well as Initial parameters of the obstacle trajectory prediction model are adjusted based on the difference between the actual trajectory and the predicted trajectory to obtain updated parameters of the obstacle trajectory prediction model.

15. A method for training an autonomous driving model in a simulation environment constructed by the method according to any one of claims 1 to 7, the method comprising: Acquire simulation input information, wherein the simulation input information includes perception information of a simulated target vehicle deployed with the autonomous driving model in the simulation environment; Inputting the simulation input information into the autonomous driving model to obtain first simulated autonomous driving strategy information output by the autonomous driving model; Determining simulation evaluation feedback information for the first simulated autonomous driving strategy information; Based on the simulation input information, the first simulation autonomous driving strategy information and the simulation evaluation feedback information, the parameters of the autonomous driving model are adjusted.

16. The method of claim 15, wherein: The simulation input information further includes a simulation intervention flag, wherein the simulation intervention flag indicates whether the first simulation automatic driving strategy information is automatic driving strategy information with human intervention. Adjusting the parameters of the autonomous driving model based on the simulation input information, the first simulation autonomous driving strategy information, and the simulation evaluation feedback information includes: Based on the simulation intervention identifier, the simulation input information, the first simulation autonomous driving strategy information and the simulation evaluation feedback information, the parameters of the autonomous driving model are adjusted.

17. The method according to claim 15 or 16, wherein: Adjusting the parameters of the autonomous driving model includes adjusting the parameters of the autonomous driving model based on reinforcement learning.

18. The method of claim 15 or 16, further comprising: Controlling the simulated target vehicle to perform autonomous driving in the simulated environment based on the adjusted autonomous driving model; For an obstacle object in a simulation environment, obtaining simulation environment sample information of the obstacle object, wherein the simulation environment sample information includes perception information of the obstacle object to the simulation target vehicle controlled based on the adjusted automatic driving model; The simulation environment sample information is processed using an obstacle trajectory prediction model to obtain a simulation prediction trajectory for the simulation environment sample information; Determining evaluation feedback information for the simulation prediction trajectory; and The parameters of the obstacle trajectory prediction model are updated based on the simulation environment sample information, the simulation predicted trajectory, and evaluation feedback information for the simulation predicted trajectory.

19. A device for training an autonomous driving model in a simulation environment constructed by the method according to any one of claims 1 to 7, the device comprising: an acquisition unit configured to acquire simulation input information, wherein the simulation input information includes perception information of a simulated target vehicle deployed with the autonomous driving model in the simulation environment; an autonomous driving strategy determination unit, configured to input the simulation input information into the autonomous driving model to obtain first simulated autonomous driving strategy information output by the autonomous driving model; an evaluation feedback unit, configured to determine simulation evaluation feedback information for the first simulated autonomous driving strategy information; A parameter adjustment unit is configured to adjust the parameters of the autonomous driving model based on the simulation input information, the first simulation autonomous driving strategy information and the simulation evaluation feedback information.

20. An electronic device, comprising: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7 and 15-18.

21. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7, 15-18.

22. A computer program product comprising a computer program, wherein: The computer program implements the method of any one of claims 1-7, 15-18 when executed by a processor.