Autonomous driving models, training methods, devices, and vehicles
Through the integrated autonomous driving model of perception-decision-making, the neural network model of the multimodal coding layer and the decision-making control layer is used to directly convert the perceptual information into an autonomous driving strategy, solving the coupling and high level of perception modules and planning modules in autonomous driving technology The problem of map update has been carefully improved, achieving efficient autonomous driving decisions and user experience.
Patent Information
- Application Number
- CN202310274740.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-03-17
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2043-03-17
AI Technical Summary
In the existing autonomous driving technology, the coupling of perception modules and planning modules leads to accumulation of errors, making it difficult to make effective decisions between prediction and planning, structured information is limited, high-precision map update costs and area are limited, resulting in poor autonomous driving effect. .
Adopting the integrated autonomous driving model of perception-decision-making, through the end-to-end neural network model of the multimodal coding layer and the decision control layer, the perceived information is directly converted into an autonomous driving strategy, reducing the dependence on high-precision maps, and using real driving The data is iteratively trained to realize autonomous driving that emphasizes perception and is light on maps.
Reduce error accumulation, solve the coupling problem between prediction and planning, overcome the defects in structured information representation and the problem of untimely update of high-precision maps, save the cost of update of high-precision maps, and improve the decision-making efficiency of autonomous driving and User experience.
Smart Images

Figure CN116881707B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to the field of autonomous driving and artificial intelligence technology, and specifically to a training method for an autonomous driving model, an autonomous driving model, an autonomous driving method implemented using the autonomous driving model, a training device for the autonomous driving model, an autonomous driving device based on the autonomous driving model, an electronic device, a computer-readable storage medium, a computer program product, and an autonomous driving vehicle. Background Art
[0002] Artificial intelligence is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.
[0003] Autonomous driving technology integrates many technologies such as recognition, decision-making, positioning, communication security, and human-computer interaction. Artificial intelligence learning can assist in generating autonomous driving strategies.
[0004] High-precision maps, also known as high-precision maps, are maps used by self-driving cars. High-precision maps have accurate vehicle location information and rich road element data information, which can help cars predict complex road information, such as slope, curvature, and heading, so as to better avoid potential risks. In other words, self-driving technology relies heavily on high-precision maps.
[0005] The methods described in this section are not necessarily methods that have been previously conceived or employed. Unless otherwise indicated, it should not be assumed that any method described in this section is considered to be prior art simply because it is included in this section. Similarly, unless otherwise indicated, the issues mentioned in this section should not be considered to have been recognized in any prior art. Summary of the invention
[0006] The present disclosure provides a training method for an autonomous driving model, an autonomous driving model, an autonomous driving method implemented using the autonomous driving model, a training device for the autonomous driving model, an autonomous driving device based on the autonomous driving model, an electronic device, a computer-readable storage medium, a computer program product, and an autonomous driving vehicle.
[0007] According to one aspect of the present disclosure, a training method for an autonomous driving model is provided. The autonomous driving model includes a multimodal coding layer and a decision control layer, wherein the multimodal coding layer and the decision control layer are connected to form an end-to-end neural network model so that the decision control layer directly predicts autonomous driving strategy information based on the output of the multimodal coding layer. The training method of the autonomous driving model includes performing a first training on the multimodal coding layer and the decision control layer, and the first training on the multimodal coding layer and the decision control layer includes: obtaining first real driving data during vehicle driving, the first real driving data including first navigation information of the vehicle and first real perception information of the vehicle's surrounding environment, the first real perception information including current perception information and historical perception information of the vehicle's surrounding environment; obtaining first real autonomous driving strategy information corresponding to the first real driving data based on the first real perception information; inputting first sample input information including the first real driving data into the multimodal coding layer to obtain a first sample implicit representation output by the multimodal coding layer; inputting first intermediate sample input information including the first sample implicit representation into the decision control layer to obtain first predicted autonomous driving strategy information output by the decision control layer; and adjusting parameters of the multimodal coding layer and the decision control layer based on the first predicted autonomous driving strategy information and the first real autonomous driving strategy information.
[0008] According to another aspect of the present disclosure, there is provided an autonomous driving model trained using the above-mentioned training method, comprising a multimodal coding layer and a decision control layer, wherein the multimodal coding layer and the decision control layer are connected to form an end-to-end neural network model, so that the decision control layer directly predicts autonomous driving strategy information based on the output of the multimodal coding layer. The first input information of the multimodal coding layer comprises navigation information of the target vehicle and perception information of the surrounding environment of the target vehicle obtained by using sensors, wherein the perception information comprises current perception information and historical perception information of the surrounding environment of the target vehicle during the driving process of the vehicle, the multimodal coding layer is configured to obtain an implicit representation corresponding to the first input information, the second input information of the decision control layer comprises the implicit representation, and the decision control layer is configured to obtain target autonomous driving strategy information based on the second input information.
[0009] According to another aspect of the present disclosure, there is provided an autonomous driving method implemented by using an autonomous driving model, including: using the above-mentioned autonomous driving model to control a target vehicle to perform autonomous driving; and acquiring real driving data during the autonomous driving process, wherein the real driving data includes navigation information of the target vehicle, real perception information of the target vehicle's surrounding environment, and real autonomous driving strategy information, and the real driving data is used to iteratively train the autonomous driving model.
[0010] According to another aspect of the present disclosure, a training device for an autonomous driving model is provided, wherein the autonomous driving model includes a multimodal coding layer and a decision control layer, wherein the multimodal coding layer and the decision control layer are connected to form an end-to-end neural network model, so that the decision control layer directly predicts autonomous driving strategy information based on the output of the multimodal coding layer. The device is configured to perform a first training on the multimodal coding layer and the decision control layer, and includes: a first real driving data acquisition unit, configured to acquire first real driving data during vehicle driving, the first real driving data including first navigation information of the vehicle and first real perception information for the vehicle's surrounding environment, the first real perception information including current perception information and historical perception information for the vehicle's surrounding environment; a real autonomous driving strategy information acquisition unit, configured to acquire first real autonomous driving strategy information corresponding to the first real driving data based on the first real perception information; a multimodal coding layer training unit, configured to input first sample input information including the first real driving data into the multimodal coding layer to acquire a first sample implicit representation output by the multimodal coding layer; a decision control layer training unit, configured to input first intermediate sample input information including the first sample implicit representation into the decision control layer to acquire first predicted autonomous driving strategy information output by the decision control layer; and a parameter adjustment unit, configured to adjust parameters of the multimodal coding layer and the decision control layer based on the first predicted autonomous driving strategy information and the first real autonomous driving strategy information.
[0011] According to another aspect of the present disclosure, an autonomous driving device based on an autonomous driving model is provided, including: a control unit, configured to control a target vehicle to perform autonomous driving using the above-mentioned autonomous driving model; and a second real driving data acquisition unit, configured to acquire real driving data during the autonomous driving process, the real driving data including navigation information of the target vehicle, real perception information of the environment surrounding the target vehicle, and real autonomous driving strategy information, and the real driving data is used to iteratively train the autonomous driving model.
[0012] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor so that the at least one processor can execute the above method.
[0013] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the above method.
[0014] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the above method when executed by a processor.
[0015] According to another aspect of the present disclosure, an autonomous driving vehicle is provided, comprising: a training device for an autonomous driving model according to an embodiment of the present disclosure, an autonomous driving device, and one of electronic devices.
[0016] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.
[0018] Figure 1 A schematic diagram showing an exemplary system in which the various methods described herein may be implemented according to an embodiment of the present disclosure;
[0019] Figure 2 A schematic diagram of an autonomous driving model according to an embodiment of the present disclosure is shown;
[0020] Figure 3 A flowchart of a method for training an autonomous driving model according to an embodiment of the present disclosure is shown;
[0021] Figure 4 A flowchart showing a partial process of a method for training an autonomous driving model according to an embodiment of the present disclosure is shown;
[0022] Figure 5 A flowchart showing a partial process of a method for training an autonomous driving model according to an embodiment of the present disclosure is shown;
[0023] Figure 6 A flowchart of a method for training an autonomous driving model according to another embodiment of the present disclosure is shown;
[0024] Figure 7 A flowchart of an automatic driving method according to an embodiment of the present disclosure is shown;
[0025] Figure 8 A flowchart of an autonomous driving method according to another embodiment of the present disclosure is shown;
[0026] Fig. 9A structural block diagram of a training device for an autonomous driving model according to an embodiment of the present disclosure is shown;
[0027] Fig.10 A structural block diagram of an automatic driving device according to an embodiment of the present disclosure is shown; and
[0028] Fig.11 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION
[0029] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.
[0030] In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, temporal relationship, or importance relationship of these elements, and such terms are only used to distinguish one element from another element. In some examples, the first element and the second element may refer to the same instance of the element, and in some cases, based on the description of the context, they may also refer to different instances.
[0031] The terms used in the description of various examples in this disclosure are only for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element can be one or more. In addition, the term "and / or" used in this disclosure covers any one of the listed items and all possible combinations.
[0032] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0033] In the related art, the optimization and rule-based algorithms in autonomous driving technology usually rely on high-precision maps and algorithm optimization for different scenarios. High-precision maps, also known as high-precision maps, are maps used by autonomous vehicles. High-precision maps have accurate vehicle location information and rich road element data information, which can help cars predict complex road information, such as slope, curvature, heading, etc., so as to better avoid potential risks. Accordingly, the application of the algorithm is limited to very local areas, and autonomous driving may fail due to map errors, and it is difficult to solve a large number of long-tail situations. In addition, the algorithms in the related art rely on a large amount of human annotation. On the one hand, these annotation methods consume a lot of manpower, and on the other hand, these annotation methods are for the purpose of perception. For example, there is a lot of background information during driving, as well as distant obstacles that are not related to driving (such as non-motor vehicles on the opposite lane). In automatic annotation for the purpose of perception, it is difficult for annotators to determine which obstacles should be identified and which should not be paid attention to, and it is difficult to directly serve the strategy optimization and driving decision of autonomous driving.
[0034] In related technologies, unmanned driving technology mainly relies on the synergy of perception modules and planning control modules. The working process of autonomous driving includes two stages: first, converting the unstructured information obtained by sensors such as cameras or radars into structured information (structured information includes obstacle information, other vehicle information, pedestrian and non-motor vehicle information, lane line information, traffic light information, other static road surface information, etc.). This information can be combined and matched with high-precision maps to accurately obtain the location information on high-precision maps. Second, prediction and decision-making are made based on structured information and related observation history. Among them, prediction includes predicting the changes in the surrounding structured environment in the future; decision-making includes generating some structured information that can be used for subsequent trajectory planning (such as changing lanes, plugging in, waiting). Third, based on structured decision information and changes in the surrounding structured environment, the trajectory of the target vehicle for a period of time in the future is planned, such as planning trajectory or control information (such as planning speed and position).
[0035] After research, it was found that autonomous driving technology based on perception-prediction-planning may face some technical problems. The first is the problem of error accumulation, because perception cannot be directly responsible for decision-making, which makes perception not necessarily capture information that plays a key role in decision-making. In addition, because the errors of perception are difficult to make up in the subsequent process (for example, obstacles in the area may not be identified), the subsequent process may find it difficult to make correct decisions when key obstacles are lost. Secondly, the coupling problem between prediction and planning cannot be solved. The behavior of surrounding obstacles, especially key obstacles that interact with the target vehicle, may be affected by the target vehicle. In other words, during the operation of the autonomous driving model, there is a coupling between the two modules of prediction and planning, which makes the streaming decision affect the final autonomous driving effect. In addition, there is also the problem of representation defects of structured information. Structured information is completely limited to artificially predefined standards. Once a new paradigm that is not clearly defined (such as the appearance of unknown obstacles, unknown states of vehicles and pedestrians, etc.) is encountered, the algorithm is likely to fail. Finally, there is the issue of reliance on high-cost maps (such as high-precision maps). Related technologies mainly rely on information such as high-precision map point clouds to locate vehicles. However, in practice, high-precision maps are only available in limited areas, which limits the actual application areas of autonomous driving. In addition, the update cost of high-precision maps is huge. Once the map and the actual road do not match, it is easy to cause decision-making failures.
[0036] Based on this, the present disclosure provides a training method for an autonomous driving model, an autonomous driving model, an autonomous driving method implemented using an autonomous driving model, a training device for an autonomous driving model, an autonomous driving device based on an autonomous driving model, an electronic device, a computer-readable storage medium, a computer program product, and an autonomous driving vehicle, which adopts a perception-decision integrated driving technology so that perception is directly responsible for decision-making, which is conducive to perceiving and capturing information that plays a key role in decision-making, reducing error accumulation, and solving the coupling problem between prediction and decision-making in related technologies. In addition, perception being directly responsible for decision-making can overcome the problem that the structured prediction information is limited by artificially predefined standards, which makes the algorithm prone to failure, and can also overcome the problem that decision-making failures are caused by untimely updates of high-precision maps and regional restrictions. Since it is free from dependence on high-precision maps, it can save the cost of updating high-precision maps. In addition, when the acquired real driving data has limited annotated real autonomous driving strategy information, it is also possible to obtain real autonomous driving strategy information corresponding to the real driving data based on the perception information in the real driving data, and complete the model training process accordingly, to achieve autonomous driving technology that focuses on perception rather than maps. In addition, using a large amount of real driving data to train the autonomous driving model can ensure decision-making efficiency, so that the autonomous driving behavior can be well aligned with the preferences of human passengers, improve user experience, and avoid the long learning process of cold start.
[0037] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.
[0038] Figure 1 FIG. 1 is a schematic diagram of an exemplary system 100 in which various methods and apparatuses described herein may be implemented according to an embodiment of the present disclosure. Figure 1 , the system 100 includes a motor vehicle 110 , a server 120 , and one or more communication networks 130 coupling the motor vehicle 110 to the server 120 .
[0039] In an embodiment of the present disclosure, the motor vehicle 110 may include a computing device according to an embodiment of the present disclosure and / or be configured to perform a method according to an embodiment of the present disclosure.
[0040] The server 120 may run one or more services or software applications that enable autonomous driving. In some embodiments, the server 120 may also provide other services or software applications, which may include non-virtualized environments and virtualized environments. Figure 1 In the configuration shown, the server 120 may include one or more components that implement the functions performed by the server 120. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. A user of the motor vehicle 110 may in turn utilize one or more client applications to interact with the server 120 to utilize the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from the system 100. Therefore, Figure 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.
[0041] Server 120 may include one or more general purpose computers, dedicated server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain a server's virtual storage device). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.
[0042] The computing units in the server 120 may run one or more operating systems including any of the above operating systems and any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.
[0043] In some embodiments, server 120 may include one or more applications to analyze and consolidate data feeds and / or event updates received from motor vehicle 110. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of motor vehicle 110.
[0044] The network 130 may be any type of network known to those skilled in the art that may support data communications using any of a variety of available protocols, including, but not limited to, TCP / IP, SNA, IPX, etc. By way of example only, one or more networks 130 may be a satellite communications network, a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (including, for example, Bluetooth, WiFi), and / or any combination of these and other networks.
[0045] The system 100 may also include one or more databases 150. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 150 may be used to store information such as audio files and video files. The data repository 150 may reside in various locations. For example, the data repository used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. The data repository 150 may be of different types. In some embodiments, the data repository used by the server 120 may be a database, such as a relational database. One or more of these databases may store, update, and retrieve data to and from the database in response to a command.
[0046] In some embodiments, one or more of the databases 150 may also be used by an application to store application data. The databases used by the application may be different types of databases, such as a key-value store, an object store, or a conventional store backed by a file system.
[0047] The motor vehicle 110 may include a sensor 111 for sensing the surrounding environment. The sensor 111 may include one or more of the following sensors: a visual camera, an infrared camera, an ultrasonic sensor, a millimeter wave radar, and a laser radar (LiDAR). Different sensors may provide different detection accuracy and range. The camera may be installed at the front, rear, or other location of the vehicle. The visual camera may capture the situation inside and outside the vehicle in real time and present it to the driver and / or passengers. In addition, by analyzing the images captured by the visual camera, information such as traffic light indications, intersection conditions, and the operating status of other vehicles may be obtained. The infrared camera may capture objects under night vision conditions. Ultrasonic sensors may be installed around the vehicle to measure the distance of objects outside the vehicle from the vehicle using the characteristics of strong ultrasonic directionality. Millimeter wave radars may be installed at the front, rear, or other location of the vehicle to measure the distance of objects outside the vehicle from the vehicle using the characteristics of electromagnetic waves. Laser radars may be installed at the front, rear, or other location of the vehicle to detect object edges and shape information, thereby identifying and tracking objects. Due to the Doppler effect, the radar device may also measure the speed change of the vehicle and the moving object.
[0048] The motor vehicle 110 may also include a communication device 112. The communication device 112 may include a satellite positioning module capable of receiving satellite positioning signals (e.g., Beidou, GPS, GLONASS, and GALILEO) from satellites 141 and generating coordinates based on these signals. The communication device 112 may also include a module for communicating with a mobile communication base station 142, and the mobile communication network may implement any suitable communication technology, such as GSM / GPRS, CDMA, LTE, and other current or evolving wireless communication technologies (e.g., 5G technology). The communication device 112 may also have a vehicle-to-vehicle or vehicle-to-everything (V2X) module, which is configured to implement, for example, vehicle-to-vehicle (V2V) communication with other vehicles 143 and vehicle-to-infrastructure (V2I) communication with infrastructure 144. In addition, the communication device 112 may also have a module configured to communicate with a user terminal 145 (including but not limited to a smartphone, a tablet computer, or a wearable device such as a watch) by, for example, using a wireless LAN or Bluetooth of the IEEE802.11 standard. Using the communication device 112, the motor vehicle 110 may also access the server 120 via the network 130.
[0049] The motor vehicle 110 may also include a control device 113. The control device 113 may include a processor that communicates with various types of computer-readable storage devices or media, such as a central processing unit (CPU) or a graphics processing unit (GPU), or other dedicated processors. The control device 113 may include an automatic driving system for automatically controlling various actuators in the vehicle. The automatic driving system is configured to control the powertrain, steering system, and braking system of the motor vehicle 110 (not shown) via multiple actuators in response to inputs from multiple sensors 111 or other input devices to control acceleration, steering, and braking, respectively, without human intervention or limited human intervention. Some processing functions of the control device 113 can be implemented through cloud computing. For example, some processing can be performed using an on-board processor, while other processing can be performed using computing resources in the cloud. The control device 113 can be configured to perform a method according to the present disclosure. In addition, the control device 113 can be implemented as an example of a computing device on the motor vehicle side (client) according to the present disclosure.
[0050] Figure 1 The system 100 may be configured and operated in various ways to enable the application of various methods and apparatuses described in the present disclosure.
[0051] According to one aspect of the present disclosure, a method for training an autonomous driving model is provided. Figure 2 A schematic diagram of an autonomous driving model 200 according to an embodiment of the present disclosure is shown; and Figure 3 A flowchart of a method 300 for training an autonomous driving model according to an embodiment of the present disclosure is shown.
[0052] First reference Figure 2 The autonomous driving model 200 includes a multimodal coding layer 210 and a decision control layer 220. The multimodal coding layer 210 and the decision control layer 220 are connected to form an end-to-end neural network model, so that the decision control layer 220 directly predicts the autonomous driving strategy information based on the output of the multimodal coding layer 210.
[0053] As described above, in the related technology, predictions can be first made based on the perception information to obtain future prediction information, and the decision control layer can then make planning predictions based on the future prediction information. That is to say, the decision control layer 220 does not make planning predictions directly based on the perception information, but makes planning predictions directly based on the future prediction information. In the embodiment of the present application, the decision control layer 220 can directly predict the autonomous driving strategy information based on the output of the multimodal coding layer 210, and the multimodal coding layer 210 is used to encode and calculate the perception information, which is equivalent to the decision control layer 220 being able to plan directly based on the perception information and predict the autonomous driving strategy information. In other words, the training method in the embodiment of the present application can learn autonomous driving technology in which perception is directly responsible for decision-making.
[0054] The training method 300 of the autonomous driving model includes first training the multimodal encoding layer 210 and the decision control layer 220. Figure 3 As shown, the first training includes:
[0055] Step S310: acquiring first real driving data during the driving process of the vehicle, the first real driving data including first navigation information of the vehicle and first real perception information of the vehicle's surrounding environment, the first real perception information including current perception information and historical perception information of the vehicle's surrounding environment;
[0056] Step S320: acquiring first real autonomous driving strategy information corresponding to the first real driving data based on the first real perception information;
[0057] Step S330: inputting first sample input information including first real driving data into the multimodal coding layer to obtain a first sample implicit representation output by the multimodal coding layer;
[0058] Step S340, inputting the first intermediate sample input information including the implicit representation of the first sample into the decision control layer to obtain the first predicted autonomous driving strategy information output by the decision control layer; and
[0059] Step S350: Adjust the parameters of the multimodal coding layer and the decision control layer based on the first predicted autonomous driving strategy information and the first real autonomous driving strategy information.
[0060] In the example, the first training can be offline pre-training, that is, during the first training process, the autonomous driving model 200 is not deployed on a real vehicle traveling on a real road scene, but the model is trained using a large amount of collected real driving data to avoid the long learning process of cold start.
[0061] The first real driving data may include driving data collected during unmanned driving, and / or driving data collected by a human driver driving a vehicle with relevant sensors. Some of these driving data are not labeled with real autonomous driving strategy information, such as trajectory or control signals (such as throttle, brake, steering amplitude, etc.), and cannot be used for model training. In the embodiment of the present application, the corresponding autonomous driving strategy information is obtained based on the perception information obtained by the sensor in the driving data to achieve pseudo-labeling of the real autonomous driving strategy information corresponding to the driving data.
[0062] In the example, the autonomous driving model 200 can adopt a Transformer network structure with an encoder and a decoder. It is understandable that the autonomous driving model 200 can also be other neural network models based on the Transformer network structure, which is not limited here. The Transformer architecture can calculate the implicit representation of the model input and output through the self-attention mechanism. In other words, the Transformer architecture can be an Encoder-Decoder model built based on this self-attention mechanism.
[0063] In the example, the first navigation information In1 of the vehicle in the first real driving data may include vectorized navigation information and vectorized map information, and the vectorized navigation information and vectorized map information may be obtained by performing vectorization operations on one or more of lane-level or road-level navigation information and coarse positioning information.
[0064] In the example, the first real perception information of the vehicle's surrounding environment in the first real driving data (for example, it may include but is not limited to In2, In3 and In4, and the following content is described as an example of perception information including In2, In3 and In4) may include perception information In2 of one or more cameras on the vehicle, perception information In3 of one or more laser radars, and perception information In4 of one or more millimeter-wave radars. It is understandable that the perception information of the vehicle's surrounding environment is not limited to the above-mentioned form. For example, it may only include perception information In2 of multiple cameras, but not perception information In3 of one or more laser radars and perception information In4 of one or more millimeter-wave radars. The perception information In2 obtained by the camera may be perception information in the form of a picture or video, and the perception information In3 obtained by the laser radar may be perception information in the form of a radar point cloud (for example, a three-dimensional point cloud). In the example, the perception information includes the current perception information x of the target vehicle's surrounding environment during the vehicle's driving process. t The historical perception information x corresponding to multiple historical moments t-Δt Here, there may be a time span of a preset duration between t and Δt.
[0065] In the example, the multimodal encoding layer 210 can encode and calculate the first real driving data to generate a corresponding implicit representation. The implicit representation can be, for example, an implicit representation in a bird's-eye view (BEV) space. For example, the perception information of the camera can be first input into a shared backbone network (Backbone) to extract the data features of each camera. Then, the perception information of multiple cameras is fused and converted to the BEV space. Next, cross-modal fusion can be performed in the BEV space to fuse the pixel-level visual data and the lidar point cloud. Finally, temporal fusion is performed to form an implicit representation of the BEV space. t .
[0066] In one example, the Transformer Encoder structure that fuses spatiotemporal information can be used to realize the projection of multi-camera input information to the implicit representation of BEV space. For example, spatiotemporal information can be utilized through a BEV query mechanism (BEV queries) with pre-set parameters for grid division. The spatial cross-attention mechanism (i.e., the BEV query mechanism extracts the required spatial features from the multi-camera features through the attention mechanism) is used to allow the BEV query mechanism to extract features from the multi-camera perspectives of interest, thereby aggregating spatial information; in addition, the historical information is fused through the temporal self-attention mechanism (i.e., the BEV features generated at each moment obtain the required temporal information from the BEV features at the previous moment), thereby aggregating temporal information.
[0067] Accordingly, the decision control layer 220 is based on the implicit representation e of the input t , obtain first predicted autonomous driving strategy information. The first predicted autonomous driving strategy information may include, for example, a planned trajectory Out1 or a control signal Out2 for the vehicle (e.g., a signal for controlling the throttle, brake, steering amplitude, etc.). In an example, the decision control layer 220 may include a decoder in a Transformer.
[0068] Since the multimodal encoding layer 210 and the decision control layer 220 of the model to be trained are connected to form an end-to-end neural network model, the perception information in the sample input information (including the above-mentioned real driving data) can be directly responsible for the decision, which can solve the coupling problem between prediction and planning in the related technology. In addition, encoding and calculating the real driving data to obtain the corresponding implicit representation can overcome the problem that the algorithm is prone to failure due to the representation defects of structured information in the related technology. In addition, since the perception information in the sample input information can be directly responsible for the decision, the perception can capture the information that is more critical to the decision through training, reducing the error accumulation caused by perception errors in model training. Furthermore, since perception is directly responsible for the decision, the automatic driving technology that emphasizes perception and light maps is realized, which can overcome the problem of decision learning failure caused by untimely updates of high-precision maps and limited areas, and because it gets rid of the dependence on high-precision maps, it can save the update cost of high-precision maps.
[0069] In addition, since the first real autonomous driving strategy information is obtained based on the first real perception information, when the labeled real autonomous driving strategy information in the sample input information is limited, the first real autonomous driving strategy information corresponding to the first real driving data is obtained through the perception information in the sample input information, and the model training process is completed accordingly. In other words, when training the autonomous driving model, if there is no or only a small amount of vehicle trajectory data or control signal data for the vehicle in the sample data, the first real autonomous driving strategy information (such as pseudo-labeled trajectory data) can also be obtained based on the first real perception information, thereby completing the model training process. As a result, autonomous driving technology that emphasizes perception and neglects maps can be realized, and using a large amount of real driving data to train the autonomous driving model can ensure decision-making efficiency, so that the autonomous driving behavior can be well aligned with the preferences of human passengers, improve user experience and safety, and avoid the long learning process of cold start.
[0070] According to some embodiments, the above-mentioned step S320 may include: inputting the first real perception information into a driving strategy prediction model (not shown in the figure) to obtain the first real automatic driving strategy information output by the driving strategy prediction model.
[0071] In the example, the first real perception information (x1, ...., x t ) (such as sensor perception information) is input into the driving strategy prediction model to predict the corresponding trajectory planning (y1,....,y t ). The predicted trajectory planning (y1,....,y t ) can be used as the first real autonomous driving strategy information in the process of training the multimodal coding layer and the decision control layer to achieve pseudo-labeling of the first real driving data.
[0072] Since the first real perception information (such as images collected by a camera or point clouds acquired by a radar) includes current perception information and historical perception information, the autonomous driving strategy information (such as driving trajectory) of the autonomous driving vehicle is implicit between the current perception information and the historical perception information. Therefore, the driving strategy prediction model can be trained based on a small amount of data annotation (i.e., driving data with annotated real trajectories). t ) predicts the corresponding trajectory planning (y1,....,y t ), thereby labeling the first real perception information without trajectory annotation with a pseudo-annotated trajectory label.
[0073] It will be understood that the driving strategy prediction model can be a model independent of the autonomous driving model 200.
[0074] According to some embodiments, the autonomous driving model 200 may further include an evaluation feedback layer 230. And the first training of the multimodal encoding layer 210 and the decision control layer 220 may further include: implicitly representing the first sample as t The evaluation feedback layer 230 is input to obtain the first sample evaluation feedback information Out3 for the first predicted autonomous driving strategy information output by the evaluation feedback layer 230. In addition, the above step S350 may include: adjusting the parameters of the multimodal coding layer 210 and the decision control layer 220 based on the first sample evaluation feedback information Out3 for the first predicted autonomous driving strategy information, the first predicted autonomous driving strategy information, and the first real autonomous driving strategy information.
[0075] In an example, the rating feedback layer 230 may include a decoder in a Transformer.
[0076] Therefore, by introducing the evaluation feedback layer 230 into the autonomous driving model 200, it is possible to learn whether the current driving behavior comes from a human driver or a model, whether the current driving is comfortable, whether the current driving violates traffic rules, and whether the current driving is dangerous driving, thereby improving the user experience. In addition, when training the model, the multimodal coding layer 210 can be further trained through the evaluation feedback layer 230 on the basis of the decision control layer 220, so that the encoding of the multimodal coding layer 210 is more accurate, and the trained decision control layer 220 can predict more optimized autonomous driving strategy information.
[0077] In the example, the parameters of the multimodal encoding layer and the decision control layer can be adjusted by reinforcement learning. For example, based on the first predicted autonomous driving strategy information (y1, ...., y t ), the first real autonomous driving strategy information And the first sample evaluation feedback information (r1,....,r t ) for reinforcement learning.
[0078] In an example, a PPO algorithm or a SAC algorithm may be used to perform the reinforcement learning.
[0079] In the example, the objective function in equation (1) below can be used to adjust the parameters of the multimodal encoding layer and the decision control layer:
[0080]
[0081] Among them, A t can indicate the advantage function at time t, and A t The feedback information (r1, ..., r t ) is obtained. α can be a hyperparameter used to adjust the size of the loss value.
[0082] When conducting reinforcement learning training on a real vehicle, the autonomous driving model may need to predict some errors or failures, or even require the target vehicle to collide with surrounding obstacles to learn based on error or collision experience. However, due to cost and safety considerations, it is impossible to allow the autonomous vehicle to have a real collision during the actual driving process.
[0083] According to some embodiments, the first real driving data may further include a first intervention flag, which may indicate whether the first real autonomous driving strategy information is driving strategy information with human intervention. And the above step S350 may include: based on the first intervention flag (i1, ..., i T ), the first sample evaluation feedback information (r1,....,r t ), the first predicted autonomous driving strategy information (y1,....,y t ) and the first real autonomous driving strategy information The parameters of the multimodal coding layer 210 and the decision control layer 220 are adjusted.
[0084] During the automatic driving of the real vehicle, the safety officer / driver can intervene at any time in a critical moment to seize control of the autonomous driving vehicle and avoid the unacceptable model training costs caused by possible collisions during the driving of the real vehicle. After the crisis is over, the control is returned to the autonomous driving vehicle. The first intervention flag is used to characterize whether the first real autonomous driving strategy information is the autonomous driving strategy information with human intervention. In other words, by introducing the first intervention flag, the model can learn the autonomous driving strategy with the intervention of the safety officer, and the driving behavior learned by the model can be well aligned with the preferences of human passengers, improving user experience and safety. Human-in-the-loop reinforcement learning can gradually learn to continuously reduce the adverse situations caused by intervention. Through this mechanism, on the one hand, the efficiency of reinforcement learning can be improved, and on the other hand, the impact of disadvantageous experience on the learning process can be reduced, thereby further improving the robustness of the trained model.
[0085] Figure 4 1 shows a flowchart of a part of the process of the training method of the autonomous driving model according to an embodiment of the present disclosure. Figure 4 As shown, the training process of the evaluation feedback layer 230 may include:
[0086] Step S410: obtaining second sample input information and true evaluation feedback information for the second sample input information, wherein the sample input information includes navigation information of the sample vehicle and current perception information and historical perception information for the surrounding environment of the sample vehicle;
[0087] Step S420: inputting the second sample input information into the multimodal coding layer to obtain an implicit representation of the second sample output by the multimodal coding layer;
[0088] Step S430, inputting the implicit representation of the second sample into the evaluation feedback layer 230, so as to obtain the predicted evaluation feedback information output by the evaluation feedback layer 230 for the second sample input information; and
[0089] Step S440: Adjust parameters of the multimodal coding layer and the evaluation feedback layer 230 based on the real evaluation feedback information and the predicted evaluation feedback information.
[0090] The second sample input information may be collected by the autonomous driving vehicle during autonomous driving (e.g., L4 level autonomous driving) or during manual driving, or may be an input sample obtained in a simulation environment. For example, the second sample input information may include sensor (e.g., camera, radar) perception information, navigation information, and other information such as lane-level maps.
[0091] Real evaluation feedback information Evaluation feedback information that can be manually fed back (passenger or driver evaluation of the driving experience of the autonomous vehicle) can, for example, indicate whether the current driving behavior comes from a human driver or a model, whether the current driving is comfortable, whether the current driving violates traffic regulations, and whether the current driving is dangerous driving.
[0092] Accordingly, the predicted evaluation feedback information (r t ) is the prediction result output by the evaluation feedback layer 230.
[0093] In an example, the objective function in the following equation (2) may be used to adjust the parameters of the multimodal encoding layer 210 and the evaluation feedback layer 230:
[0094]
[0095] In the example, feedback modeling can be used to learn a function to estimate the evaluation feedback information. In other words, the model itself can estimate the expected benefit obtained by the current driving trajectory (i.e., the prediction result output by the evaluation feedback layer 230 mentioned above). For example, the following equation (3) can be used to determine (r t ):
[0096] r t =R(x t ,…,x t-l+1 ) Equation (3)
[0097] Among them, (x t ,…,x t-l+1 ) can be the second sample input information.
[0098] Figure 5 A flowchart of a part of the training method of the autonomous driving model according to an embodiment of the present disclosure is shown. According to some embodiments, the autonomous driving model 200 may also include a future prediction layer, and the first real perception information may include future perception information for the vehicle's surrounding environment. And, as Figure 5 As shown, the first training of the multimodal coding layer and the decision control layer may also include:
[0099] Step S510: acquiring future real information about the vehicle's surrounding environment based on the future perception information;
[0100] Step S520: inputting the implicit representation of the first sample into the future prediction layer to obtain future prediction information output by the future prediction layer; and
[0101] Step S530: adjusting parameters of the multimodal coding layer and the future prediction layer based on the future real information and the future prediction information.
[0102] Future Perception Informationx t+ΔtIt can be the current perception information x t The perception information after that, and there may be a time span of a preset time length between t and Δt. Accordingly, the future real information may include information with annotations (such as detection boxes) corresponding to the future perception information.
[0103] The future prediction layer can include the decoder in the Transformer.
[0104] Therefore, when performing model training, the multimodal coding layer 210 can be further trained through the future prediction layer on the basis of the decision control layer 220, so that the encoding of the multimodal coding layer 210 is more accurate, thereby enabling the decision control layer 220 to predict more optimized autonomous driving strategy information.
[0105] According to some embodiments, the future prediction layer and the decision control layer may share the same network structure, that is, the output of the network structure may include future prediction information and autonomous driving strategy information. Exemplarily, the future prediction layer and the decision control layer may share the same decoder in the Transformer.
[0106] According to some embodiments, the future prediction information may include at least one of the following: future prediction perception information about the vehicle's surrounding environment (e.g., sensor information at a certain point in the future); sensor information at a certain moment in the future includes camera input information or radar input information at a certain moment in the future); and an implicit representation of the future prediction corresponding to the future predicted perception information (For example, the implicit representation in the BEV space corresponding to the sensor information at a certain moment in the future).
[0107] According to some embodiments, further reference is made to Figure 2 The future prediction layer may include at least one of a first future prediction layer 240 and a second future prediction layer 250. The first future prediction layer 240 may be configured to implicitly predict the first sample of the input t Indicates output of future predicted perception information Out4 (e.g. sensor information at a certain moment in the future The sensor information at a certain moment in the future includes camera input information or radar input information at a certain moment in the future); the second future prediction layer 250 can be configured to implicitly represent e based on the first sample input t Output future prediction implicit representation Out5( For example, an implicit representation of the BEV space at some point in the future).
[0108] Figure 6A flowchart of a method 600 for training an autonomous driving model according to another embodiment of the present disclosure is shown. The method 600 for training an autonomous driving model includes the steps of first training the multimodal encoding layer 210 and the decision control layer 220 in the method 300 for training an autonomous driving model, and according to some embodiments, the method 600 for training an autonomous driving model may further include:
[0109] Step S610: obtaining second real driving data in a process of controlling the target vehicle to perform automatic driving using the automatic driving model obtained by the first training, wherein the second real driving data includes second navigation information of the target vehicle, second real perception information of the surrounding environment of the target vehicle, and second real automatic driving strategy information as well as
[0110] The second training of the autonomous driving model is performed based on the second real driving data. The second training includes the following steps S621 to S623:
[0111] Step S621: inputting the third sample input information including the second real driving data into the multimodal coding layer to obtain the third sample implicit representation output by the multimodal coding layer;
[0112] Step S622: input the second intermediate sample input information including the third sample implicit representation into the decision control layer to obtain the second predicted automatic driving strategy information (y1, ..., y t );as well as
[0113] Step S623: Based on the second predicted automatic driving strategy information (y1, ..., y t ) and the second real autonomous driving strategy information Adjust the parameters of the multimodal encoding layer and the decision control layer.
[0114] In the example, the first training may be an offline pre-training, that is, during the first training process, the autonomous driving model 200 is not deployed on a real vehicle traveling on a real road scene or a simulated vehicle traveling on a simulated road scene. Accordingly, the second training may be a training performed by obtaining second real driving data collected during the driving process of the vehicle controlled by the autonomous driving model obtained by the first training, that is, during the second training process, the autonomous driving model 200 is deployed on a real vehicle traveling on a real road scene or a simulated vehicle traveling on a simulated road scene.
[0115] During the second training process, the second navigation information of the target vehicle in the second real driving data may include vectorized navigation information and vectorized map information. The vectorized navigation information and vectorized map information may be obtained by vectorizing one or more of the lane-level or road-level navigation information and the coarse positioning information. The second real perception information may include the perception information of one or more cameras on the vehicle in a real road scene, the perception information of one or more laser radars, and the perception information of one or more millimeter-wave radars. It is understandable that the perception information of the target vehicle's surrounding environment is not limited to the above-mentioned form. For example, it may only include the perception information of multiple cameras, but not the perception information of one or more laser radars and the perception information of one or more millimeter-wave radars. The perception information obtained by the camera may be perception information in the form of pictures or videos, and the perception information obtained by the laser radar may be perception information in the form of radar point clouds (such as three-dimensional point clouds). Second real autonomous driving strategy information It may include the planned trajectory of the autonomous driving vehicle or control signals for the vehicle (such as signals for controlling the throttle, brakes, steering amplitude, etc.) collected in real road scenarios.
[0116] As a result, the autonomous driving model can be trained in real road scenarios, simulated road scenarios and offline pre-training scenarios, realizing training of massive data and multiple scenarios, thereby improving the efficiency of model training while further improving the accuracy of model training results.
[0117] According to some embodiments, when the autonomous driving model includes an evaluation feedback layer 230, performing a second training on the autonomous driving model based on the second real driving data may further include: inputting the third sample implicit representation into the evaluation feedback layer 230 to obtain the second sample evaluation feedback information (r1, ..., r2) output by the evaluation feedback layer 230 for the second predicted autonomous driving strategy information. t ). Wherein, based on the second sample evaluation feedback information (r1,....,r t ), the second predicted autonomous driving strategy information (y1,....,y t ) and the second real autonomous driving strategy information The parameters of the multimodal coding layer 210 and the decision control layer 220 are adjusted.
[0118] Second real autonomous driving strategy information It can be the trajectory data of manual driving. Accordingly, the second predicted automatic driving strategy information (y1,...,y t ) is the prediction result (trajectory planning) output by the decision control layer 220.
[0119] The second sample evaluation feedback information (r1,....,r t ) For example, it can indicate whether the current driving behavior comes from a human driver or a model, whether the current driving is comfortable, whether the current driving violates traffic regulations, and whether the current driving is dangerous driving.
[0120] Therefore, by further utilizing sample evaluation feedback information to coordinately adjust the parameters of the multimodal coding layer 210 and the decision control layer 220, the learning effect of the multimodal coding layer 210 and the decision control layer 220 can be further improved, thereby improving the user experience.
[0121] In the example, the parameters of the multimodal encoding layer and the decision control layer can be adjusted by reinforcement learning. For example, based on the second predicted autonomous driving strategy information (y1, ...., y t ), second real autonomous driving strategy information And the second sample evaluation feedback information (r1,....,r t ) for reinforcement learning.
[0122] In an example, a PPO algorithm or a SAC algorithm may be used to perform the reinforcement learning.
[0123] In an example, the objective function in the following equation (3) may be used to adjust the parameters of the multimodal coding layer 210 and the decision control layer 220:
[0124]
[0125] Among them, A t can indicate the advantage function at time t, and A t The feedback information (r1, ..., r t ) is obtained. α can be a hyperparameter used to adjust the size of the loss value.
[0126] According to some embodiments, when the autonomous driving model includes an evaluation feedback layer, the second real driving data may further include a second intervention identifier (i1, ..., i T ), the second intervention identifier (i1,....,i T ) can characterize whether the second real autonomous driving strategy information is the autonomous driving strategy information with human intervention. The second training of the autonomous driving model based on the second real driving data may also include: inputting the third sample implicit representation into the evaluation feedback layer to obtain the second sample evaluation feedback information output by the evaluation feedback layer for the second predicted autonomous driving strategy information. T), the second sample evaluation feedback information for the second predicted autonomous driving strategy information (r1,....,r t ), the second predicted autonomous driving strategy information (y1,....,y t ) and the second real autonomous driving strategy information The parameters of the multimodal coding layer 210 and the decision control layer 220 are adjusted.
[0127] In the example, the parameters of the multimodal encoding layer and the decision control layer can be adjusted by using feedback reinforcement learning and human-in-the-loop learning. For example, based on the second sample evaluation feedback information (r1, ..., r t ), the second intervention marker (i1,....,i t ), the second predicted autonomous driving strategy information (y1,....,y t ), second real autonomous driving strategy information And the second sample input information (x1,....,x t ) is used to learn the five-tuple data including .
[0128] Among them, when the second intervention mark (i1,....,i T ) is true, indicating that the autonomous driving vehicle is manually control and is no longer controlled by the control signal from the autonomous driving model; when the intervention flag (i1,....,i T ) is not true, indicating that the autonomous driving vehicle is controlled by the control signal sent by the autonomous driving model rather than by the human Control.
[0129] In an example, the objective function in the following equation (5) may be used to adjust the parameters of the multimodal encoding layer and the evaluation feedback layer:
[0130]
[0131] Where λ1 and λ2 can be hyperparameters indicating the weighting of the corresponding components. T ) is true value 1, and false value 0. t can indicate the advantage function at time t, and A t The feedback information (r1, ..., r t )get.
[0132] According to some other embodiments, when the autonomous driving model includes an evaluation feedback layer, the second real driving data may further include a second intervention identifier (i1, ..., i t), the second intervention identifier (i1,....,i t ) can characterize whether the second real autonomous driving strategy information is the autonomous driving strategy information with human intervention, and the second real driving data includes real evaluation feedback information for the second real autonomous driving strategy information. And the second training of the autonomous driving model based on the second real driving data can also include: inputting the third sample implicit representation into the evaluation feedback layer to obtain the second sample evaluation feedback information for the second predicted autonomous driving strategy information output by the evaluation feedback layer. And based on the second intervention identifier, the second sample evaluation feedback information and the real evaluation feedback information for the second predicted autonomous driving strategy information, as well as the second predicted autonomous driving strategy information and the second real autonomous driving strategy information, the parameters of the multimodal coding layer and the decision control layer are adjusted.
[0133] Real evaluation feedback information for the second real autonomous driving strategy information in the second real driving data It can be manual feedback evaluation feedback information (passenger or driver's evaluation of the driving experience of the autonomous vehicle), for example, it can indicate whether the current driving behavior comes from a human driver or a model, whether the current driving is comfortable, whether the current driving violates traffic regulations, and whether the current driving is dangerous driving.
[0134] In an example, the objective function as in equation (5) above may be used to adjust the parameters of the multimodal encoding layer 210 and the evaluation feedback layer 230 .
[0135] In addition, feedback modeling can be used to learn a function to estimate the evaluation feedback information. In other words, the model itself can estimate the expected benefit obtained by the current driving trajectory (i.e., the prediction result output by the evaluation feedback layer 230 mentioned above). For example, the above equation (3) can be used to determine (r t ). Then, the objective function as in the above equation (5) can be used to adjust the parameters of the multimodal coding layer 210 and the evaluation feedback layer 230.
[0136] It will be understood that the actual driving data during the driving process using the autonomous driving model may include the vehicle's navigation information, current perception information and historical perception information of the vehicle environment, actual autonomous driving strategy information, as well as evaluation feedback information and intervention identification, wherein the evaluation feedback information may be the evaluation feedback information of the safety officer / driver, or the evaluation feedback information predicted by the evaluation feedback layer.
[0137] According to some embodiments, the evaluation feedback information (e.g., evaluation feedback information, first sample evaluation feedback information, or second sample evaluation feedback information) may include at least one of the following: driving comfort information, driving safety information, driving efficiency, whether the driving lights are used in a civilized manner, driving behavior source information, and whether traffic rules are violated.
[0138] According to some embodiments, second real driving data of a target vehicle being controlled to perform autonomous driving by the autonomous driving model obtained by the first training may be obtained at preset time intervals, and the autonomous driving model may be subjected to a second training based on the newly obtained second real driving data.
[0139] The preset time may be, for example, half a day, a day, half a month, a month, etc., and may be set according to actual needs without limitation.
[0140] Thus, the autonomous driving model can be continuously iterated and trained based on driving data in real vehicle driving and / or simulation scenarios, so as to continuously optimize the autonomous driving model. It is understandable that while the second training is performed iteratively online, the autonomous driving model can also be trained offline based on the sample input information used in the first training.
[0141] According to some embodiments, the model training method may further include: after performing a second training on the autonomous driving model based on second real driving data, performing a first training again on the autonomous driving model that includes at least a multimodal encoding layer and a decision control layer.
[0142] In the example, offline pre-training can be performed first, and then online training can be performed. Based on the model obtained through online training, offline training and online training can be performed. Thus, the model can be continuously optimized through multiple iterative trainings.
[0143] According to some embodiments, obtaining second real driving data in a process in which the target vehicle is controlled to perform autonomous driving using the autonomous driving model obtained by the first training may include:
[0144] Acquire second real driving data in a process of controlling a target vehicle to perform automatic driving using the automatic driving model obtained by the first training in a real driving scenario, and / or
[0145] Acquire second real driving data in a process of controlling a target vehicle to perform autonomous driving using the autonomous driving model obtained through the first training in a simulated driving scenario.
[0146] In the example, the second real driving data obtained by executing the automatic driving in the real driving scene can be used as the main data, and the second real driving data obtained by executing the automatic driving in the simulated driving scene can be used as the auxiliary data. The simulation sample input information can be set in various ways as required, so as to use the simulation environment to mine more diverse long-tail samples and expand the richness of the training samples. In other words, the amount of real vehicle driving data used in the training process of the automatic driving model can be greater than the amount of simulated vehicle driving data.
[0147] It will be appreciated that both the offline pre-training stage and the online training stage may include training based on driving data in a simulation environment.
[0148] According to some embodiments, the autonomous driving strategy information may include a target planning trajectory.
[0149] According to another aspect of the present disclosure, an autonomous driving model is provided. The autonomous driving model may be trained using the model training method according to an embodiment of the present disclosure.
[0150] like Figure 2 As shown, the autonomous driving model 200 includes a multimodal coding layer 210 and a decision control layer 220, and the multimodal coding layer 210 and the decision control layer 220 are connected to form an end-to-end neural network model, so that the decision control layer 220 directly obtains the autonomous driving strategy information based on the output of the multimodal coding layer 210. The first input information of the multimodal coding layer 210 includes the navigation information of the target vehicle and the perception information of the surrounding environment of the target vehicle obtained by the sensor, and the perception information includes the current perception information and historical perception information of the surrounding environment of the target vehicle during the driving process of the vehicle. The multimodal coding layer 210 is configured to obtain the implicit representation corresponding to the first input information. The second input information of the decision control layer 220 includes the implicit representation, and the decision control layer is configured to obtain the target autonomous driving strategy information based on the second input information.
[0151] The details of the autonomous driving model 200 have been described in detail above, so they will not be repeated here. Since the multimodal encoding layer 210 and the decision control layer 220 of the model to be trained are connected to form an end-to-end neural network model, the perception information in the sample information can be directly responsible for the decision, which can solve the coupling problem between the prediction and planning of the trained autonomous driving model. In addition, the introduction of implicit representation can overcome the problem that the algorithm is prone to failure due to the representation defects of structured information. In addition, since the perception information in the sample information can be directly responsible for the decision, perception can capture information that is more critical to the decision, reducing the error accumulation caused by perception errors in the trained model. Furthermore, since perception is directly responsible for decision-making, an autonomous driving technology that emphasizes perception and neglects maps is realized, which can overcome the problem of decision failure caused by untimely updates of high-precision maps and limited areas. Since it is free from dependence on high-precision maps, the update cost of high-precision maps can be reduced.
[0152] According to some embodiments, the autonomous driving model 200 may further include a future prediction layer, which is configured to obtain future prediction information for the surrounding environment of the target vehicle based on an implicit representation corresponding to the input first input information.
[0153] According to some embodiments, further reference is made to Figure 2 The future prediction layer may include at least one of a first future prediction layer 240 and a second future prediction layer 250. The first future prediction layer 240 may be configured to implicitly predict the first sample of the input t Indicates output of future predicted perception information Out4 (e.g. sensor information at a certain moment in the future The sensor information at a certain moment in the future includes camera input information or radar input information at a certain moment in the future); the second future prediction layer 250 can be configured to implicitly represent e based on the first sample input t Output future prediction implicit representation Out5( For example, an implicit representation of the BEV space at some point in the future).
[0154] Exemplarily, the future prediction layer may include, but is not limited to, a decoder in a Transformer.
[0155] Figure 2 As shown in FIG. 1 , the future prediction layer and the decision control layer 220 are two independent network structures. It can be understood that the future prediction layer and the decision control layer can also share the same network structure, that is, the output of the network structure can include future prediction information and autonomous driving strategy information. Exemplarily, the future prediction layer and the decision control layer can share the same decoder in the Transformer.
[0156] According to some embodiments, the future prediction information may include at least one of the following: future prediction perception information about the vehicle's surrounding environment (e.g., sensor information at a certain point in the future); sensor information at a certain moment in the future includes camera input information or radar input information at a certain moment in the future); and an implicit representation of the future prediction corresponding to the future predicted perception information (For example, the implicit representation in the BEV space corresponding to the sensor information at a certain moment in the future).
[0157] According to another aspect of the present disclosure, an autonomous driving method implemented using an autonomous driving model is provided.
[0158] Figure 7 FIG. 7 is a flowchart of an automatic driving method 700 according to an embodiment of the present disclosure. Figure 7 As shown, the automatic driving method 700 includes:
[0159] Step S710, using the above-mentioned automatic driving model 200 to control the target vehicle to perform automatic driving; and
[0160] Step S720: Acquire real driving data during the autonomous driving process. The real driving data includes navigation information of the target vehicle, real perception information of the environment around the target vehicle, and real autonomous driving strategy information. The real driving data is used to iteratively train the autonomous driving model.
[0161] The navigation information of the target vehicle in the real driving data may include vectorized navigation information and vectorized map information, and the vectorized navigation information and vectorized map information may be obtained by vectorizing one or more of the lane-level or road-level navigation information and the coarse positioning information. The real perception information may include the perception information of one or more cameras on the vehicle in the real road scene, the perception information of one or more laser radars, and the perception information of one or more millimeter-wave radars. It is understandable that the perception information of the surrounding environment of the target vehicle is not limited to the above-mentioned one form, for example, it may only include the perception information of multiple cameras, but not the perception information of one or more laser radars and the perception information of one or more millimeter-wave radars. The perception information obtained by the camera may be the perception information in the form of pictures or videos, and the perception information obtained by the laser radar may be the perception information in the form of radar point clouds (e.g., three-dimensional point clouds). The real autonomous driving strategy information may include the planned trajectory of the autonomous driving vehicle collected in the real road scene or the control signal for the vehicle (e.g., the signal for controlling the throttle, brake, steering amplitude, etc.).
[0162] According to some embodiments of the present application, the autonomous driving strategy information (such as the planned trajectory) predicted by the autonomous driving model can be used to control the target vehicle to perform autonomous driving.
[0163] Figure 8 FIG. 8 is a flowchart of an autonomous driving method 800 according to another embodiment of the present disclosure. According to some embodiments, Figure 8 As shown, the autonomous driving method 800 includes steps S810 and S820 which are similar to steps S710 and S720 in the autonomous driving method 700 respectively:
[0164] Step S810, using the automatic driving model 200 obtained through iterative training to control the target vehicle to perform automatic driving; and
[0165] Step S820: acquiring real driving data during the autonomous driving process, where the real driving data includes navigation information of the target vehicle, real perception information of the surrounding environment of the target vehicle, and real autonomous driving strategy information, and the real driving data is used to iteratively train the autonomous driving model; and
[0166] Step S830, use the autonomous driving model obtained by iterative training to control the target vehicle to perform autonomous driving again. Thus, during the actual vehicle driving process, the autonomous driving task and the model training task can be performed simultaneously. In the example, the planned trajectory predicted by the autonomous driving model 200 or the control signal for the vehicle (such as the signal for controlling the throttle, brake, steering amplitude, etc.) can be used to control the target vehicle to perform autonomous driving again. For example, the control strategy module in the autonomous driving vehicle can be used to interpret the trajectory planning to obtain the control signal for the vehicle; or the neural network can be used to directly output the control signal for the vehicle based on the implicit representation.
[0167] According to some embodiments, in the autonomous driving method 700 or the autonomous driving method 800, real driving data of a process in which an autonomous driving model is used to control a target vehicle to perform autonomous driving can be obtained at preset time intervals, and the autonomous driving model can be iteratively trained based on the newly acquired real driving data.
[0168] According to some embodiments, in the autonomous driving method 700 or the autonomous driving method 800, the real driving data may include an intervention flag, and the intervention flag can characterize whether the real autonomous driving strategy information is autonomous driving strategy information with human intervention.
[0169] During the actual driving process, the safety officer can intervene at any time in critical moments to seize control of the autonomous driving vehicle, which can avoid possible collisions during the actual driving. After the crisis is over, the control will be returned to the autonomous driving vehicle. The intervention flag is used to characterize whether the real autonomous driving strategy information is the autonomous driving strategy information with human intervention. In other words, by introducing the intervention flag, the model can learn the autonomous driving strategy with the intervention of the safety officer, and the driving behavior learned by the model can be well aligned with the preferences of human passengers, improving user experience and safety. Human-in-the-loop reinforcement learning can gradually learn to reduce the adverse situations caused by intervention. Through this mechanism, on the one hand, the efficiency of reinforcement learning can be improved, and on the other hand, the impact of disadvantageous experience on the learning process can be reduced, thereby further improving the robustness of the trained model.
[0170] The training method of the autonomous driving model provided in the embodiments of the present application has the following advantages:
[0171] There is a lack of labeled data for autonomous driving. We use the existing L2+ and L4 models to quickly pre-train the model so that it can meet certain standards before being installed on the vehicle. After the vehicle is installed, it is further updated and iterated through continuous reinforcement learning. Compared with traditional reinforcement learning, this solution has the following advantages:
[0172] a. Risk-Free Learning. Unlike traditional reinforcement learning, which requires some high-cost behaviors to learn, HRL technology can learn to avoid collisions, violations and other risks completely risk-free under the protection of safety officers, so that the entire training process can be learned not only in a simulation environment, but also in a real environment at the same time.
[0173] b. It enables driving behavior to be well aligned with the preferences of human passengers, and can comprehensively consider issues such as efficiency, comfort, and safety to provide the best solution.
[0174] c. Extremely low training costs. Once the entire process is set up, the training and migration costs will be very low. Autonomous vehicles only need to collect road test data when migrating to different cities and regions.
[0175] d. Advantages of massive data + large models. Using massive data for pre-training can avoid the long cold start learning process, allowing us to make full use of larger models and achieve better results.
[0176] According to another aspect of the present disclosure, a training device for an autonomous driving model is provided. The autonomous driving model includes a multimodal coding layer and a decision control layer, and the multimodal coding layer and the decision control layer are connected to form an end-to-end neural network model, so that the decision control layer directly obtains autonomous driving strategy information based on the output of the multimodal coding layer.
[0177] Fig. 9 The structural block diagram of the training device 900 of the autonomous driving model according to an embodiment of the present disclosure is shown. The training device 900 of the autonomous driving model is configured to perform a first training on the multimodal encoding layer and the decision control layer, and includes:
[0178] A first real driving data acquisition unit 910 is configured to acquire first real driving data during the driving process of the vehicle, wherein the first real driving data includes first navigation information of the vehicle and first real perception information of the vehicle's surrounding environment, wherein the first real perception information includes current perception information and historical perception information of the vehicle's surrounding environment;
[0179] A real autonomous driving strategy information acquisition unit 920 is configured to acquire first real autonomous driving strategy information corresponding to the first real driving data based on the first real perception information;
[0180] The multimodal coding layer training unit 930 is configured to input the first sample input information including the first real driving data into the multimodal coding layer to obtain the first sample implicit representation output by the multimodal coding layer;
[0181] A decision control layer training unit 940 is configured to input first intermediate sample input information including the implicit representation of the first sample into the decision control layer to obtain first predicted autonomous driving strategy information output by the decision control layer; and
[0182] The parameter adjustment unit 950 is configured to adjust the parameters of the multimodal coding layer and the decision control layer based on the first predicted autonomous driving strategy information and the first real autonomous driving strategy information.
[0183] According to another aspect of the present disclosure, an automatic driving device based on an automatic driving model is provided.
[0184] Fig.10 FIG. 1 shows a structural block diagram of an automatic driving device 1000 according to an embodiment of the present disclosure. Fig.10 As shown, the automatic driving device 1000 includes:
[0185] A control unit 1010 is configured to control the target vehicle to perform autonomous driving using the autonomous driving model 200; and
[0186] The second real driving data acquisition unit 1020 is configured to acquire real driving data during the autonomous driving process, wherein the real driving data includes navigation information of the target vehicle, real perception information of the surrounding environment of the target vehicle, and real autonomous driving strategy information, and the real driving data is used to iteratively train the autonomous driving model.
[0187] It should be understood that Fig. 9 The various modules or units of the apparatus 900 shown in FIG. 9 can be used in conjunction with the reference Figure 3 Therefore, the operations, features and advantages described above for the method 300 are also applicable to the apparatus 900 and the modules and units included therein; and Fig.10 The various modules or units of the apparatus 1000 shown in FIG. 1 can be used in conjunction with the reference Figure 7 The steps in the method 700 described above correspond to each other. Therefore, the operations, features and advantages described above for the method 700 are also applicable to the device 1000 and the modules and units included therein. For the sake of brevity, some operations, features and advantages are not repeated here.
[0188] Although specific functionality is discussed above with reference to specific modules, it should be noted that the functionality of the various units discussed herein may be separated into multiple units, and / or at least some functionality of multiple units may be combined into a single unit.
[0189] It should also be understood that various techniques may be described herein in the general context of software hardware elements or program modules. Fig. 9 and Fig.10 The various units described can be implemented in hardware or in hardware in combination with software and / or firmware. For example, these units can be implemented as computer program codes / instructions, which are configured to be executed in one or more processors and stored in a computer-readable storage medium. Alternatively, these units can be implemented as hardware logic / circuits. For example, in some embodiments, one or more of units 910 to 950 and units 1010 to 1020 can be implemented together in a system on chip (System on Chip, SoC). SoC may include an integrated circuit chip (which includes a processor (e.g., a central processing unit (CPU), a microcontroller, a microprocessor, a digital signal processor (DSP), etc.), a memory, one or more communication interfaces, and / or one or more components in other circuits), and may optionally execute the received program code and / or include embedded firmware to perform functions.
[0190] According to another aspect of the present disclosure, an electronic device is also provided, including: at least one processor; and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the autonomous driving method or the training method of the autonomous driving model according to an embodiment of the present disclosure.
[0191] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is also provided, and the computer instructions are used to enable the computer to execute the autonomous driving method or the training method of the autonomous driving model according to the embodiment of the present disclosure.
[0192] According to another aspect of the present disclosure, a computer program product is also provided, including a computer program, wherein the computer program, when executed by a processor, implements the autonomous driving method or the training method of the autonomous driving model according to the embodiment of the present disclosure.
[0193] According to another aspect of the present disclosure, an autonomous driving vehicle is also provided, comprising a training device 900 of an autonomous driving model according to an embodiment of the present disclosure, an autonomous driving device 1000, and one of the above-mentioned electronic devices.
[0194] refer to Fig.11 , a block diagram of an electronic device 1100 that can be used as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0195] like Fig.11 As shown, the electronic device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 1108 into a random access memory (RAM) 1103. In the RAM 1103, various programs and data required for the operation of the electronic device 1100 can also be stored. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.
[0196] Multiple components in the electronic device 1100 are connected to the I / O interface 1105, including: an input unit 1106, an output unit 1107, a storage unit 1108, and a communication unit 1109. The input unit 1106 can be any type of device that can input information to the electronic device 1100. The input unit 1106 can receive input digital or character information, and generate key signal input related to user settings and / or function control of the electronic device, and can include but is not limited to a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone, and / or a remote controller. The output unit 1107 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 1108 can include but is not limited to a disk and an optical disk. The communication unit 1109 allows the electronic device 1100 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver and / or a chipset, such as a Bluetooth device, an 802.11 device, a WiFi device, a WiMax device, a cellular communication device and / or the like.
[0197] The computing unit 1101 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1101 performs the various methods and processes described above, such as methods (or processes) 300 to 800. For example, in some embodiments, the methods (or processes) 300 to 800 may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1108. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program is loaded into the RAM 1103 and executed by the computing unit 1101, one or more steps of the methods (or processes) 300 to 800 described above may be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to execute the methods (or processes) 300 to 800 in any other appropriate manner (eg, by means of firmware).
[0198] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0199] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0200] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0201] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0202] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.
[0203] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.
[0204] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0205] Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present invention is not limited by these embodiments or examples, but only by the claims after authorization and their equivalent scope. Various elements in the embodiments or examples can be omitted or replaced by their equivalent elements. In addition, each step can be performed in an order different from that described in the present disclosure. Further, the various elements in the embodiments or examples can be combined in various ways. It is important that with the evolution of technology, many elements described herein can be replaced by equivalent elements that appear after the present disclosure.
Claims
1. A training method for an autonomous driving model, the autonomous driving model comprising a multimodal coding layer and a decision control layer, the multimodal coding layer and the decision control layer being connected to form an end-to-end neural network model, so that the decision control layer directly predicts autonomous driving strategy information based on an output of the multimodal coding layer, the method comprising first training the multimodal coding layer and the decision control layer, in, The first training of the multimodal coding layer and the decision control layer includes: Acquire first real driving data during the driving of the vehicle, the first real driving data including first navigation information of the vehicle and first real perception information of the vehicle's surrounding environment, the first real perception information including current perception information and historical perception information of the vehicle's surrounding environment; Acquire first real autonomous driving strategy information corresponding to the first real driving data based on the first real perception information; Inputting first sample input information including the first real driving data into the multimodal coding layer to obtain a first sample implicit representation output by the multimodal coding layer that combines spatial and temporal information; Inputting first intermediate sample input information including the implicit representation of the first sample into the decision control layer to obtain first predicted autonomous driving strategy information output by the decision control layer; and Based on the first predicted autonomous driving strategy information and the first real autonomous driving strategy information, the parameters of the multimodal coding layer and the decision control layer are adjusted.
2. The method according to claim 1, wherein: The autonomous driving model further includes a future prediction layer, wherein the first real perception information includes future perception information of the vehicle's surrounding environment, The first training of the multimodal coding layer and the decision control layer further includes: Acquire future real information about the vehicle's surrounding environment based on the future perception information; Inputting the first sample implicit representation into the future prediction layer to obtain future prediction information output by the future prediction layer; and Based on the future real information and the future predicted information, parameters of the multimodal coding layer and the future prediction layer are adjusted.
3. The method according to claim 2, wherein: The future prediction information includes at least one of the following: Future predicted perception information for the vehicle's surroundings; and A future prediction implicit representation corresponding to the future prediction perceptual information.
4. The method according to claim 3, wherein: The future prediction layer includes at least one of a first future prediction layer and a second future prediction layer, The first future prediction layer is configured to output the future prediction perceptual information based on the input first sample implicit representation, and the second future prediction layer is configured to output the future prediction implicit representation based on the input first sample implicit representation.
5. The method according to any one of claims 2 to 4, wherein: The future prediction layer and the decision control layer share the same network structure.
6. The method according to any one of claims 1 to 4, wherein: Acquiring first real autonomous driving strategy information of the first real driving data based on the first real perception information includes: The first real perception information is input into a driving strategy prediction model to obtain first real automatic driving strategy information output by the driving strategy prediction model.
7. The method according to any one of claims 1 to 4, wherein: The autonomous driving model further includes an evaluation feedback layer, and the first training of the multimodal encoding layer and the decision control layer further includes: inputting the first sample implicit representation into the evaluation feedback layer to obtain first sample evaluation feedback information output by the evaluation feedback layer for the first predicted autonomous driving strategy information, Among them, based on the first sample evaluation feedback information for the first predicted autonomous driving strategy information, the first predicted autonomous driving strategy information and the first real autonomous driving strategy information, the parameters of the multimodal coding layer and the decision control layer are adjusted.
8. The method according to claim 7, wherein: The first real driving data further includes a first intervention flag, and the first intervention flag can indicate whether the first real autonomous driving strategy information is driving strategy information with human intervention. Among them, based on the first intervention identifier, the first sample evaluation feedback information for the first predicted autonomous driving strategy information, the first predicted autonomous driving strategy information and the first real autonomous driving strategy information, the parameters of the multimodal coding layer and the decision control layer are adjusted.
9. The method according to claim 7, wherein: The training process of the evaluation feedback layer includes: Acquire second sample input information and true evaluation feedback information for the second sample input information; Inputting second sample input information into the multimodal coding layer to obtain a second sample implicit representation output by the multimodal coding layer; Inputting the second sample implicit representation into the evaluation feedback layer to obtain predicted evaluation feedback information output by the evaluation feedback layer for the second sample input information; and Based on the real evaluation feedback information and the predicted evaluation feedback information, parameters of the multimodal coding layer and the evaluation feedback layer are adjusted.
10. The method according to any one of claims 1 to 4, further comprising: Acquire second real driving data in a process of controlling a target vehicle to perform automatic driving using the automatic driving model obtained by the first training, wherein the second real driving data includes second navigation information of the target vehicle, second real perception information of a surrounding environment of the target vehicle, and second real automatic driving strategy information; as well as Performing a second training on the autonomous driving model based on the second real driving data includes: Inputting third sample input information including the second real driving data into the multimodal coding layer to obtain a third sample implicit representation output by the multimodal coding layer; Inputting the second intermediate sample input information including the implicit representation of the third sample into the decision control layer to obtain the second predicted automatic driving strategy information output by the decision control layer; as well as Based on the second predicted autonomous driving strategy information and the second real autonomous driving strategy information, the parameters of the multimodal coding layer and the decision control layer are adjusted.
11. The method according to claim 10, wherein: When the autonomous driving model includes an evaluation feedback layer, performing a second training on the autonomous driving model based on the second real driving data further includes: inputting the third sample implicit representation into the evaluation feedback layer to obtain second sample evaluation feedback information output by the evaluation feedback layer for the second predicted autonomous driving strategy information, Among them, based on the second sample evaluation feedback information for the second predicted autonomous driving strategy information, the second predicted autonomous driving strategy information and the second real autonomous driving strategy information, the parameters of the multimodal coding layer and the decision control layer are adjusted.
12. The method according to claim 10, wherein: When the autonomous driving model includes an evaluation feedback layer, the second real driving data further includes a second intervention flag, and the second intervention flag can indicate whether the second real autonomous driving strategy information is autonomous driving strategy information with human intervention. The performing a second training on the automatic driving model based on the second real driving data further includes: inputting the third sample implicit representation into the evaluation feedback layer to obtain second sample evaluation feedback information output by the evaluation feedback layer for the second predicted autonomous driving strategy information, Among them, based on the second intervention identifier, the second sample evaluation feedback information for the second predicted autonomous driving strategy information, the second predicted autonomous driving strategy information and the second real autonomous driving strategy information, the parameters of the multimodal coding layer and the decision control layer are adjusted.
13. The method according to claim 10, wherein: When the autonomous driving model includes an evaluation feedback layer, the second real driving data further includes a second intervention flag, and the second intervention flag can indicate whether the second real autonomous driving strategy information is autonomous driving strategy information with human intervention, and the second real driving data includes real evaluation feedback information for the second real autonomous driving strategy information. The performing a second training on the automatic driving model based on the second real driving data further includes: inputting the third sample implicit representation into the evaluation feedback layer to obtain second sample evaluation feedback information output by the evaluation feedback layer for the second predicted autonomous driving strategy information, Among them, based on the second intervention identifier, the second sample evaluation feedback information and the real evaluation feedback information for the second predicted autonomous driving strategy information, and the second predicted autonomous driving strategy information and the second real autonomous driving strategy information, the parameters of the multimodal coding layer and the decision control layer are adjusted.
14. The method according to claim 13, wherein: The evaluation feedback information includes at least one of the following: Driving comfort information, driving safety information, driving efficiency, whether the driving lights are used in a civilized manner, information on the source of driving behavior, and whether traffic rules are violated.
15. The method according to claim 10, wherein: At preset time intervals, second real driving data of the target vehicle being controlled to perform autonomous driving by the autonomous driving model obtained by the first training is obtained, and the autonomous driving model is subjected to a second training based on the newly obtained second real driving data.
16. The method according to claim 10, further comprising: After the autonomous driving model is trained for the second time based on the second real driving data, the autonomous driving model including at least the multimodal encoding layer and the decision control layer is trained for the first time again.
17. The method according to claim 10, wherein obtaining second real driving data in a process in which the target vehicle is controlled to perform autonomous driving using the autonomous driving model obtained by the first training comprises: Acquire second real driving data in a process of controlling a target vehicle to perform automatic driving using the automatic driving model obtained by the first training in a real driving scenario, and / or Acquire second real driving data in a process of controlling a target vehicle to perform autonomous driving using the autonomous driving model obtained through the first training in a simulated driving scenario.
18. The method according to any one of claims 1 to 4, wherein: The autonomous driving strategy information includes the target planning trajectory.
19. An autonomous driving model trained by the training method according to any one of claims 1 to 18, comprising a multimodal coding layer and a decision control layer, wherein the multimodal coding layer and the decision control layer are connected to form an end-to-end neural network model, so that the decision control layer directly predicts autonomous driving strategy information based on the output of the multimodal coding layer, in, The first input information of the multimodal coding layer includes navigation information of the target vehicle and perception information of the surrounding environment of the target vehicle obtained by using a sensor, wherein the perception information includes current perception information and historical perception information of the surrounding environment of the target vehicle during the driving process of the vehicle, and the multimodal coding layer is configured to obtain an implicit representation corresponding to the first input information, The second input information of the decision control layer includes the implicit representation, and the decision control layer is configured to obtain target autonomous driving strategy information based on the second input information.
20. An autonomous driving method implemented using an autonomous driving model, comprising: Using the autonomous driving model described in claim 19 to control the target vehicle to perform autonomous driving; as well as Acquire real driving data during the autonomous driving process, wherein the real driving data includes navigation information of the target vehicle, real perception information of the target vehicle's surrounding environment, and real autonomous driving strategy information, and the real driving data is used to iteratively train the autonomous driving model.
21. The method according to claim 20, further comprising: The autonomous driving model obtained by iterative training is used to control the target vehicle to perform autonomous driving again.
22. The method according to claim 20 or 21, wherein: At preset time intervals, real driving data of a target vehicle being controlled by an autonomous driving model during the autonomous driving process is acquired, and the autonomous driving model is iteratively trained based on the newly acquired real driving data.
23. The method according to claim 20 or 21, wherein: The real driving data includes an intervention flag, and the intervention flag can indicate whether the real autonomous driving strategy information is autonomous driving strategy information with human intervention.
24. A training device for an autonomous driving model, the autonomous driving model comprising a multimodal coding layer and a decision control layer, the multimodal coding layer and the decision control layer being connected to form an end-to-end neural network model, so that the decision control layer directly predicts autonomous driving strategy information based on an output of the multimodal coding layer, the device being configured to perform a first training on the multimodal coding layer and the decision control layer, and comprising: A first real driving data acquisition unit is configured to acquire first real driving data during the driving process of the vehicle, wherein the first real driving data includes first navigation information of the vehicle and first real perception information of the vehicle's surrounding environment, wherein the first real perception information includes current perception information and historical perception information of the vehicle's surrounding environment; a real autonomous driving strategy information acquisition unit, configured to acquire first real autonomous driving strategy information corresponding to the first real driving data based on the first real perception information; a multimodal coding layer training unit, configured to input first sample input information including the first real driving data into the multimodal coding layer, so as to obtain a first sample implicit representation output by the multimodal coding layer that combines spatial and temporal information; a decision control layer training unit, configured to input first intermediate sample input information including the implicit representation of the first sample into the decision control layer to obtain first predicted autonomous driving strategy information output by the decision control layer; as well as A parameter adjustment unit is configured to adjust parameters of the multimodal coding layer and the decision control layer based on the first predicted autonomous driving strategy information and the first real autonomous driving strategy information.
25. An automatic driving device based on an automatic driving model, comprising: A control unit configured to control the target vehicle to perform autonomous driving using the autonomous driving model according to claim 19; as well as The second real driving data acquisition unit is configured to acquire real driving data during the autonomous driving process, wherein the real driving data includes navigation information of the target vehicle, real perception information of the surrounding environment of the target vehicle, and real autonomous driving strategy information, and the real driving data is used to iteratively train the autonomous driving model.
26. An electronic device comprising: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-18 or 20-23.
27. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-18 or 20-23.
28. A computer program product comprising a computer program, wherein: The computer program implements the method of any one of claims 1-18 or 20-23 when executed by a processor.
29. An autonomous driving vehicle comprising: One of a training device for an autonomous driving model according to claim 24, an autonomous driving device according to claim 25, and an electronic device according to claim 26.
