A safe and efficient evolutionary method and device for autonomous driving with deep integration of data mechanisms

Abstract

The present application relates to the field of autonomous driving technology, and more specifically, to a method and device for the safe and efficient evolution of autonomous driving with deep fusion of data mechanisms, which can, to a certain extent, solve the problem that a large amount of expert data is still needed for support, and there is a hidden concern that safety is difficult to guarantee, which limits its application in the field of autonomous driving. The method includes: designing an explicit space planning architecture to calculate and output an optimized autonomous driving strategy; mapping explicit space state quantities to latent space state quantities through a representation mapping model; using the latent space state quantities for planning calculations, policy network learning, and potential dynamics network learning; and performing value network alignment and policy alignment through explicit space-latent space fusion planning, thereby outputting the current optimal autonomous driving strategy after fusion.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and more specifically, to a safe and efficient evolutionary method and device for autonomous driving with deep integration of data mechanisms. Background Art

[0002] Faced with a complex and ever-changing external environment, high-level autonomous driving vehicles are required to have an extremely high level of intelligence. Having safe and efficient self-evolution capabilities is crucial to promoting the development of high-level autonomous driving technology. This capability can enable the system to continuously improve its performance level, enabling it to cope with various challenges and meet the needs of autonomous driving in different scenarios.

[0003] Most of the current autonomous driving decision-making and control algorithms rely mainly on optimization and rules. Although these algorithms make full use of expert experience and domain knowledge, artificially defined rules are difficult to flexibly respond to various highly complex external environments, which hinders the realization of truly high-level autonomous driving. At the same time, methods based on generative deep learning large models have demonstrated the potential for achieving general artificial intelligence.

[0004] However, these methods still require the support of a large amount of expert data, and there is a hidden concern that safety is difficult to guarantee, which limits their application in the field of autonomous driving. The current trend of technological development shows that methods that combine expert experience knowledge with generative deep learning models will provide new possibilities for achieving breakthroughs in high-level autonomous driving technology. This fusion approach is expected to combine the advantages of traditional methods and deep learning large model methods, opening up new ideas and directions for the further development of autonomous driving technology. Summary of the invention

[0005] In order to solve the problem that the existing technology still requires a large amount of expert data support and there is a hidden concern that safety is difficult to guarantee, which limits its application in the field of autonomous driving, the present application provides a safe and efficient evolution method and device for autonomous driving with deep integration of data mechanisms.

[0006] The embodiment of the present application is implemented as follows:

[0007] In a first aspect, the present application provides a method and device for safe and efficient evolution of autonomous driving with deep integration of data mechanisms, including:

[0008] Design an explicit spatial planning architecture to calculate and output optimization-based autonomous driving strategies;

[0009] By characterizing the mapping model, the spatial state quantity Mapping to latent space state quantity ;

[0010] The latent space state quantity Used for planning calculations, policy network learning, and potential dynamics network learning;

[0011] Through explicit space-latent space fusion planning, value network alignment and strategy alignment are performed to output the current optimal autonomous driving strategy after fusion.

[0012] In a possible implementation, after the value network alignment and strategy alignment are performed through the explicit space-latent space fusion planning module to output the fused current optimal autonomous driving strategy steps, it also includes recording experience to update the neural network and further perform strategy learning.

[0013] In one possible implementation, the spatial state quantity It is the state space with actual physical meaning that is input into the autonomous driving algorithm during its actual operation, and it is related to the algorithm details.

[0014] In one possible implementation, an explicit spatial planning architecture is designed to calculate and output an optimization-based autonomous driving strategy, further comprising:

[0015] Construct nonlinear programming problems and design cost functions;

[0016] Constraints are used to indicate state violations that should not occur in explicit space planning, and the constraints include vehicle kinematic constraints, control increment constraints, collision constraints, and safe following constraints;

[0017] The nonlinear optimization problem is constructed in the optimization component, and the optimization-based autonomous driving strategy is obtained and output .

[0018] In a possible implementation, the representation mapping model is used to represent the state quantity from the explicit space To the latent space state quantity The mapping relationship between them.

[0019] In a possible implementation, the explicit space-latent space fusion planning includes value estimation fusion and output strategy alignment:

[0020] Through the value estimation fusion, a mechanism-based cost function that is relatively accurate but has limited performance can be combined with a deep neural network-based value function that has good final performance but is not accurate at the beginning, thereby leveraging the advantages of both the data approach and the mechanism approach.

[0021] Through the output strategy alignment, the security of the output strategy can be guaranteed, and the data-driven method can be prevented from outputting unreasonable strategies.

[0022] In one possible implementation, when planning in the latent space, the strategy parameters are updated according to the latent dynamics model, the value model, and the reward model;

[0023] Potential dynamics model Defined as:

[0024] ;

[0025] Reward Model Defined as:

[0026] ;

[0027] Value Model Defined as:

[0028] ;

[0029] Policy Network Defined as:

[0030] ;

[0031] in, is the potential state quantity at time i+1;

[0032] Through the potential dynamics model , value model and reward model , calculate the total return of the sampled trajectory in the latent space planning process for:

[0033] ;

[0034] Among them, H is the shallow spatial prediction time domain, For the alignment strategy, is the value discount factor.

[0035] In a possible implementation, in the method of using a potential dynamics model , value model and reward model , calculate the total return of the sampled trajectory in the latent space planning process After the steps, also include:

[0036] The cost function in spatial planning and Combined and through parameters Adjust weight:

[0037] ;

[0038] Among them, the parameters Adjusts linearly as training progresses;

[0039] set up is the strategy output obtained by latent space planning, which obeys the normal distribution ;

[0040] Where I is the identity matrix;

[0041] Sort the rewards obtained from the sampled trajectories, select the top k rewards and iterate, and get the new parameters after the iteration. and :

[0042] ;

[0043] ;

[0044] in, Controls the sharpness of the weights. After a fixed number of iterations, the planning process terminates and samples trajectories from the final normalized distribution of rewards over the action sequence. represents the sampling trajectory under continuous a action;

[0045] Will , and Strategy Network Output Relative alignment, alignment strategy :

[0046] ;

[0047] Among them, the parameters is the weight coefficient, which is dynamically adjusted according to the collision risk situation. The collision risk is determined by the relative Euler distance between the autonomous vehicle and the surrounding traffic vehicles. calculate;

[0048] The planning process will predict the time domain N in the explicit space p The above process is repeated until N p Step Alignment Strategy The first action in a sequence of actions chosen as the plan.

[0049] In a possible implementation, the mapping model , explicit dynamics network, latent dynamics model and reward model All of them are trained by supervised learning based on the empirical data obtained from exploration;

[0050] The value network is trained as follows:

[0051] ;

[0052] in For the strategy network, is the value discount factor. In order to alleviate the problem of overestimation of the value network, the target value network is defined ,in The parameters of are not updated in real time during the training process, but are updated at fixed intervals. Updated to ;

[0053] The policy network updates parameters by maximizing the value, and its formula is:

[0054] ;

[0055] Among them, sg() is the stop gradient return operator, Indicates that the input of the potential state at time i will not bring about the gradient update of the entire loss. is the strategy discount factor, Indicates the trajectory.

[0056] In the second aspect, the present application provides a safe and efficient evolution device for autonomous driving with deep integration of data mechanisms, including:

[0057] A design module for designing an explicit spatial planning architecture to calculate and output optimization-based autonomous driving strategies;

[0058] The mapping module is used to represent the spatial state quantity through the mapping model. Mapping to latent space state quantity ;

[0059] A learning module is used to transform the latent space state quantity Used for planning calculations, policy network learning, and potential dynamics network learning;

[0060] The output module is used to perform value network alignment and strategy alignment through explicit space-latent space fusion planning, thereby outputting the current optimal autonomous driving strategy after fusion.

[0061] The technical solution provided by this application can at least achieve the following beneficial effects:

[0062] The safe and efficient evolution method for autonomous driving with deep integration of data mechanisms provided in this application combines the explicit space planning results with the latent space planning results by combining expert experience knowledge with a generative deep learning large model, so that the autonomous driving algorithm can safely and efficiently evolve itself and improve its intelligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0064] Figure 1 It is a flowchart of a safe and efficient evolution method for autonomous driving with deep integration of data mechanisms shown in an exemplary embodiment of the present application;

[0065] Figure 2 It is a flow chart of a safe and efficient evolutionary model of autonomous driving with deep integration of data mechanisms shown in an exemplary embodiment of the present application;

[0066] Figure 3 is a flow chart of a visible space-latent space fusion planning module shown in an exemplary embodiment of the present application;

[0067] Figure 4 is a flowchart of a strategy alignment process in a typical autonomous driving scenario shown in an exemplary embodiment of the present application;

[0068] Figure 5 It is a structural diagram of a safe and efficient evolution device for autonomous driving with deep integration of data mechanisms shown in an exemplary embodiment of the present application.

[0069] Reference numerals:

[0070] 1. Design module; 2. Mapping module; 3. Learning module 4. Output module. DETAILED DESCRIPTION

[0071] In order to make the purpose, implementation mode and advantages of the present application clearer, the exemplary implementation mode of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, not all of the embodiments. It should be understood that the specific embodiments described here are only used to explain the present application and are not used to limit the present application.

[0072] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their ordinary and common meanings.

[0073] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0074] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0075] Before explaining the safe and efficient evolution method of autonomous driving with deep integration of data mechanisms provided in the embodiment of the present application, the application scenarios and implementation environment of the embodiment of the present application are first introduced.

[0076] The present application relates to the field of autonomous driving technology, and more specifically, to a safe and efficient evolutionary method and device for autonomous driving with deep integration of data mechanisms.

[0077] Next, the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems will be described in detail through embodiments and in combination with the accompanying drawings. The embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments.

[0078] Figure 1 It is a flowchart of a safe and efficient evolution method for autonomous driving with deep integration of data mechanisms shown in an exemplary embodiment of the present application.

[0079] In an exemplary embodiment, Figure 1 As shown, a safe and efficient evolution method for autonomous driving with deep fusion of data mechanisms is provided. In this embodiment, the method may include the following steps:

[0080] Step 100: Design an explicit spatial planning architecture to calculate and output an optimization-based autonomous driving strategy.

[0081] Step 200: By characterizing the mapping model, the spatial state quantity Mapping to latent space state quantity .

[0082] Step 300: The latent space state quantity Used for planning calculations, policy network learning, and latent dynamics network learning.

[0083] Step 400: Through explicit space-latent space fusion planning, value network alignment and strategy alignment are performed to output the current optimal autonomous driving strategy after fusion.

[0084] Figure 2 It is a flow chart of a safe and efficient evolutionary model for autonomous driving with deep integration of data mechanisms shown in an exemplary embodiment of the present application.

[0085] In one possible implementation, Figure 2 As shown in the figure, the safe and efficient evolutionary model of autonomous driving with deep integration of data mechanism is as follows: first, the explicit space planning architecture is designed to calculate and output the optimized autonomous driving strategy; then, the explicit space state quantity is mapped to the latent space state quantity through the representation mapping model; then, the latent space state quantity is used for planning calculation, policy network learning and potential dynamics network learning; finally, the value network alignment and policy alignment are performed through the explicit space-latent space fusion planning module, so as to output the current optimal autonomous driving strategy after integration, and record the experience to update the neural network for further policy learning.

[0086] Figure 3 is a flow chart of a visible space-latent space fusion planning module shown in an exemplary embodiment of the present application, Figure 4 It is a flowchart of a strategy alignment process in a typical autonomous driving scenario shown in an exemplary embodiment of the present application.

[0087] In one possible implementation, Figure 3 and Figure 4 As shown in the figure, the specific implementation steps of the safe and efficient evolutionary method of autonomous driving with deep integration of data mechanisms are as follows:

[0088] 1. Display space planning framework

[0089] Explicit space state It is the state space with actual physical meaning that is input by the autonomous driving algorithm during actual operation. This definition is related to the details of the algorithm. An embodiment not limited to this is:

[0090] .

[0091] Where n is the number of traffic vehicles within the observation range; x e ,y e are the horizontal and vertical coordinates of the vehicle in the global coordinate system respectively; is the vehicle's heading angle, v e is the speed of the vehicle; x i ,y i are the horizontal and vertical coordinates of the pericycle in the global coordinate system (i=1,…,n), is the direction of the chariot (i=1,…,n), v i is the speed of the surrounding vehicle (i=1,…,n).

[0092] The autonomous driving system is required to reach the destination safely and comfortably with the highest possible efficiency. Based on this goal, a nonlinear programming problem is constructed and the cost function is designed as follows:

[0093] .

[0094] Among them, Nc is the control time domain, Np is the explicit space prediction time domain, is the terminal posture after Np time domain, is the predicted trajectory of the vehicle in the Np time domain, is the number of traffic vehicles, is the control action output in the Np time domain, is the control action output increment in the Np time domain, It is obtained through a state machine designed based on expert knowledge, that is, when there is no traffic vehicle in the target lane and the lane change conditions are met, the corresponding position of the target lane is set to the terminal posture after Np time domain, is the predicted trajectory of the traffic vehicle in the Np time domain, and its result is output by the explicit dynamics network.

[0095] Constraints are used to indicate state violations that should not occur in explicit space planning. There are four types of constraints, including vehicle kinematic constraints, control increment constraints, collision constraints, and safe following constraints.

[0096] The constraint equations are defined as follows:

[0097] .

[0098] in, is a mechanism-based vehicle dynamics model. is the vehicle dynamics model state, and To control the upper and lower limits of the increment, is the predicted y coordinate value of the vehicle in front of this lane in the Np time domain, To ensure safe following distance, is the safety radius.

[0099] The above nonlinear optimization problem is constructed in the optimization component. By solving the above problem, the optimization-based autonomous driving strategy is obtained and output .

[0100] 2. Representation Mapping Model

[0101] The mapping model is used to characterize the state quantity from the explicit space To the latent space state quantity The mapping relationship between them can be expressed as:

[0102] .

[0103] 3. Planning framework for the integration of visible space and latent space

[0104] The process of explicit space-latent space fusion planning mainly includes two parts: value estimation fusion and output strategy alignment.

[0105] Through value estimation fusion, a mechanism-based cost function that is relatively accurate but has limited performance can be combined with a deep neural network-based value function that ultimately has good performance but is not accurate at the beginning, thereby leveraging the advantages of both data methods and mechanism methods.

[0106] By aligning the output strategies, we can ensure the security of the output strategies and prevent the data-driven methods from outputting unreasonable strategies.

[0107] When planning in the latent space, the strategy parameters are updated according to the latent dynamics model, value model and reward model.

[0108] Potential dynamics model is defined as:

[0109] .

[0110] Reward Model is defined as:

[0111] .

[0112] Value Model is defined as:

[0113] .

[0114] Policy Network is defined as:

[0115] .

[0116] in, is the potential state quantity at time i+1.

[0117] Through the potential dynamics model , value model and reward model , calculate the total return of the sampled trajectory in the latent space planning process for:

[0118] ,

[0119] Among them, H is the shallow spatial prediction time domain, For the alignment strategy, is the value discount factor.

[0120] In the planning process of the latent space, all the models involved are in the training process, so at the beginning of self-evolution, the above returns is not accurate. At this time, the cost function in spatial planning will be displayed and Combined and through parameters Adjust weight:

[0121] .

[0122] Among them, the parameters Adjusts linearly as training progresses.

[0123] set up is the strategy output obtained by latent space planning. The strategy obeys the normal distribution .

[0124] Where I is the identity matrix.

[0125] Sort the rewards obtained from the sampled trajectories, select the top k rewards and iterate, and get the new parameters after the iteration. and :

[0126] ;

[0127] .

[0128] in, Controls the sharpness of the weights. After a fixed number of iterations, the planning process terminates and samples trajectories from the final normalized distribution of rewards over the action sequence. Represents the sampling trajectory under continuous a action.

[0129] Will , and Strategy Network Output Align to get the alignment strategy :

[0130] .

[0131] Among them, the parameters is a weight coefficient that is dynamically adjusted according to the collision risk. The collision risk is determined by the relative Euler distance between the autonomous vehicle and the surrounding traffic vehicles. calculate.

[0132] The planning process will be carried out in the explicit space prediction domain Np, and the above process will be repeated until Np steps are aligned. The first action in a sequence of actions chosen as the plan.

[0133] 4. Model training

[0134] The mapping model described in the present invention , explicit dynamics network, latent dynamics model and reward model All of them are trained through supervised learning based on the empirical data obtained from exploration.

[0135] The value network described in the present invention is trained by the following formula:

[0136] .

[0137] in For the strategy network, is the value discount factor. In order to alleviate the problem of overestimation of the value network, the target value network is defined ,in The parameters of are not updated in real time during the training process, but are updated at fixed intervals. Updated to .

[0138] The policy network updates parameters by maximizing the value, and its formula is:

[0139] .

[0140] Among them, sg() is the stop gradient return operator, Indicates that the input of the potential state at time i will not bring about the gradient update of the entire loss. is the strategy discount factor, Indicates the trajectory.

[0141] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the instructions, these steps are not necessarily executed in sequence according to the order of the instructions. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the above-mentioned embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the steps or stages in other steps.

[0142] Corresponding to the embodiment of the aforementioned method for safe and efficient evolution of autonomous driving with deep integration of data mechanisms, adopting the same technical concept, the present application also provides an embodiment of a device for safe and efficient evolution of autonomous driving with deep integration of data mechanisms.

[0143] Figure 5 It is a structural diagram of a safe and efficient evolution device for autonomous driving with deep integration of data mechanisms shown in an exemplary embodiment of the present application.

[0144] In an exemplary embodiment, Figure 5 As shown in the figure, the safe and efficient evolution device for autonomous driving with deep integration of data mechanisms includes:

[0145] Design module 1, used to design the explicit space planning architecture to calculate and output the optimization-based autonomous driving strategy;

[0146] Mapping module 2 is used to represent the spatial state quantity by mapping the model Mapping to latent space state quantity ;

[0147] Learning module 3, used to transform the latent space state quantity Used for planning calculations, policy network learning, and potential dynamics network learning;

[0148] The output module 4 is used to perform value network alignment and strategy alignment through explicit space-latent space fusion planning, so as to output the current optimal autonomous driving strategy after fusion.

[0149] For the specific definition of the safe and efficient evolution device for autonomous driving with deep integration of data mechanisms, please refer to the definition of the safe and efficient evolution method for autonomous driving with deep integration of data mechanisms above, which will not be repeated here. Each module in the above-mentioned safe and efficient evolution device for autonomous driving with deep integration of data mechanisms can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0150] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0151] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A safe and efficient evolutionary method for autonomous driving with deep integration of data and mechanism, characterized by: include: Design a spatial planning architecture to calculate and output the optimization-based autonomous driving strategy and the spatial state quantity s t It is the state space with actual physical meaning that is input into the autonomous driving algorithm during its actual operation. This definition is related to the details of the algorithm and can be expressed as: Where n is the number of traffic vehicles within the observation range; x e ,y e are the horizontal and vertical coordinates of the vehicle in the global coordinate system respectively; is the vehicle's heading angle, v e is the speed of the vehicle; x i ,y i are the horizontal and vertical coordinates of the pericycle in the global coordinate system (i=1,…,n), is the direction of the chariot (i=1,…,n), v i is the speed of the surrounding vehicle (i=1,…,n); By characterizing the mapping model, the explicit space state quantity s t Mapping to the latent space state z t The representation mapping model is used to represent the state quantity s from the explicit space t To the latent space state z t The mapping relationship between them can be expressed as: z t =h θ (s t ); The latent space state quantity z t Used for planning calculations, policy network learning, and potential dynamics network learning; Through explicit space-latent space fusion planning, value network alignment and strategy alignment are performed to output the current optimal autonomous driving strategy after fusion, including value estimation fusion and output strategy alignment: Through the value estimation fusion, a mechanism-based cost function that is relatively accurate but has limited performance can be combined with a deep neural network-based value function that has good final performance but is not accurate at the beginning, thereby leveraging the advantages of both the data approach and the mechanism approach. Through the output strategy alignment, the security of the output strategy can be guaranteed, and the data-driven method can be prevented from outputting unreasonable strategies.

2. The method for safe and efficient evolution of autonomous driving with deep fusion of data mechanisms as claimed in claim 1, characterized in that: After the value network alignment and strategy alignment are performed through the explicit space-latent space fusion planning module to output the fused current optimal autonomous driving strategy steps, it also includes recording experience to update the neural network and further perform strategy learning.

3. The method for safe and efficient evolution of autonomous driving with deep fusion of data mechanisms as claimed in claim 2, characterized in that: Design an explicit spatial planning architecture to compute and output optimization-based autonomous driving strategies, further including: Construct nonlinear programming problems and design cost functions; Constraints are used to indicate state violations that should not occur in explicit space planning, and the constraints include vehicle kinematic constraints, control increment constraints, collision constraints, and safe following constraints; The nonlinear optimization problem is constructed in the optimization component, and the optimization-based autonomous driving strategy is obtained and outputted as a st .

4. The safe and efficient evolutionary method for autonomous driving with deep fusion of data mechanisms as claimed in claim 3 is characterized in that: The representation mapping model is used to represent the state quantity s from the explicit space t To the latent space state z t The mapping relationship between them.

5. The method for safe and efficient evolution of autonomous driving with deep fusion of data mechanisms as claimed in claim 4, characterized in that: When planning in latent space, the strategy parameters are updated according to the latent dynamics model, value model and reward model; Potential kinetic model θ Defined as: from i+1 =dθ(z t ,and t ); Reward Model R θ Defined as: Value Model Q θ Defined as: Policy Network π θ Defined as: π=πθ(z i+1 ); Among them, Z i+1 is the potential state quantity at time i+1; Through the potential dynamics model θ , value model Q θ and the reward model R θ , calculate the total return J of the sampling trajectory in the latent space planning process Γ for: Among them, H is the shallow spatial prediction time domain, a t is the alignment strategy, and γ is the value discount factor.

6. The method for safe and efficient evolution of autonomous driving with deep fusion of data mechanisms as claimed in claim 5, characterized in that: In the latent kinetic model d θ , value model Q θ and the reward model R θ , calculate the total return J of the sampling trajectory in the latent space planning process Γ After the steps, also include: The cost function J in the explicit spatial planning πMPC With J Γ The weight is adjusted by the parameter υ: f Γ =υ·J Γ +(1-υ)J πMPC ; Among them, the parameter υ is adjusted linearly with the training process; Assume a zt is the strategy output obtained by latent space planning, which obeys the normal distribution Where I is the identity matrix; Sort the rewards obtained from the sampled trajectories, select the first k rewards and iterate, and get the new parameter μ after the iteration. j and σ j : in, Controls the sharpness of the weights. After a fixed number of iterations, the planning process terminates and samples trajectories from the final normalized distribution of rewards over the action sequence. represents the sampling trajectory under continuous a action; A st 、a zt With the policy network π θ The output π is aligned, and the alignment strategy a is obtained. t : a t =υ π ·a st +(1-y π (a) zt +π) / 2; Among them, the parameter υ π is the weight coefficient, which is dynamically adjusted according to the collision risk situation. The collision risk is determined by the relative Euler distance d between the autonomous vehicle and the surrounding traffic vehicles. safety calculate; The planning process will predict the time domain N in the explicit space p The above process is repeated until N p Step alignment strategy t The first action in a sequence of actions chosen as the plan.

7. The method for safe and efficient evolution of autonomous driving with deep fusion of data mechanisms as claimed in claim 6, characterized in that: The mapping model h θ , explicit dynamics network, latent dynamics model d θ and reward model R θ All of them are trained by supervised learning based on the empirical data obtained through exploration; The value network is trained as follows: where π θ is the strategy network, γ is the value discount factor, and in order to alleviate the problem of overestimation of the value network, the target value network Q is defined θ- , where Q θ The parameters of Q are not updated in real time during the training process, but are updated at fixed intervals. θ- Update to Q θ ; The policy network updates parameters by maximizing the value, and its formula is: Among them, sg() is the stop gradient return operator, (sg(z i )) indicates that the input of the potential state at time i will not bring about the gradient update of the entire loss, λ is the strategy discount factor, and Γ represents the trajectory.

8. A safe and efficient evolutionary device for autonomous driving with deep integration of data mechanisms, characterized in that: include: Design module, used to design the explicit space planning architecture to calculate and output the optimization-based autonomous driving strategy, explicit space state quantity s t It is the state space with actual physical meaning that is input into the autonomous driving algorithm during its actual operation. This definition is related to the details of the algorithm and can be expressed as: Where n is the number of traffic vehicles within the observation range; x e ,y e are the horizontal and vertical coordinates of the vehicle in the global coordinate system respectively; is the vehicle's heading angle, v e is the speed of the vehicle; x i ,y i are the horizontal and vertical coordinates of the pericycle in the global coordinate system (i=1,…,n), is the direction of the chariot (i=1,…,n), v i is the speed of the surrounding vehicle (i=1,…,n); The mapping module is used to represent the spatial state quantity s by mapping the model. t Mapping to the latent space state z t The representation mapping model is used to represent the state quantity s from the explicit space t To the latent space state z t The mapping relationship between them can be expressed as: <h2 style=";text-align:left;direction:ltr">z<h2 style=";text-align:left;direction:ltr"> t <h2 style=";text-align:left;direction:ltr"> =hθ(s<h2 style=";text-align:left;direction:ltr"> t <h2 style=";text-align:left;direction:ltr"> ); A learning module is used to transform the latent space state quantity z t Used for planning calculations, policy network learning, and potential dynamics network learning; The output module is used to perform value network alignment and strategy alignment through explicit space-latent space fusion planning, so as to output the current optimal autonomous driving strategy after fusion, including value estimation fusion and output strategy alignment: Through the value estimation fusion, a mechanism-based cost function that is relatively accurate but has limited performance can be combined with a deep neural network-based value function that has good final performance but is not accurate at the beginning, thereby leveraging the advantages of both the data approach and the mechanism approach. Through the output strategy alignment, the security of the output strategy can be guaranteed, and the data-driven method can be prevented from outputting unreasonable strategies.

Images