An automatic driving method, system, device and storage medium of a vehicle

Abstract

This application discloses an autonomous driving method, system, device, and storage medium for vehicles, applied in the field of machine learning technology. The method includes: planning a reference path and determining static navigation information reflecting the vehicle's static path; obtaining a perception state input reflecting the vehicle's own driving state through feature extraction based on the detection content of onboard and roadside sensors; determining correlation features reflecting historical decision-making information during the vehicle's interaction with the environment; receiving behavior prediction information from the roadside system reflecting the future behavior of each vehicle within the detection area of ​​the roadside sensors; and inputting the static navigation information, perception state input, correlation features, and behavior prediction information into a deep reinforcement learning model to obtain and execute the driving strategy output by the deep reinforcement learning model. Applying the solution of this application can effectively realize an autonomous driving method for vehicles, ensuring driving safety.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and in particular to an autonomous driving method, system, device, and storage medium for vehicles. Background Technology

[0002] In recent years, autonomous driving has become a research area of ​​great interest to both academia and industry. Autonomous vehicles, based on real-time driving environment information acquired by onboard sensors, utilize intelligent decision-making technology to achieve safe and automatic driving without human control. This effectively improves traffic efficiency, reduces accident rates and energy consumption, and is a key technology for enhancing the intelligence level of road traffic.

[0003] Deep reinforcement learning is a new paradigm for intelligent decision-making in autonomous vehicles. It attempts to find the optimal strategy that maximizes expected returns by balancing the utility of the current strategy with exploration of unknown spaces, enabling the driving strategy to learn itself during the interaction between the vehicle and its environment. Deep reinforcement learning-based autonomous vehicle decision-making technology can be described as a closed-loop optimization process. At each moment, the vehicle maps its driving state s to a corresponding driving action a based on its driving strategy. The vehicle executes the driving action to complete the state transition, while the environment provides corresponding reward feedback r to guide the updating of the driving strategy. Because the training process of deep reinforcement learning driving strategies is unconstrained by supervised information, it has the potential to surpass human driving performance.

[0004] However, before the full industrialization of autonomous vehicles, there is still a long transition period. During this period, human-driven cars and autonomous vehicles will inevitably coexist in complex traffic scenarios such as highways and cities. For this mixed traffic flow, considering the randomness of the driving environment and the uncertainty of human driving behavior, vehicles not only need to react promptly to dynamic changes in the environment, but also need to observe the behavioral intentions of surrounding human-driven vehicles in order to ensure driving safety through interactive negotiation. Therefore, driving strategies based on deep reinforcement learning must not only process driving environment information, but also capture complex traffic dynamics, including predicting the driving behavior of surrounding vehicles and adjusting driving behavior based on historical states. Therefore, ensuring the safety of driving strategies during the learning and execution process is a significant research challenge for deep reinforcement learning in the field of autonomous driving.

[0005] To address the safety issues of autonomous driving strategies, existing research has proposed corresponding solutions from the perspectives of vehicle-side, road-side, and vehicle-road cooperation.

[0006] For vehicle-side decision-making, the learning process of driving strategies typically employs reward shaping or prior-driven approaches to design corresponding safety reinforcement learning frameworks based on specific driving scenarios or tasks. Reward shaping directly modifies the objective optimization function of the driving strategy by integrating risk into the reward, thereby reducing safety violations. Prior-driven approaches improve driving safety by modifying the driving strategy exploration process, and can be further divided into action masking mechanisms and safety checkers. Because these methods rely on expert experience and lack theoretical safety guarantees, they cannot achieve sufficient safety. Furthermore, since vehicle-side decision-making depends on environmental perception information from the vehicle's first-person perspective, it requires significant onboard computing resources for perception processing. Limitations and blind spots in onboard sensors can also lead to inaccurate or incomplete perception data, posing significant safety risks.

[0007] For roadside decision-making, a common approach is to consider vehicle-to-vehicle (V2V) communication technology to improve the efficiency and safety of autonomous vehicles. This involves combining onboard sensor information with shared data between connected vehicles, and then having a roadside computing center coordinate and make decisions for the entire vehicle group to enhance traffic safety. This method can overcome limitations in onboard computing resources and sensor range. However, when utilizing shared vehicle data, V2V technology limitations such as communication latency must be considered, and all vehicles must be equipped with this interconnectivity capability. In the real world, it may take decades for all vehicles to become connected. Until then, human drivers will still be on the road, unable to directly control all vehicles, making this approach unsuitable for mixed traffic scenarios.

[0008] In conclusion, how to effectively implement autonomous driving methods for vehicles and ensure driving safety is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0009] The purpose of this invention is to provide an autonomous driving method, system, device, and storage medium for vehicles, so as to effectively realize the autonomous driving method for vehicles and ensure driving safety.

[0010] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0011] An autonomous driving method for a vehicle includes:

[0012] A reference path is planned from the starting point to the destination, and static navigation information reflecting the vehicle's static path is determined based on the reference path.

[0013] Based on the detection content of the vehicle-mounted sensor and the detection content received from the roadside sensor, feature extraction is used to obtain the perception state input quantity that reflects the vehicle's own driving state.

[0014] Identify the relevance features used to reflect historical decision-making information during the interaction between vehicles and the environment;

[0015] The detection content of the vehicle-mounted sensing device is sent to the roadside system, and the behavior prediction information reflecting the future behavior of each vehicle in the detection area of ​​the roadside sensing device is received from the roadside system.

[0016] The static navigation information, the perception state input, the correlation features, and the behavior prediction information are input into the deep reinforcement learning model to obtain the driving strategy output by the deep reinforcement learning model.

[0017] Implement the driving strategy described above.

[0018] Preferably, the behavior prediction information received from the roadside system, which reflects the future behavior of each vehicle within the detection area of ​​the roadside sensing device, is the behavior prediction information determined by the roadside system through the following operations:

[0019] Based on the detection data from the vehicle-mounted sensors and the roadside sensors, the historical trajectory sequence of each detected vehicle is determined. ;

[0020] For the historical trajectory sequence Linear transformation and normalization are performed to obtain the corresponding low-dimensional fusion features. ;

[0021] The obtained low-dimensional fusion features The signal is passed to the filtering model to attenuate noise information in the frequency domain.

[0022] The output of the filtering model is sent to the MLP-based behavior prediction model to obtain behavior prediction information output by the behavior prediction model, which reflects the future behavior of each vehicle within the detection area of ​​the roadside sensing device.

[0023] Where N is the sequence length, This refers to the environmental context information of the i-th trajectory point in the historical trajectory sequence. Vehicle observation information for the i-th trajectory point in the historical trajectory sequence; Low-dimensional fusion features The i-th feature in.

[0024] Preferably, the filtering model is composed of multiple filtering sub-models stacked together, and each filtering sub-model includes a filtering layer, a first random deactivation layer, a second random deactivation layer, a feedforward layer, a first normalization layer, and a second normalization layer.

[0025] The filtering layer is used to perform a fast Fourier transform on the input, optimize it using SGD to attenuate noise information in the frequency domain, and transform the optimization result back to the time domain using an inverse fast Fourier transform.

[0026] The first random deactivation layer is used to receive the output of the filter layer, and after residual connection, it is normalized by the first normalization layer. The result is used as the input of the feedforward layer.

[0027] The feedforward layer is used to extract nonlinear features from its own input through an activation function in the MLP structure, and the result is used as the input of the second random deactivation layer.

[0028] The second random deactivation layer is used to receive the output of the feedforward layer, and after residual connection, it is normalized by the second normalization layer. The result is used as the output of the filter sub-model.

[0029] Preferably, the loss function of the behavior prediction model based on MLP Set to:

[0030] ;

[0031] in, For model parameters, For indicator functions, Let t be the state of the vehicle in the k-th predicted trajectory at time t in the future. To predict the mean, Let K be the prediction variance, and K be the total number of predicted trajectories. Let T be the output probability of the k-th predicted trajectory, and T be the length of the predicted trajectory. M represents the actual ground trajectory, and M is the total number of actual ground trajectories.

[0032] The behavior prediction information output by the behavior prediction model, which reflects the future behavior of each vehicle within the detection area of ​​the roadside sensing device, is obtained. ;

[0033] Where H represents the total number of vehicles in the detected area. The output of the behavior prediction model represents the predicted trajectory with the highest probability for the j-th vehicle.

[0034] Preferably, the step of planning a reference path from the departure point to the destination point, and determining static navigation information reflecting the vehicle's static path based on the reference path, includes:

[0035] Based on the navigation map, a reference path w is planned from the starting point to the destination, where w = [w0, w1, ..., w...]. G];

[0036] Based on the vehicle's current location, d path points are extracted from the reference path w to determine the static navigation information reflecting the vehicle's static path as [w]. q w q+1 , ..., w q+d ];

[0037] Among them, w0 to w G This represents the path points from the starting point to the destination, where w0 is the path point from the starting point, and w... G These are waypoints for the destination location;

[0038] w q d represents the path point closest to the vehicle's current location among all the path points from the vehicle's current location to the destination location, where d is a positive integer.

[0039] Preferably, the step of obtaining the perception state input quantity reflecting the vehicle's own driving state through feature extraction based on the detection content of the on-board sensing device and the received detection content of the roadside sensing device includes:

[0040] Based on the detection data from the vehicle-mounted sensors and the data received from the roadside sensors, a first BEV map is constructed through data fusion.

[0041] Extract the environmental semantic information from the first BEV graph to obtain the second BVE graph;

[0042] The low-dimensional features of the second BVE map are extracted to obtain the perception state input s, which reflects the vehicle's own driving state.

[0043] Preferably, the determination of the correlation features used to reflect historical decision-making information during the interaction between the vehicle and the environment includes:

[0044] During the interaction between the vehicle and the environment, various driving samples are recorded, and a historical behavior sequence based on the driving samples is constructed.

[0045] Using the historical behavior sequence as input to the first learning model, a correlation feature c is determined to reflect the historical decision-making information in the process of vehicle-environment interaction.

[0046] Here, the recorded driving samples are τ, and τ = (s, a, r), and the constructed historical behavior sequence based on the driving samples is {τ}. t-l ,...,τ t-i ,...,τ t-1};

[0047] s is the perceived state input, a is the driving strategy, r is the environmental reward feedback in the deep reinforcement learning model, t represents the current time, and τ t-1 To τ t-l This represents the driving samples from time t-1 to time tl, where l is the set length of the historical behavior sequence.

[0048] Preferably, the deep reinforcement learning model is a deep reinforcement learning model with an attention enhancement mechanism, using weights for input quantities associated with the current state of the vehicle and the driving strategy.

[0049] Preferably, the attention matrix used in the attention enhancement mechanism is represented as follows:

[0050] ;

[0051] Where output is the output of the attention matrix, softmax is the activation function, K0 is the key of all vehicles, V0 is the value of all vehicles, Q is a single query from a vehicle, T0 represents the matrix transpose operation, and d k Let K0 be the dimension of the key vector.

[0052] Preferred options also include:

[0053] After obtaining the driving strategy output by the deep reinforcement learning model, it is determined whether the obtained driving strategy meets the preset safe driving rules.

[0054] If so, then execute the driving strategy described above;

[0055] If not, the driving strategy is adjusted based on the safe driving rules, and the adjusted driving strategy is executed.

[0056] Preferably, the safe driving rules include:

[0057] When V m >V limit And a m When >0, a m Set to 0;

[0058] Among them, V m a is the output speed of the driving strategy. m V is the output acceleration of the driving strategy. limit This refers to the speed limit for traffic.

[0059] Preferably, the safe driving rules also include:

[0060] When D p -T min V m <d min At that time, am Set to a min ;

[0061] When V m >V rcm At that time, a m Set to a max ;

[0062] Among them, D p d represents the distance between the current vehicle and the vehicle in front. min For the minimum safe distance, T min For the minimum collision time, a min For minimum deceleration, V rcm For the recommended speed, a max This is the maximum acceleration.

[0063] An autonomous driving system for a vehicle includes:

[0064] The static navigation information determination module is used to plan a reference path from the starting point to the destination, and determine static navigation information reflecting the vehicle's static path based on the reference path.

[0065] The perception state input quantity determination module is used to obtain the perception state input quantity reflecting the vehicle's own driving state by feature extraction based on the detection content of the vehicle-mounted sensing device and the detection content received from the roadside sensing device.

[0066] The correlation feature determination module is used to determine the correlation features that reflect historical decision-making information in the process of vehicle-environment interaction;

[0067] The behavior prediction information determination module is used to send the detection content of the vehicle-mounted sensing device to the roadside system, and receive the behavior prediction information fed back by the roadside system, which reflects the future behavior of each vehicle in the detection area of ​​the roadside sensing device.

[0068] The driving strategy output module is used to input the static navigation information, the perception state input, the correlation features and the behavior prediction information as input to the deep reinforcement learning model to obtain the driving strategy output by the deep reinforcement learning model.

[0069] The driving strategy execution module is used to execute the driving strategy.

[0070] An autonomous driving device for a vehicle includes:

[0071] Memory, used to store computer programs;

[0072] A processor for executing the computer program to implement the steps of the autonomous driving method for a vehicle as described above.

[0073] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the autonomous driving method for a vehicle as described above.

[0074] By applying the technical solution provided in this invention, a global static path is used as the decision guide, i.e., a reference path from the starting point to the destination is planned, and static navigation information reflecting the vehicle's static path is determined based on the reference path, which helps reduce the training difficulty of deep reinforcement learning models. When obtaining the perception state, this application not only relies on environmental perception information from the vehicle's first-person perspective, but also uses feature extraction based on the detection content of onboard sensors and the received detection content of roadside sensors to obtain a perception state input reflecting the vehicle's own driving state, making the obtained perception state input more accurate. Furthermore, this application considers that the vehicle driving process has significant temporal characteristics, causing the decision made by the driving strategy at a certain moment to depend on the correlation characteristics between historical behaviors over a previous period. Therefore, to further enhance the expressive power of the input features, this application also determines the correlation characteristics reflecting the historical decision information during the interaction between the vehicle and the environment. Addressing the uncertainty of human driving behavior, this application predicts the future driving behavior of surrounding vehicles through a road test system, which can enhance decision-making safety. This application utilizes a roadside system for behavior prediction, receiving behavior prediction information from the roadside system that reflects the future behavior of each vehicle within the detection area of ​​the roadside sensing equipment. This enhances the safety of the subsequent driving strategy. By inputting static navigation information, perception state input, correlation features, and behavior prediction information into a deep reinforcement learning model, a driving strategy output by the deep reinforcement learning model can be obtained and then executed.

[0075] In summary, the solution proposed in this application can effectively realize autonomous driving methods for vehicles and ensure driving safety. Attached Figure Description

[0076] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0077] Figure 1 This is a flowchart illustrating the implementation of an autonomous driving method for a vehicle according to the present invention.

[0078] Figure 2 This is a schematic diagram of the framework of the behavior prediction model and the filtering model in a specific embodiment of the present invention;

[0079] Figure 3 A schematic diagram of the framework of the attention enhancement mechanism in a specific embodiment of the present invention;

[0080] Figure 4 This is a schematic diagram of the structure of an autonomous driving system for a vehicle according to the present invention;

[0081] Figure 5 This is a schematic diagram of the structure of an autonomous driving device for a vehicle according to the present invention. Detailed Implementation

[0082] The core of this invention is to provide an autonomous driving method for vehicles, which can effectively realize autonomous driving and ensure driving safety.

[0083] To enable those skilled in the art to better understand the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0084] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating an implementation of an autonomous driving method for a vehicle according to the present invention. The autonomous driving method for the vehicle may include the following steps:

[0085] Step S101: Plan a reference path from the starting point to the destination, and determine static navigation information to reflect the vehicle's static path based on the reference path.

[0086] Specifically, a reference path from the starting point to the destination can be planned using high-precision maps. This reference path is typically represented by a series of intermediate waypoints. When determining static navigation information reflecting the vehicle's static path based on the reference path, a certain number of waypoints can usually be selected from the current position within the reference path. This application reduces the difficulty of real-time decision-making by using static navigation information as input to a deep reinforcement learning model.

[0087] In one specific embodiment of the present invention, step S101 may specifically include:

[0088] Based on the navigation map, a reference path w is planned from the starting point to the destination, where w = [w0, w1, ..., w...]. G ];

[0089] Based on the vehicle's current location, d path points are extracted from the reference path w to determine the static navigation information [w]. q w q+1 , ..., w q+d ];

[0090] Among them, w0 to w G This represents the path points from the starting point to the destination, where w0 is the path point from the starting point, and w... G These are waypoints for the destination location;

[0091] w q d represents the path point closest to the vehicle's current position among all the path points leading to the destination position. d is the next closest path point from the current position. d is a positive integer.

[0092] Understandably, in this implementation, G represents the total number of waypoints, determined by the planning step size and total distance. When determining the static navigation information used to reflect the vehicle's static path, the value of d can be set and adjusted as needed.

[0093] Step S102: Based on the detection content of the vehicle-mounted sensing device and the detection content received from the roadside sensing device, feature extraction is performed to obtain the perception state input quantity that reflects the vehicle's own driving state.

[0094] The solution in this application does not only rely on environmental perception information from the vehicle's first-person perspective, but also extracts features based on the detection content of the on-board sensors and the detection content received from the roadside sensors. This makes the obtained perception state input reflecting the vehicle's own driving state more accurate, improves the learning efficiency of the deep reinforcement learning model and its robustness to changing environments.

[0095] In one specific embodiment of the present invention, step S102 may specifically include:

[0096] Based on the detection data from the vehicle-mounted sensors and the data received from the roadside sensors, a first BEV map is constructed through data fusion.

[0097] Extract the environmental semantic information from the first BEV graph to obtain the second BVE graph;

[0098] Low-dimensional features of the second BVE map are extracted to obtain the perception state input s, which reflects the vehicle's own driving state.

[0099] In this implementation, considering that constructing a BEV (Birds Eye Views) map as an intermediate representation of the perception state can avoid the influence of environmental differences and improve robustness to changing environments, and then extracting low-dimensional features from it can reduce the training difficulty of driving strategies.

[0100] The detection content of vehicle-mounted sensing devices can be various, such as visual images detected by vehicle-mounted cameras and radar point clouds detected by radar devices.

[0101] The construction of a BEV map relies on information received in real time by sensing devices to generate input states, which is therefore limited by the sensing range of these devices, meaning it cannot describe environmental information beyond their observation distance. Therefore, the solution in this application utilizes high-precision maps for fusion positioning and adds the detection content from roadside sensors to the vehicle-mounted sensing information via V2I (Vehicle-to-Infrastructure) wireless communication technology. In other words, the detection content from the roadside sensors can be transmitted to the vehicle via V2I technology, and the corresponding processor on the vehicle constructs the first BEV map based on the detection content from both the vehicle-mounted and roadside sensors.

[0102] Specifically, based on the detection data from the vehicle-mounted sensors and the data received from the roadside sensors, a first BEV map can be constructed using techniques such as target detection, data fusion, and semantic segmentation.

[0103] When extracting environmental semantic information from the first BEV map, specific environmental semantic information such as the controlled vehicle, background buildings, drivable areas, traffic signs, and traffic participants can be extracted to reduce the impact of environmental differences. The remaining information in the first BEV map, such as road space layout and most of the information required for vehicle decision-making, can be retained.

[0104] In this implementation, low-dimensional features of the second BVE map are further extracted to obtain the perceptual state input 's' reflecting the vehicle's own driving state. For example, a CNN (Convolutional Neural Network) encoder can be used to extract low-dimensional potential key features from the second BVE map, such as the distance to the vehicle ahead, the distance to the nearest stop sign, the distance to the lane center, the status of the nearest traffic light, etc. The extracted results are used as the perceptual state input 's' for the driving strategy, thus obtaining the perceptual state input 's' reflecting the vehicle's own driving state. Since extracting low-dimensional features from the second BVE map achieves dimensionality reduction of the state space, it can accelerate the subsequent deep reinforcement learning policy optimization process and improve sample learning efficiency.

[0105] Furthermore, it should be noted that when using methods such as CNN encoders for feature extraction, the CNN encoder can be pre-trained using supervised learning or trained simultaneously with a deep reinforcement learning model via backpropagation gradient updates. In practical applications, to avoid large-scale data collection in the early stages, the common approach is to train the CNN encoder and the deep reinforcement learning model simultaneously. Moreover, in some implementations, other methods besides CNN encoders can also be used, as long as the required low-dimensional features can be effectively extracted from the second BVE map, without affecting the implementation of this invention.

[0106] Step S103: Determine the correlation features used to reflect historical decision-making information in the process of vehicle-environment interaction.

[0107] This application takes into account that the vehicle driving process has significant temporal characteristics, which means that the decision made by the driving strategy at a certain moment may depend on the correlation characteristics between historical behaviors in a previous time period. Therefore, in order to further enhance the expressive power of the input features, the solution of this application identifies the correlation features that reflect the historical decision-making information in the process of vehicle-environment interaction, and uses them as input to the deep reinforcement learning model.

[0108] The specific method for determining the correlation characteristics used to reflect historical decision-making information during the interaction between the vehicle and the environment can be set according to actual needs, for example, in one specific embodiment of the present invention.

[0109] Step S103 may specifically include:

[0110] During the interaction between the vehicle and the environment, various driving samples are recorded, and a historical behavior sequence based on the driving samples is constructed.

[0111] Using historical behavior sequences as input to the first learning model, we determine the correlation features c that reflect historical decision-making information during the interaction between the vehicle and the environment.

[0112] Here, the recorded driving samples are τ, and τ = (s, a, r), and the constructed historical behavior sequence based on the driving samples is {τ}. t-l ,...,τ t-i ,...,τ t-1};

[0113] s represents the perceptual state input, a represents the driving strategy, r represents the environmental reward feedback in the deep reinforcement learning model, t represents the current time step, and τ represents the current time step. t-1 To τ t-l This represents the driving samples from time t-1 to time tl, where l is the set length of the historical behavior sequence.

[0114] In this implementation, the driving state, decision-making actions, and environmental rewards at each moment are recorded as a driving sample, i.e., τ = (s, a, r). Historical decision-making information during the vehicle-environment interaction process can be used, for example, an RNN (Recurrent Neural Network) can be employed to determine a sequence of historical behaviors {τ}. t-l ,...,τ t-i ,...,τ t-1 The relevance characteristic c of}.

[0115] t represents the current time, i.e., the current decision time, and l is the set length of the historical behavior sequence to characterize the hidden correlation characteristics between historical behaviors in the sequential decision-making process of deep reinforcement learning. Furthermore, in other implementations, the first learning model may not use an RNN network model; for example, LSTM (Long Short-Term Memory), GRU (Gated Recurrent Unit), and their variants can be selected. For instance, in a specific scenario, considering the balance between representational power and computational complexity, GRU can be used to extract the correlation feature c.

[0116] Step S104: Send the detection content of the vehicle-mounted sensor to the roadside system, and receive the behavior prediction information from the roadside system that reflects the future behavior of each vehicle in the detection area of ​​the roadside sensor.

[0117] This application takes into account the uncertainty of human driving behavior. Therefore, the solution in this application enhances decision-making safety by predicting the future driving behavior of surrounding vehicles. Furthermore, considering that vehicle-centric behavior prediction methods are limited by the range of onboard sensors and computing resources, and that their computational complexity and scalability become bottlenecks with increasing traffic density, this application constructs a scenario-centric behavior prediction model within a roadside system. This relies on the roadside system to achieve efficient and accurate vehicle behavior prediction, alleviating the computational burden on the vehicle and enhancing decision-making predictability.

[0118] Specifically, V2I communication technology can be used to send the detection data of vehicle-mounted sensors to the roadside system. The roadside system can then combine the detection data of the vehicle-mounted sensors with the detection data of the roadside sensors to determine the historical trajectory sequence of each vehicle. In turn, it can use a behavior prediction model to predict the future behavior of each vehicle in the detection area, that is, predict the future trajectory of each vehicle.

[0119] In one specific embodiment of the present invention, the behavior prediction information received from the roadside system, which reflects the future behavior of each vehicle within the detection area of ​​the roadside sensing device, is the behavior prediction information determined by the roadside system through the following operations:

[0120] Based on the detection data from the vehicle-mounted sensors and the roadside sensors, the historical trajectory sequences of each detected vehicle are determined. ;

[0121] Historical trajectory sequence Linear transformation and normalization are performed to obtain the corresponding low-dimensional fusion features. ;

[0122] The obtained low-dimensional fusion features It is passed to the filtering model to attenuate noise information in the frequency domain;

[0123] The output of the filtering model is sent to the MLP-based behavior prediction model to obtain behavior prediction information that reflects the future behavior of each vehicle within the detection area of ​​the roadside sensing device.

[0124] In this implementation, considering the impact of communication interference and sensor errors, data noise in the vehicle's historical trajectory sequence can lead to overfitting in the behavior prediction model, affecting prediction performance. Therefore, this application leverages the filtering concept from digital signal processing to enhance the robustness of the solution by introducing a filtering model. Commonly used behavior prediction models include structures such as MLP (Multilayer Perceptron), RNN, CNN, and Transformer. This application can use the simple MLP as the backbone network to construct a filtered and enhanced behavior prediction model.

[0125] Specifically, based on the detection data from the vehicle-mounted sensors and the roadside sensors, the historical trajectory sequence of each detected vehicle can be determined, represented as follows: N is the sequence length. This refers to the environmental context information of the i-th trajectory point in the historical trajectory sequence. This includes vehicle observation information for the i-th trajectory point in the historical trajectory sequence, such as the vehicle's position, speed, and steering angle.

[0126] Historical trajectory sequence By performing linear transformation and normalization, the corresponding low-dimensional fusion features can be obtained, represented as follows: . Low-dimensional fusion features The i-th feature in the model is then used to obtain the low-dimensional fused feature. This low-dimensional fused feature is then passed to the filtering model for noise attenuation.

[0127] In one specific embodiment of the present invention, the filtering model is composed of multiple stacked filtering sub-models, each filtering sub-model including a filtering layer, a first random deactivation layer, a second random deactivation layer, a feedforward layer, a first normalization layer, and a second normalization layer; see reference. Figure 2 This is a schematic diagram illustrating the framework of a behavior prediction model and a filtering model in one specific implementation. It should be noted that... Figure 2 Only a single filter sub-model is shown in the figure.

[0128] The filtering layer is used to perform a fast Fourier transform on the input, optimize it using SGD to attenuate noise information in the frequency domain, and transform the optimization result back to the time domain using an inverse fast Fourier transform.

[0129] The first random deactivation layer is used to receive the output of the filter layer, and after residual connection, it is normalized by the first normalization layer. The result is used as the input of the feedforward layer.

[0130] The feedforward layer is used in the MLP structure to extract nonlinear features from its own input through an activation function, and the result is used as the input of the second random deactivation layer.

[0131] The second random deactivation layer receives the output of the feedforward layer and, after residual connection, is normalized by the second normalization layer. The result is then used as the output of the filter sub-model.

[0132] For example, the input of the l-th filter layer is represented as x l After performing a Fast Fourier Transform, it can be represented as X. l =f(x l Here, f() can be a one-dimensional FFT (Fast Fourier Transform) operation.

[0133] After performing a Fast Fourier Transform, the frequency domain representation is adjusted using a learnable filter F. For example, specifically... =F⊙X l The dot product ⊙ represents element-wise multiplication. The filter F can be optimized using SGD (Stochastic Gradient Descent) to adaptively attenuate noise information in the frequency domain. Since this dot product in the frequency domain is equivalent to a circular convolution in the time domain, it has a larger receptive field and can better capture the periodic features of the sequence.

[0134] Finally, the frequency domain representation is transformed back to the time domain using inverse FFT, which can be expressed as: Here ( ) can be a one-dimensional inverse FFT operation. This is the denoised sequence representation.

[0135] Furthermore, to avoid gradient vanishing and unstable training problems, and to prevent overfitting, this application will... After the input to the Dropout layer, residual connections are performed. That is, the first random deactivation layer receives the output of the filtering layer, and after the residual connections, it is normalized by the first normalization layer. The result obtained from the first normalization layer serves as the input to the feedforward layer, which can be expressed as: In this formula That is the first random deactivation layer. This is the first normalization layer.

[0136] In the feedforward layer, MLP and ReLU activation functions can be combined to further extract information from the sequence. Extracting nonlinear features from this can be represented as: W1, b1, W2, and b2 are all trainable parameters.

[0137] After the output of the feedforward layer is sent to the second random deactivation layer, residual connections are performed, and then the first normalization layer normalizes the result. The result can be used as the output of the filter sub-model, that is, the output of the l-th filter sub-model in this example.

[0138] As described above, this application can use a simple MLP as the backbone network to construct a filtered and enhanced behavior prediction model.

[0139] For example, the output sequence of the filtering model is Then the output of the behavior prediction model can be represented as a Gaussian distribution of the vehicle's future trajectory: In this formula Let N represent the distribution, where N is a Gaussian distribution. Let this be the state of the vehicle at time t in the k-th predicted trajectory. To predict the mean, The variance is the prediction variance. The probability distribution model of the predicted roadside trajectory can be described using a Gaussian Mixture Model (GMM), expressed as:

[0140] Among them Let K be the probability, and K be the total number of predicted trajectories. Let T be the output probability of the k-th predicted trajectory, and T be the length of the predicted trajectory.

[0141] MLP is used to map the input to a probability distribution representation of the predicted trajectory, which can be optimized using SGD to maximize the likelihood estimate of the vehicle's predicted trajectory. For example, the training data is represented as... , If the actual trajectory is on the ground, then in one specific embodiment of the present invention, the model optimization objective is, i.e.

[0142] Loss function of behavior prediction model based on MLP It can be specifically set as follows:

[0143] ;

[0144] in, For model parameters, For indicator functions, Let t be the state of the vehicle in the k-th predicted trajectory at time t in the future. To predict the mean, Let K be the prediction variance, and K be the total number of predicted trajectories. Let T be the output probability of the k-th predicted trajectory, and T be the length of the predicted trajectory. M represents the actual ground trajectory, and M represents the total number of actual ground trajectories.

[0145] The trained behavior prediction model can be deployed on a roadside system to predict the real-time trajectories of vehicles within the detection area. Assuming there are H vehicles in the controlled area, the roadside behavior prediction model can infer the future trajectory distribution of each vehicle and output the predicted trajectory with the highest probability. By aggregating the predicted trajectories of all vehicles, information about the future behavior of surrounding vehicles can be obtained.

[0146] In one specific embodiment of the present invention, the behavior prediction information output by the behavior prediction model, which reflects the future behavior of each vehicle within the detection area of ​​the roadside sensing device, can be represented as follows: Where H represents the total number of vehicles within the detected area. The output of the behavior prediction model represents the predicted trajectory with the highest probability for the j-th vehicle.

[0147] Step S105: Input the static navigation information, perception state input, correlation features and behavior prediction information into the deep reinforcement learning model to obtain the driving strategy output by the deep reinforcement learning model.

[0148] Step S106: Implement the driving strategy.

[0149] In this application, static navigation information w, perceived state input s, correlation features c, and behavior prediction information I are combined to form the input state. , and serve as input to deep reinforcement learning models, in which This is the concatenation operator.

[0150] When using deep reinforcement learning models to implement vehicle motion control, algorithms that can be employed include PPO (Proximal Policy Optimization), DDPG (Deep Deterministic Policy Gradient), TD3 (Twin Delayed Deep Deterministic Policy Gradient), SAC (Soft Actor-Critic), and others. Considering hyperparameters and convergence performance, this invention, for example, can use PPO for training.

[0151] Within the PPO framework, deep reinforcement learning models can learn from driving samples generated during each interaction between the vehicle and its environment. Importance sampling is used to improve sample efficiency, and the objective function is pruned to prevent excessive differences before and after policy updates, thus ensuring policy optimization performance. Assuming the model is... , among them For model parameters, then based on Generate a set of driving samples Here, T represents the total number of steps in one interaction process. During the update process, a small batch of samples is randomly selected each time. As training samples, Update parameters according to the following objective optimization function, based on the batch size. :

[0152] .

[0153] Among them The model parameters are obtained from the k-th iteration update. For strategy The probability distribution can be obtained from the samples. Approaching, For strategy The probability distribution, through samples To approximate, clip() is a clipping operation to prevent... and The difference is too large, specifically:

[0154]

[0155] in, These are limiting parameters. Let the action advantage function be expressed as:

[0156]

[0157] Among them For state Take action at the time Reward feedback, For value function, This is the reward decay factor. When k reaches the set maximum number of iterations, one round of optimization ends, and the vehicle continues to interact with the environment and perform the next round of strategy optimization until the maximum number of optimizations is reached.

[0158] Furthermore, in one specific embodiment of the present invention, the deep reinforcement learning model is a deep reinforcement learning model with an attention enhancement mechanism to increase the weight of inputs associated with the current state of the vehicle and the driving strategy.

[0159] Specifically, an attention module can be designed in the policy network of PPO to enable the deep reinforcement learning model to pay more attention to the inputs related to the current vehicle state and decision, that is, to increase the weight of the inputs associated with the current vehicle state and driving strategy.

[0160] See also Figure 3 This is a schematic diagram of the attention enhancement mechanism, where the static navigation information w, the perceptual state input s, the associative features c, and the classification... Figure 3 The vehicle's own state is considered in the roadside system's behavior prediction information, which can be further divided into the vehicle's own state and the surrounding vehicle states, and categorized into these two parts respectively. Figure 3 The system calculates the self-vehicle state and the surrounding vehicle states. Then, it normalizes the values ​​and passes them to the MLP (Multi-Level Processing), and the MLP output is further passed to the attention module. In one specific embodiment of this invention, the attention matrix used by the attention enhancement mechanism can be specifically represented as:

[0161] ;

[0162] Here, K0 represents the keys for all vehicles, and V0 represents the values ​​for all vehicles, both of which can be obtained through linear projection of the ego vehicle's state information and that of surrounding vehicles. Q represents a single query from a vehicle, which can be obtained through linear projection of the ego vehicle's state information. output is the output of the attention matrix, and softmax is the activation function. Here, T0 represents the matrix transpose operation, and d k Let K0 be the dimension of the key vector.

[0163] The output data of the attention module is concatenated and linearly transformed, then added to the output of the MLP of the self-vehicle state. After normalization, it is passed to the output MLP layer, and finally the action value is output, that is, the driving strategy.

[0164] Furthermore, in one specific embodiment of the present invention, it may also include:

[0165] After obtaining the driving strategy output by the deep reinforcement learning model, it is determined whether the obtained driving strategy meets the preset safe driving rules;

[0166] If so, then execute the driving strategy;

[0167] If not, adjust the driving strategy based on safe driving rules and execute the adjusted driving strategy.

[0168] To further improve driving safety, this implementation method, after obtaining the driving strategy output by the deep reinforcement learning model, will further determine whether the obtained driving strategy meets the preset safe driving rules.

[0169] If the conditions are met, the obtained driving strategy is normal and can be executed directly. If the conditions are not met, from a safety perspective, the driving strategy will be adjusted based on safe driving rules, and the adjusted driving strategy will be executed.

[0170] Of course, specific safe driving rules can be set and selected according to actual needs. For example, a commonly considered safe driving rule is a speed limit rule. In one specific embodiment of the present invention, safe driving rules may include:

[0171] When V m >V limit And a m When >0, a m Set to 0;

[0172] Among them, V m For the output speed of the driving strategy, a m V is the output acceleration of the driving strategy. limit This refers to the speed limit for traffic.

[0173] It can be seen that when V m >V limit If the speed exceeds the traffic speed limit, the output speed of the driving strategy is replaced with 0 to prevent the vehicle from speeding.

[0174] Furthermore, in one specific embodiment of the present invention, the safe driving rules may further include:

[0175] When D p -T min V m <d min At that time, a m Set to a min ;

[0176] When V m >V rcm At that time, a m Set to a max ;

[0177] Among them, D p d represents the distance between the current vehicle and the vehicle in front. min For the minimum safe distance, T min For the minimum collision time, a min For minimum deceleration, V rcm For the recommended speed, a max This is the maximum acceleration.

[0178] This implementation further supplements the safe driving rules from the perspectives of maintaining a safe following distance and overtaking safely.

[0179] When it is determined that D p -T min V m <d min When this occurs, it indicates that the following distance between the current vehicle and the vehicle in front is too low, therefore a... m Set to a min This means controlling the vehicle to perform minimum deceleration in order to maintain a safe distance from the vehicle in front.

[0180] And when V m >V rcm At that time, a m Set to a max V is used to ensure that the vehicle performs maximum acceleration when changing lanes or overtaking, until the recommended speed is reached. rcm It can be preset and dynamically adjusted according to the actual situation.

[0181] By applying the technical solution provided in this invention, a global static path is used as the decision guide, i.e., a reference path from the starting point to the destination is planned, and static navigation information reflecting the vehicle's static path is determined based on the reference path, which helps reduce the training difficulty of deep reinforcement learning models. When obtaining the perception state, this application not only relies on environmental perception information from the vehicle's first-person perspective, but also uses feature extraction based on the detection content of onboard sensors and the received detection content of roadside sensors to obtain a perception state input reflecting the vehicle's own driving state, making the obtained perception state input more accurate. Furthermore, this application considers that the vehicle driving process has significant temporal characteristics, causing the decision made by the driving strategy at a certain moment to depend on the correlation characteristics between historical behaviors over a previous period. Therefore, to further enhance the expressive power of the input features, this application also determines the correlation characteristics reflecting the historical decision information during the interaction between the vehicle and the environment. Addressing the uncertainty of human driving behavior, this application predicts the future driving behavior of surrounding vehicles through a road test system, which can enhance decision-making safety. This application utilizes a roadside system for behavior prediction, receiving behavior prediction information from the roadside system that reflects the future behavior of each vehicle within the detection area of ​​the roadside sensing equipment. This enhances the safety of the subsequent driving strategy. By inputting static navigation information, perception state input, correlation features, and behavior prediction information into a deep reinforcement learning model, a driving strategy output by the deep reinforcement learning model can be obtained and then executed.

[0182] In summary, the solution proposed in this application can effectively realize autonomous driving methods for vehicles and ensure driving safety.

[0183] Corresponding to the above method embodiments, this invention also provides an autonomous driving system for vehicles, which can be referred to in conjunction with the above description.

[0184] See Figure 4 The diagram shown is a structural schematic of an autonomous driving system for a vehicle according to the present invention, comprising:

[0185] The static navigation information determination module 401 is used to plan a reference path from the starting point to the destination, and determine static navigation information reflecting the static path of the vehicle based on the reference path.

[0186] The perception state input quantity determination module 402 is used to obtain the perception state input quantity reflecting the vehicle's own driving state by feature extraction based on the detection content of the vehicle-mounted sensing device and the detection content received from the roadside sensing device.

[0187] The correlation feature determination module 403 is used to determine the correlation features that reflect historical decision-making information in the process of vehicle-environment interaction;

[0188] The behavior prediction information determination module 404 is used to send the detection content of the vehicle-mounted sensing device to the roadside system and receive the behavior prediction information fed back by the roadside system, which reflects the future behavior of each vehicle in the detection area of ​​the roadside sensing device.

[0189] The driving strategy output module 405 is used to input static navigation information, perception state input, correlation features and behavior prediction information as input to the deep reinforcement learning model to obtain the driving strategy output by the deep reinforcement learning model.

[0190] Driving strategy execution module 406 is used to execute driving strategies.

[0191] In one specific embodiment of the present invention, the behavior prediction information received by the behavior prediction information determination module 404 from the roadside system, which reflects the future behavior of each vehicle within the detection area of ​​the roadside sensing device, is the behavior prediction information determined by the roadside system through the following operations:

[0192] Based on the detection data from the vehicle-mounted sensors and the roadside sensors, the historical trajectory sequences of each detected vehicle are determined. ;

[0193] Historical trajectory sequence Linear transformation and normalization are performed to obtain the corresponding low-dimensional fusion features. ;

[0194] The obtained low-dimensional fusion features It is passed to the filtering model to attenuate noise information in the frequency domain;

[0195] The output of the filtering model is sent to the MLP-based behavior prediction model to obtain behavior prediction information that reflects the future behavior of each vehicle within the detection area of ​​the roadside sensing device.

[0196] Where N is the sequence length, This refers to the environmental context information of the i-th trajectory point in the historical trajectory sequence. Vehicle observation information for the i-th trajectory point in the historical trajectory sequence; Low-dimensional fusion features The i-th feature in.

[0197] In one specific embodiment of the present invention, the filtering model is composed of multiple filtering sub-models stacked together. Each filtering sub-model includes a filtering layer, a first random deactivation layer, a second random deactivation layer, a feedforward layer, a first normalization layer, and a second normalization layer.

[0198] The filtering layer is used to perform a fast Fourier transform on the input, optimize it using SGD to attenuate noise information in the frequency domain, and transform the optimization result back to the time domain using an inverse fast Fourier transform.

[0199] The first random deactivation layer is used to receive the output of the filter layer, and after residual connection, it is normalized by the first normalization layer. The result is used as the input of the feedforward layer.

[0200] The feedforward layer is used in the MLP structure to extract nonlinear features from its own input through an activation function, and the result is used as the input of the second random deactivation layer.

[0201] The second random deactivation layer receives the output of the feedforward layer and, after residual connection, is normalized by the second normalization layer. The result is then used as the output of the filter sub-model.

[0202] In one specific embodiment of the present invention, the loss function of the behavior prediction model based on MLP is... Set to:

[0203] ;

[0204] in, For model parameters, For indicator functions, Let t be the state of the vehicle in the k-th predicted trajectory at time t in the future. To predict the mean, Let K be the prediction variance, and K be the total number of predicted trajectories. Let T be the output probability of the k-th predicted trajectory, and T be the length of the predicted trajectory. M represents the actual ground trajectory, and M is the total number of actual ground trajectories.

[0205] The behavior prediction model outputs behavior prediction information that reflects the future behavior of each vehicle within the detection area of ​​the roadside sensor. ;

[0206] Where H represents the total number of vehicles in the detected area. The output of the behavior prediction model represents the predicted trajectory with the highest probability for the i-th vehicle.

[0207] In one specific embodiment of the present invention, the static navigation information determination module 401 is specifically used for:

[0208] Based on the navigation map, a reference path w is planned from the starting point to the destination, where w = [w0, w1, ..., w...]. G ];

[0209] Based on the vehicle's current location, d path points are extracted from the reference path w to determine the static navigation information [w]. i w i+1 , ..., w i+d ];

[0210] Among them, w0 to w G This represents the path points from the starting point to the destination, where w0 is the path point from the starting point, and w... G These are waypoints for the destination location;

[0211] w i d represents the path point closest to the vehicle's current location among all the path points from the vehicle's current location to the destination location, where d is a positive integer.

[0212] In one specific embodiment of the present invention, the sensing state input quantity determination module 402 is specifically used for:

[0213] Based on the detection data from the vehicle-mounted sensors and the data received from the roadside sensors, a first BEV map is constructed through data fusion.

[0214] Extract the environmental semantic information from the first BEV graph to obtain the second BVE graph;

[0215] Low-dimensional features of the second BVE map are extracted to obtain the perception state input s, which reflects the vehicle's own driving state.

[0216] In one specific embodiment of the present invention, the correlation feature determination module 403 is specifically used for:

[0217] During the interaction between the vehicle and the environment, various driving samples are recorded, and a historical behavior sequence based on the driving samples is constructed.

[0218] Using historical behavior sequences as input to the first learning model, we determine the correlation features c that reflect historical decision-making information during the interaction between the vehicle and the environment.

[0219] Here, the recorded driving samples are τ, and τ = (s, a, r), and the constructed historical behavior sequence based on the driving samples is {τ}. t-l ,...,τ t-i ,...,τ t-1};

[0220] s represents the perceptual state input, a represents the driving strategy, r represents the environmental reward feedback in the deep reinforcement learning model, t represents the current time step, and τ represents the current time step. t-1 To τ t-l This represents the driving samples from time t-1 to time tl, where l is the set length of the historical behavior sequence.

[0221] In one specific embodiment of the present invention, the deep reinforcement learning model is a deep reinforcement learning model with an attention enhancement mechanism, using weights of input quantities associated with the current state of the vehicle and the driving strategy.

[0222] In one specific embodiment of the present invention, the attention matrix used in the attention enhancement mechanism is represented as follows:

[0223] ;

[0224] Where output is the output of the attention matrix, softmax is the activation function, K0 is the key of all vehicles, V0 is the value of all vehicles, Q is a single query from a vehicle, T0 represents the matrix transpose operation, and d k Let K0 be the dimension of the key vector.

[0225] In one specific embodiment of the present invention, the driving strategy execution module 406 is further configured to:

[0226] After obtaining the driving strategy output by the deep reinforcement learning model, it is determined whether the obtained driving strategy meets the preset safe driving rules;

[0227] If so, then execute the driving strategy;

[0228] If not, adjust the driving strategy based on safe driving rules and execute the adjusted driving strategy.

[0229] In one specific embodiment of the present invention, safe driving rules include:

[0230] When V m >V limit And a m When >0, a m Set to 0;

[0231] Among them, V m For the output speed of the driving strategy, a m V is the output acceleration of the driving strategy. limit This refers to the speed limit for traffic.

[0232] In one specific embodiment of the present invention, the safe driving rules further include:

[0233] When D p -Tmin V m <d min At that time, a m Set to a min ;

[0234] When V m >V rcm At that time, a m Set to a max ;

[0235] Among them, D p d represents the distance between the current vehicle and the vehicle in front. min For the minimum safe distance, T min For the minimum collision time, a min For minimum deceleration, V rcm For the recommended speed, a max This is the maximum acceleration.

[0236] Corresponding to the above methods and system embodiments, this invention also provides an autonomous driving device for a vehicle and a computer-readable storage medium, which can be referred to in conjunction with the above.

[0237] See also Figure 5 The vehicle's autonomous driving equipment may include:

[0238] Memory 501 is used to store computer programs;

[0239] Processor 502 is configured to execute a computer program to implement the steps of the autonomous driving method for a vehicle as described in any of the above embodiments.

[0240] The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the autonomous driving method for a vehicle as described in any of the above embodiments. The computer-readable storage medium referred to herein includes random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0241] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0242] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0243] This article uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the technical solutions and core ideas of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the protection scope of the present invention.

Claims

1. An automatic driving method of a vehicle, characterized by, The method comprises the following steps: planning a reference path from a starting location to a destination location, and determining static navigation information reflecting a static path of a vehicle according to the reference path; extracting features from the detection content of the on-board sensing device and the received detection content of the roadside sensing device to obtain a perception state input quantity reflecting the driving state of the vehicle itself; determining a relevance feature reflecting historical decision information of the vehicle in the interaction with the environment; sending the detection content of the on-board sensing device to the roadside system and receiving behavior prediction information reflecting the future behavior of each vehicle in the detection area of the roadside sensing device fed back by the roadside system; inputting the static navigation information, the perception state input quantity, the relevance feature and the behavior prediction information as input quantities into a deep reinforcement learning model to obtain a driving strategy output by the deep reinforcement learning model; executing the driving strategy; the behavior prediction information reflecting the future behavior of each vehicle in the detection area of the roadside sensing device fed back by the roadside system is determined by the roadside system through the following operations: Based on the detection data from the vehicle-mounted sensors and the roadside sensors, the historical trajectory sequence of each detected vehicle is determined. ; performing linear transformation and normalization processing on the historical trajectory sequence to obtain corresponding low-dimensional fusion features ; The obtained respective low-dimensional fusion features are transmitted to a filtering model to attenuate noise information in the frequency domain by the filtering model. sending the output of the filtering model to an MLP-based behavior prediction model to obtain behavior prediction information reflecting the future behavior of each vehicle in the detection area of the roadside sensing device output by the behavior prediction model; wherein N is a sequence length, is the environmental context information of the i-th trajectory point in the historical trajectory sequence, is the vehicle observation information of the i-th trajectory point in the historical trajectory sequence; is the low-dimensional fusion feature is the i-th feature quantity in the low-dimensional fusion feature the filtering model is stacked by a plurality of filtering sub-models, and each filtering sub-model comprises a filtering layer, a first random inactivation layer, a second random inactivation layer, a feedforward layer, a first normalization layer and a second normalization layer; the filtering layer is used to perform fast Fourier transform on the input, optimize it by SGD to attenuate noise information in the frequency domain, and transform the optimized result back to the time domain by inverse fast Fourier transform; the first random inactivation layer is used to receive the output of the filtering layer, normalize it by the first normalization layer after residual connection, and obtain the result as the input of the feedforward layer; the feedforward layer is used to extract nonlinear features from its own input by an activation function under the MLP structure, and obtain the result as the input of the second random inactivation layer; the second random inactivation layer is used to receive the output of the feedforward layer, normalize it by the second normalization layer after residual connection, and obtain the result as the output of the filtering sub-model; extracting features from the detection content of the on-board sensing device and the received detection content of the roadside sensing device to obtain a perception state input quantity reflecting the driving state of the vehicle itself, comprising: constructing a first BEV graph by data fusion according to the detection content of the on-board sensing device and the received detection content of the roadside sensing device; extracting environmental semantic information of the first BEV graph to obtain a second BVE graph; extracting low-dimensional features of the second BVE graph by a CNN encoder to realize state space dimension reduction and obtain a perception state input quantity s reflecting the driving state of the vehicle itself.

2. The automatic driving method of a vehicle according to claim 1, characterized by, a loss function of the behavior prediction model based on MLP is set to: ； wherein, is a model parameter, is an indicator function, is the state of the vehicle at time t in the kth predicted trajectory, is the predicted mean, is the predicted variance, K is the total number of predicted trajectories, is the output probability of the kth predicted trajectory, T is the length of the predicted trajectory, is the ground truth trajectory, M is the total number of ground truth trajectories; obtaining behavior prediction information for reflecting future behaviors of each vehicle within a detection area of a roadside sensing device output by the behavior prediction model as ; wherein H is the total number of vehicles in the detected region, is the predicted trajectory output by the behavior prediction model that represents the greatest probability of the jth vehicle.

3. The automatic driving method of a vehicle according to claim 1, characterized by, The planning of the reference path from the starting location to the destination location, and the determination of the static navigation information reflecting the static path of the vehicle according to the reference path, comprises: Based on the navigation map, a reference path w from the start location to the destination location is planned, and w = [w0, w1,..., wn-1, wn], where w0is the start location, wn is the destination location, and w1, w2,..., wn-1 are the intermediate locations. G ] Based on the current position of the vehicle, d path points are intercepted from the reference path w, and static navigation information reflecting the static path of the vehicle is determined as [w q , w q+1 ,..., w q+d ] wherein w0 to w G represent the respective waypoints from the start location to the destination location, wherein w0 is the waypoint of the start location, w G is the waypoint of the destination location; w q represents the path point closest to the current position of the vehicle among the path points from the current position of the vehicle to the destination position, and d is a positive integer.

4. The automatic driving method of a vehicle according to claim 1, characterized by, The determination of the relevance feature reflecting the historical decision information in the interaction process of the vehicle and the environment comprises: In the interaction process of the vehicle and the environment, each driving sample is recorded, and a historical behavior sequence based on the driving sample is constructed; The historical behavior sequence is taken as an input of the first learning model, and the relevance feature c reflecting the historical decision information in the interaction process of the vehicle and the environment is determined. Wherein, the recorded driving sample is τ, and τ=(s, a, r, ), the constructed history behavior sequence based on the driving sample is {τ t-l ,..., τ t-i ,..., τ t-1}; s is the perception state input, a is the driving strategy, r is the environmental reward feedback in the deep reinforcement learning model, t represents the current time, τ t-1 to τ t-l represents the driving sample from t-1 time to t-1 time, and l is the set history behavior sequence length.

5. The automatic driving method of a vehicle according to claim 1, characterized by, The deep reinforcement learning model is a deep reinforcement learning model provided with an attention enhancement mechanism, and the weight of the input associated with the current state of the vehicle and the driving strategy.

6. The automatic driving method of a vehicle according to claim 5, characterized by, The attention matrix adopted by the attention enhancement mechanism is represented as: ； where output is the output of the attention matrix, softmax is the activation function, K0is the keys of all vehicles, V0is the values of all vehicles, Q is a single query from a vehicle, T0denotes the matrix transpose operation, d k is the dimension of the key vector K0.

7. The automatic driving method of a vehicle according to any one of claims 1 to 6, characterized by, Further comprising: After obtaining the driving strategy output by the deep reinforcement learning model, it is judged whether the obtained driving strategy satisfies the preset safe driving rule; If yes, the driving strategy is executed; If no, the driving strategy is adjusted based on the safe driving rule, and the driving strategy after the adjustment is executed.

8. The automatic driving method of a vehicle according to claim 7, characterized by, The safe driving rule comprises: When V m > V limit and a m > 0, set a m to 0; where V m is the output speed of the driving strategy, a m is the output acceleration of the driving strategy, V limit is the traffic speed limit value.

9. The automatic driving method of a vehicle according to claim 8, characterized by, The safe driving rule further comprises: When D p - T min V m <d min is set to a m a min ; When V m > V rcm , set a m to a max ; where D p is the current vehicle-to-vehicle distance, d min is the minimum safe distance, T min is the minimum collision time, a min is the minimum deceleration, V rcm is the recommended speed, a max is the maximum acceleration.

10. An automatic driving system of a vehicle, characterized by, Comprising: A static navigation information determination module is configured to plan a reference path from a starting location to a destination location, and determine static navigation information reflecting a static path of a vehicle according to the reference path; A perception state input determination module is configured to obtain a perception state input reflecting a driving state of the vehicle through feature extraction according to detection content of a vehicle-mounted sensing device and detection content received from a roadside sensing device; A relevance feature determination module is configured to determine a relevance feature reflecting historical decision information in an interaction process of the vehicle and the environment; A behavior prediction information determination module is configured to send the detection content of the vehicle-mounted sensing device to a roadside system, and receive behavior prediction information reflecting future behaviors of each vehicle in a detection area of the roadside sensing device fed back by the roadside system; A driving strategy output module is configured to input the static navigation information, the perception state input, the relevance feature and the behavior prediction information as inputs into a deep reinforcement learning model, and obtain a driving strategy output by the deep reinforcement learning model; A driving strategy execution module is configured to execute the driving strategy. The behavior prediction information reflecting future behaviors of each vehicle in a detection area of the roadside sensing device fed back by the roadside system is behavior prediction information determined by the roadside system through the following operations: Based on the detection data from the vehicle-mounted sensors and the roadside sensors, the historical trajectory sequence of each detected vehicle is determined. ; performing linear transformation and normalization processing on the historical trajectory sequence to obtain corresponding low-dimensional fusion features ; The obtained respective low-dimensional fusion features are transmitted to a filtering model to attenuate noise information in the frequency domain by the filtering model. The output of the filtering model is sent to an MLP-based behavior prediction model to obtain behavior prediction information reflecting future behaviors of each vehicle in a detection area of the roadside sensing device output by the behavior prediction model; wherein N is a sequence length, is the environmental context information of the i-th trajectory point in the historical trajectory sequence, is the vehicle observation information of the i-th trajectory point in the historical trajectory sequence; is the i-th feature in the low-dimensional fusion feature ​ The filtering model is stacked by a plurality of filtering sub-models, and each filtering sub-model comprises a filtering layer, a first random inactivation layer, a second random inactivation layer, a feedforward layer, a first normalization layer and a second normalization layer; The filter layer is used to optimize the input after fast Fourier transform by SGD to attenuate the noise information in the frequency domain, and transform the optimization result back to the time domain by inverse fast Fourier transform; The first random inactivation layer is used to receive the output of the filter layer, and is normalized by the first normalization layer after residual connection, and the result is taken as the input of the feedforward layer; The feedforward layer is used to extract nonlinear features from its own input by an activation function under the MLP structure, and the result is taken as the input of the second random inactivation layer; The second random inactivation layer is used to receive the output of the feedforward layer, and is normalized by the second normalization layer after residual connection, and the result is taken as the output of the filter submodel; According to the detection content of the vehicle-mounted sensing device and the received detection content of the roadside sensing device, the perception state input quantity reflecting the driving state of the vehicle itself is obtained through feature extraction, including: According to the detection content of the vehicle-mounted sensing device and the received detection content of the roadside sensing device, a first BEV map is constructed by data fusion; The environmental semantic information of the first BEV map is extracted to obtain a second BVE map; The low-dimensional features of the second BVE map are extracted by a CNN encoder to realize state space dimension reduction, and the perception state input quantity s reflecting the driving state of the vehicle itself is obtained.

11. An automatic driving device of a vehicle, characterized by comprising: Including: A memory for storing a computer program; A processor for executing the computer program to implement the steps of the automatic driving method of the vehicle according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the automatic driving method of the vehicle according to any one of claims 1 to 9.

Images