An artificial intelligence-based method and system for predicting behavior of an autonomous vehicle

Abstract

The application provides an automatic driving vehicle behavior prediction method and system based on artificial intelligence. The method first acquires multi-modal driving data of a sample automatic driving vehicle and performs preprocessing. Feature extraction is performed on data of different modalities, and weighted fusion is performed based on a cross-modal attention mechanism. An initial dynamic graph network model and a traffic participant behavior model are constructed, the behavior of a traffic participant is taken as a node, the interaction relationship between nodes is determined, and the weight of an edge is dynamically updated to obtain an updated dynamic graph network model. Based on the attention mechanism, the attention distribution of a traffic participant to surrounding obstacles is calculated, and a multi-modal feature vector is constructed, which is input into the dynamic graph network model to obtain updated node features. A high-level decision model is constructed according to the updated node features, and multi-modal data of a to-be-tested automatic driving vehicle are input into the model to generate a result. The application improves the accuracy of automatic driving vehicle behavior prediction by fusing multi-modal data.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to an artificial intelligence-based method and system for predicting the behavior of autonomous vehicles. Background Technology

[0002] With the rapid development of autonomous driving technology, vehicle behavior prediction has become a key element in realizing advanced driving functions.

[0003] The invention patent with authorization announcement number CN118025203B discloses an autonomous vehicle behavior prediction system and method that integrates complex networks and graph Transformers. First, a complex network for vehicle interaction is constructed, and the original map information data, vehicle motion data, and local interaction information are used as multi-layer inputs to the model. Then, by using the graph convolution and multi-head node attention mechanism of the graph Transformer, the complex relationships between vehicles are modeled and learned to obtain the global and local correlations of vehicles. Finally, by training the prediction model, the graph Transformer model is made to have good generalization and robustness, and can accurately output the behavior of autonomous vehicles.

[0004] This technology primarily relies on single-modal data (such as vehicle motion data or road information) for modeling, making it difficult to fully utilize the complementary information between multimodal data (such as data from cameras, LiDAR, radar, and other sensors). Furthermore, this technology lacks generalization ability and robustness in complex traffic scenarios, making it difficult to adapt to diverse application scenarios. Therefore, there is an urgent need for a method for predicting the behavior of autonomous vehicles that can integrate multimodal data and expand application scenarios. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide an artificial intelligence-based method and system for predicting the behavior of autonomous vehicles, which can comprehensively capture the complexity of vehicle behavior and thus improve the accuracy of predicting the behavior of autonomous vehicles.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] An artificial intelligence-based method for predicting the behavior of autonomous vehicles includes:

[0008] Multimodal driving data of sample autonomous vehicles is acquired, and the multimodal driving data is preprocessed to obtain preprocessed data; the multimodal driving data includes: motion data, road data, traffic participant data, and environmental data;

[0009] Feature extraction is performed on the preprocessed data of different modalities to obtain data features of each modality, and the data features are weighted and fused based on the cross-modal attention mechanism to obtain fused features;

[0010] An initial dynamic graph network model and a traffic participant behavior model are constructed; the output of the traffic participant behavior model is the traffic participant behavior.

[0011] The traffic participants' behaviors are used as nodes in the initial dynamic graph network model, and the edges between nodes are determined as interaction relationships. The weights of the edges are dynamically updated according to the fusion features to obtain the updated dynamic graph network model.

[0012] Based on the attention mechanism, the attention distribution of traffic participants to surrounding nearby obstacles is calculated according to the updated dynamic graph network model.

[0013] A multimodal feature vector is constructed based on the node's features and the attention distribution, and the multimodal feature vector is input into the dynamic graph network model to obtain the updated node features;

[0014] A high-level decision model is constructed based on the characteristics of the updated nodes, and the multimodal driving data of the autonomous vehicle under test is input into the high-level decision model to obtain the prediction results.

[0015] Preferably, the expression for the fusion feature is: Among them, F i F represents the feature of the i-th mode; 融合 The fusion feature; α i For F i The weight.

[0016] Preferably, the traffic participant behavior model includes a motor vehicle behavior model, a non-motor vehicle behavior model, and a pedestrian behavior model; the motor vehicle behavior model is constructed based on a longitudinal following model, a lateral lane-changing model, and an intersection interaction model, and is used to describe the motion behavior of motor vehicles; the non-motor vehicle behavior model is constructed by combining driver characteristics, vehicle characteristics, and traffic characteristics; the pedestrian behavior model is constructed based on pedestrian dynamic parameters and cluster behavior characteristics.

[0017] Preferably, the dynamic update formula for the edge weight is: w ij =f(F 融合 ,d ij ,v ij ); where w ij d represents the edge weight between node i and node j; ij v is the distance between node i and node j; ijLet f be the velocity difference between node i and node j; f() is the edge weight update function.

[0018] Preferably, the update formula for the features of the node is: in, For the features of node i in the (l+1)th layer, Let W be the set of neighbors of node i. (l) Let d be the weight matrix of the l-th layer. i and d j Let be the degrees of nodes i and j, respectively, and σ() be the activation function.

[0019] Preferably, the formula for calculating the attention distribution is: Among them, A ij Let f(h) be the attention weight of node i to node j. i ,h k ) is the interaction function between node i and node j.

[0020] Preferably, the prediction results include: lane change decision, acceleration decision, and deceleration decision.

[0021] An artificial intelligence-based autonomous vehicle behavior prediction system includes:

[0022] The data acquisition and processing module is used to acquire multimodal driving data of sample autonomous vehicles and preprocess the multimodal driving data to obtain preprocessed data; the multimodal driving data includes: motion data, road data, traffic participant data and environmental data;

[0023] The feature extraction and fusion module is used to extract features from the preprocessed data of different modalities to obtain data features of each modality, and to perform weighted fusion of the data features based on the cross-modal attention mechanism to obtain fused features;

[0024] The model building module is used to construct an initial dynamic graph network model and a traffic participant behavior model; the output of the traffic participant behavior model is the traffic participant behavior.

[0025] The model update module is used to take the behavior of the traffic participants as nodes in the initial dynamic graph network model, determine the edges between the nodes as interaction relationships, and dynamically update the weights of the edges according to the fusion features to obtain the updated dynamic graph network model.

[0026] The attention calculation module is used to calculate the attention distribution of traffic participants to surrounding nearby obstacles based on the updated dynamic graph network model, according to the attention mechanism.

[0027] The feature update module is used to construct a multimodal feature vector based on the features of the node and the attention distribution, and input the multimodal feature vector into the dynamic graph network model to obtain the updated node features;

[0028] The model prediction module is used to construct a high-level decision model based on the features of the updated nodes, and input the multimodal driving data of the autonomous vehicle under test into the high-level decision model to obtain the prediction results.

[0029] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0030] This invention provides an artificial intelligence-based method and system for predicting the behavior of autonomous vehicles. The method includes: acquiring multimodal driving data of sample autonomous vehicles and preprocessing the multimodal driving data to obtain preprocessed data; the multimodal driving data includes motion data, road data, traffic participant data, and environmental data; extracting features from the preprocessed data of different modalities to obtain data features for each modality, and weighting and fusing the data features based on a cross-modal attention mechanism to obtain fused features; constructing an initial dynamic graph network model and a traffic participant behavior model; the output of the traffic participant behavior model is the traffic participant behavior; and the traffic participant behavior is then... The behavior of traffic participants is used as nodes in the initial dynamic graph network model, and the edges between nodes are defined as interaction relationships. The weights of the edges are dynamically updated according to the fused features to obtain the updated dynamic graph network model. Based on the attention mechanism, the attention distribution of traffic participants to surrounding nearby obstacles is calculated according to the updated dynamic graph network model. Multimodal feature vectors are constructed based on the features of the nodes and the attention distribution, and the multimodal feature vectors are input into the dynamic graph network model to obtain the updated node features. A high-level decision model is constructed based on the updated node features, and the multimodal driving data of the autonomous vehicle under test is input into the high-level decision model to obtain the prediction result. By fusing multimodal driving data (motion data, road data, traffic participant data, and environmental data), this invention can comprehensively capture the complexity of vehicle behavior, thereby improving the accuracy of autonomous vehicle behavior prediction. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1A flowchart of the method provided in an embodiment of the present invention;

[0033] Figure 2 This is a system structure diagram provided for an embodiment of the present invention. Detailed Implementation

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] The purpose of this invention is to provide an artificial intelligence-based method and system for predicting the behavior of autonomous vehicles, which can comprehensively capture the complexity of vehicle behavior and thus improve the accuracy of predicting the behavior of autonomous vehicles.

[0036] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0037] Figure 1 The method flowchart provided in the embodiments of the present invention is as follows: Figure 1 As shown, this invention provides an artificial intelligence-based method for predicting the behavior of autonomous vehicles, comprising:

[0038] Step 100: Acquire multimodal driving data of sample autonomous vehicles and preprocess the multimodal driving data to obtain preprocessed data; the multimodal driving data includes: motion data, road data, traffic participant data and environmental data;

[0039] Step 200: Extract features from the preprocessed data of different modalities to obtain data features of each modality, and perform weighted fusion of the data features based on the cross-modal attention mechanism to obtain fused features;

[0040] Step 300: Construct the initial dynamic graph network model and the traffic participant behavior model; the output of the traffic participant behavior model is the traffic participant behavior.

[0041] Step 400: Use the behavior of traffic participants as nodes in the initial dynamic graph network model, determine the edges between nodes as interaction relationships, and dynamically update the weights of the edges according to the fusion features to obtain the updated dynamic graph network model.

[0042] Step 500: Based on the attention mechanism, calculate the attention distribution of traffic participants to surrounding neighboring obstacles according to the updated dynamic graph network model;

[0043] Step 600: Construct multimodal feature vectors based on node features and attention distribution, and input the multimodal feature vectors into the dynamic graph network model to obtain the updated node features;

[0044] Step 700: Construct a high-level decision model based on the updated node features, and input the multimodal driving data of the autonomous vehicle under test into the high-level decision model to obtain the prediction results.

[0045] Specifically, step 100 in this embodiment includes:

[0046] Step 101: Data Acquisition

[0047] To obtain multimodal driving data from sample autonomous vehicles, data collection is required using various sensor devices on the autonomous vehicles. Specifically, this includes:

[0048] Motion data is collected through the vehicle's internal sensors (such as IMU inertial measurement unit, GPS, vehicle speed sensor, etc.) to obtain dynamic information such as the vehicle's speed, acceleration, direction, and turning radius.

[0049] Road data is obtained through cameras, LiDAR, and high-precision maps, capturing static and dynamic road information such as road topology, lane line information, traffic signs, and traffic lights.

[0050] Traffic participant data is collected by using sensors such as lidar, millimeter-wave radar, and cameras to detect and track surrounding motor vehicles, non-motor vehicles, and pedestrians, and to record their location, speed, acceleration, size, and other characteristics.

[0051] Environmental data is collected through environmental sensors (such as light sensors, temperature sensors, humidity sensors, etc.) to obtain environmental information such as weather, light intensity, and slipperiness.

[0052] This embodiment uses multiple sensors to work together to collect the above data in real time, forming the raw input of multimodal driving data.

[0053] Step 102: Data Cleaning

[0054] This embodiment considers that the collected multimodal driving data may contain noise, outliers, or incomplete data; therefore, data cleaning is required to ensure data quality. Specifically, this includes:

[0055] Noise in sensor data is filtered, for example, Kalman filtering is used for IMU data, and noise is denoised for LiDAR point cloud data.

[0056] Outliers, such as GPS drift or erroneous data caused by sensor malfunctions, are detected and removed using statistical methods or machine learning models.

[0057] For missing data (such as sensor data lost at certain times), it can be supplemented by interpolation algorithms or prediction models based on historical data.

[0058] Since multimodal data comes from different sensors, the sampling frequency and timestamps may be different, so it is necessary to time-align the data to ensure that all data are synchronized under the same time reference.

[0059] Step 103: Data Formatting

[0060] The cleaned multimodal driving data needs to be formatted for subsequent feature extraction and modeling. Specifically, this includes:

[0061] Transform data from different sensors (such as LiDAR point clouds and camera images) into a unified coordinate system, such as the vehicle coordinate system or the global coordinate system.

[0062] Data can be segmented based on time windows or spatial regions, such as dividing continuous driving data into segments of fixed time lengths or dividing point cloud data into different spatial regions.

[0063] Image data is preprocessed (e.g., normalization, cropping, enhancement), point cloud data is downsampled or segmented, and numerical data is normalized or standardized to ensure consistent scale across different modalities.

[0064] Step 104: Data Storage and Management

[0065] Preprocessed multimodal driving data needs to be stored and managed for subsequent feature extraction and modeling. Specifically, this includes:

[0066] Store the preprocessed data in a high-performance database or file system, such as using HDF5 format to store multimodal data, or using a distributed storage system to manage large-scale data.

[0067] The stored data is labeled, for example, the categories of traffic participants (motor vehicles, non-motor vehicles, pedestrians), behaviors (acceleration, deceleration, lane changes, etc.), and characteristics of road scenes (intersections, straight roads, curves, etc.).

[0068] Index the stored data to facilitate fast retrieval and querying, for example, by timestamp, geographic location, or traffic scenario type.

[0069] Through the above steps, this embodiment completes the acquisition and preprocessing of multimodal driving data of sample autonomous vehicles, laying the foundation for subsequent feature extraction and behavior prediction model construction.

[0070] Preferably, the expression for the fusion feature is: Among them, F i F represents the feature of the i-th mode; 融合 The fusion feature; α i For F i The weight.

[0071] Specifically, in step 200 of this embodiment, the weight allocation is achieved through adaptive learning via a neural network. In other words, the cross-modal attention mechanism dynamically adjusts the weight allocation based on the contribution of different modal features. For example, when a certain modality (such as visual data) is more important in the current scene, its weight will automatically increase, while the weights of other modalities will decrease accordingly.

[0072] Preferably, the traffic participant behavior model includes a motor vehicle behavior model, a non-motor vehicle behavior model, and a pedestrian behavior model; the motor vehicle behavior model is constructed based on a longitudinal following model, a lateral lane-changing model, and an intersection interaction model, and is used to describe the motion behavior of motor vehicles; the non-motor vehicle behavior model is constructed by combining driver characteristics, vehicle characteristics, and traffic characteristics; the pedestrian behavior model is constructed based on pedestrian dynamic parameters and cluster behavior characteristics.

[0073] Specifically, the traffic participant behavior model in this embodiment is used to predict the behavior (such as acceleration, deceleration, lane changing, and avoidance) of traffic participants (such as motor vehicles, non-motor vehicles, and pedestrians). This model can be built based on deep learning methods, specifically employing a multilayer perceptron (MLP) or recurrent neural network (RNN) structure. Inputs include the dynamic features of traffic participants (such as speed, acceleration, position, and orientation) and environmental features (such as road topology and information about nearby obstacles). The model structure includes:

[0074] Input layer: Receives feature vectors X of traffic participants, including motion features, environmental features, and interaction features.

[0075] Hidden layer: Extracts behavioral features of traffic participants through multi-layer fully connected networks or time series modeling (such as LSTM or GRU).

[0076] Output layer: Outputs the behavior categories of traffic participants (such as acceleration, deceleration, lane change, etc.) or continuous behavior parameters (such as target speed, target position, etc.).

[0077] The model in this embodiment is trained through supervised learning, using labeled traffic participant behavior data as the training set, and optimizing the cross-entropy loss or mean squared error loss function.

[0078] Furthermore, the initial dynamic graph network model in this embodiment is used to model the interaction relationships between traffic participants. Its structure is based on a dynamic graph neural network (DGNN), where nodes represent traffic participants and edges represent the interaction relationships between them. The specific construction process is as follows:

[0079] Node initialization: The output of the traffic participant behavior model (such as behavior category or behavior parameters) is used as the node feature h. i Each node corresponds to a traffic participant.

[0080] Edge initialization: Initialize the edge weight w based on information such as distance, speed difference, and relative position between traffic participants. ij , representing the interaction strength between node i and node j.

[0081] Dynamic graph update: Through the propagation mechanism of dynamic graph networks (such as graph convolutional networks GCN or graph attention networks GAT), node features and edge weights are dynamically updated to capture the real-time interaction relationships between traffic participants.

[0082] Output: The initial dynamic graph network model outputs updated node features and edge weights, providing input for subsequent attention distribution calculations and high-level decision models.

[0083] Through the above process, the traffic participant behavior model and the initial dynamic graph network model in this embodiment jointly construct a dynamic and interactive behavior prediction framework, which can effectively capture the individual behavior of traffic participants and their mutual influence.

[0084] Specifically, in step 400 of this embodiment, the output of the traffic participant behavior model is used as the node features in the initial dynamic graph network model. Specifically, each traffic participant (such as a vehicle, pedestrian, or non-motorized vehicle) is represented as a node, and its feature vector h... i This includes behavioral information of traffic participants (such as acceleration, deceleration, lane changing, etc.), motion states (such as speed, acceleration, position, etc.), and environmental features (such as information about surrounding obstacles). These features are extracted through a traffic participant behavior model and used as the initial input to the nodes, forming the node set of the dynamic graph network. Edges between nodes represent the interaction relationships between traffic participants, with initial weights w. ij Based on the relative relationship characteristics between two nodes (such as distance d) ij Speed ​​difference v ij (relative direction, etc.) and multimodal fusion features F 融合 A joint decision.

[0085] Preferably, the dynamic update formula for the edge weight is: w ij=f(F 融合 ,d ij ,v ij ); where w ij d represents the edge weight between node i and node j; ij v is the distance between node i and node j; ij Let f be the velocity difference between node i and node j; f() is the edge weight update function.

[0086] The input to the edge weight update function f() includes the node features h of traffic participants i and j. i and h j The relative relationship between the two (e.g., distance d) ij Speed ​​difference v ij (relative direction, etc.), and multimodal fusion features F 融合 These inputs are modeled using a learnable neural network, as shown in the following formula:

[0087] w ij =f(h) i ,h j ,d ij ,v ij ,F 融合 )

[0088] Where f is the edge weight update function, implemented by a multilayer perceptron (MLP), the specific structure of which includes:

[0089] Input layer: Receive node features h i h j Relative relation characteristics d ij v ij and fusion feature F 融合 The concatenated vector.

[0090] Hidden layers: High-dimensional interactive features are extracted through several fully connected layers. The activation function is usually ReLU or LeakyReLU.

[0091] Output layer: Output edge weight w ij , representing the interaction strength between node i and node j, is normalized to the range [0,1] using the Sigmoid or Softmax function.

[0092] During training, the edge weight update function is optimized through backpropagation, aiming to minimize the loss function of the behavior prediction task. This ensures that the edge weights accurately reflect the interaction relationships between traffic participants and their contribution to behavior prediction. In this embodiment, the edge weights dynamically adapt to changes in the traffic scene, capturing real-time interaction characteristics.

[0093] Preferably, the update formula for the features of the node is: in, For the features of node i in the (l+1)th layer, Let W be the set of neighbors of node i. (l) Let d be the weight matrix of the l-th layer. i and d j Let be the degrees of nodes i and j, respectively, and σ() be the activation function.

[0094] Preferably, the formula for calculating the attention distribution is: Among them, A ij Let f(h) be the attention weight of node i to node j. i ,h k ) is the interaction function between node i and node j.

[0095] Specifically, in this embodiment, the interaction relationship function is a formula for calculating the cosine similarity of the features of two nodes.

[0096] Specifically, in step 500 of this embodiment, the features of each traffic participant node and its edge weights with neighboring obstacle nodes are obtained from the updated dynamic graph network model. These edge weights dynamically reflect the strength of the interaction between the traffic participant and the obstacle. Then, through an attention mechanism, the features of the traffic participant node are associated with the features of its neighboring obstacle nodes, and a learnable attention function (such as a function based on feature similarity or interaction relationship) is used to calculate the attention score of each neighboring obstacle for the traffic participant. Next, the attention scores of all neighboring obstacles are normalized (e.g., through a softmax function) to generate an attention distribution representing the degree of attention paid by the traffic participant to different obstacles. Finally, these attention distributions are used to measure the priority of the traffic participant to surrounding obstacles in the current scene, thereby providing a basis for subsequent behavior prediction and decision-making.

[0097] Further, in step 600 of this embodiment, firstly, based on the node's features (such as the behavioral features and motion state features of traffic participants) and attention distribution, the features of the node and its neighboring nodes (such as obstacle nodes) are weighted and aggregated. The weights are determined by the attention distribution, representing the importance of each neighboring node to the current node. Next, the aggregated features of the neighboring nodes are fused with the current node's own features. Fusion methods include feature concatenation, weighted summation, or nonlinear transformation through a neural network, thereby generating a feature vector containing multimodal information. This multimodal feature vector integrates the node's own feature information and the interaction information of neighboring nodes, enabling a more comprehensive representation of the node's state and environmental relationships. Finally, the generated multimodal feature vector is used as input and passed to the next layer of the dynamic graph network model. The node's features are further updated through the network's propagation mechanism (such as graph convolution or graph attention mechanism), allowing it to dynamically reflect the changes in node behavior and interaction in the traffic scene.

[0098] Furthermore, in step 700 of this embodiment, the core of the high-level decision model is to extract global information and predict driving behavior based on the updated node features. This model is typically built using a deep learning framework, combining the output features of a dynamic graph network model to capture the dynamic interaction between the autonomous vehicle and surrounding traffic participants. Specifically, the updated node features (such as the behavioral features, motion state features, and interaction features of the autonomous vehicle and surrounding traffic participants) are further processed, and time-series features are extracted through fully connected layers, attention mechanisms, or recurrent neural networks (such as LSTM and GRU) to model the dynamic changes in the traffic scene. The structure of the high-level decision model may include an input layer (receiving node features), a hidden layer (extracting high-dimensional features), and an output layer (generating driving decisions or behavior prediction results). In this embodiment, the multimodal driving data of the autonomous vehicle under test is input into the high-level decision model. The multimodal data includes the vehicle's motion state (such as speed, acceleration, direction, etc.), environmental perception information (such as obstacle positions, traffic signals, road topology, etc.), and interaction features with surrounding traffic participants. After being processed by the preceding dynamic graph network model, this data has been integrated into updated node features, comprehensively reflecting the state and interactions of autonomous vehicles in the current traffic scenario. The high-level decision-making model utilizes this input data, combined with historical information and scenario dynamics, to generate predictions of future driving behavior.

[0099] Preferably, the prediction results include: lane change decision, acceleration decision, and deceleration decision.

[0100] Optionally, the high-level decision-making model in this embodiment outputs prediction results for the autonomous vehicle based on the input multimodal driving data. The prediction results may include driving behaviors (such as acceleration, deceleration, lane changing, and parking), trajectory planning (such as future driving paths), or high-level decisions (such as choosing overtaking, following, or avoidance strategies). Through deep analysis of node features and combined with the global dynamics of the traffic scene, the model generates optimal decisions that conform to the current environment and driving objectives. Ultimately, the prediction results can be used to guide the control module of the autonomous vehicle to achieve safe and efficient driving behavior.

[0101] Figure 2 The system structure diagram provided in the embodiments of the present invention is as follows: Figure 2 As shown, the present invention also provides an artificial intelligence-based autonomous vehicle behavior prediction system, comprising:

[0102] The data acquisition and processing module is used to acquire multimodal driving data of sample autonomous vehicles and preprocess the multimodal driving data to obtain preprocessed data; the multimodal driving data includes: motion data, road data, traffic participant data and environmental data;

[0103] The feature extraction and fusion module is used to extract features from the preprocessed data of different modalities to obtain data features of each modality, and to perform weighted fusion of the data features based on the cross-modal attention mechanism to obtain fused features;

[0104] The model building module is used to construct an initial dynamic graph network model and a traffic participant behavior model; the output of the traffic participant behavior model is the traffic participant behavior.

[0105] The model update module is used to take the behavior of the traffic participants as nodes in the initial dynamic graph network model, determine the edges between the nodes as interaction relationships, and dynamically update the weights of the edges according to the fusion features to obtain the updated dynamic graph network model.

[0106] The attention calculation module is used to calculate the attention distribution of traffic participants to surrounding nearby obstacles based on the updated dynamic graph network model, according to the attention mechanism.

[0107] The feature update module is used to construct a multimodal feature vector based on the features of the node and the attention distribution, and input the multimodal feature vector into the dynamic graph network model to obtain the updated node features;

[0108] The model prediction module is used to construct a high-level decision model based on the features of the updated nodes, and input the multimodal driving data of the autonomous vehicle under test into the high-level decision model to obtain the prediction results.

[0109] The beneficial effects of this invention are as follows:

[0110] (1) By integrating multimodal driving data (motion data, road data, traffic participant data and environmental data), this invention can comprehensively capture the complexity of vehicle behavior, thereby improving the accuracy of predicting autonomous vehicle behavior.

[0111] (2) This invention utilizes a dynamic graph network model, which can update the interaction relationships between nodes and the weights of edges in real time, adapting to changes in the dynamic traffic environment. This dynamic update capability enables the model to better reflect changes in actual traffic scenarios, improving the timeliness and reliability of predictions.

[0112] (3) This invention uses a cross-modal attention mechanism to weightedly fuse data features from different modalities, which can effectively integrate information from different sensors, improve information utilization, and enhance the model’s comprehensive judgment ability.

[0113] (4) By constructing a traffic participant behavior model, this invention can better understand and predict the behavior of traffic participants (such as pedestrians, non-motorized vehicles, etc.), thereby enhancing the robustness of the system in complex traffic scenarios.

[0114] (5) By constructing a high-level decision-making model, this invention can generate more reasonable driving decisions based on updated node features and multimodal feature vectors. This provides stronger decision support capabilities for autonomous driving systems, improving the safety and efficiency of autonomous driving.

[0115] (6) The method of the present invention can adapt to different traffic environments and scenarios, has strong generalization ability, and can perform effective behavior prediction in a variety of complex situations.

[0116] (7) This invention can effectively reduce the occurrence of traffic accidents and improve traffic safety through accurate behavior prediction and reasonable decision support.

[0117] (8) This invention promotes the development of intelligent transportation systems: This invention provides technical support for the realization of intelligent transportation systems and promotes the progress and application of autonomous driving technology.

[0118] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0119] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An artificial intelligence-based automatic driving vehicle behavior prediction method, characterized by, The method comprises the following steps: acquiring multi-modal driving data of a sample autonomous vehicle, and performing data preprocessing on the multi-modal driving data to obtain preprocessed data; the multi-modal driving data comprises motion data, road data, traffic participant data, and environmental data; extracting features from the preprocessed data of different modalities respectively to obtain data features of each modality, and performing weighted fusion on the data features based on a cross-modal attention mechanism method to obtain fused features; constructing an initial dynamic graph network model and a traffic participant behavior model; the output of the traffic participant behavior model is traffic participant behavior; taking the traffic participant behavior as a node in the initial dynamic graph network model, determining the edges between nodes as interaction relationships, and dynamically updating the weights of the edges according to the fused features to obtain an updated dynamic graph network model; based on an attention mechanism, calculating the attention distribution of a traffic participant to surrounding adjacent obstacles according to the updated dynamic graph network model; constructing a multi-modal feature vector according to the features of the nodes and the attention distribution, and inputting the multi-modal feature vector into the dynamic graph network model to obtain updated features of the nodes; constructing a high-level decision-making model according to the updated features of the nodes, and inputting multi-modal driving data of a to-be-tested autonomous vehicle into the high-level decision-making model to obtain a prediction result. 2.The AI-based automated driving vehicle behavior prediction method of claim 1, wherein, The expression of the fusion feature is: wherein F i is the feature of the i-th modality; F 融合 is the fusion feature; α i is the weight of F i . 3.The AI-based automated driving vehicle behavior prediction method of claim 1, wherein, The traffic participant behavior model comprises a motor vehicle behavior model, a non-motor vehicle behavior model, and a pedestrian behavior model; the motor vehicle behavior model is constructed based on a longitudinal following model, a lateral lane-changing model, and an intersection interaction model, and is used to describe the motion behavior of a motor vehicle; the non-motor vehicle behavior model is constructed in combination with driver characteristics, vehicle characteristics, and traffic characteristics; and the pedestrian behavior model is constructed based on pedestrian dynamics parameters and crowd behavior characteristics. 4.The AI-based automated driving vehicle behavior prediction method of claim 2, wherein, The dynamic updating formula of the weight of the edge is: w ij = f(F 融合 ,d ij ,v ij ); wherein w ij is the weight of the edge between node i and node j; d ij is the distance between node i and node j; v ij is the speed difference of node i and node j; and f() is an edge weight updating function. 5.The AI-based automated driving vehicle behavior prediction method of claim 4, wherein, The update formula of the feature of the node is: Wherein, is the feature of node i at the l+1 layer, is the neighbor set of node i, W (l) is the weight matrix of the l layer, d i and d j are the degrees of node i and node j respectively, and σ() is an activation function. 6.The AI-based automated driving vehicle behavior prediction method of claim 5, wherein, The calculation formula of the attention distribution is: Wherein, A ij is the attention weight of node i to node j, f(h i , h k ) is the interaction function of node i and node j. 7.The AI-based automated driving vehicle behavior prediction method of claim 1, wherein, The prediction result comprises a lane-changing decision, an acceleration decision, and a deceleration decision.

8. An artificial intelligence-based autonomous vehicle behavior prediction system, characterized by, The method comprises the following steps: a data acquisition and processing module is configured to acquire multi-modal driving data of a sample autonomous vehicle, and perform data preprocessing on the multi-modal driving data to obtain preprocessed data; the multi-modal driving data comprises motion data, road data, traffic participant data, and environmental data; a feature extraction and fusion module is configured to extract features from the preprocessed data of different modalities respectively to obtain data features of each modality, and perform weighted fusion on the data features based on a cross-modal attention mechanism method to obtain fused features; a model construction module is configured to construct an initial dynamic graph network model and a traffic participant behavior model; the output of the traffic participant behavior model is traffic participant behavior; a model updating module is configured to take the traffic participant behavior as a node in the initial dynamic graph network model, determine the edges between nodes as interaction relationships, and dynamically update the weights of the edges according to the fused features to obtain an updated dynamic graph network model; an attention calculation module is configured to calculate the attention distribution of a traffic participant to surrounding adjacent obstacles based on an attention mechanism and according to the updated dynamic graph network model; a feature updating module, configured to construct a multi-modal feature vector according to the features of the nodes and the attention distribution, and input the multi-modal feature vector into the dynamic graph network model to obtain updated features of the nodes; a model prediction module, configured to construct a high-level decision-making model according to the updated features of the nodes, and input multi-modal driving data of a to-be-tested autonomous vehicle into the high-level decision-making model to obtain a prediction result.

Images