Automatic driving vehicle behavior prediction method and system based on artificial intelligence

Abstract

The invention provides an artificial intelligence-based automatic driving vehicle behavior prediction method and system. The method comprises the following steps of: firstly, acquiring multi-mode driving data of a sample automatic driving vehicle and preprocessing the multi-mode driving data; feature extraction is carried out on data of different modals, and weighted fusion is carried out based on a cross-modal attention mechanism; and constructing an initial dynamic graph network model and a traffic participant behavior model, taking traffic participant behaviors as nodes, determining an interaction relationship between the nodes, and dynamically updating the weight of the edge to obtain an updated dynamic graph network model. On the basis of an attention mechanism, attention distribution of the traffic participants to surrounding obstacles is calculated, and a multi-modal feature vector is constructed and input into the dynamic graph network model to obtain updated node features; and constructing a high-level decision model according to the updated node features, and inputting the multi-modal data of the to-be-tested autonomous vehicle into the model to generate a result. According to the invention, by fusing the multi-modal data, the accuracy of behavior prediction of the automatic driving vehicle is improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to an artificial intelligence-based autonomous driving vehicle behavior prediction method and system. Background Art

[0002] With the rapid development of autonomous driving technology, vehicle behavior prediction has become a key link in realizing advanced driving functions.

[0003] The invention patent with authorization announcement number CN118025203B discloses a system and method for predicting the behavior of an autonomous driving vehicle that integrates a complex network and a graph transformer. First, a complex network of vehicle interactions is constructed, and the original map information data, vehicle motion data, and local interaction information are used as multi-layer inputs of the model. Then, with the help of the graph convolution and multi-head node attention mechanism of the graph transformer, the complex relationships between vehicles are modeled and learned to obtain the global and local correlations of the vehicles. Finally, by training the prediction model, the graph transformer model has good generalization and robustness, and can accurately output the behavior of the autonomous driving vehicle.

[0004] This technology mainly relies on single-modal data (such as vehicle motion data or road information) for modeling, and it is difficult to fully utilize the complementary information between multi-modal data (such as sensor data such as cameras, lidar, radar, etc.). In addition, this technology lacks generalization and robustness in complex traffic scenarios, and it is difficult to adapt to diverse application scenarios. Therefore, there is an urgent need for an autonomous driving vehicle behavior prediction method that can integrate multi-modal data and expand application scenarios. Summary of the invention

[0005] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide an artificial intelligence-based autonomous driving vehicle behavior prediction method and system, which can fully capture the complexity of vehicle behavior, thereby improving the accuracy of autonomous driving vehicle behavior prediction.

[0006] To achieve the above object, the present invention provides the following solutions:

[0007] A method for predicting behavior of an autonomous driving vehicle based on artificial intelligence, comprising:

[0008] Acquire multimodal driving data of a sample autonomous driving vehicle, and perform data preprocessing on the multimodal driving data to obtain preprocessed data; the multimodal driving data includes: motion data, road data, traffic participant data, and environment data;

[0009] Extracting features from the preprocessed data of different modalities respectively to obtain data features of each modality, and weighted fusion of the data features based on a cross-modal attention mechanism method to obtain fusion features;

[0010] Constructing an initial dynamic graph network model and a traffic participant behavior model; the output of the traffic participant behavior model is the traffic participant behavior;

[0011] The behaviors of the traffic participants are used as nodes in the initial dynamic graph network model, and the edges between the nodes are determined as interactive relationships, and the weights of the edges are dynamically updated according to the fusion features to obtain an updated dynamic graph network model;

[0012] Based on the attention mechanism, the attention distribution of the traffic participants to the surrounding adjacent obstacles is calculated according to the updated dynamic graph network model;

[0013] Constructing a multimodal feature vector according to the feature of the node and the attention distribution, and inputting the multimodal feature vector into the dynamic graph network model to obtain the feature of the updated node;

[0014] A high-level decision model is constructed according to the characteristics of the updated nodes, and the multimodal driving data of the autonomous driving vehicle to be tested is input into the high-level decision model to obtain a prediction result.

[0015] Preferably, the expression of the fusion feature is: Among them, F i is the characteristic of the i-th mode; F 融合 is the fusion feature; α i F i The weight of .

[0016] Preferably, the traffic participant behavior model includes a motor vehicle behavior model, a non-motor vehicle behavior model and a pedestrian behavior model; the motor vehicle behavior model is constructed based on a longitudinal following model, a lateral lane change model and an intersection interaction model, and the motor vehicle behavior model is used to describe the movement behavior of a motor vehicle; the non-motor vehicle behavior model is constructed in combination with driver characteristics, vehicle characteristics and traffic characteristics; the pedestrian behavior model is constructed based on pedestrian dynamic parameters and cluster behavior characteristics.

[0017] Preferably, the dynamic update formula of the edge weight is: ij =f(F 融合 ,d ij ,v ij ), where w ij is the edge weight between node i and node j; d ij is the distance between node i and node j; v ijis the speed difference between node i and node j; f() is the edge weight update function.

[0018] Preferably, the updating formula of the node feature is: in, is the feature of node i at layer l+1, is the neighbor set of node i, W (l) is the weight matrix of the lth layer, d i and d j are the degrees of node i and node j respectively, and σ() is the activation function.

[0019] Preferably, the calculation formula for the attention distribution is: Among them, A ij is the attention weight of node i to node j, f(h i ,h k ) is the interaction relationship function between node i and node j.

[0020] Preferably, the prediction results include: lane change decision, acceleration decision and deceleration decision.

[0021] An artificial intelligence-based autonomous driving vehicle behavior prediction system, comprising:

[0022] A data acquisition and processing module, used to obtain multimodal driving data of a sample autonomous driving vehicle, and perform data preprocessing on the multimodal driving data to obtain preprocessed data; the multimodal driving data includes: motion data, road data, traffic participant data and environmental data;

[0023] A feature extraction and fusion module, used to extract features from the preprocessed data of different modalities respectively to obtain data features of each modality, and to perform weighted fusion on the data features based on a cross-modal attention mechanism method to obtain fused features;

[0024] A model building module is used to build an initial dynamic graph network model and a traffic participant behavior model; the output of the traffic participant behavior model is the traffic participant behavior;

[0025] A model updating module, used to use the behaviors of the traffic participants as nodes in the initial dynamic graph network model, determine the edges between the nodes as interactive relationships, and dynamically update the weights of the edges according to the fusion features to obtain an updated dynamic graph network model;

[0026] An attention calculation module, used for calculating the attention distribution of traffic participants to surrounding adjacent obstacles based on the attention mechanism and the updated dynamic graph network model;

[0027] A feature updating module, used for constructing a multimodal feature vector according to the feature of the node and the attention distribution, and inputting the multimodal feature vector into the dynamic graph network model to obtain the feature of the updated node;

[0028] The model prediction module is used to construct a high-level decision model according to the characteristics of the updated nodes, and input the multimodal driving data of the autonomous driving vehicle to be tested into the high-level decision model to obtain a prediction result.

[0029] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0030] The present invention provides an artificial intelligence-based autonomous driving vehicle behavior prediction method and system, the method comprising: obtaining multimodal driving data of a sample autonomous driving vehicle, and performing data preprocessing on the multimodal driving data to obtain preprocessed data; the multimodal driving data comprises: motion data, road data, traffic participant data and environmental data; performing feature extraction on the preprocessed data of different modes respectively to obtain data features of each mode, and performing weighted fusion on the data features based on a cross-modal attention mechanism method to obtain fused features; constructing an initial dynamic graph network model and a traffic participant behavior model; the output of the traffic participant behavior model is the traffic participant behavior; the traffic participant The behavior of objects is used as nodes in the initial dynamic graph network model, and the edges between nodes are determined as interactive relationships, and the weights of the edges are dynamically updated according to the fusion features to obtain an updated dynamic graph network model; based on the attention mechanism, the attention distribution of traffic participants to surrounding adjacent obstacles is calculated according to the updated dynamic graph network model; a multimodal feature vector is constructed according to the characteristics of the node and the attention distribution, and the multimodal feature vector is input into the dynamic graph network model to obtain the characteristics of the updated node; a high-level decision model is constructed according to the characteristics of the updated node, and the multimodal driving data of the autonomous driving vehicle to be tested is input into the high-level decision model to obtain a prediction result. By fusing multimodal driving data (motion data, road data, traffic participant data and environmental data), the present invention can fully capture the complexity of vehicle behavior, thereby improving the accuracy of autonomous driving vehicle behavior prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0032] Figure 1A flow chart of a method provided by an embodiment of the present invention;

[0033] Figure 2 A system structure diagram provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] The purpose of the present invention is to provide an artificial intelligence-based autonomous driving vehicle behavior prediction method and system, which can fully capture the complexity of vehicle behavior, thereby improving the accuracy of autonomous driving vehicle behavior prediction.

[0036] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0037] Figure 1 A flow chart of a method provided by an embodiment of the present invention, such as Figure 1 As shown, the present invention provides an artificial intelligence-based automatic driving vehicle behavior prediction method, comprising:

[0038] Step 100: Acquire multimodal driving data of a sample autonomous driving vehicle, and perform data preprocessing on the multimodal driving data to obtain preprocessed data; the multimodal driving data includes: motion data, road data, traffic participant data, and environment data;

[0039] Step 200: extracting features from preprocessed data of different modalities respectively to obtain data features of each modality, and performing weighted fusion on the data features based on a cross-modal attention mechanism method to obtain fused features;

[0040] Step 300: constructing an initial dynamic graph network model and a traffic participant behavior model; the output of the traffic participant behavior model is the traffic participant behavior;

[0041] Step 400: The behaviors of traffic participants are used as nodes in the initial dynamic graph network model, and the edges between the nodes are determined as interactive relationships, and the weights of the edges are dynamically updated according to the fusion features to obtain an updated dynamic graph network model;

[0042] Step 500: Based on the attention mechanism, calculate the attention distribution of the traffic participants to the surrounding adjacent obstacles according to the updated dynamic graph network model;

[0043] Step 600: construct a multimodal feature vector according to the node features and attention distribution, and input the multimodal feature vector into the dynamic graph network model to obtain updated node features;

[0044] Step 700: Construct a high-level decision model based on the characteristics of the updated nodes, and input the multimodal driving data of the autonomous driving vehicle to be tested into the high-level decision model to obtain a prediction result.

[0045] Specifically, step 100 of this embodiment includes:

[0046] Step 101: Data Collection

[0047] In order to obtain multimodal driving data of the sample autonomous driving vehicle, it is necessary to collect data through various sensor devices of the autonomous driving vehicle. Specifically, it includes:

[0048] Motion data collects dynamic information such as the vehicle's speed, acceleration, direction, turning radius, etc. through the vehicle's internal sensors (such as IMU inertial measurement unit, GPS, vehicle speed sensor, etc.).

[0049] Road data, using cameras, lidar and high-precision maps to obtain static and dynamic road information such as road topology, lane information, traffic signs, and traffic lights.

[0050] Traffic participant data, through sensors such as lidar, millimeter-wave radar and cameras, detects and tracks surrounding motor vehicles, non-motor vehicles and pedestrians, and records their position, speed, acceleration, size and other characteristics.

[0051] Environmental data, collects environmental information such as weather, light, and wetness through environmental sensors (such as light sensors, temperature sensors, humidity sensors, etc.).

[0052] This embodiment collects the above data in real time through the collaborative work of multiple sensors to form the original input of multi-modal driving data.

[0053] Step 102: Data cleaning

[0054] This embodiment considers that the collected multimodal driving data contains noise, outliers or incomplete data, so data cleaning is required to ensure data quality. Specifically, it includes:

[0055] Filter the noise in the sensor data, such as using Kalman filtering on IMU data and denoising the lidar point cloud data.

[0056] Detect and remove outliers through statistical methods or machine learning models, such as erroneous data caused by GPS drift and sensor failure.

[0057] Missing data (such as missing sensor data at certain moments) can be supplemented through interpolation algorithms or prediction models based on historical data.

[0058] Since multimodal data comes from different sensors, the sampling frequency and timestamp may be different, so the data needs to be time-aligned to ensure that all data are synchronized under the same time base.

[0059] Step 103: Data Formatting

[0060] The cleaned multimodal driving data needs to be formatted for subsequent feature extraction and modeling. Specifically, it includes:

[0061] Convert data from different sensors (such as lidar point clouds and camera images) into a unified coordinate system, such as a vehicle coordinate system or a global coordinate system.

[0062] Segment the data according to time windows or spatial regions, such as segmenting continuous driving data into segments of fixed time length, or segmenting point cloud data into different spatial regions.

[0063] Preprocess the image data (such as normalization, cropping, and enhancement), downsample or block the point cloud data, and normalize or standardize the numerical data to ensure that the scales of different modal data are consistent.

[0064] Step 104: Data Storage and Management

[0065] The pre-processed multimodal driving data needs to be stored and managed for subsequent feature extraction and modeling. Specifically, it includes:

[0066] Store the preprocessed data in an efficient database or file system, such as using the HDF5 format to store multimodal data, or using a distributed storage system to manage large-scale data.

[0067] The stored data is labeled, for example, the categories of traffic participants (motor vehicles, non-motor vehicles, pedestrians), behaviors (acceleration, deceleration, lane changes, etc.) and characteristics of road scenes (intersections, straight roads, curves, etc.).

[0068] Create indexes for stored data to facilitate fast retrieval and query, such as indexing by timestamp, geographic location, or traffic scene type.

[0069] Through the above steps, this embodiment completes the acquisition and preprocessing of the multimodal driving data of the sample autonomous driving vehicle, laying the foundation for the subsequent feature extraction and construction of the behavior prediction model.

[0070] Preferably, the expression of the fusion feature is: Among them, F i is the characteristic of the i-th mode; F 融合 is the fusion feature; α i F i The weight of .

[0071] Specifically, in step 200 of this embodiment, the weight distribution is learned through neural network adaptive learning. Specifically, the cross-modal attention mechanism dynamically adjusts the weight distribution according to the contribution of different modal features. For example, when a certain modality (such as visual data) is more important in the current scene, its weight will automatically increase, while the weights of other modalities will decrease accordingly.

[0072] Preferably, the traffic participant behavior model includes a motor vehicle behavior model, a non-motor vehicle behavior model and a pedestrian behavior model; the motor vehicle behavior model is constructed based on a longitudinal following model, a lateral lane change model and an intersection interaction model, and the motor vehicle behavior model is used to describe the movement behavior of a motor vehicle; the non-motor vehicle behavior model is constructed in combination with driver characteristics, vehicle characteristics and traffic characteristics; the pedestrian behavior model is constructed based on pedestrian dynamic parameters and cluster behavior characteristics.

[0073] Specifically, the traffic participant behavior model of this embodiment is used to predict the behavior (such as acceleration, deceleration, lane change, avoidance, etc.) of traffic participants (such as motor vehicles, non-motor vehicles and pedestrians). The model can be constructed based on a deep learning method, specifically using a multi-layer perceptron (MLP) or a recurrent neural network (RNN) structure. The input includes the dynamic characteristics of traffic participants (such as speed, acceleration, position, direction, etc.) and environmental characteristics (such as road topology, adjacent obstacle information, etc.). The model structure includes:

[0074] Input layer: Receives the feature vector X of traffic participants, including motion features, environmental features, and interaction features.

[0075] Hidden layer: The behavioral characteristics of traffic participants are extracted through a multi-layer fully connected network or time series modeling (such as LSTM or GRU).

[0076] Output layer: Outputs the behavior categories of traffic participants (such as acceleration, deceleration, lane change, etc.) or continuous behavior parameters (such as target speed, target position, etc.).

[0077] The model of this embodiment is trained through supervised learning, using labeled traffic participant behavior data as a training set to optimize the cross entropy loss or mean square error loss function.

[0078] Furthermore, the initial dynamic graph network model of this embodiment is used to model the interaction relationship between traffic participants. Its structure is based on the dynamic graph neural network (DGNN), where nodes represent traffic participants and edges represent the interaction relationship between traffic participants. The specific construction process is as follows:

[0079] Node initialization: The output of the traffic participant behavior model (such as behavior category or behavior parameter) is used as the node feature h i , each node corresponds to a traffic participant.

[0080] Edge initialization: Initialize the edge weight w based on the distance, speed difference, relative position and other information between traffic participants ij , represents the interaction strength between node i and node j.

[0081] Dynamic graph update: Through the propagation mechanism of the dynamic graph network (such as graph convolutional network GCN or graph attention network GAT), the node features and edge weights are dynamically updated to capture the real-time interaction relationship between traffic participants.

[0082] Output: The initial dynamic graph network model outputs updated node features and edge weights, providing input for subsequent attention distribution calculations and high-level decision models.

[0083] Through the above process, the traffic participant behavior model and the initial dynamic graph network model of this embodiment jointly construct a dynamic and interactive behavior prediction framework, which can effectively capture the individual behaviors of traffic participants and their mutual influence.

[0084] Specifically, in step 400 of this embodiment, the output of the traffic participant behavior model is used as the node feature in the initial dynamic graph network model. Specifically, each traffic participant (such as a vehicle, pedestrian, non-motor vehicle, etc.) is represented as a node, and its feature vector h i , including the behavior information of traffic participants (such as acceleration, deceleration, lane change and other behavior categories or behavior parameters), motion state (such as speed, acceleration, position, etc.) and environmental characteristics (such as surrounding obstacle information). These features are extracted through the traffic participant behavior model and used as the initial input of the node to form the node set of the dynamic graph network. The edges between the nodes represent the interaction relationship between the traffic participants, and the initial weight of the edge w ij The relative relationship characteristics between two nodes (such as distance d ij , speed difference v ij , relative direction, etc.) and multimodal fusion feature F 融合 Joint decision.

[0085] Preferably, the dynamic update formula of the edge weight is: ij=f(F 融合 ,d ij ,v ij ), where w ij is the edge weight between node i and node j; d ij is the distance between node i and node j; v ij is the speed difference between node i and node j; f() is the edge weight update function.

[0086] The input of the edge weight update function f() includes the node features h of traffic participants i and j i and h j , the relative relationship characteristics between the two (such as the distance d ij , speed difference v ij , relative direction, etc.), and multimodal fusion feature F 融合 These inputs are modeled by a learnable neural network with the following formula:

[0087] w ij =f(h i ,h j ,d ij ,v ij ,F 融合 )

[0088] Among them, f is the edge weight update function, which is implemented by a multi-layer perceptron (MLP). The specific structure includes:

[0089] Input layer: receives node features h i 、h j , relative relationship feature d ij 、v ij , and the fusion feature F 融合 The stitching vector.

[0090] Hidden layer: High-dimensional interaction features are extracted through several fully connected layers, and the activation function usually uses ReLU or LeakyReLU.

[0091] Output layer: output edge weight w ij , represents the interaction strength between node i and node j, which is normalized to the range of [0,1] through Sigmoid or Softmax function.

[0092] During the training process, the edge weight update function is optimized through back propagation, with the goal of minimizing the loss function of the behavior prediction task, so that the edge weight can accurately reflect the interaction relationship between traffic participants and their contribution to behavior prediction. In this way, the edge weight of this embodiment dynamically adapts to changes in traffic scenarios and captures real-time interactive characteristics.

[0093] Preferably, the updating formula of the node feature is: in, is the feature of node i at layer l+1, is the neighbor set of node i, W (l) is the weight matrix of the lth layer, d i and d j are the degrees of node i and node j respectively, and σ() is the activation function.

[0094] Preferably, the calculation formula for the attention distribution is: Among them, A ij is the attention weight of node i to node j, f(h i ,h k ) is the interaction relationship function between node i and node j.

[0095] Specifically, the interaction relationship function in this embodiment is a calculation formula for calculating the cosine similarity of features of two nodes.

[0096] Specifically, in step 500 of this embodiment, the features of each traffic participant node and the edge weights between it and the adjacent obstacle nodes are obtained from the updated dynamic graph network model. These edge weights have dynamically reflected the strength of the interaction relationship between the traffic participant and the obstacle. Then, through the attention mechanism, the features of the traffic participant node are associated with the features of its adjacent obstacle nodes, and a learnable attention function (such as a function based on feature similarity or interaction relationship) is used to calculate the attention score of each adjacent obstacle to the traffic participant. Then, the attention scores of all adjacent obstacles are normalized (for example, through the Softmax function) to generate an attention distribution, which represents the degree of attention of the traffic participant to different obstacles. Finally, these attention distributions are used to measure the priority of the traffic participant to the surrounding obstacles in the current scene, thereby providing a basis for subsequent behavior prediction and decision-making.

[0097] Furthermore, in step 600 of this embodiment, first, based on the characteristics of the node (such as the behavioral characteristics of traffic participants, motion state characteristics, etc.) and the attention distribution, the characteristics of the node and its neighboring nodes (such as obstacle nodes) are weighted and aggregated, wherein the weight is determined by the attention distribution, indicating the importance of each neighboring node to the current node. Next, the aggregated neighboring node features are fused with the features of the current node itself, and the fusion method includes feature splicing, weighted summation, or nonlinear transformation through a neural network, thereby generating a feature vector containing multimodal information. The multimodal feature vector integrates the feature information of the node itself and the interactive information of the neighboring nodes, and can more comprehensively represent the state of the node and the environmental relationship. Finally, the generated multimodal feature vector is used as input and passed to the next layer of the dynamic graph network model, and the node features are further updated through the network's propagation mechanism (such as graph convolution or graph attention mechanism) to dynamically reflect the behavior and interactive changes of the nodes in the traffic scene.

[0098] Furthermore, in step 700 of this embodiment, the core of the high-level decision model is to extract global information and predict driving behavior based on the updated node features. The model is usually constructed using a deep learning framework, combined with the output features of a dynamic graph network model, to capture the dynamic interaction between the autonomous driving vehicle and the surrounding traffic participants. Specifically, the updated node features (such as the behavioral features, motion state features, interaction features, etc. of the autonomous driving vehicle and the surrounding traffic participants) are further processed, and time series features are extracted through a fully connected layer, an attention mechanism, or a recursive neural network (such as LSTM, GRU) to model the dynamic changes of traffic scenes. The structure of the high-level decision model may include an input layer (receiving node features), a hidden layer (extracting high-dimensional features), and an output layer (generating driving decisions or behavior prediction results). In this embodiment, the multimodal driving data of the autonomous driving vehicle to be tested is input into the high-level decision model. The multimodal data includes the vehicle's motion state (such as speed, acceleration, direction, etc.), environmental perception information (such as obstacle location, traffic signals, road topology, etc.), and interaction features with surrounding traffic participants. After being processed by the previous dynamic graph network model, these data have been integrated into updated node features, which can fully reflect the state and interaction relationship of the autonomous driving vehicle in the current traffic scene. The high-level decision model uses these input data, combined with historical information and scene dynamics, to generate predictions for future driving behavior.

[0099] Preferably, the prediction results include: lane change decision, acceleration decision and deceleration decision.

[0100] Optionally, the high-level decision model of this embodiment outputs the prediction results of the autonomous driving vehicle based on the input multimodal driving data. The prediction results may include driving behaviors (such as acceleration, deceleration, lane change, parking, etc.), trajectory planning (such as future driving paths), or high-level decisions (such as choosing strategies such as overtaking, following, or avoiding). The model generates the optimal decision that meets the current environment and driving goals through in-depth analysis of node features and combined with the global dynamics of the traffic scene. Ultimately, the prediction results can be used to guide the control module of the autonomous driving vehicle to achieve safe and efficient driving behavior.

[0101] Figure 2 A system structure diagram provided for an embodiment of the present invention, such as Figure 2 As shown, the present invention also provides an automatic driving vehicle behavior prediction system based on artificial intelligence, comprising:

[0102] A data acquisition and processing module, used to obtain multimodal driving data of a sample autonomous driving vehicle, and perform data preprocessing on the multimodal driving data to obtain preprocessed data; the multimodal driving data includes: motion data, road data, traffic participant data and environmental data;

[0103] A feature extraction and fusion module, used to extract features from the preprocessed data of different modalities respectively to obtain data features of each modality, and to perform weighted fusion on the data features based on a cross-modal attention mechanism method to obtain fused features;

[0104] A model building module is used to build an initial dynamic graph network model and a traffic participant behavior model; the output of the traffic participant behavior model is the traffic participant behavior;

[0105] A model updating module, used to use the behaviors of the traffic participants as nodes in the initial dynamic graph network model, determine the edges between the nodes as interactive relationships, and dynamically update the weights of the edges according to the fusion features to obtain an updated dynamic graph network model;

[0106] An attention calculation module, used for calculating the attention distribution of traffic participants to surrounding adjacent obstacles based on the attention mechanism and the updated dynamic graph network model;

[0107] A feature updating module, used for constructing a multimodal feature vector according to the feature of the node and the attention distribution, and inputting the multimodal feature vector into the dynamic graph network model to obtain the feature of the updated node;

[0108] The model prediction module is used to construct a high-level decision model according to the characteristics of the updated nodes, and input the multimodal driving data of the autonomous driving vehicle to be tested into the high-level decision model to obtain a prediction result.

[0109] The beneficial effects of the present invention are as follows:

[0110] (1) By fusing multimodal driving data (motion data, road data, traffic participant data, and environmental data), the present invention can fully capture the complexity of vehicle behavior, thereby improving the accuracy of autonomous driving vehicle behavior prediction.

[0111] (2) The present invention utilizes a dynamic graph network model that can update the interaction relationships between nodes and the weights of edges in real time to adapt to changes in the dynamic traffic environment. This dynamic update capability enables the model to better reflect changes in actual traffic scenarios and improve the timeliness and reliability of predictions.

[0112] (3) The present invention performs weighted fusion of data features of different modalities through a cross-modal attention mechanism, which can effectively integrate information from different sensors, improve information utilization, and enhance the comprehensive judgment ability of the model.

[0113] (4) By constructing a traffic participant behavior model, the present invention can better understand and predict the behavior of traffic participants (such as pedestrians, non-motor vehicles, etc.), thereby enhancing the robustness of the system in complex traffic scenarios.

[0114] (5) By constructing a high-level decision model, the present invention can generate more reasonable driving decisions based on updated node features and multimodal feature vectors. This provides a stronger decision support capability for the autonomous driving system and improves the safety and efficiency of autonomous driving.

[0115] (6) The method of the present invention can adapt to different traffic environments and scenarios, has strong generalization ability, and can perform effective behavior prediction in a variety of complex situations.

[0116] (7) The present invention can effectively reduce the occurrence of traffic accidents and improve traffic safety through accurate behavior prediction and reasonable decision support.

[0117] (8) The present invention promotes the development of intelligent transportation systems: The present invention provides technical support for the implementation of intelligent transportation systems and promotes the advancement and application of autonomous driving technology.

[0118] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0119] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for predicting the behavior of an autonomous driving vehicle based on artificial intelligence, characterized in that: include: Acquire multimodal driving data of a sample autonomous driving vehicle, and perform data preprocessing on the multimodal driving data to obtain preprocessed data; The multimodal driving data includes: motion data, road data, traffic participant data and environment data; Extracting features from the preprocessed data of different modalities respectively to obtain data features of each modality, and weighted fusion of the data features based on a cross-modal attention mechanism method to obtain fusion features; Constructing an initial dynamic graph network model and a traffic participant behavior model; the output of the traffic participant behavior model is the traffic participant behavior; The behaviors of the traffic participants are used as nodes in the initial dynamic graph network model, and the edges between the nodes are determined as interactive relationships, and the weights of the edges are dynamically updated according to the fusion features to obtain an updated dynamic graph network model; Based on the attention mechanism, the attention distribution of the traffic participants to the surrounding adjacent obstacles is calculated according to the updated dynamic graph network model; Constructing a multimodal feature vector according to the feature of the node and the attention distribution, and inputting the multimodal feature vector into the dynamic graph network model to obtain the feature of the updated node; A high-level decision model is constructed according to the characteristics of the updated nodes, and the multimodal driving data of the autonomous driving vehicle to be tested is input into the high-level decision model to obtain a prediction result.

2. The method for predicting the behavior of an autonomous driving vehicle based on artificial intelligence according to claim 1, characterized in that: The expression of the fusion feature is: Among them, F i is the characteristic of the i-th mode; F 融合 is the fusion feature; α i F i The weight of .

3. The method for predicting the behavior of an autonomous driving vehicle based on artificial intelligence according to claim 1, characterized in that: The traffic participant behavior model includes a motor vehicle behavior model, a non-motor vehicle behavior model and a pedestrian behavior model; the motor vehicle behavior model is constructed based on a longitudinal following model, a lateral lane change model and an intersection interaction model, and the motor vehicle behavior model is used to describe the movement behavior of a motor vehicle; the non-motor vehicle behavior model is constructed in combination with driver characteristics, vehicle characteristics and traffic characteristics; the pedestrian behavior model is constructed based on pedestrian dynamic parameters and cluster behavior characteristics.

4. The method for predicting the behavior of an autonomous driving vehicle based on artificial intelligence according to claim 2, characterized in that: The dynamic update formula of the edge weight is: ij =f(F 融合 ,d ij ,v ij ), where w ij is the edge weight between node i and node j; d ij is the distance between node i and node j; v ij is the speed difference between node i and node j; f() is the edge weight update function.

5. The method for predicting the behavior of an autonomous driving vehicle based on artificial intelligence according to claim 4, characterized in that: The update formula of the node feature is: in, is the feature of node i at layer l+1, is the neighbor set of node i, W (l) is the weight matrix of the lth layer, d i and d j are the degrees of node i and node j respectively, and σ() is the activation function.

6. The method for predicting behavior of an autonomous driving vehicle based on artificial intelligence according to claim 5, characterized in that: The calculation formula of the attention distribution is: Among them, A ij is the attention weight of node i to node j, f(h i ,h k ) is the interaction relationship function between node i and node j.

7. The method for predicting behavior of an autonomous driving vehicle based on artificial intelligence according to claim 1, characterized in that: The prediction results include: lane change decision, acceleration decision and deceleration decision.

8. An artificial intelligence-based autonomous driving vehicle behavior prediction system, characterized in that: include: A data acquisition and processing module, used to obtain multimodal driving data of a sample autonomous driving vehicle, and perform data preprocessing on the multimodal driving data to obtain preprocessed data; The multimodal driving data includes: motion data, road data, traffic participant data and environment data; A feature extraction and fusion module, used to extract features from the preprocessed data of different modalities respectively to obtain data features of each modality, and to perform weighted fusion on the data features based on a cross-modal attention mechanism method to obtain fused features; A model building module is used to build an initial dynamic graph network model and a traffic participant behavior model; the output of the traffic participant behavior model is the traffic participant behavior; A model updating module, used to use the behaviors of the traffic participants as nodes in the initial dynamic graph network model, determine the edges between the nodes as interactive relationships, and dynamically update the weights of the edges according to the fusion features to obtain an updated dynamic graph network model; An attention calculation module, used for calculating the attention distribution of traffic participants to surrounding adjacent obstacles based on the attention mechanism and the updated dynamic graph network model; A feature updating module, used for constructing a multimodal feature vector according to the feature of the node and the attention distribution, and inputting the multimodal feature vector into the dynamic graph network model to obtain the feature of the updated node; The model prediction module is used to construct a high-level decision model according to the characteristics of the updated nodes, and input the multimodal driving data of the autonomous driving vehicle to be tested into the high-level decision model to obtain a prediction result.

Images