A Driving Context Reasoning Method Based on Metadata-Driven and Causal Analysis Theory

Abstract

This invention discloses a driving situation reasoning method based on metadata-driven and causal analysis theory. The method comprises: first, integrating multi-source perception metadata; second, analyzing the causal cognitive mechanism of driving situations in human-machine collaborative systems; third, constructing a driving situation reasoning model; and fourth, constructing a human-machine collaborative situation evolution evaluation system. Beneficial effects include: aiding in data management and organization; improving data retrieval efficiency; making human-machine collaborative driving system decisions interpretable; reducing computational complexity; enabling autonomous driving systems to make more comprehensive and reasonable decisions; and enabling autonomous driving systems to flexibly adjust and adapt to different driving situations and dynamic and uncertain environmental conditions.

Description

Technical Field

[0001] This invention relates to a driving situation reasoning method, and more particularly to a driving situation reasoning method based on metadata-driven and causal analysis theory. Background Technology

[0002] With the development of technologies such as machine vision and artificial intelligence, the application of various sensors in autonomous vehicles is gradually maturing. Autonomous vehicles rely on a large number of sensors to perceive their surroundings and make corresponding decisions and actions. Sensors collect various information to create environmental models, helping autonomous vehicles identify and understand elements such as roads, traffic signs, vehicles, and pedestrians in their vicinity. This provides the necessary information and capabilities for autonomous vehicles to achieve safe and efficient autonomous driving. With the continuous development and innovation of sensor technologies, the performance of autonomous vehicles will be further improved, bringing more convenience and safety to people's travel. In autonomous driving research, environmental perception is considered the first step in intelligent driving, and also one of the most challenging topics. Accurately identifying information in a scene and making correct inferences about the driving situation is a necessary guarantee for the safety of autonomous vehicles.

[0003] Currently, in the field of environmental perception for autonomous driving, multi-sensor data fusion technology, which comprehensively considers the advantages and disadvantages of sensors and utilizes redundancy and complementarity to acquire vehicle environmental information, is the mainstream solution. However, as the number of onboard sensors in autonomous vehicles increases, their data formats also become more diverse: on the one hand, the data formats and structures differ, such as point clouds, images, and videos, exhibiting heterogeneity; on the other hand, the data sources differ, such as visual cameras, LiDAR, millimeter-wave radar, and inertial navigation systems, demonstrating heterogeneity. Furthermore, currently, a single onboard sensor can only accurately describe the characteristics of the measured object in one aspect, which has limitations. The increase in the number of sensors also leads to a significant increase in computational difficulty, placing higher demands on the computing power of onboard computing platforms. In addition, current research on environmental perception lacks data-driven technologies and driving situation cognition technologies that better align with human cognitive mechanisms for environmental perception problems based on multi-source sensors.

[0004] Patents related to scene perception using metadata and driving situation reasoning methods based on metadata-driven and causal analysis theories are extremely rare. Chinese patent CN202210318950.3 discloses an interpretable intelligent agent decision-making method, which extracts features within the application scenario to establish a knowledge base and knowledge base configuration file, symbolically represents the knowledge base, and constructs a causal reasoning network. Chinese patent CN202211082826.8 discloses an autonomous driving decision-making method based on multi-sensor data fusion. This method fuses lane image data obtained and trained by a camera with obstacle position and distance data obtained and trained by a lidar sensor to correct the road condition information of the vehicle and use it as the basis for autonomous driving decisions. Chinese patent CN202110090412.9 discloses a system and method for risk object identification through causal reasoning. This system and method include receiving at least one image of the driving scene of a self-aware vehicle and analyzing at least one image to detect and track dynamic objects within the driving scene of the self-aware vehicle. The three invention patents mentioned above have respectively achieved the management and modeling of multi-source heterogeneous data, the fusion of multi-source data, and the reasoning method based on causal cognition theory. However, they cannot achieve multi-source sensor data management and scene cognition for autonomous driving systems, as well as driving situation reasoning based on multi-source sensor data and causal cognition theory. Summary of the Invention

[0005] The main objective of this invention is to solve the problems of high computational load, low computational efficiency, and poor computational effect in the fusion of multi-source heterogeneous sensor data under complex and realistic driving environments.

[0006] Another objective of this invention is to improve the weak cognitive reasoning ability of autonomous vehicles in complex environments due to the imperfections of artificial intelligence.

[0007] To achieve the above objectives and solve the above problems, this invention provides a driving situation reasoning method based on metadata-driven and causal analysis theory.

[0008] The driving situation reasoning method based on metadata-driven and causal analysis theory provided by this invention includes the following steps:

[0009] The first step is to integrate multi-source sensing metadata. The specific steps are as follows:

[0010] Step 1: Construction of the metadata representation framework;

[0011] Step 2: Integration of agent metadata;

[0012] Step 3: Integration of traffic contextual metadata;

[0013] The second step is to analyze the causal cognitive mechanism of the human-machine collaborative system in driving situations. The specific steps are as follows:

[0014] Step 1: Analysis of the driver's causal cognitive mechanism;

[0015] Step 2: Analysis of the causal cognitive mechanism of the self-driving system;

[0016] Step 3: Analysis of the causal cognitive coupling mechanism of the human-machine collaborative system;

[0017] Step 4: Comprehensive model of causal analysis for human-machine collaborative systems;

[0018] The third step is to construct a driving scenario reasoning model, and the specific steps are as follows:

[0019] Step 1: Multi-objective driving scenario learning framework;

[0020] Step 2: Extended driving context network based on memory metadata;

[0021] Step 3: Dynamic fuzzy logic reasoning;

[0022] Step 4: Metadata Fusion Reasoning;

[0023] The fourth step is to construct an evaluation system for the evolution of human-machine collaborative scenarios. The specific steps are as follows:

[0024] Step 1: Analysis of the evolutionary patterns of human-machine collaboration scenarios;

[0025] Step 2: Contextual reasoning based on spatiotemporal patterns;

[0026] Step 3: Target Situation Prediction;

[0027] Step 4: Integrate the results of the situational evolution assessment.

[0028] The specific steps in Step 1 are as follows:

[0029] Step 1, Metadata Modeling, involves modeling metadata to describe the data characteristics, attributes, and data sources and fields associated with scene perception functions. Metadata modeling comprises three parts: 1) For each specific data source, the metadata model includes the data source name, connection information, and data processing algorithm interfaces corresponding to different data structures; 2) For any specific function within the scene perception function based on multi-source sensor data, such as object detection and recognition, the metadata model includes the function name, the data names associated with that function, and the association and order information of various data types under that function; 3) For any function and its corresponding specific data fields from any of its data sources, such as partial image data acquired by the camera for data fusion, the metadata model includes data source information, data field information, and function information, forming a function-data field association model. Therefore, the input signals for this step are the data sources from each sensor and the data call signals based on the functions; the output signals are the access relationships between each function and the data source.

[0030] Step 2: Metadata Baseline Table Construction. This step describes the basic information of each data source by establishing a technical metadata information table to store the data source information. The data source table includes an auto-incrementing primary key, a data source ID for identification, a data source name to distinguish different data sources, the adapter name for parsing the data source, and the data connection address of the data source. The names of multi-source sensor data sources are: camera data source, LiDAR data source, millimeter-wave radar data source, inertial sensor IMU data source, and GPS data source.

[0031] Step 3: Metadata Mapping Table Construction. This step describes the relationship between functions and data sources by establishing a metadata mapping table. The metadata mapping table includes an auto-incrementing primary key, function IDs and names used to distinguish different functions, associated data source IDs and names when the same function is associated with one or more data sources, and the order of data sources when the same function is associated with multiple data sources.

[0032] Step 4: Domain Mapping Table Construction. This step describes the relationship between functions and data fields by establishing a domain mapping table. The implementation of a certain function only requires some information from different data sources. This table specifically constructs the mapping relationship between functions and specific fields. The domain mapping table includes an auto-incrementing primary key, data field name, data input / output type, function ID and name associated with the data field, and data source ID and name to which the data field belongs.

[0033] In multi-sensor data sources, data is stored in the following format:

[0034] 1) Camera data source:

[0035] Each frame of data captured by the camera is stored in the form of a two-dimensional array, with each element storing [RGB color data, grayscale data], and each frame of data is accompanied by a timestamp;

[0036] 2) LiDAR data source:

[0037] The raw data of the lidar is received through the user data packet protocol and stored in the form of data packets. The data format is as follows: the data length is fixed at 1248 bytes. The first 48 bytes are the data packet identifier, the next 1200 bytes are 12 groups of data packets with a length of 100 bytes each. Each data packet contains information on the rotation angle, distance, and reflection intensity of the laser beam. The last 6 bytes are a 2-byte timestamp and a 4-byte lidar model.

[0038] 3) Millimeter-wave radar data source:

[0039] The raw data from the millimeter-wave radar is received via the User Data Packet Protocol and stored in the form of data packets. The data format is as follows: the first 32 bytes of the millimeter-wave radar data packet are the start frame identifier, followed by n data packets. Each data packet contains target distance, velocity, angle, target size, and target intensity information. Finally, there is a 2-byte timestamp and a 4-byte end frame identifier.

[0040] 4) Inertial Measurement Unit (IMU) data source:

[0041] The data is stored in a CSV file with the following headers: timestamp, acceleration X, acceleration Y, acceleration Z, gyroscope X, gyroscope Y, and gyroscope Z.

[0042] 5) GPS data source:

[0043] The data is stored in a CSV file with the following headers: timestamp, longitude, latitude, and altitude.

[0044] In the multi-sensor data source domain mapping table, the data field names are:

[0045] 1) From the camera data source: element position information, RGB color information, grayscale information, and time information;

[0046] 2) Data from the lidar data source includes: time, X coordinate, Y coordinate, Z coordinate, and the reflection intensity of the point;

[0047] 3) Data sources from millimeter-wave radar: time, target ID, range, velocity, angle, target size, and target intensity;

[0048] 4) Data from the inertial sensor (IMU): time, vehicle acceleration, vehicle angular velocity, and vehicle orientation;

[0049] 5) GPS data source: longitude, latitude, altitude and time of the vehicle's location.

[0050] The specific steps in the first and second steps are as follows:

[0051] Step 1: Intelligent Agent Metadata Engine. In the environmental perception process, the collection and calculation of the vehicle's motion and location data is a crucial step. This step establishes an intelligent agent metadata engine to address the data retrieval needs of the intelligent agents, enabling unified scheduling of this data. The intelligent agent metadata engine receives data query or modification signals from various functions, accesses GPS and IMU data through the metadata model, and then completes data extraction and modification. Therefore, the input signal of this step is the environmental perception process's demand for intelligent agent data; the output signal is the data extracted and modified according to functional requirements.

[0052] Step 2, Data Query Interface: When the implementation of specific functions in the environmental perception process requires vehicle data as input, the data query interface receives the ID corresponding to the function, queries the metadata associated with the specific function through the metadata model, and finds the corresponding single database data or several database data based on the metadata. Therefore, the input signal of this step is the ID corresponding to each function; the output signal is the data extracted according to the functional requirements.

[0053] Step 3, Data Change Interface: This step is used to feed back the data processed by each function to the respective data sources. To achieve environmental perception, it is necessary to fuse the agent data with the driving scenario data for the next step of calculation. At the same time, the agent data also needs to be sent back to the data source as a data backup for subsequent data analysis and algorithm improvement, data verification and debugging, training and machine learning, as well as safety and traceability. Therefore, the input signal of this step is the ID corresponding to each function; the output signal is the data that has been processed to a certain extent and is used for backup.

[0054] Step 4: Data interaction with heterogeneous data sources for the intelligent agent. After querying data by function, the data collected by the IMU is read in text form. A CSV file parsing library is used to read and parse the acceleration, angular velocity, and orientation data of the IMU, as well as the longitude, latitude, altitude, and time information of the vehicle collected by GPS. The parsed data is then input into the multi-sensor data fusion algorithm corresponding to the specific function to complete the subsequent heterogeneous data interaction.

[0055] The specific steps in step three of the first step are as follows:

[0056] Section 1: Traffic Context Metadata Engine. This section establishes a traffic context metadata engine to address the data retrieval needs of traffic contexts. It is used for unified scheduling of different needs of various functions for multi-source sensor data in traffic contexts during the real-time process of environmental perception. The traffic context metadata engine receives data query signals from various functions, accesses data through the metadata model, and then searches for the data. The searched and processed data is then transmitted as the output signal of this section to the specific functional algorithm module.

[0057] Step Two: Security Module Interface. This step involves approving data acquired through communication to ensure data security. The data sources, data formats, communication protocols, and encryption protocols for vehicle-to-vehicle and vehicle-to-cloud communication are pre-classified and organized. When external information is received, the data is approved sequentially according to the classification results. Only after passing four approvals can the data transmission continue; otherwise, the data is invalid. Through data approval and screening, security filtering of data is achieved, removing untrusted data and ensuring information security.

[0058] Section 3: Data Adaptation Interface. Due to the large number of sensor types and the diverse types of traffic scene data collected, such as image data, point cloud data, location data, and vehicle motion parameters, the data exhibits strong heterogeneity. Therefore, it is necessary to fuse the queried data. Multi-source sensor data is stored in binary or text format. Based on the adapter information in the metadata benchmark table, the corresponding adapter is matched to the data of different formats to parse the data format.

[0059] Step 4 involves data interaction from heterogeneous traffic scene data sources. Different sensors use different coordinate systems. First, sensor parameter calibration transforms the data from each sensor into the same coordinate system for processing. Second, the data from different sensors undergoes correction, noise reduction, filtering, and distortion correction to eliminate system errors and noise. Next, feature information is extracted from the data from different sensors, and the multi-source sensor data is matched in time and space to correlate the data collected by different sensors. Then, a Kalman filter algorithm is used to fuse the correlated data, inputting it into the algorithm in matrix format to obtain more accurate environmental information. Finally, state estimation is performed on the fused data to obtain an accurate description of the vehicle's surrounding environment, including the position, speed, and attitude of surrounding vehicles, as well as the position, speed, and shape of surrounding obstacles. Therefore, the input signal for this step is multi-source heterogeneous traffic scene sensor data; the output signal is the fused traffic scene description.

[0060] The specific steps in step one of the second step are as follows:

[0061] Section 1: Based on the transfer entropy causal relationship model of multivariate time series data, common human-machine causal cognitive characteristic variables during driving include: steering wheel angle, steering wheel speed, brake pedal position and rate of change, accelerator pedal position and rate of change, turn signal usage time; vehicle longitudinal speed and acceleration, lateral speed and acceleration, vehicle yaw angle, yaw rate; distance to the vehicle in front, relative speed, TTC, lane departure degree, and the status of the vehicle in front's lights; traffic light status, traffic signs, pedestrian movement status, and pedestrian distance;

[0062] The causal relationship of the above human-machine causal cognitive characteristic variables is established by the method of transfer entropy. For any two variables X and Y, the transfer entropy formula is obtained by information entropy and conditional entropy.

[0063] Information entropy is expressed by formula (1):

[0064] Η(X)=-∑Ρ(x)log2Ρ(x) (1)

[0065] In the formula, x represents all possible values ​​of variable X, and P(x) represents the probability that random variable X takes the value x.

[0066] Conditional entropy is expressed by formula (2):

[0067] Η(X|Y)=-∑∑Ρ(x,y)log2Ρ(x|y) (2)

[0068] The transfer entropy is obtained from H(X) and H(Y), and formula (3) defines the transfer entropy T from variable Y to X. Y→X :

[0069]

[0070] In the formula, x t and y t Let x represent the values ​​of two variables at time t. t (k) =[x t ,x t-1 ,…,x t-k+1 [x] represents all values ​​of x over the past k time steps, y t (l)=[x t ,x t-1 ,…,x t-l+1 Let ] represent all values ​​of y over the past l moments. The value of the transfer entropy reflects the relationship of information flow between the two variables. Since the transfer entropy of the two variables is asymmetrical, the causal relationship between the two variables needs to be expressed by equation (4):

[0071] T X,Y =T X→Y -T Y→X (4)

[0072] If T X,Y If X < 0, then variable X is the result of variable Y; otherwise, variable X is the cause of variable Y. The causal relationship of the human-machine causal cognition characteristic variables summarized in this section is calculated to obtain the causal relationship matrix N. This matrix is ​​used as the adjacency matrix in the graph structure to construct the edges between nodes in the graph.

[0073] Step 2: Convolutional Feature Extraction Model Based on Multivariate Time Series Data. The values ​​of the variables mentioned in Step 1 of Step 1 of the second step are derived from real driving scenarios and are strongly correlated with driving rules and driving habits. They change regularly over time, so it is necessary to periodically process the multivariate time series data. This step uses multiple depthwise separable convolutions of different scales to extract information of the same feature at different times, and uses a first convolution with a kernel of 1×m to aggregate information from different channels, where m is the number of channels, thereby obtaining the information merging of the features of the time series data over all time periods. This process is shown in equations (5) and (6):

[0074] h i =ReLU(W i *x+b i (5)

[0075]

[0076] In the formula h i W represents the output of the i-th layer of the neural network. i Let b be the weight vector. i The vector represents the bias vector, * represents the convolution operation, and ReLU is a non-linear activation function. Indicates concatenation operation;

[0077] Step 3: Graph Attention Network Model Based on Data Fusion. This step uses the causal relationship matrix between the time-series variables of human-machine causal cognition characteristics obtained in Step 1 and the feature matrix under each variable node obtained in Step 2 to construct a graph structure. In this step, the graph attention network is used to represent the structure and node content of the graph in a unified framework, and different weights are assigned to the domain nodes.

[0078] In a graph attention network, the nodes in the graph are represented as shown in equation (7):

[0079]

[0080] Where z i (l+1) Let N(i) be the output representation of node i, N(i) be the neighboring nodes of i, σ be a nonlinear function, and α be the output representation of node i. ijThe value represents the importance of neighboring nodes, where W is the weight. The node importance coefficient is calculated from both attribute values ​​and topological distance. Regarding attribute values, the attention coefficient α... ij Equation (8) represents:

[0081]

[0082] In the formula, W xi W xj Let the weights be the weights of the i-th and j-th inputs. This is the scaling factor;

[0083] Regarding topological distance, the target node is calculated based on neighboring nodes and edges between nodes, using t-order neighboring nodes to obtain the proximity matrix M:

[0084] M = (B + B) 2 +…+B t ) / t (9)

[0085] In the formula, B is the transition matrix, M is the t-th order topological dependence between the two nodes, and t is a hyperparameter;

[0086] After normalizing the attention coefficients and adding the topological weights M and the LeakyReLU activation function, the attention coefficients are expressed as follows:

[0087]

[0088] Where δ is the activation function, M ij Let t represent the topological correlation between the two nodes.

[0089] The collected time-series data of human-machine causal cognitive characteristic variables are used as input, and the graph attention network finally outputs driver operation data with a dimension of 1 as the prediction result.

[0090] Step 4: Analysis of Driver's Causal Cognitive Mechanism in Typical Scenarios. In the following typical driving scenarios: 1) Driving along the lane: straight driving, curve driving, following another vehicle, and vehicles cutting in / out; 2) Lane changing: overtaking, terrain change, and lane closure; 3) Driving through traffic light intersections: straight driving, turning, and U-turn; 4) Ramp driving; 5) Parking. The first three steps of Step 1 in the second step use time-series data of multivariate human-machine causal cognitive characteristic variables as input to obtain causal inferences about driver behavior. In each typical driving scenario, the inferred driver operational characteristics are categorized and summarized according to driving common sense, cognitive logic, and legal provisions to obtain driver operational patterns in different scenarios, thereby analyzing the driver's causal cognitive mechanism.

[0091] The specific steps in step two of the second step are as follows:

[0092] Section 1: A Causal Framework for Autonomous Driving Cognition Based on Multivariate Time-Series Data. In the understanding and response of autonomous driving systems to driving situations, the multivariate variables used for causal cognitive mechanism analysis include: throttle opening, braking pressure, steering wheel angle, and rate of change of steering wheel angle; vehicle longitudinal speed and acceleration, lateral speed and acceleration, vehicle yaw angle, and yaw rate; distance to the vehicle in front, relative speed, TTC, lane departure degree, and status of the vehicle in front's lights; traffic light status, traffic signs, pedestrian movement status, and pedestrian distance.

[0093] This step uses Lasso regression to select variables and reduce variable dimensionality. At the same time, a regression model needs to be established to perform causal analysis between all input variables and the target variable. Assuming that the throttle opening is the response variable, the time series of this variable is expressed as Equation (11):

[0094] Y = (y1, y2, ..., y n ) T (11)

[0095] List the other variables as covariates, as in equation (12):

[0096] X = (X 1 ,X 2 ,...,X n (12)

[0097] The ordinary linear model is expressed as equation (13):

[0098] Y=Xβ+ε (13)

[0099] For each of these covariates X j ,X i =(X1) i X2 i ,…,X n I ) T ε is the random error term, β is the regression coefficient, and an L1 penalty term is added to the ordinary linear model for variable selection, thereby achieving dimensionality reduction of the variable data. The Lasso estimate for the ordinary linear model is given by equation (14):

[0100]

[0101] In the formula, the second term on the right side of the equal sign is the L1 penalty term, and λ is the adjustment coefficient. Using Lasso regression can easily calculate causal relationships under complex dimensions and initially screen out variables that have a strong causal relationship with the response variables of the autonomous driving system.

[0102] Step 2: Cognitive Perceptron Based on Multivariate Time-Series Data. A multi-output multilayer perceptron neural network model is constructed using the TensorFlow deep learning framework. The input layer, hidden layer, and output layer of the model are defined. The output y is the weighted sum of the nodes connected to y in the previous layer, as shown in Equation (15):

[0103]

[0104] In the formula, x i w is the input to the previous layer node. i Here, is the weight coefficient, b is the bias, and Act is the nonlinear activation function used to trigger node activation. The loss function C(w,b) is constructed using the mean squared error. The model is trained using multivariate time series data, and the weights and biases are adjusted through the backpropagation algorithm to minimize the loss function. The backpropagation process is shown in Equations 16 and 17.

[0105]

[0106]

[0107] Where α is the learning rate, and this gradient algorithm is iterated until the cost converges;

[0108] The input signal for this step is the selected multivariate time-series data with strong causal relationships, and the output signal is the trained multilayer perceptron model.

[0109] Step 3: Based on multivariate time-series data, a convolutional neural network is used to preprocess the multivariate time-series variable data for analyzing the causal cognitive mechanism of autonomous driving. One-dimensional data is normalized, and multi-dimensional data is standardized. A convolutional neural network model is constructed: a convolutional neural network model suitable for processing mixed multivariate time-series data is built. The model's input conforms to the characteristics of one-dimensional and multi-dimensional data. One-dimensional convolutional layers are used to process one-dimensional data, and two-dimensional or three-dimensional convolutional layers are used to process multi-dimensional data. The data processed by the convolutional layers is input into pooling layers to extract key features, and then the key features are input into fully connected layers to capture global features and establish the relationship between features and output. Finally, the processed dataset is input into the model for training, and the model is optimized to improve its performance. The input signal in this step is the selected multivariate time-series data with strong causal relationships, and the output signal is the trained multilayer perceptron model.

[0110] Step 4: Analysis of the causal cognitive mechanism of the autonomous driving system in typical scenarios. In typical driving scenarios, the first three steps of Step 2 of the second step take time-series data of multivariate autonomous driving causal cognitive characteristic variables as input to obtain predictions of the autonomous driving system response. In each type of typical driving scenario, the predicted autonomous driving system response characteristics are classified and summarized according to vehicle dynamics models, driving common sense, cognitive logic, and legal provisions to obtain the autonomous driving system response patterns in different scenarios, and then analyze the causal cognitive mechanism of the autonomous driving system.

[0111] The specific steps in step three of the second step are as follows:

[0112] Step 1: Human-Machine Causal Relationship Feature Extraction. This involves extracting driver operation data, autonomous driving system response data, and driving scenario data from the data collected by sensors. Specifically, these data types include:

[0113] 1) Driver operation data: torque applied to the steering wheel by the driver, pressure applied to the accelerator pedal by the driver, pressure applied to the brake pedal by the driver, and driver's gear shifting operations;

[0114] 2) Automated driving system response data: steering wheel signal, braking signal, transmission signal, gear shift signal;

[0115] 3) Driving scenario data: The vehicle's position, speed, acceleration, and attitude data, as well as an accurate description of the vehicle's surrounding environment after processing by the metadata engine, namely the position, speed, and attitude of surrounding vehicles, and the position, speed, and shape of surrounding obstacles;

[0116] Based on the strength of the causal relationships obtained in steps one and two of the second step, the data is filtered to obtain data with stronger causal relationships, thereby obtaining data features of important scene elements in the driving scenario; and the causal relationships between the data are uniformly labeled, including: the vehicle in front decelerates, the vehicle in front brakes, the vehicle in front passes a zebra crossing, the vehicle in front appears a pedestrian, and the vehicle in front brakes when the traffic light turns red. The labels are then summarized to obtain the tag features of the scene elements.

[0117] Step Two: Human-Machine Causal Cognitive Feature Representation Learning Model, which uses the data features x of subjects in the same scene. sd and label features x sl The features are processed to maintain dimensionality consistency, and then concatenated to obtain new object features t. s The same operation is performed on the object features to obtain the object features t. o The object features are fed into a Transformer-based feature representation learning model to obtain t. s 'and t o';Data features of subject and object are processed by u so =Union(x sd ,x od The combined operation yields the combo box features. Finally, the learned subject object features, combo box features, and learned object features are concatenated to obtain the relation feature x. tri =[t s ',uso,t o After training on various types of data, all relation features are obtained. The relation features obtained in this step will be entered into the relation classifier for classification.

[0118] Step 3: Generating a model for the causal cognitive coupling mechanism. In typical driving scenarios, the actions of the driver and the autonomous driving system are responses based on a comprehensive judgment of the driving scenario, requiring all scene features in the typical scenario. Simultaneously, the driver and the autonomous driving system need to cooperate to complete the driving task. Therefore, for each typical driving scenario, all features X within the scenario are extracted. sc The object characteristics X of the subject s and the object characteristics X o Here, the subject and object are any two objects within the scene that have a causal relationship, such as the vehicle in front and the vehicle itself, or the lane line and the vehicle itself. Then, the subject features and scene features are fused to obtain the subject object feature X of the fused scene. s '=f(X s ,X sc Similarly, the object features X of the fused scene are obtained by processing the object features. o '=f(X o ,X sc ), then X s 'and X o 'Feature representation learning model based on Transformer is fed into the model, the learned object features are concatenated, two objects are randomly selected to form a relationship pair, and the distribution network of all subjects, relationships and objects in each scenario is output to establish a causal cognition coupling mechanism generation model.'

[0119] The specific steps in step four of the second step are as follows:

[0120] Step 1: Multivariate Data Causal Relationship Model in Typical Scenarios. Using the human-machine causal cognition characteristic variables from Step 1 as model input, an initial directed acyclic graph is constructed. Each node represents a variable, and directed edges represent causal relationships between variables. For each node, a conditional probability table is constructed based on the strength of the causal relationship determined in Step 1. This table reflects the probability distribution of each node given its parent node, representing the causal relationship between the node and its given parent node. This constitutes a Bayesian network model based on the human-machine causal cognition characteristic variables. Similarly, the same operation is performed on the autonomous driving system causal cognition characteristic variables from Step 2 to construct a Bayesian network model based on the autonomous driving system causal cognition characteristic variables.

[0121] Examine the variables in two Bayesian network models and identify the common variables between them. Establish directed edges between the common variables in the two models to represent the causal relationship between them. Connect the two models through these edges, establish a conditional probability table for the common variables, and revise the conditional probability tables of the nodes involving the common variables in the two models to ensure that the variables shared in the two models have consistent parameters when calculating the conditional probabilities, thereby obtaining a multivariate data causal relationship model in a typical scenario.

[0122] Step 2: Human-Machine Cognitive Efficiency Model in Typical Scenarios. This step defines metrics for evaluating human-machine cognitive efficiency, including driver response time, autonomous driving system response time, and response quality. Experiments are conducted in various typical scenarios to collect driver behavior data, perception data, and vehicle state data, extracting data features. Support Vector Machine (SVM) algorithms are used to correlate the extracted features with corresponding cognitive efficiency metrics, completing data training. The training results are used to filter data with high correlation to cognitive efficiency. The main relationship classifications and the distribution networks of all subjects, relationships, and objects in typical scenarios are obtained from the human-machine causal cognitive feature representation learning model. Subjects or objects associated with each data type in typical scenarios are identified. Regression analysis is used to calculate the subject and object features associated with cognitive efficiency and each response data, resulting in a human-machine cognitive efficiency model with causal cognitive coupling in typical scenarios.

[0123] Section 3, the causal cognition analysis model of the human-machine collaborative system, integrates the multivariate data causal relationship model and the human-machine cognitive efficiency model in typical scenarios to establish a unified model framework. Based on the Bayesian network model based on the causal cognitive feature variables of human-machine and the Bayesian network model based on the causal cognitive feature variables of the autonomous driving system, it is extended into a comprehensive causal cognitive network. Considering the common variables of the two sections, directed edges are established to connect the common variables in the two models to represent the causal relationship between them.

[0124] Data features of the human-machine collaborative system in various typical scenarios are extracted, and the support vector machine algorithm is used to associate the extracted features with the measurement index of human-machine cognitive effectiveness. The conditional probabilities between each node and its child nodes are determined using the data training results. At the same time, the conditional probability tables of nodes involving common variables are revised to ensure that the variables shared in the two models have consistent parameters when calculating the conditional probabilities, and the scope of application of the model is continuously expanded and updated.

[0125] The specific steps in step one of the third step are as follows:

[0126] Step 1: Summarizing and Semantic Analysis of Driving Situation Elements. Based on the composition of the driving situation, the elements of the driving situation are summarized, mainly into the following categories:

[0127] 1) Driving mode: Driving mode refers to the way and behavior used to control the driving of a car, such as following the car and changing lanes;

[0128] 2) Driving intention: Driving intention refers to the driver's intention and purpose during the driving process. It indicates that the driver hopes to achieve a certain goal through specific actions or movements. For example, when the driver turns on the turn signal and approaches an intersection, it can be inferred that his driving intention is to prepare to turn.

[0129] 3) Driving behavior: Driving behavior refers to the movement of a vehicle under the control of a driver or an automated driving system, including acceleration, braking, and steering;

[0130] 4) Traffic participants and traffic signs;

[0131] 5) Road conditions and weather conditions;

[0132] First, a convolutional neural network model is used to pre-train entity elements in the driving scenario to identify and classify various entity elements. Second, a support vector machine model is used to train a relation extraction model to identify the relationship between entity elements and rule elements in the driving scenario, such as the rule that vehicles cannot cross the stop line when the traffic light is red. A pre-trained semantic role labeling model is used to label the semantic roles in the sentence. Finally, a semantic interpretation model is used to convert the text or speech information into a semantic representation that the machine can understand.

[0133] Section Two: Attribute Constraint Specifications. In specific driving scenarios, the driving scenario elements, their attributes, and the constraints of each element are specified. The constraints of each element include traffic rules, road conditions, vehicle status, and driver behavior. Taking the situation where a vehicle approaches a traffic light intersection along a straight lane when the traffic light is red, and the vehicle slows down, as an example, the possible driving behaviors corresponding to the vehicle slowing down are low-speed driving or stopping. Since the vehicle is in the straight lane, the driving behaviors are only normal driving or stopping to wait for the traffic light. Driving is restricted by the straight traffic light, and a red light prohibits vehicles from passing. Therefore, the purpose of slowing down the vehicle is to stop and wait for the traffic light to change.

[0134] Step 3, Logical Resolution Completeness Analysis: First, each constraint is transformed into a logical expression or propositional logic form, ensuring that each constraint can be accurately expressed as a logical statement, and using logical symbols to represent relationships. Second, the completeness of the constructed logical expression is checked, i.e., whether it covers all possible situations and constraints, ensuring that the logical expression can encompass all possibilities in the driving situation. The relationships between elements in the logical expression are checked to ensure that the logical relationships accurately reflect the actual relationships between elements in the driving situation. Finally, the accuracy and completeness of the logical expression are verified by testing actual driving situation cases. If the logical expression can correctly predict and describe the elements and constraints in the driving situation, then the logical resolution can be considered complete.

[0135] The specific steps in step two of the third step are as follows:

[0136] Step 1: Transient and Long-Term Memory Structure. To construct an extended semantic network for driving situations, this step adopts a memory model that mimics the brain. Transient memory is responsible for temporarily storing reasoning information, while long-term memory stores all received facts and high-probability inferences.

[0137] Long-term memory, as a large extended semantic network, stores factual evidence obtained from input text and information with high probability after reasoning. Because the traffic rules generated by long-term memory and traffic experience trained on a large amount of driving data are difficult to retrieve, they are indexed and retrieved through metadata.

[0138] Transient memory is a small, extended semantic network that extracts entities and relationships of driving context elements from the input text, constructs them simply, and extracts related entities and relationships with the probability of occurrence from long-term memory. For the needs of reasoning, reverse edges are added to the relationships in transient memory to represent another voice.

[0139] After the transient memory is constructed, the text information is reasoned, and the reasoning results are sent to other sub-modules. The transient memory network is automatically deleted. If the relationship obtained by the transient memory reasoning is factual, then the relationship and related entities are directly stored in long-term memory. If it is a deduced conclusion, if the probability is greater than 80%, it is stored in long-term memory; otherwise, it is not stored in long-term memory.

[0140] Step 2: Memorizing the definition of a network;

[0141] After semantic analysis of the current driving situation, the driving situation is represented in sentence form S={S1,S2,…,S…} n The presentation extracts the subject headword or object headword from a series of sentences into V' = {v k |k=1,2,…,n}, extract the relational terms as E'={e d (v i ,v j )|d=1,2,…,n}. G'={V',E'} constitutes an extended semantic network containing S, where V' is a vertex and E' is an edge;

[0142] Step 3: Memorize the definitions of vertices and edges;

[0143] The vertices in the extended semantic network include the entity name, identification code, and related attributes of the sentence. The entity is the subject and object of the sentence, representing the conceptual entity in the driving context, such as car, lane line, and pedestrian. Taking car as the representative, car belongs to a category. In the multi-vehicle driving context, in order to distinguish the vehicles, each vehicle needs to be assigned a specific identification code for differentiation and identification. The vehicle type, speed, and distance information are recorded in the related attributes. For convenient querying and retrieval, the entity name, entity name ID, identification code ID, and attribute ID information are stored in the metadata baseline table, metadata mapping table, and domain mapping table.

[0144] In extended semantic networks, edges represent partial order relations between two entities. Key properties of partial order relations include:

[0145] 1) Relation Name: Use the relational conjunction in the sentence as the relation name;

[0146] 2) Factual basis indicator: Used to indicate whether the edges of the semantic web can be used as factual basis for the reasoning model;

[0147] 3) Probability of Relationship Occurrence: By setting a certain probability for the relationship, the authenticity of the sentence can be measured. When this probability is higher than the set threshold, the edge of relationship 2 will be retained; otherwise, it will be deleted.

[0148] 4) The occurrence and duration of the relationship: The occurrence and duration of the relationship will have a significant impact on causal reasoning and need to be stored to determine the order in which the relationship occurs;

[0149] 5) Unigram and binary relations: In extended semantic networks, edges represent all binary relations, which include direction and purpose. For sentences with only subject-verb structure, such as "the car stopped", in the case of no object, it is assumed that there is a virtual object, which is represented by the subject itself. That is, the relation starts from the subject and returns to the subject.

[0150] 6) Multiple relationships between two vertices: There may be multiple relationships between any two elements in a driving scenario. In this case, there can be multiple edges between two fixed points.

[0151] Similarly, the relation name, relation name ID, and relation attribute ID are stored in the metadata base table, metadata mapping table, and domain mapping table;

[0152] Section 4: Adaptive Bayesian Driving Context Network. The set of entity events in the extended semantic network based on driving context is E = [E i Based on the metadata baseline table, query and extract event E sequentially. i And select all those containing E i Two-node Bayesian confidence networks are added to the Bayesian confidence network set B. Entity events are grouped according to the relevance of factual evidence, and their corresponding two-node Bayesian confidence network sets are merged. Two-node Bayesian confidence networks that are related to factual evidence are selected to reduce the complexity of the inference model. Finally, the selected two-node Bayesian confidence networks are combined into a whole Bayesian confidence network based on common nodes, and the conditional probabilities are updated.

[0153] The specific steps in step three of the third step are as follows:

[0154] Section 1, Reasoning Pattern Hierarchy: When using known driving situation elements to perform logical reasoning about the driving situation, the following reasoning pattern (18) is divided into two layers: inner and outer.

[0155]

[0156] The inner layer consists of the “X is CX” layer and the “Y is CY” layer, while the outer layer consists of the “(X,Y)is CX∩CY” layer. The inner layer uses a general reasoning pattern to find the relevant information CX and CY of the driving situation elements X and Y, while the outer layer seeks the association information of the driving situation elements X and Y based on the inner layer.

[0157] Section Two: Graph Method for Reasoning Mode. First, a rule base is established using the constraint rules of the driving context element attributes. The dynamic fuzzy reasoning system is represented by a directed graph structure. For any node, if the directed edge entering the node is not 0, the node is the conclusion of the previous node; if the directed edge leaving the node is not 0, the node is the premise of the next node. For input objects X and Y, nodes with a dynamic fuzzy similarity higher than a specified threshold are first searched and marked. After finding a node, the propagation proceeds along the directed edge, and the nodes traversed are marked and their information is recorded in CX and CY. If a node is marked twice, the information of that node and subsequent nodes is recorded in CX∩CY.

[0158] Step 3: Abstraction of the reasoning pattern set. For input objects X and Y, match nodes in the directed graph of the rule base whose dynamic fuzzy similarity reaches a specified threshold, and add all child nodes under that node to set D. x D y In the problem, find the intersection of two sets, C = D. x ∩D y If C is an empty set, then X and Y are not associated in the current rule base; if C is not an empty set, then X and Y are associated in the current rule base, and the association result is the element in C.

[0159] Step Four: Multi-Source Reasoning Extension Mechanism. In complex driving scenarios, the relationships between elements are not always one-to-one. More often, multiple elements are mutually constrained; for example, traffic signs, road conditions, and the vehicle ahead all affect the vehicle's speed. Therefore, connecting several objects for reasoning is equally important. For graph methods, when the number of input elements is n, it is necessary to propagate the record starting from the node marked n times. For set methods, it is necessary to find n sets D. n The intersection of.

[0160] The specific steps in step four of the third step are as follows:

[0161] Step 1: Metadata Integration Logic;

[0162] First, the metadata of the extended semantic network is defined, including the attributes of nodes and edges and the relationship information between them. Based on the definition of the metadata, the extended semantic network is stored in the format of a graph database, and the attributes and relationships of nodes and edges are mapped to the fields in the graph structure of the data storage.

[0163] For the reasoning knowledge base, the metadata includes rules, concepts, logical relationships and semantic relationships. Based on the definition of the metadata, a storage knowledge base containing data tables and graph structures is established. The data of the dynamic fuzzy logic reasoning knowledge base is loaded into the data storage, and according to the definition of the metadata, the rules, concepts and logical relationships are mapped to the corresponding data tables and fields in the graph structure in the data storage.

[0164] Analyze the common information in the two data sources and determine their field mapping relationships in each data source. Through the mapping relationship, associate fields with the same meaning in different data sources. Based on the field mapping relationship in the metadata, create a joint query statement to retrieve information from both data sources simultaneously.

[0165] Step 2: Driving Context Memory Reconstruction. As the driving task progresses, the system continuously receives new information. Using the information analysis module, the new information is used to construct an extended semantic network. The system then calls the relevant Bayesian driving context network in the extended driving context network, merges the new extended semantic network with the original extended semantic network, recalculates the probabilities, updates the Bayesian driving context network based on the newly calculated probabilities, and sends the results back to the database.

[0166] Step 3: Inference knowledge base update. Because the experience information stored in the database cannot reflect the diversity of real driving situations, as driving progresses, the number of nodes in the Bayesian driving situation network increases, and the probabilities represented by the edges also change. When the probability is lower than 0.20, the corresponding inference rule in the inference knowledge base is deleted; when the probability of the new edge between nodes is higher than 0.60, the new inference rule is added to the inference knowledge base.

[0167] Step 4: Inference Model Integration. Due to the diversity of driving scenarios and the gradual accumulation of data, new inference rules will be added to the inference knowledge base, and there will be redundancy in structure and storage. In order to make the inference model comprehensive and concise, it is necessary to merge common nodes periodically and integrate the inference model.

[0168] The specific steps in step one of the fourth step are as follows:

[0169] Step 1: Design of causal correlation indicators;

[0170] To confirm the effectiveness of the causal analysis model for the human-machine collaborative system, it is necessary to design relevant indicators to evaluate the model's causal analysis efficiency.

[0171] In the car-following scenario, the causal relationship analysis model of the human-machine cooperative system concludes that there is a causal relationship between the speed and distance of the vehicle in front and the operating state of the human-machine co-driving system of the vehicle itself. In this scenario, let the evaluation factor set be A = [the force applied by the driver to the brake pedal, the force applied by the driver to the accelerator pedal, the braking signal of the autonomous driving system, and the acceleration signal of the autonomous driving system], and use the dangerous collision time TTC under the car-following condition as the evaluation.

[0172] In lane-changing scenarios, the causal relationship analysis model of the human-machine collaborative system concludes that there is a causal relationship between the speed and distance of the vehicle in front in the original lane, the speed and distance of the vehicles in front and behind in the target lane, and the operating state of the human-machine co-driving system of this vehicle. In this scenario, let the evaluation factor set be A = [force applied by the driver to the brake pedal, force applied by the driver to the accelerator pedal, steering torque applied by the driver to the steering wheel, braking signal of the automatic driving system, acceleration signal of the automatic driving system, steering signal of the automatic driving system], and the dangerous collision time TTC in lane-changing conditions is used as the evaluation.

[0173] Step 2: Construction of the fuzzy comprehensive evaluation system;

[0174] First, the weights W of each evaluation factor are determined using principal component analysis and factor analysis. Then, the evaluation factor A in the human-machine collaborative causal cognition analysis model is treated as a fuzzy mapping to the comment set V, and a fuzzy evaluation matrix R is established. Based on fuzzy theory, the weights of the evaluation factors and the fuzzy evaluation matrix are calculated to obtain the driver's situational cognition comprehensive evaluation B, and the evaluation result is obtained according to the maximum membership principle.

[0175] B = W·R (19);

[0176] The specific steps in step two of the fourth step are as follows:

[0177] Step 1: Vehicle Dynamics Internal Model Construction. A linear two-degree-of-freedom vehicle dynamics model is proposed based on the following assumptions: only the lateral motion along the y-axis and the yaw angle motion around the z-axis are considered; the steering wheel angle is equated to the front wheel angle, and the deformation and interference of the suspension and steering systems are ignored; the effect of the self-centering torque is ignored, and it is assumed that each tire has a linear elasticity F. Y =k*α, where the lateral stiffness k is a constant for a given tire;

[0178] Linear two-degree-of-freedom vehicle dynamics model:

[0179]

[0180]

[0181] Where M is the mass of the car, δ f For the front wheel steering angle, C f Cr Let l represent the lateral stiffness of the front and rear wheels, l1 be the distance from the center of gravity to the front axle, l2 be the distance from the center of gravity to the rear axle, and Ω be the lateral stiffness. z V is the yaw rate of the vehicle. x V y The velocity of the vehicle along the x-axis and y-axis;

[0182] Section 2: Derivation of Model Prediction Equations. To predict the future state of the driving scenario, model predictive control is used to predict the future state of the vehicle. The prediction equations of the controller are as follows:

[0183]

[0184] Where, x k x represents the state or variable of the system at time step k; k+1 A represents the state vector at the next time step k+1; d The state transition matrix describes how the system state changes within a time step; u k The control input vector represents the control input or operation of the system at time step k; B d The control input matrix describes the effect of the control input within one time step; D d This is the direct action matrix, representing the effect of control inputs directly on state variables, without going through the state transition matrix; y k C is the output vector, representing the system's output or measurement at time step k; d The output matrix describes how the state vector is mapped to the output vector; D d,dis Let be the disturbance vector, representing the disturbance or interference to the system at time step k;

[0185] Formula (22) can be rearranged into formula (23):

[0186] X=FX0+ΦU+E (23)

[0187] Where X is the state vector, representing the state or variable of the system at the current time step; F is the state transition matrix, describing how the system state changes within a time step; X0 is the initial state vector, representing the state or variable of the system at the initial time step; Φ is the control input matrix, describing the effect of the control input within a time step; U is the control input vector, representing the control input or operation of the system at the current time step; and E is the disturbance vector, representing the disturbance or interference of the system at the current time step.

[0188] The variables and coefficient matrix in formula (23) are as follows:

[0189]

[0190] Step 3: Spatiotemporal evolution of the inference model. This step integrates the vehicle dynamics model with the model prediction equations. By inputting the vehicle parameters in the current state, the prediction results for the vehicle state at future moments can be obtained.

[0191] The specific steps in step three of the fourth step are as follows:

[0192] Step 1: Metadata-driven spatiotemporal data integration. During driving, the metadata-driven engine calls up the current sensor data, vehicle status data, vehicle-to-vehicle communication data, and driving scenario reasoning model for subsequent calculations.

[0193] Step 2: Target Intent Analysis. An adaptive Bayesian driving context network is constructed. The vehicle's attitude prediction for future moments is obtained through joint calculation using the vehicle dynamics model and model predictive control. The predicted vehicle state is input into the transient memory module for inference. By comparing it with long-term memory data, reasonable inference results are retained and unreasonable inference results are ignored. The vehicle state in the reasonable inference results is combined with the driving context for analysis to obtain the driving intent of the human-machine collaborative system in the current driving context.

[0194] Step 3: Driving risk assessment, establishing a driving risk field based on driving risk field theory:

[0195]

[0196] Among them, E s_j Let E be the resultant vector of the driving risk field at position j of the vehicle. v_j Let E be the resultant vector of the kinetic field strength at position j of the vehicle. R_j Let E be the resultant vector of the static energy field at position j of the vehicle. D_j Let F be the resultant vector of the field strength at position j of the vehicle; j Let M be the resultant vector of the driving risk field at position j of the vehicle, i.e., the driving risk. j Let v be the equivalent mass of vehicle j. j Let R be the speed of vehicle j. j Let be the road factor at vehicle j, k2 be a given constant, and θ be the road factor at vehicle j. j Let D be the angle between the direction of the vehicle's velocity j and the direction of the electric field. ri For driver risk factors;

[0197] The system will input scenario data and driving data from the current moment to the predicted time into the driving risk field for calculation, and obtain the driving risk at the current moment and within the predicted time, thereby judging whether the decision of the human-machine collaborative system is reasonable.

[0198] The specific steps in step four of the fourth procedure are as follows:

[0199] Step 1: Updating the metadata list. As driving scenarios continue to expand, data content, data fields, and data types are constantly being updated from various data sources. The metadata list should be updated along with the data updates to facilitate subsequent data retrieval and storage.

[0200] Step 2: Updating the causal cognition logic. The existing causal cognition model is trained based on previously acquired driving data and is difficult to cover the complete driving situation. As driving data is collected and stored, the causal cognition model will be trained together with the original data after a certain driving mileage interval, so that the causal cognition logic is constantly updated.

[0201] Step 3: Updating the reasoning knowledge base. The expansion of driving scenarios causes the long-term memory extended semantic network to continuously add or delete nodes and edges, and continuously correct the probabilities. At the same time, the reasoning rule base is expanded. The reasoning knowledge base is updated through the continuous memory reconstruction process.

[0202] Step 4: Integration of Contextual Evolution Assessment Results. This step integrates the reasoning results based on spatiotemporal patterns, risk assessment results, and driving parameters of the human-machine collaborative system, and inputs them into the decision-making module to complete subsequent decision-making and planning operations.

[0203] The beneficial effects of this invention are:

[0204] 1) The metadata-based data management method described in this invention can provide detailed descriptions and structured information about the data, which helps in the management and organization of the data.

[0205] 2) The metadata-based data retrieval method described in this invention, through the provided indexes and tags, helps to quickly locate and access specific data. Metadata allows for a rapid understanding of the data's location and attributes, avoiding traditional traversal search methods and improving data retrieval efficiency.

[0206] 3) This invention proposes a causal cognitive mechanism method based on multivariate time-series data, which quantifies the causal relationship between driving data and makes the decision-making of human-machine cooperative driving system interpretable.

[0207] 4) The long-term memory and transient memory structure proposed in this invention, which simulates human memory, separates the data storage module from the data calculation module, thereby reducing the complexity of the calculation.

[0208] 5) The dynamic fuzzy logic reasoning model proposed in this invention considers the complex relationships between multiple factors and variables in real driving situations, and takes into account multiple objectives and constraints, enabling the autonomous driving system to make more comprehensive and reasonable decisions.

[0209] 6) The driving situation reasoning method based on metadata-driven and causal analysis theory described in this invention can achieve adaptive updates of causal cognitive logic and reasoning knowledge base as driving data increases through a memory reconstruction model, enabling the autonomous driving system to flexibly adjust and adapt to different driving situations and adapt to dynamic and uncertain environmental conditions. Attached Figure Description

[0210] Figure 1 This is a schematic diagram illustrating the overall steps of the driving scenario reasoning method described in this invention.

[0211] Figure 2 This is a schematic diagram of the overall architecture of the driving scenario reasoning method described in this invention.

[0212] Figure 3 This is a schematic diagram of the overall architecture of the first step described in this invention.

[0213] Figure 4 This is a schematic diagram of the overall architecture of the second step described in this invention.

[0214] Figure 5 This is a schematic diagram of the overall architecture of the third step described in this invention.

[0215] Figure 6 This is a schematic diagram of the overall architecture of the fourth step described in this invention.

[0216] Figure 7 This is a flowchart of the algorithm for step four in the second step of the present invention.

[0217] Figure 8 This is a flowchart of the algorithm for step four in the third step of the present invention.

[0218] Figure 9 This is a flowchart of the algorithm for step three in the fourth step of the present invention.

[0219] Figure 10 This is an example diagram of the calculation process in step two of the first step of the present invention.

[0220] Figure 11 This is an example diagram showing the calculation results of step four in the third step of the present invention.

[0221] Figure 12 This is an example diagram of the calculation results in step three of the fourth step of the present invention. Detailed Implementation

[0222] Please see Figures 1 to 12 As shown:

[0223] The driving situation reasoning method based on metadata-driven and causal analysis theory provided by this invention is as follows:

[0224] The first step is to integrate multi-source sensing metadata;

[0225] The second step is to analyze the causal cognitive mechanism of the human-machine collaborative system in driving situations.

[0226] The third step is to construct a driving scenario reasoning model;

[0227] The fourth step is to construct an evaluation system for the evolution of human-machine collaborative scenarios.

[0228] The process of integrating multi-source sensing metadata in the first step is as follows:

[0229] Step 1: Construction of the metadata representation framework. Step 1 consists of four parts.

[0230] Step 1: Metadata Modeling. This step involves modeling metadata to describe the data characteristics, attributes, and data sources and fields associated with the scene perception function. Metadata modeling comprises three parts: For each specific data source, the metadata model includes the data source name, connection information, and data processing algorithm interfaces corresponding to different data structures; for any specific function in the scene perception function based on multi-source sensor data, such as object detection and recognition, the metadata model includes the function name, the data names associated with that function, and the association and order information of various types of data under that function; for any function and any specific data field of its corresponding data source, such as partial image data acquired by the camera required for the data fusion function, the metadata model includes the data source information, data field information, and function information, forming a function-data field association model. Therefore, the input signals for this step are the data source data input from each sensor and the data call signals based on the function; the output signal is the access relationship between each function and the data source.

[0231] Step Two: Metadata Baseline Table Construction. This step involves creating a table to store the basic information of each data source's technical metadata. The data source table includes an auto-incrementing primary key, a data source ID for identification, a data source name to distinguish different data sources, the name of the adapter used to parse the data, and the data connection address of the data source. The multi-source sensor data source names include: camera data source, LiDAR data source, millimeter-wave radar data source, inertial sensor (IMU) data source, and GPS data source.

[0232] Step 3: Metadata Mapping Table Construction. This step describes the relationship between functions and data sources by establishing a metadata mapping table. The metadata mapping table includes an auto-incrementing primary key, function IDs and names to distinguish different functions, associated data source IDs and names when the same function is associated with one or more data sources (e.g., the data fusion function needs to use camera image data, LiDAR point cloud data, millimeter-wave radar point cloud data, GPS positioning data, and IMU vehicle monitoring data), and the order of data sources when the same function is associated with multiple data sources.

[0233] Step 4: Domain Mapping Table Construction. This step describes the relationship between functions and data fields by establishing a domain mapping table. If a function only requires partial information from different data sources, this table will specifically construct the mapping relationship between the function and the specific field. The domain mapping table includes an auto-incrementing primary key, data field name, data input / output type, the function ID and name associated with the data field, and the data source ID and name to which the data field belongs.

[0234] In multi-sensor data sources, data is stored in the following format:

[0235] 1) Camera data source:

[0236] Each frame of data captured by the camera is stored in the form of a two-dimensional array, with each element storing [RGB color data, grayscale data], and each frame of data is accompanied by a timestamp.

[0237] 2) LiDAR data source:

[0238] Raw lidar data is received via User Datagram Protocol (UDP) and stored in data packets. The data format is as follows: the data length is fixed at 1248 bytes. The first 48 bytes are the data packet identifier, and the next 1200 bytes are 12 groups of data packets, each 100 bytes long. Each data packet contains information on the laser beam's rotation angle, distance, and reflection intensity. The last 6 bytes are a 2-byte timestamp and a 4-byte lidar model number.

[0239] 4) Millimeter-wave radar data source:

[0240] The raw data from the millimeter-wave radar is received via the User Datagram Protocol and stored in the form of data packets. The data format is as follows: the first 32 bytes of the millimeter-wave radar data packet are the start frame identifier, followed by n data packets. Each data packet contains target distance, velocity, angle, target size, and target intensity information. Finally, there is a 2-byte timestamp and a 4-byte end frame identifier.

[0241] 4) Inertial Measurement Unit (IMU) data source:

[0242] The data is stored in a CSV file with the following headers: timestamp, acceleration X, acceleration Y, acceleration Z, gyroscope X, gyroscope Y, and gyroscope Z.

[0243] 5) GPS data source:

[0244] The data is stored in a CSV file with the following headers: timestamp, longitude, latitude, and altitude.

[0245] In the multi-sensor data source domain mapping table, the data field names are:

[0246] 1) From the camera data source: element position information, RGB color information, grayscale information, and time information;

[0247] 2) Data from the lidar data source includes: time, X coordinate, Y coordinate, Z coordinate, and the reflection intensity of the point;

[0248] 3) Data sources from millimeter-wave radar: time, target ID, range, velocity, angle, target size, and target intensity;

[0249] 4) Data from the inertial sensor (IMU): time, vehicle acceleration, vehicle angular velocity, and vehicle orientation;

[0250] 5) GPS data source: longitude, latitude, altitude and time of the vehicle's location.

[0251] Step 2: Integration of agent metadata. Step 2 consists of four steps.

[0252] Component 1: Agent Metadata Engine. In the environmental perception process, the collection and calculation of the vehicle's motion and location data is a crucial step. This component establishes an agent metadata engine to address the data retrieval needs of the agent, enabling unified scheduling of this data. The agent metadata engine receives data query or modification signals from various functions, and through the metadata model, accesses GPS and IMU data, thereby completing data extraction and modification. Therefore, the input signal for this component is the environmental perception process's demand for agent data; the output signal is the data extracted and modified according to functional requirements.

[0253] Step Two: Data Query Interface. When the implementation of specific functions in the environmental perception process requires vehicle data as input, the data query interface receives the ID corresponding to the function, queries the metadata associated with the specific function through the metadata model, and finds the corresponding single-database or multi-database data based on the metadata. Therefore, the input signal of this step is the ID corresponding to each function; the output signal is the data extracted according to the functional requirements.

[0254] Step 3: Data Change Interface. This step is used to feed back the data processed by each function to the respective data sources. Achieving environmental perception requires fusing agent data with driving scenario data for further computation. Simultaneously, agent data also needs to be sent back to the data sources as backups for subsequent data analysis and algorithm improvement, data verification and debugging, training and machine learning, as well as safety and traceability. Therefore, the input signal for this step is the ID corresponding to each function; the output signal is the processed data used for backup.

[0255] Step 4: Data Interaction with Heterogeneous Data Sources for the Intelligent Agent. After retrieving data by function, the data collected by the IMU is read in text format. A CSV file parsing library is used to read and parse the IMU's acceleration, angular velocity, and orientation data, along with the vehicle's longitude, latitude, altitude, and time information collected by GPS. The parsed data is then input into the multi-sensor data fusion algorithm corresponding to the specific function to complete the subsequent heterogeneous data interaction.

[0256] Step 3: Integration of traffic contextual metadata. Step 3 consists of four steps.

[0257] Step 1: Traffic Context Metadata Engine. Autonomous driving systems need to perceive and understand the vehicle's surroundings in real time to make accurate decisions and ensure safe driving. Therefore, the collection and computation of traffic context data is a crucial part of environmental perception. This step establishes a traffic context metadata engine to address the data retrieval needs of various functions during the real-time environmental perception process, unifying the scheduling of different requirements for multi-source sensor data on traffic contexts. The traffic context metadata engine receives data query signals from various functions, accesses and retrieves data through a metadata model, and then outputs the retrieved and processed data as the output signal to the specific functional algorithm module.

[0258] Section Two: Security Module Interface. In autonomous vehicles, in addition to collecting information through sensors such as cameras and radar, the vehicle also communicates with surrounding vehicles using an independent short-range communication module, and interacts with the cloud platform using 4G network modules, 5G network modules, and the vehicle's own WLAN (Wireless Local Area Network) module. During data interaction, there are potential security issues such as data leakage, unauthorized access, tampering, and damage. Therefore, this section approves the data acquired through communication to ensure data security. The data sources, data formats, communication protocols, and encryption protocols for vehicle-to-vehicle and vehicle-to-cloud communication are pre-classified and organized. When external information is received, the data is approved sequentially according to the classification results. Only after passing four approvals can data transmission continue; otherwise, the data is invalid. Through data approval and screening, secure filtering of data is achieved, removing untrusted data and ensuring information security.

[0259] Step 3: Data Adaptation Interface. Due to the large variety of sensors and the diverse types of traffic scene data collected, such as image data, point cloud data, location data, and vehicle motion parameters, the data exhibits significant heterogeneity. Therefore, data fusion processing is required. Multi-source sensor data is stored in binary or text format. Based on the adapter information in the metadata benchmark table, appropriate adapters are matched to different data formats to parse the data.

[0260] Step 4: Data Interaction from Heterogeneous Data Sources in Traffic Scenario. Different sensors use different coordinate systems. First, sensor parameter calibration transforms the data from each sensor into the same coordinate system for processing. Second, the data from different sensors undergoes correction, noise reduction, filtering, and distortion correction to eliminate system errors and noise from different sensors. Next, feature information is extracted from the data from different sensors, and the multi-source sensor data is matched in time and space to correlate the data collected by different sensors. Then, a Kalman filter algorithm is used to fuse the correlated data, inputting it into the algorithm in matrix format to obtain more accurate environmental information. Finally, state estimation is performed on the fused data to obtain an accurate description of the vehicle's surrounding environment, including the position, speed, and attitude of surrounding vehicles, as well as the position, speed, and shape of surrounding obstacles. Therefore, the input signal for this step is multi-source heterogeneous sensor data of the traffic scenario; the output signal is the fused traffic scenario description.

[0261] exist Figure 3 An exemplary implementation of the first step is shown. The first step ultimately outputs multi-dimensional time-series data and communication data acquired by the sensor.

[0262] The second step involves analyzing the causal cognitive mechanism of the human-machine collaborative system in driving situations as follows:

[0263] Step 1: Analysis of the Driver's Causal Cognitive Mechanism. Step 1 consists of four parts.

[0264] Section 1: Transition Entropy Causal Relationship Model Based on Multivariate Time-Series Data. Common human-machine causal cognitive characteristic variables during driving include: steering wheel angle, steering wheel speed, brake pedal position and rate of change, accelerator pedal position and rate of change, turn signal usage time; vehicle longitudinal speed and acceleration, lateral speed and acceleration, vehicle yaw angle, yaw rate; distance to the vehicle in front, relative speed, TTC, lane departure degree, status of the vehicle in front's lights; traffic light status, traffic signs, pedestrian movement status, pedestrian distance.

[0265] The causal relationship of the above human-machine causal cognitive characteristic variables is established by using the transfer entropy method. For any two variables X and Y, the transfer entropy formula is obtained by using information entropy and conditional entropy.

[0266] Information entropy is expressed by formula (1):

[0267] Η(X)=-∑Ρ(x)log2Ρ(x) (1)

[0268] In the formula, x represents all possible values ​​of variable X, and P(x) represents the probability that random variable X takes the value x.

[0269] Conditional entropy is expressed by formula (2):

[0270] Η(X|Y)=-∑∑Ρ(x,y)log2Ρ(x|y) (2)

[0271] The transfer entropy is obtained from H(X) and H(Y), and formula (3) defines the transfer entropy T from variable Y to X. Y→X :

[0272]

[0273] In the formula, x t and y t Let x represent the values ​​of two variables at time t. t (k) =[x t ,x t-1 ,…,x t-k+1 [] represents all values ​​of x over the past k time points, y t (l)=[x t ,x t-1 ,…,x t-l+1 [] represents all values ​​of y over the past l time steps. The value of the transfer entropy reflects the relationship of information flow between two variables. Since the transfer entropy of the two variables is asymmetric, the causal relationship between the two variables needs to be expressed by equation (4):

[0274] T X,Y =T X→Y -T Y→X (4)

[0275] If T X,Y If the value of X is less than 0, then variable X is the result of variable Y; otherwise, variable X is the cause of variable Y. The causal relationship of the human-machine causal cognitive characteristic variables summarized in this section is calculated to obtain the causal relationship matrix N. This matrix is ​​then used as the adjacency matrix in the graph structure to construct the edges between nodes in the graph.

[0276] Step 2: Convolutional Feature Extraction Model Based on Multivariate Time Series Data. The values ​​of the variables mentioned in Step 1 of Step 1 of the second step are derived from real driving scenarios and are strongly correlated with driving rules and driving habits. They change regularly over time, so periodic processing of multivariate time series data is required. This step uses multiple depthwise separable convolutions of different scales to extract information of the same feature at different times, and uses a first convolution with a kernel of 1×m (where m is the number of channels) to aggregate the information of different channels, thereby obtaining the information merging of the features of the time series data over all time periods. This process is shown in equations (5) and (6):

[0277] h i =ReLU(W i *x+b i (5)

[0278]

[0279] In the formula h i W represents the output of the i-th layer of the neural network. i Let b be the weight vector. i The vector represents the bias vector, * represents the convolution operation, and ReLU is a non-linear activation function. This indicates a chain operation.

[0280] Step 3: Graph Attention Network Model Based on Data Fusion. This step utilizes the causal relationship matrix between temporal variables of human-machine causal cognition characteristics obtained in Step 1 and the feature matrix under each variable node obtained in Step 2 to construct a graph structure. In this step, a graph attention network is used to represent the graph structure and node content within a unified framework, and different weights are assigned to domain nodes.

[0281] In a graph attention network, the nodes in the graph are represented as shown in equation (7):

[0282]

[0283] Where z i (l+1)Let N(i) be the output representation of node i, N(i) be the neighboring nodes of i, σ be a nonlinear function, and α be the output representation of node i. ij The value represents the importance of neighboring nodes, where W is the weight. The node importance coefficient is calculated from both attribute values ​​and topological distance. Regarding attribute values, the attention coefficient α... ij Equation (8) represents:

[0284]

[0285] In the formula, W xi W xj Let the weights be the weights of the i-th and j-th inputs. This is the scaling factor.

[0286] Regarding topological distance, the target node is calculated based on neighboring nodes and edges between nodes, using t-order neighboring nodes to obtain the proximity matrix M:

[0287] M = (B + B) 2 +…+B t ) / t (9)

[0288] In the formula, B is the transition matrix, M is the t-th order topological dependence between the two nodes, and t is a hyperparameter.

[0289] After normalizing the attention coefficients and adding the topological weights M and the LeakyReLU activation function, the attention coefficients are expressed as follows:

[0290]

[0291] Where δ is the activation function, M ij Let t represent the topological correlation between the two nodes.

[0292] The collected time-series data of human-machine causal cognitive characteristic variables are used as input, and the graph attention network finally outputs driver operation data with a dimension of 1 as the prediction result.

[0293] Step Four: Analysis of Driver's Causal Cognitive Mechanism in Typical Scenarios. In the following typical driving scenarios: 1) Driving along the lane: straight driving, curve driving, following another vehicle, and cutting in / out of lanes ahead; 2) Lane changing: overtaking, terrain change driving, and lane closure driving; 3) Driving through traffic light intersections: straight driving, turning, and U-turn; 4) Ramp driving; 5) Parking driving. The first three steps of Step One in the second section use time-series data of multiple human-machine causal cognitive characteristic variables as input to obtain causal inferences about driver behavior. In each typical driving scenario, the inferred driver operational characteristics are categorized and summarized according to driving common sense, cognitive logic, and legal provisions to obtain driver operational patterns in different scenarios, and then the driver's causal cognitive mechanism is analyzed.

[0294] Step Two: Analysis of the Causal Cognitive Mechanism of the Self-Driving System. Step Two consists of four parts.

[0295] Section 1: A Causal Framework for Autonomous Driving Cognition Based on Multivariate Time-Series Data. In the understanding and response of autonomous driving systems to driving situations, the multivariate variables used for causal cognitive mechanism analysis include: throttle opening, braking pressure, steering wheel angle, and rate of change of steering wheel angle; vehicle longitudinal speed and acceleration, lateral speed and acceleration, vehicle yaw angle, and yaw rate; distance to the vehicle in front, relative speed, TTC (Traffic Traction Control), lane departure degree, and the status of the vehicle in front's lights; traffic light status, traffic signs, pedestrian movement status, and pedestrian distance.

[0296] This step uses Lasso regression to select variables, thereby reducing the dimensionality of the variables. Furthermore, only one regression model needs to be established to perform causal analysis between all input variables and the target variable. Assuming the throttle opening is the response variable, its time series representation is given by equation (11):

[0297] Y = (y1, y2, ..., y n ) T (11)

[0298] List the other variables as covariates, as in equation (12):

[0299] X = (X 1 ,X 2 ,...,X n (12)

[0300] The ordinary linear model is expressed as equation (13):

[0301] Y=Xβ+ε (13)

[0302] For each of these covariates Xj ,X i =(X1) i X2 i ,…,X n I ) T ε is the random error term, and β is the regression coefficient. An L1 penalty term is added to the ordinary linear model for variable selection, thereby achieving dimensionality reduction of the variable data. The Lasso estimate for the ordinary linear model is given by equation (14):

[0303]

[0304] In the formula, the second term on the right side of the equation is the L1 penalty term, and λ is the adjustment coefficient. Lasso regression can be used to conveniently calculate causal relationships under complex dimensions and to initially screen out variables with strong causal relationships to the response variables of autonomous driving systems.

[0305] Section 2. Cognitive Perceptron Based on Multivariate Time-Series Data. A multi-output multilayer perceptron neural network model is constructed using the TensorFlow deep learning framework. The input layer, hidden layer, and output layer of the model are defined. The output y is the weighted sum of the nodes connected to y in the previous layer, as shown in Equation (15):

[0306]

[0307] In the formula, x i w is the input to the previous layer node. i denoted by , b represents the weight coefficient, and Act is the non-linear activation function used to trigger node activation. A loss function C(w,b) is constructed using the mean squared error. The model is trained using multivariate time-series data, and the weights and biases are adjusted using the backpropagation algorithm to minimize the loss function. The backpropagation process is shown in Equations 16 and 17.

[0308]

[0309]

[0310] Here, α is the learning rate, and the gradient algorithm is iterated until the cost converges.

[0311] The input signal in this step is the selected multivariate time-series data with strong causal relationships, and the output signal is the trained multilayer perceptron model.

[0312] Step 3: Convolutional Neural Networks Based on Multivariate Temporal Data. This step involves preprocessing the multivariate temporal variable data for analyzing the causal cognitive mechanism of autonomous driving. One-dimensional data is normalized, and multivariate data is standardized. A convolutional neural network model is then constructed, suitable for processing mixed multivariate temporal data. The model's input conforms to the characteristics of both one-dimensional and multivariate data. One-dimensional convolutional layers are used to process one-dimensional data, while two-dimensional or three-dimensional convolutional layers are used to process multivariate data. The data processed by the convolutional layers is input into pooling layers to extract key features. These key features are then input into fully connected layers to capture global features and establish the relationship between features and output. Finally, the processed dataset is input into the model for training and optimization to improve model performance. The input signal in this step is the selected multivariate temporal data with strong causal relationships, and the output signal is the trained multilayer perceptron model.

[0313] Step Four: Analysis of the Intrinsic Cognitive Mechanism of Autonomous Driving Systems in Typical Scenarios. In typical driving scenarios, the first three steps of Step Two in Step Two use time-series data of multivariate causal cognitive characteristic variables of autonomous driving as input to obtain predictions of the autonomous driving system's response. Under each type of typical driving scenario, the predicted autonomous driving system response characteristics are classified and summarized according to vehicle dynamics models, driving common sense, cognitive logic, and legal provisions to obtain the response patterns of autonomous driving systems in different scenarios, and then analyze the causal cognitive mechanism of autonomous driving systems.

[0314] Step 3: Analysis of the causal cognitive coupling mechanism of the human-machine collaborative system. Step 3 consists of three parts.

[0315] Step 1: Extraction of Human-Machine Causal Relationship Features. From the data collected by sensors, driver operation data, autonomous driving system response data, and driving scenario data are extracted. These data types specifically include:

[0316] 1) Driver operation data: torque applied to the steering wheel by the driver, pressure applied to the accelerator pedal by the driver, pressure applied to the brake pedal by the driver, and driver's gear shifting operations;

[0317] 2) Automated driving system response data: steering wheel signal, braking signal, transmission signal, gear shift signal;

[0318] 3) Driving scenario data: The vehicle's position, speed, acceleration, and attitude data, as well as an accurate description of the vehicle's surrounding environment after processing by the metadata engine, namely the position, speed, and attitude of surrounding vehicles, and the position, speed, and shape of surrounding obstacles;

[0319] Based on the strength of the causal relationships obtained in steps one and two of the second step, the data is filtered to obtain data with stronger causal relationships, thereby acquiring data features of important scene elements in the driving scenario. Furthermore, the causal relationships between the data are standardized and labeled, including: the vehicle in front slows down, the vehicle in front brakes, the vehicle in front passes a zebra crossing and slows down, the appearance of a pedestrian in front and slows down, and the traffic light turning red and causing the vehicle to brake, etc. These labels are then summarized to obtain the tag features of the scene elements.

[0320] Step Two: Human-Machine Causal Cognitive Feature Representation Learning Model. This involves representing the data features x of subjects within the same scene. sd and label features x sl The features are processed to maintain dimensionality consistency, and then concatenated to obtain new object features t. s The same operation is performed on the object features to obtain the object features t. o The object features are fed into a Transformer-based feature representation learning model to obtain t. s 'and t o ';Data features of subject and object are processed by u so =Union(x sd ,x od The combined operation yields the combo box features. Finally, the learned subject object features, combo box features, and learned object features are concatenated to obtain the relation feature x. tri =[t s ',uso,t o After training on various types of data, all relation features are obtained. The relation features obtained in this step will then be used for classification in a relation classifier.

[0321] Step 3: Generating a Model for the Causal-Cognitive Coupling Mechanism. In typical driving scenarios, the actions of the driver and the autonomous driving system are responses based on a comprehensive judgment of the driving scenario, requiring all scenario features within the typical scenario. Simultaneously, the driver and the autonomous driving system need to cooperate to complete the driving task. Therefore, for each typical driving scenario, all features X within the scenario are extracted. sc The object characteristics X of the subject s and the object characteristics X o Here, the subject and object are any two objects within the scene that have a causal relationship, such as the vehicle in front and the vehicle itself, or the lane line and the vehicle itself. Then, the subject features and scene features are fused to obtain the subject object feature X of the fused scene. s '=f(X s ,X sc Similarly, the object features X of the fused scene are obtained by processing the object features. o '=f(Xo ,X sc ), then X s 'and X o 'Feature representation learning model based on Transformer is fed into the model, the learned object features are concatenated, two objects are randomly selected to form a relationship pair, and the distribution network of all subjects, relationships and objects in each scenario is output to establish a causal cognition coupling mechanism generation model.'

[0322] Step 4: Comprehensive Model of Causal Analysis for Human-Machine Collaborative Systems. Step 4 consists of three parts.

[0323] Step 1: Multivariate Causal Relationship Model in Typical Scenarios. Using the human-machine causal cognition characteristic variables from Step 1 in typical scenarios as model input, an initial directed acyclic graph is constructed. Each node represents a variable, and directed edges represent causal relationships between variables. For each node, a conditional probability table is constructed based on the strength of the causal relationship determined in Step 1. This table reflects the probability distribution of each node given its parent node, representing the causal relationship between the node and its given parent node. This constitutes a Bayesian network model based on the human-machine causal cognition characteristic variables. Similarly, the same operation is performed on the causal cognition characteristic variables of the autonomous driving system in typical scenarios from Step 2 to construct a Bayesian network model based on these variables.

[0324] Examine the variables in two Bayesian network models and identify their common variables. Establish directed edges between the common variables in the two models to represent their causal relationships, connecting the two models through these edges. Build conditional probability tables for the common variables, and refine the conditional probability tables for nodes involving common variables in both models to ensure that the shared variables have consistent parameters when calculating conditional probabilities, thus obtaining a multivariate causal relationship model for typical scenarios.

[0325] Step Two: Human-Machine Cognitive Efficiency Model in Typical Scenarios. Define metrics for evaluating human-machine cognitive efficiency, including driver response time, autonomous driving system response time, and response quality. Conduct experiments in various typical scenarios, collecting driver behavior data, perception data, and vehicle state data, and extracting data features. Use a support vector machine algorithm to associate the extracted features with the corresponding cognitive efficiency metrics to complete data training. Utilize the training results to filter data with high correlation to cognitive efficiency. Use the main relationship classifications and the distribution networks of all subjects, relationships, and objects in typical scenarios obtained from the human-machine causal cognitive feature representation learning model to identify subjects or objects associated with each data type in typical scenarios. Use regression analysis to calculate the subject and object features associated with cognitive efficiency and each response data, obtaining a human-machine cognitive efficiency model with causal cognitive coupling in typical scenarios.

[0326] Step 3: Causal Cognition Analysis Model for Human-Machine Collaborative Systems. This step integrates multivariate data causal relationship models and human-machine cognitive efficiency models in typical scenarios to establish a unified model framework. Based on Bayesian network models of causal cognitive feature variables from both human-machine and autonomous driving systems, these models are extended into a comprehensive causal cognitive network. Considering the common variables in both steps, directed edges are established to connect these common variables, representing the causal relationships between them.

[0327] Data features of the human-machine collaborative system were extracted from experimental data in various typical scenarios. Support Vector Machine (SVM) algorithms were then used to correlate these features with metrics of human-machine cognitive effectiveness. The conditional probabilities between each node and its child nodes were determined using the training results. Simultaneously, the conditional probability tables for nodes involving common variables were revised to ensure consistent parameters for shared variables in both models when calculating conditional probabilities, continuously expanding and updating the model's applicability.

[0328] exist Figure 7 An exemplary implementation of step four in the second step is presented.

[0329] exist Figure 4 An exemplary implementation of the second step is shown. The second step ultimately outputs a causal cognitive network for a human-machine collaborative system.

[0330] The process of constructing the driving scenario reasoning model in the third step is as follows:

[0331] Step 1: Multi-objective driving scenario learning framework. Step 1 consists of three parts.

[0332] Step 1: Summarizing and Semantic Analysis of Driving Context Elements. Based on the composition of the driving context, the elements are summarized into the following categories:

[0333] 1) Driving mode: Driving mode refers to the way and behavior used to control the driving of a car, such as following the car and changing lanes;

[0334] 2) Driving Intent: Driving intent refers to the driver's intention and purpose during the driving process. It indicates the driver's desire to achieve a certain goal through specific actions or movements. For example, when a driver turns on their turn signal and approaches an intersection, it can be inferred that their driving intent is to prepare to turn;

[0335] 3) Driving behavior: Driving behavior refers to the movement of a vehicle under the control of a driver or an automated driving system, including acceleration, braking, steering, etc.

[0336] 4) Traffic participants and traffic signs;

[0337] 5) Road conditions and weather conditions.

[0338] First, a convolutional neural network model is pre-trained to identify and classify entity elements in the driving context. Second, a support vector machine model is used to train a relation extraction model to represent the relationships between entity elements and rule elements in the driving context, such as the rule that vehicles cannot cross the stop line when the traffic light is red. A pre-trained semantic role labeling model is then used to label the semantic roles in sentences. Finally, a semantic interpretation model is used to transform text or speech information into a machine-understandable semantic representation.

[0339] Section Two: Attribute Constraint Specifications. In specific driving scenarios, the driving scenario elements, their attributes, and the constraints on each element are defined. These constraints include traffic rules, road conditions, vehicle status, and driver behavior. An example is a vehicle approaching a traffic light intersection in a straight-ahead lane when the traffic light is red. The possible driving behaviors corresponding to this deceleration are slow driving or stopping. Since the vehicle is in the straight-ahead lane, the driving behavior is limited to normal driving or stopping to wait for the traffic light. Driving is restricted by the straight-ahead traffic light, and a red light prohibits vehicles from proceeding. Therefore, the purpose of slowing down is to stop and wait for the traffic light to change.

[0340] Step 3: Logical Resolution Completeness Analysis. First, each constraint is transformed into a logical expression or propositional logic form, ensuring that each constraint can be precisely expressed as a logical statement, and using logical symbols to represent relationships. Second, the completeness of the constructed logical expression is checked, i.e., whether it covers all possible situations and constraints, ensuring that the logical expression can encompass all possibilities in the driving scenario. The relationships between elements in the logical expression are checked to ensure that the logical relationships accurately reflect the actual relationships between elements in the driving scenario. Finally, the accuracy and completeness of the logical expression are verified by testing actual driving scenario cases. If the logical expression can correctly predict and describe the elements and constraints in the driving scenario, then the logical resolution can be considered complete.

[0341] Step Two: Extended Driving Context Network Based on Memory Metadata. Step Two consists of four parts.

[0342] Step 1: Transient and Long-Term Memory Structure. To construct an extended semantic network for driving situations, this step adopts a memory model that mimics the brain: transient memory is responsible for temporarily storing reasoning information, while long-term memory stores all received facts and high-probability inferences.

[0343] Long-term memory, as a large, extended semantic network, stores factual evidence obtained from input text and highly probable information derived through reasoning. Because the traffic rules generated by long-term memory and traffic experience gained from training on large amounts of driving data are difficult to retrieve, they are indexed and accessed through metadata.

[0344] Transient memory is a small, extended semantic network that extracts entities and relationships of driving context elements from input text and constructs them simply. It also extracts related entities and relationships with the potential to occur from long-term memory. For the needs of reasoning, reverse edges are added to the relationships in transient memory to represent another voice.

[0345] After the transient memory is constructed, the text information is reasoned, and the reasoning results are sent to other sub-modules. The transient memory network is then automatically deleted. If the relationship obtained by the transient memory reasoning is based on facts, then the relationship and related entities are directly stored in long-term memory; if it is a deduced conclusion, it is stored in long-term memory if the probability is greater than 80%, otherwise it is not stored in long-term memory.

[0346] Step 2: Memorizing the definition of a network.

[0347] After semantic analysis of the current driving situation, the driving situation is represented in sentence form S={S1,S2,…,S…} n The presentation extracts the subject headword or object headword from a series of sentences into V' = {v k |k=1,2,…,n}, extract the relational terms as E'={e d (v i ,v j )|d=1,2,…,n}. G'={V',E'} constitutes an extended semantic network containing S, where V' is the vertex and E' is the edge.

[0348] Step 3: Memorize the definitions of vertices and edges.

[0349] The vertices in the extended semantic network include entity names, identifiers, and related attributes in a sentence. Entities are the subjects and objects of a sentence, representing conceptual entities in a driving context, such as vehicles, lane lines, and pedestrians. Taking a vehicle as an example, a vehicle belongs to a category. In multi-vehicle driving scenarios, to distinguish vehicles, each vehicle needs to be assigned a specific identifier for differentiation and identification. Information such as vehicle type, speed, and distance is recorded in related attributes. For convenient querying and retrieval, information such as entity name, entity name ID, identifier ID, and attribute ID is stored in a metadata baseline table, a metadata mapping table, and a domain mapping table.

[0350] In extended semantic networks, edges represent partial order relations between two entities. Key properties of partial order relations include:

[0351] 1) Relation Name: Use the relational conjunction in the sentence as the relation name;

[0352] 2) Factual basis indicator: Used to indicate whether the edges of the semantic web can be used as factual basis for the reasoning model;

[0353] 3) Probability of relation occurrence: By setting a certain probability for the relation, the authenticity of the sentence can be measured. When this probability is higher than a certain threshold, the edge of relation 2 will be retained; otherwise, it will be deleted.

[0354] 4) The occurrence and duration of the relationship: The occurrence and duration of the relationship will have a significant impact on causal reasoning and need to be stored to determine the order in which the relationship occurs;

[0355] 5) Unary and Binary Relations: In extended semantic networks, edges represent all binary relations, which include direction and purpose. For sentences with only subject-verb structures, such as "The car stopped," in the absence of an object, a hypothetical object is assumed to exist, and the subject itself represents this hypothetical object; that is, the relation starts from the subject and returns to the subject.

[0356] 6) Multiple relationships between two vertices: There may be multiple relationships between any two elements in a driving scenario. In this case, there may be multiple edges between two fixed points.

[0357] Similarly, the relation name, relation name ID, and relation attribute ID are stored in the metadata base table, metadata mapping table, and domain mapping table.

[0358] Section 4: Adaptive Bayesian Driving Context Network. The set of entity events in the extended semantic network based on driving context is E = [E...]. i Based on the metadata baseline table, query and extract event E sequentially. i And select all those containing E i The two-node Bayesian confidence networks are added to the Bayesian confidence network set B. Entity events are grouped according to the relevance of factual evidence, and their corresponding two-node Bayesian confidence network sets are merged. Two-node Bayesian confidence networks that are relevant to the factual evidence are selected to reduce the complexity of the inference model. Finally, the selected two-node Bayesian confidence networks are combined into a unified Bayesian confidence network based on common nodes, and the conditional probabilities are updated.

[0359] Step 3: Dynamic Fuzzy Logic Reasoning. This step consists of the following four parts.

[0360] Step 1: Hierarchical Structure of Reasoning Patterns. When using known elements of the driving situation to perform logical reasoning about the driving situation, the following reasoning pattern (Equation 18) is divided into two layers: inner and outer.

[0361]

[0362] The inner layer consists of the "Xis CX" layer and the "Yis CY" layer, while the outer layer is the "(X,Y)is CX∩CY" layer. The inner layer uses a general reasoning model to find the relevant information CX and CY of the driving situation elements X and Y, while the outer layer, based on the inner layer, seeks the correlation information between the driving situation elements X and Y.

[0363] Section Two: Graph Method for Reasoning Patterns. First, a rule base is established using the constraint rules of driving scenario element attributes, and the dynamic fuzzy reasoning system is represented by a directed graph structure. For any node, if the directed edge leading to the node is not zero, the node is the conclusion of the previous node; if the directed edge leading to the node is not zero, the node is the premise of the next node. For input objects X and Y, nodes with a dynamic fuzzy similarity higher than a specified threshold are first searched and marked. After finding a node, the process propagates along the directed edges, marking the nodes traversed and recording the information in CX and CY. If a node is marked twice, the information of that node and subsequent nodes is recorded in CX∩CY.

[0364] Step 3: Abstraction of Reasoning Pattern Sets. For input objects X and Y, match nodes in the directed graph of the rule base whose dynamic fuzzy similarity reaches a specified threshold, and add all child nodes under that node to set D. x D y In the problem, find the intersection of two sets, C = D. x ∩D y If C is an empty set, then X and Y are not associated in the current rule base; if C is not an empty set, then X and Y are associated in the current rule base, and the association result is the element in C.

[0365] Step Four: Multi-Source Reasoning Expansion Mechanism. In complex driving scenarios, the relationships between elements are not always one-to-one. More often, multiple elements are mutually constrained; for example, traffic signs, road conditions, and the vehicle ahead all affect the vehicle's speed. Therefore, connecting multiple objects for reasoning is equally important. For graph methods, when the number of input elements is n, it is necessary to propagate records starting from the node that has been marked n times; for set methods, it is necessary to find n sets D. n The intersection of.

[0366] Step 4: Metadata Fusion Reasoning. This step consists of four parts.

[0367] Step 1: Metadata Integration Logic

[0368] First, the metadata of the extended semantic network is defined, including the attributes of nodes and edges and the relationships between them. Based on the metadata definition, the extended semantic network is stored in a graph database format, and the attributes and relationships of nodes and edges are mapped to fields in the graph structure of the data storage.

[0369] For a reasoning knowledge base, metadata should include rules, concepts, logical relationships, and semantic relationships. Based on the metadata definition, a stored knowledge base containing data tables and a graph structure is established. The data from the dynamic fuzzy logic reasoning knowledge base is loaded into the data store, and according to the metadata definition, rules, concepts, and logical relationships are mapped to the corresponding data tables and fields in the graph structure within the data store.

[0370] Analyze the common information in the two data sources and determine their field mapping relationships in each data source. Using these mapping relationships, associate fields with the same meaning from different data sources. Based on the field mapping relationships in the metadata, create a union query to retrieve information from both data sources simultaneously.

[0371] Step Two: Driving Context Memory Reconstruction. As the driving task progresses, the system continuously receives new information. Using the information analysis module, it constructs an extended semantic network from this new information and calls upon the relevant Bayesian driving context network within the extended driving context network. The new extended semantic network is then merged with the original extended semantic network, probabilities are recalculated, and the Bayesian driving context network is updated based on the newly calculated probabilities before being fed back into the database.

[0372] Step 3: Updating the Inference Knowledge Base. Because the experiential information stored in the database cannot reflect the diversity of real-world driving scenarios, the number of nodes in the Bayesian driving scenario network increases as driving progresses, and the probabilities represented by edges also change. When the probability falls below 0.20, the corresponding inference rule is deleted from the inference knowledge base; when the probability of a new edge between nodes exceeds 0.60, the new inference rule is added to the inference knowledge base.

[0373] Step 4: Inference Model Integration. Due to the diversity of driving scenarios, and the gradual accumulation of data, new inference rules will be added to the inference knowledge base, resulting in structural and storage redundancy. To ensure the inference model is comprehensive yet concise, common nodes need to be merged periodically to integrate the inference model.

[0374] exist Figure 8 An exemplary implementation of step four in step three is shown in the figure.

[0375] exist Figure 5 An exemplary implementation of the third step is shown in the figure.

[0376] The process of constructing the human-machine collaborative scenario evolution evaluation model in step four is as follows:

[0377] Step 1: Analysis of the Evolutionary Patterns of Human-Machine Collaboration Situations. This step consists of three parts.

[0378] Step 1: Design of causal correlation indicators.

[0379] To confirm the effectiveness of the causal analysis model for the human-machine collaborative system, it is necessary to design relevant indicators to evaluate the model's causal analysis efficiency.

[0380] In the car-following scenario, the causal relationship analysis model of the human-machine collaborative system concludes that there is a causal relationship between the speed and distance of the vehicle in front and the operating state of the human-machine collaborative driving system of this vehicle. In this scenario, let the evaluation factor set be A = [force applied by the driver to the brake pedal, force applied by the driver to the accelerator pedal, braking signal of the autonomous driving system, acceleration signal of the autonomous driving system], and the dangerous collision time TTC under the car-following condition be used as the evaluation.

[0381] In lane-changing scenarios, a causal relationship analysis model of the human-machine collaborative system concludes that there is a causal relationship between the speed and distance of the vehicle in front in the original lane, the speed and distance of the vehicles in front and behind in the target lane, and the operating state of the vehicle's human-machine collaborative driving system. In this scenario, let the evaluation factor set be A = [force applied by the driver to the brake pedal, force applied by the driver to the accelerator pedal, steering torque applied by the driver to the steering wheel, braking signal of the autonomous driving system, acceleration signal of the autonomous driving system, steering signal of the autonomous driving system], and the dangerous collision time (TTC) under lane-changing conditions be used as the evaluation criterion.

[0382] Step 2: Construction of Fuzzy Comprehensive Evaluation System

[0383] First, the weights W of each evaluation factor are determined using principal component analysis and factor analysis. Then, the evaluation factor A in the human-machine collaborative causal cognition analysis model is treated as a fuzzy mapping to the comment set V, and a fuzzy evaluation matrix R is established. Based on fuzzy theory, the driver's situational cognition comprehensive evaluation B is obtained by calculating the weights of the evaluation factors and the fuzzy evaluation matrix, and the evaluation result is obtained according to the principle of maximum membership.

[0384] B = W·R ( 19)

[0385] Step 2: Contextual reasoning based on spatiotemporal patterns.

[0386] Step 1: Vehicle Dynamics Internal Model Construction. A linear two-degree-of-freedom vehicle dynamics model is proposed based on the following assumptions: only the lateral motion along the y-axis and the yaw angle motion around the z-axis are considered; the steering wheel angle is equated to the front wheel angle, and the deformation and interference of the suspension and steering systems are ignored; the effect of the self-centering torque is ignored, and it is assumed that each tire has linear elasticity F.Y =k*α, where the lateral stiffness k is a constant for a given tire.

[0387] Linear two-degree-of-freedom vehicle dynamics model:

[0388]

[0389]

[0390] Where M is the mass of the car, δ f For the front wheel steering angle, C f C r Let l represent the lateral stiffness of the front and rear wheels, l1 be the distance from the center of gravity to the front axle, l2 be the distance from the center of gravity to the rear axle, and Ω be the lateral stiffness of the front and rear wheels. z V is the yaw rate of the vehicle. x V y Let x be the vehicle's speed along the x-axis and y-axis.

[0391] Section Two: Derivation of Model Prediction Equations. To predict the future state of the driving scenario, model predictive control is used to predict the vehicle's future behavior. The controller's prediction equations are as follows:

[0392]

[0393] Where, x k This represents the state or variable of the system at time step k. k+1 This represents the state vector at the next time step k+1. A d This is the state transition matrix, which describes how the system state changes within a time step. k The control input vector represents the control input or operation of the system at time step k. d The control input matrix describes the effect of the control input within one time step. (D) d This is the direct action matrix, representing the effect of control inputs directly on state variables, without going through the state transition matrix. k This is the output vector, representing the system's output or measurement at time step k. C d The output matrix describes how the state vector is mapped to the output vector. (D) d,dis Let be the disturbance vector, representing the disturbance or interference to the system at time step k.

[0394] Formula (22) can be rearranged into formula (23):

[0395] X=FX0+ΦU+E (23)

[0396] Where X is the state vector, representing the system's state or variable at the current time step. F is the state transition matrix, describing how the system state changes within a time step. X0 is the initial state vector, representing the system's state or variable at the initial time step. Φ is the control input matrix, describing the effect of the control input within a time step. U is the control input vector, representing the control input or operation of the system at the current time step. E is the disturbance vector, representing the disturbance or interference to the system at the current time step.

[0397] The variables and coefficient matrix in formula (23) are as follows:

[0398]

[0399] Step 3: Spatiotemporal Evolution of the Inference Model. By fusing the vehicle dynamics model with the model prediction equations and inputting the vehicle parameters in the current state, the prediction results for the vehicle state at future moments can be obtained.

[0400] Step 3: Target Situation Prediction. This step consists of three parts.

[0401] Step 1: Metadata-Driven Spatiotemporal Data Integration. During driving, the metadata-driven engine accesses current sensor data, vehicle status data, vehicle-to-vehicle communication data, and driving scenario inference models for subsequent calculations.

[0402] Step Two: Target Intent Analysis. An adaptive Bayesian driving scenario network is constructed. Through joint calculation using a vehicle dynamics model and model predictive control, the vehicle's future attitude predictions are obtained. The predicted vehicle state is input into a transient memory module for inference. By comparing this prediction with long-term memory data, reasonable inference results are retained, while unreasonable ones are ignored. The vehicle's state from the reasonable inference results is then analyzed in conjunction with the driving scenario to derive the driving intent of the human-machine collaborative system in the current driving situation.

[0403] Step 3: Driving Risk Assessment. Establish a driving risk field based on driving risk field theory:

[0404]

[0405] Among them, E s_j Let E be the resultant vector of the driving risk field at position j of the vehicle. v_j Let E be the resultant vector of the kinetic field strength at position j of the vehicle. R_j Let E be the resultant vector of the static energy field at position j of the vehicle. D_j Let F be the resultant vector of the field strength at position j of the vehicle. j Let M be the resultant vector of the driving risk field at position j of the vehicle, i.e., the driving risk. j Let v be the equivalent mass of vehicle j.j Let R be the speed of vehicle j. j Let be the road factor at vehicle j, k2 be a given constant, and θ be the road factor at vehicle j. j Let D be the angle between the direction of the vehicle's velocity j and the direction of the electric field. ri For driver risk factors.

[0406] The system will input scenario data and driving data from the current moment to the predicted time into the driving risk field for calculation, and obtain the driving risk at the current moment and within the predicted time, thereby judging whether the decision of the human-machine collaborative system is reasonable.

[0407] exist Figure 9 An exemplary implementation of step three in step four is shown in the figure.

[0408] Step 4: Integration of Context Evolution Assessment Results. This step consists of four parts.

[0409] Step 1: Updating the Metadata List. As driving scenarios continue to expand, data content, data fields, and data types from various data sources are constantly being updated. The metadata list should be updated accordingly to facilitate subsequent data retrieval and storage.

[0410] Step Two: Updating the Causal Cognition Logic. Existing causal cognition models are trained based on previously acquired driving data and are insufficient to cover complete driving scenarios. As driving data is collected and stored, the causal cognition model will be trained using newly collected data and existing data at regular intervals, continuously updating the causal cognition logic.

[0411] Step 3: Updating the Reasoning Knowledge Base. Expanding the driving context causes the long-term memory extended semantic network to continuously add or delete nodes and edges, and to continuously adjust probabilities, while simultaneously expanding the reasoning rule base. This continuous process of memory reconstruction completes the update of the reasoning knowledge base.

[0412] Step 4: Integration of Contextual Evolution Assessment Results. The inference results based on spatiotemporal patterns, risk assessment results, and driving parameters from the human-machine collaborative system are integrated and input into the decision-making module to complete subsequent decision-making and planning operations.

[0413] Figure 6 An exemplary implementation of the fourth step is shown in the figure.

Claims

1. A driving situation reasoning method based on metadata-driven and causal analysis theory, characterized in that: The method includes the following steps: The first step is to integrate multi-source sensing metadata. The specific steps are as follows: Step 1: Construction of the metadata representation framework; Step 2: Integration of agent metadata; Step 3: Integration of traffic contextual metadata; The second step is to analyze the causal cognitive mechanism of the human-machine collaborative system in driving situations. The specific steps are as follows: Step 1: Analysis of the driver's causal cognitive mechanism; Step 2: Analysis of the causal cognitive mechanism of the self-driving system; Step 3: Analysis of the causal cognitive coupling mechanism of the human-machine collaborative system; Step 4: Comprehensive model of causal analysis for human-machine collaborative systems; The third step is to construct a driving scenario reasoning model, and the specific steps are as follows: Step 1: Multi-objective driving scenario learning framework; Step 2: Extended driving context network based on memory metadata; Step 3: Dynamic fuzzy logic reasoning; The specific steps in step three are as follows: Section 1, Reasoning Pattern Hierarchy: When using known driving situation elements to perform logical reasoning about the driving situation, the following reasoning pattern (18) is divided into two layers: inner and outer. The inner layer consists of the "Xis CX" layer and the "Yis CY" layer, while the outer layer consists of the "(X,Y)is CX∩CY" layer. The inner layer uses a general reasoning model to find the relevant information CX and CY of the driving situation elements X and Y, while the outer layer seeks the association information of the driving situation elements X and Y based on the inner layer. Section Two: Graph Method for Reasoning Mode. First, a rule base is established using the constraint rules of the driving context element attributes. The dynamic fuzzy reasoning system is represented by a directed graph structure. For any node, if the directed edge entering the node is not 0, the node is the conclusion of the previous node; if the directed edge leaving the node is not 0, the node is the premise of the next node. For input objects X and Y, nodes with a dynamic fuzzy similarity higher than a specified threshold are first searched and marked. After finding a node, the propagation proceeds along the directed edge, and the nodes traversed are marked and their information is recorded in CX and CY. If a node is marked twice, the information of that node and subsequent nodes is recorded in CX∩CY. Step 3: Abstraction of the reasoning pattern set. For input objects X and Y, match nodes in the directed graph of the rule base whose dynamic fuzzy similarity reaches a specified threshold, and add all child nodes under that node to set D. x D y In the problem, find the intersection of two sets, C = D. x ∩D y If C is an empty set, then X and Y are not associated in the current rule base; if C is not an empty set, then X and Y are associated in the current rule base, and the association result is the element in C. Step Four: Multi-Source Reasoning Extension Mechanism. In complex driving scenarios, the relationships between elements are not always one-to-one. More often, multiple elements are mutually constrained; for example, traffic signs, road conditions, and the vehicle ahead all affect the vehicle's speed. Therefore, connecting several objects for reasoning is equally important. For graph methods, when the number of input elements is n, it is necessary to propagate the record starting from the node marked n times. For set methods, it is necessary to find n sets D. n The intersection; Step 4: Metadata Fusion Reasoning; The specific steps in step four are as follows: Step 1: Metadata Integration Logic; First, the metadata of the extended semantic network is defined, including the attributes of nodes and edges and the relationship information between them. Based on the definition of the metadata, the extended semantic network is stored in the format of a graph database, and the attributes and relationships of nodes and edges are mapped to the fields in the graph structure of the data storage. For the reasoning knowledge base, the metadata includes rules, concepts, logical relationships and semantic relationships. Based on the definition of the metadata, a storage knowledge base containing data tables and graph structures is established. The data of the dynamic fuzzy logic reasoning knowledge base is loaded into the data storage, and according to the definition of the metadata, the rules, concepts and logical relationships are mapped to the corresponding data tables and fields in the graph structure in the data storage. Analyze the common information in the two data sources and determine their field mapping relationships in each data source. Through the mapping relationship, associate fields with the same meaning in different data sources. Based on the field mapping relationship in the metadata, create a joint query statement to retrieve information from both data sources simultaneously. Step 2: Driving Context Memory Reconstruction. As the driving task progresses, the system continuously receives new information. Using the information analysis module, the new information is used to construct an extended semantic network. The system then calls the relevant Bayesian driving context network in the extended driving context network, merges the new extended semantic network with the original extended semantic network, recalculates the probabilities, updates the Bayesian driving context network based on the newly calculated probabilities, and sends the results back to the database. Step 3: Inference knowledge base update. Because the experience information stored in the database cannot reflect the diversity of real driving situations, as driving progresses, the number of nodes in the Bayesian driving situation network increases, and the probabilities represented by the edges also change. When the probability is lower than 0.20, the corresponding inference rule in the inference knowledge base is deleted; when the probability of the new edge between nodes is higher than 0.60, the new inference rule is added to the inference knowledge base. Step 4: Inference Model Integration. Due to the diversity of driving scenarios and the gradual accumulation of data, new inference rules will be added to the inference knowledge base, and there will be redundancy in structure and storage. In order to make the inference model comprehensive and concise, it is necessary to merge common nodes periodically and integrate the inference model. The fourth step is to construct an evaluation system for the evolution of human-machine collaborative scenarios. The specific steps are as follows: Step 1: Analysis of the evolutionary patterns of human-machine collaboration scenarios; Step 2: Contextual reasoning based on spatiotemporal patterns; Step 3: Target Situation Prediction; Step 4: Integrate the results of the situational evolution assessment.

2. The driving situation reasoning method based on metadata-driven and causal analysis theory according to claim 1, characterized in that: The steps in the first step are as follows: The specific steps in Step 1 are as follows: Step 1, Metadata Modeling: This step involves modeling metadata to describe the data characteristics, attributes, and data sources and fields associated with scene perception functions. Metadata modeling comprises three parts: For each specific data source, the metadata model includes the data source name, connection information, and a data source model representing the data processing algorithm interface corresponding to different data structures; for any specific function within the scene perception function based on multi-source sensor data, such as object detection and recognition, the metadata model includes the function name, the data name associated with that function, and the association and sequence information of various data types under that function, forming a function-data source association model; for any function and any specific data field of its corresponding data source, such as partial image data acquired by the camera required for data fusion, the metadata model includes a function-data field association model containing data source information, data field information, and function information. Therefore, the input signals for this step are the data source data input from each sensor and the data call signals based on the function. The output signals represent the access relationships between each function and the data source; Step 2: Metadata Baseline Table Construction. This step describes the basic information of each data source by establishing a technical metadata information table to store the data source information. The data source table includes an auto-incrementing primary key, a data source ID for identification, a data source name to distinguish different data sources, the adapter name for parsing the data source, and the data connection address of the data source. The names of multi-source sensor data sources are: camera data source, LiDAR data source, millimeter-wave radar data source, inertial sensor IMU data source, and GPS data source. Step 3: Metadata Mapping Table Construction. This step describes the relationship between functions and data sources by establishing a metadata mapping table. The metadata mapping table includes an auto-incrementing primary key, function IDs and names used to distinguish different functions, associated data source IDs and names when the same function is associated with one or more data sources, and the order of data sources when the same function is associated with multiple data sources. Step 4: Domain Mapping Table Construction. This step describes the relationship between functions and data fields by establishing a domain mapping table. The implementation of a certain function only requires some information from different data sources. This table specifically constructs the mapping relationship between functions and specific fields. The domain mapping table includes an auto-incrementing primary key, data field name, data input / output type, function ID and name associated with the data field, and data source ID and name to which the data field belongs. In multi-sensor data sources, data is stored in the following format: 1) Camera data source: Each frame of data captured by the camera is stored in the form of a two-dimensional array, with each element storing [RGB color data, grayscale data], and each frame of data is accompanied by a timestamp; 2) LiDAR data source: The raw data of the lidar is received through the user data packet protocol and stored in the form of data packets. The data format is as follows: the data length is fixed at 1248 bytes. The first 48 bytes are the data packet identifier, the next 1200 bytes are 12 groups of data packets with a length of 100 bytes each. Each data packet contains information on the rotation angle, distance, and reflection intensity of the laser beam. The last 6 bytes are a 2-byte timestamp and a 4-byte lidar model. 3) Millimeter-wave radar data source: The raw data from the millimeter-wave radar is received via the User Data Packet Protocol and stored in the form of data packets. The data format is as follows: the first 32 bytes of the millimeter-wave radar data packet are the start frame identifier, followed by n data packets. Each data packet contains target distance, velocity, angle, target size, and target intensity information. Finally, there is a 2-byte timestamp and a 4-byte end frame identifier. 4) Inertial Measurement Unit (IMU) data source: The data is stored in a CSV file with the following headers: timestamp, acceleration X, acceleration Y, acceleration Z, gyroscope X, gyroscope Y, and gyroscope Z. 5) GPS data source: The data is stored in a CSV file with the following headers: timestamp, longitude, latitude, and altitude. In the multi-sensor data source domain mapping table, the data field names are: 1) From the camera data source: element position information, RGB color information, grayscale information, and time information; 2) Data from the lidar data source includes: time, X coordinate, Y coordinate, Z coordinate, and the reflection intensity of the point; 3) Data sources from millimeter-wave radar: time, target ID, range, velocity, angle, target size, and target intensity; 4) Data from the inertial sensor (IMU): time, vehicle acceleration, vehicle angular velocity, and vehicle orientation; 5) GPS data source: longitude, latitude, altitude, and time of the vehicle's location; The specific steps in the first and second steps are as follows: Step 1: Intelligent Agent Metadata Engine. In the environmental perception process, the collection and calculation of the vehicle's motion and location data is a crucial step. This step establishes an intelligent agent metadata engine to address the data retrieval needs of the intelligent agents, enabling unified scheduling of this data. The intelligent agent metadata engine receives data query or modification signals from various functions, accesses GPS and IMU data through the metadata model, and then completes data extraction and modification. Therefore, the input signal of this step is the environmental perception process's demand for intelligent agent data; the output signal is the data extracted and modified according to functional requirements. Step 2, Data Query Interface: When the implementation of specific functions in the environmental perception process requires vehicle data as input, the data query interface receives the ID corresponding to the function, queries the metadata associated with the specific function through the metadata model, and finds the corresponding single database data or several database data based on the metadata. Therefore, the input signal of this step is the ID corresponding to each function; the output signal is the data extracted according to the functional requirements. Step 3, Data Change Interface: This step is used to feed back the data processed by each function to each data source. To achieve environmental perception, it is necessary to fuse the agent data with the driving scenario data for the next step of calculation. At the same time, the agent data also needs to be sent back to the data source as a data backup for subsequent data analysis and algorithm improvement, data verification and debugging, training and machine learning, as well as safety and traceability. Therefore, the input signal of this step is the ID corresponding to each function. The output signal is data that has been processed to a certain extent and is used for backup; Step 4: Data interaction with heterogeneous data sources for the intelligent agent. After querying data by function, the data collected by the IMU is read in text form. A CSV file parsing library is used to read and parse the acceleration, angular velocity, and orientation data of the IMU, as well as the vehicle longitude, latitude, altitude, and time information collected by the GPS. The parsed data is then input into the multi-sensor data fusion algorithm corresponding to the specific function to complete the subsequent heterogeneous data interaction. The specific steps in step three of the first step are as follows: Section 1: Traffic Context Metadata Engine. This section establishes a traffic context metadata engine to address the data retrieval needs of traffic contexts. It is used for unified scheduling of different needs of various functions for multi-source sensor data in traffic contexts during the real-time process of environmental perception. The traffic context metadata engine receives data query signals from various functions, accesses data through the metadata model, and then searches for the data. The searched and processed data is then transmitted as the output signal of this section to the specific functional algorithm module. Step Two: Security Module Interface. This step involves approving data acquired through communication to ensure data security. The data sources, data formats, communication protocols, and encryption protocols for vehicle-to-vehicle and vehicle-to-cloud communication are pre-classified and organized. When external information is received, the data is approved sequentially according to the classification results. Only after passing four approvals can the data transmission continue; otherwise, the data is invalid. Through data approval and screening, security filtering of data is achieved, removing untrusted data and ensuring information security. Section 3: Data Adaptation Interface. Due to the large number of sensor types and the diverse types of traffic scene data collected, including image data, point cloud data, location data, and vehicle motion parameters, the data exhibits strong heterogeneity. Therefore, it is necessary to fuse the queried data. Multi-source sensor data is stored in binary or text format. Based on the adapter information in the metadata benchmark table, the corresponding adapter is matched to the data of different formats to parse the data format. Step 4: Data interaction from heterogeneous data sources in traffic scenarios. Different sensors have different coordinate systems. First, sensor parameter calibration is used to convert the data from each sensor to the same coordinate system for processing. Second, the data from different sensors are corrected, denoised, filtered, and distortion corrected to eliminate system errors and noise from different sensors. Subsequently, feature information is extracted from the data of different sensors, and the multi-source sensor data is matched in time and space to associate the data collected by different sensors. The Kalman filter algorithm is then used to fuse the associated data and input it into the algorithm in matrix format to obtain more accurate environmental information. Finally, by performing state estimation on the fused data, an accurate description of the vehicle's surrounding environment is obtained, namely the position, speed, and attitude of surrounding vehicles, as well as the position, speed, and shape of surrounding obstacles. Therefore, the input signal of this step is multi-source heterogeneous sensor data of the traffic scene; the output signal is the fused description of the traffic scene.

3. The driving situation reasoning method based on metadata-driven and causal analysis theory according to claim 1, characterized in that: The steps in the second step are as follows: The specific steps in step one of the second step are as follows: Section 1: Based on the transfer entropy causal relationship model of multivariate time series data, common human-machine causal cognitive characteristic variables during driving include: steering wheel angle, steering wheel speed, brake pedal position and rate of change, accelerator pedal position and rate of change, turn signal usage time; vehicle longitudinal speed and acceleration, lateral speed and acceleration, vehicle yaw angle, yaw rate; distance to the vehicle in front, relative speed, TTC, lane departure degree, and the status of the vehicle in front's lights; traffic light status, traffic signs, pedestrian movement status, and pedestrian distance; The causal relationship of the above human-machine causal cognitive characteristic variables is established by the method of transfer entropy. For any two variables X and Y, the transfer entropy formula is obtained by information entropy and conditional entropy. Information entropy is expressed by formula (1): Η(X)=-∑Ρ(x)log2Ρ(x) (1) In the formula, x represents all possible values ​​of variable X, and P(x) represents the probability that random variable X takes the value x. Conditional entropy is expressed by formula (2): Η(X|Y)=-∑∑Ρ(x,y)log2Ρ(x|y) (2) The transfer entropy is obtained from H(X) and H(Y), and formula (3) defines the transfer entropy T from variable Y to X. Y→X : In the formula, x t and y t Let x represent the values ​​of two variables at time t. t (k) =[x t ,x t-1 ,…,x t-k+1 [x] represents all values ​​of x over the past k time points, y t (l)=[x t ,x t-1 ,…,x t-l+1 Let ] represent all values ​​of y over the past l moments. The value of the transfer entropy reflects the relationship of information flow between the two variables. Since the transfer entropy of the two variables is asymmetrical, the causal relationship between the two variables needs to be expressed by equation (4): T X,Y =T X→Y -T Y→X (4) If T X,Y If X < 0, then variable X is the result of variable Y; otherwise, variable X is the cause of variable Y. The causal relationship of the human-machine causal cognition characteristic variables summarized in this section is calculated to obtain the causal relationship matrix N. This matrix is ​​used as the adjacency matrix in the graph structure to construct the edges between nodes in the graph. Step 2: Convolutional Feature Extraction Model Based on Multivariate Time Series Data. The values ​​of the variables mentioned in Step 1 of Step 1 of the second step are derived from real driving scenarios and are strongly correlated with driving rules and driving habits. They change regularly over time, so it is necessary to periodically process the multivariate time series data. This step uses multiple depthwise separable convolutions of different scales to extract information of the same feature at different times, and uses a first convolution with a kernel of 1×m to aggregate information from different channels, where m is the number of channels, thereby obtaining the information merging of the features of the time series data over all time periods. This process is shown in equations (5) and (6): h i =ReLU(W i *x+b i ) (5) In the formula h i W represents the output of the i-th layer of the neural network. i Let b be the weight vector. i The vector represents the bias vector, * represents the convolution operation, and ReLU is a non-linear activation function. Indicates concatenation operation; Step 3: Graph Attention Network Model Based on Data Fusion. This step uses the causal relationship matrix between the time-series variables of human-machine causal cognition characteristics obtained in Step 1 and the feature matrix under each variable node obtained in Step 2 to construct a graph structure. In this step, the graph attention network is used to represent the structure and node content of the graph in a unified framework, and different weights are assigned to the domain nodes. In a graph attention network, the nodes in the graph are represented as shown in equation (7): Where z i (l+1) Let N(i) be the output representation of node i, N(i) be the neighboring nodes of i, σ be a nonlinear function, and α be the output representation of node i. ij The value represents the importance of neighboring nodes, where W is the weight. The node importance coefficient is calculated from both attribute values ​​and topological distance. Regarding attribute values, the attention coefficient α... ij Equation (8) represents: In the formula, W xi W xj Let the weights be the weights of the i-th and j-th inputs. This is the scaling factor; Regarding topological distance, the target node is calculated based on neighboring nodes and edges between nodes, using t-order neighboring nodes to obtain the proximity matrix M: M=(B+B 2 +…+B t ) / t (9) In the formula, B is the transition matrix, M is the t-th order topological dependence between the two nodes, and t is a hyperparameter; After normalizing the attention coefficients and adding the topological weights M and the LeakyReLU activation function, the attention coefficients are expressed as follows: Where δ is the activation function, M ij The topological correlation between the two nodes is t-order. The collected time-series data of human-machine causal cognitive characteristic variables are used as input, and the graph attention network finally outputs driver operation data with a dimension of 1 as the prediction result. Step 4: Analysis of Driver's Causal Cognitive Mechanism in Typical Scenarios. In the following typical driving scenarios: 1) Driving along the current lane: straight driving, curve driving, following another vehicle, and vehicles cutting in / out; 2) Lane changing: overtaking, terrain change, and lane closure; 3) Driving through traffic light intersections: straight driving, turning, and U-turn; 4) Ramp driving; 5) Parking. The first three steps of Step 1 in the second step use time-series data of multiple human-machine causal cognitive characteristic variables as input to obtain causal inferences about driver behavior. In each typical driving scenario, the inferred driver operational characteristics are categorized and summarized according to driving common sense, cognitive logic, and legal provisions to obtain driver operational patterns in different scenarios, thereby analyzing the driver's causal cognitive mechanism. The specific steps in step two of the second step are as follows: Section 1: A Causal Framework for Autonomous Driving Cognition Based on Multivariate Time-Series Data. In the understanding and response of autonomous driving systems to driving situations, the multivariate variables used for causal cognitive mechanism analysis include: throttle opening, braking pressure, steering wheel angle, and rate of change of steering wheel angle; vehicle longitudinal speed and acceleration, lateral speed and acceleration, vehicle yaw angle, and yaw rate; distance to the vehicle in front, relative speed, TTC, lane departure degree, and status of the vehicle in front's lights; traffic light status, traffic signs, pedestrian movement status, and pedestrian distance. This step uses Lasso regression to select variables and reduce variable dimensionality. At the same time, a regression model needs to be established to perform causal analysis between all input variables and the target variable. Assuming that the throttle opening is the response variable, the time series of this variable is expressed as Equation (11): Y=(y1,y2,...,y n ) T (11) List the other variables as covariates, as in equation (12): X=(X 1 ,X 2 ,...,X n ) (12) The ordinary linear model is expressed as equation (13): Y=Xβ+ε (13) For each of these covariates X j ,X i =(X1) i X2 i ,…,X n I ) T ε is the random error term, β is the regression coefficient, and an L1 penalty term is added to the ordinary linear model for variable selection, thereby achieving dimensionality reduction of the variable data. The Lasso estimate for the ordinary linear model is given by equation (14): In the formula, the second term on the right side of the equal sign is the L1 penalty term, and λ is the adjustment coefficient. Using Lasso regression can easily calculate causal relationships under complex dimensions and initially screen out variables that have a strong causal relationship with the response variables of the autonomous driving system. Step 2: Cognitive Perceptron Based on Multivariate Time-Series Data. A multi-output multilayer perceptron neural network model is constructed using the TensorFlow deep learning framework. The input layer, hidden layer, and output layer of the model are defined. The output y is the weighted sum of the nodes connected to y in the previous layer, as shown in Equation (15): In the formula, x i w is the input to the previous layer node. i Here, is the weight coefficient, b is the bias, and Act is the nonlinear activation function used to trigger node activation. The loss function C(w,b) is constructed using the mean squared error. The model is trained using multivariate time series data, and the weights and biases are adjusted through the backpropagation algorithm to minimize the loss function. The backpropagation process is shown in Equations 16 and 17. Where α is the learning rate, and this gradient algorithm is iterated until the cost converges; The input signal for this step is the selected multivariate time-series data with strong causal relationships, and the output signal is the trained multilayer perceptron model. Step 3: Based on multivariate time-series data, a convolutional neural network is used to preprocess the multivariate time-series variable data for analyzing the causal cognitive mechanism of autonomous driving. One-dimensional data is normalized, and multi-dimensional data is standardized. A convolutional neural network model is constructed: a convolutional neural network model suitable for processing mixed multivariate time-series data is built. The model's input conforms to the characteristics of one-dimensional and multi-dimensional data. One-dimensional convolutional layers are used to process one-dimensional data, and two-dimensional or three-dimensional convolutional layers are used to process multi-dimensional data. The data processed by the convolutional layers is input into pooling layers to extract key features, and then the key features are input into fully connected layers to capture global features and establish the relationship between features and output. Finally, the processed dataset is input into the model for training, and the model is optimized to improve its performance. The input signal in this step is the selected multivariate time-series data with strong causal relationships, and the output signal is the trained multilayer perceptron model. Step 4: Analysis of the causal cognitive mechanism of the autonomous driving system in typical scenarios. In typical driving scenarios, the first three steps of step 2 of the second step take time-series data of multivariate autonomous driving causal cognitive characteristic variables as input to obtain predictions of the autonomous driving system response. In each type of typical driving scenario, the predicted autonomous driving system response characteristics are classified and summarized according to the vehicle dynamics model, driving common sense, cognitive logic, and legal provisions to obtain the autonomous driving system response rules in different scenarios, and then analyze the causal cognitive mechanism of the autonomous driving system. The specific steps in step three of the second step are as follows: Step 1: Human-Machine Causal Relationship Feature Extraction. This involves extracting driver operation data, autonomous driving system response data, and driving scenario data from the data collected by sensors. Specifically, these data types include: 1) Driver operation data: torque applied to the steering wheel by the driver, pressure applied to the accelerator pedal by the driver, pressure applied to the brake pedal by the driver, and driver's gear shifting operations; 2) Automated driving system response data: steering wheel signal, braking signal, transmission signal, gear shift signal; 3) Driving scenario data: The vehicle's position, speed, acceleration, and attitude data, as well as an accurate description of the vehicle's surrounding environment after processing by the metadata engine, namely the position, speed, and attitude of surrounding vehicles, and the position, speed, and shape of surrounding obstacles; Based on the strength of the causal relationships obtained in steps one and two of the second step, the data is filtered to obtain data with stronger causal relationships, thereby obtaining data features of important scene elements in the driving scenario; and the causal relationships between the data are uniformly labeled, including: the vehicle in front decelerates, the vehicle in front brakes, the vehicle in front passes a zebra crossing, the vehicle in front appears a pedestrian, and the vehicle in front brakes when the traffic light turns red. The labels are then summarized to obtain the tag features of the scene elements. Step Two: Human-Machine Causal Cognitive Feature Representation Learning Model, which uses the data features x of subjects in the same scene. sd and label features x sl The features are processed to maintain dimensionality consistency, and then concatenated to obtain new object features t. s The same operation is performed on the object features to obtain the object features t. o The object features are fed into a Transformer-based feature representation learning model to obtain t. s 'and t o ';Data features of subject and object are processed by u so =Union(x sd ,x od The combined operation yields the combo box features. Finally, the learned subject object features, combo box features, and learned object features are concatenated to obtain the relation feature x. tri =[t s ',uso,t o After training on various types of data, all relation features are obtained. The relation features obtained in this step will be entered into the relation classifier for classification. Step 3: Generating a model for the causal cognitive coupling mechanism. In typical driving scenarios, the actions of the driver and the autonomous driving system are responses based on a comprehensive judgment of the driving scenario, requiring all scene features in the typical scenario. Simultaneously, the driver and the autonomous driving system need to cooperate to complete the driving task. Therefore, for each typical driving scenario, all features X within the scenario are extracted. sc The object characteristics X of the subject s and the object characteristics X o Here, the subject and object are any two objects within the scene that have a causal relationship, such as the vehicle in front and the vehicle itself, or the lane line and the vehicle itself. Then, the subject features and scene features are fused to obtain the subject object feature X of the fused scene. s '=f(X s ,X sc Similarly, the object features X of the fused scene are obtained by processing the object features. o '=f(X o ,X sc ), then X s 'and X o 'Input the Transformer-based feature representation learning model, concatenate the learned object features, randomly select two objects to form a relationship pair, output the distribution network of all subjects, relationships and objects in each scenario, and establish a causal cognition coupling mechanism generation model;' The specific steps in step four of the second step are as follows: Step 1: Multivariate Data Causal Relationship Model in Typical Scenarios. Using the human-machine causal cognition characteristic variables from Step 1 as model input, an initial directed acyclic graph is constructed. Each node represents a variable, and directed edges represent causal relationships between variables. For each node, a conditional probability table is constructed based on the strength of the causal relationship determined in Step 1. This table reflects the probability distribution of each node given its parent node, representing the causal relationship between the node and its given parent node. This constitutes a Bayesian network model based on the human-machine causal cognition characteristic variables. Similarly, the same operation is performed on the autonomous driving system causal cognition characteristic variables from Step 2 to construct a Bayesian network model based on the autonomous driving system causal cognition characteristic variables. Examine the variables in two Bayesian network models and identify the common variables between them. Establish directed edges between the common variables in the two models to represent the causal relationship between them. Connect the two models through these edges, establish a conditional probability table for the common variables, and revise the conditional probability tables of the nodes involving the common variables in the two models to ensure that the variables shared in the two models have consistent parameters when calculating the conditional probabilities, thereby obtaining a multivariate data causal relationship model in a typical scenario. Step 2: Human-Machine Cognitive Efficiency Model in Typical Scenarios. This step defines metrics for evaluating human-machine cognitive efficiency, including driver response time, autonomous driving system response time, and response quality. Experiments are conducted in various typical scenarios to collect driver behavior data, perception data, and vehicle state data, extracting data features. Support Vector Machine (SVM) algorithms are used to correlate the extracted features with corresponding cognitive efficiency metrics, completing data training. The training results are used to filter data with high correlation to cognitive efficiency. The main relationship classifications and the distribution networks of all subjects, relationships, and objects in typical scenarios are obtained from the human-machine causal cognitive feature representation learning model. Subjects or objects associated with each data type in typical scenarios are identified. Regression analysis is used to calculate the subject and object features associated with cognitive efficiency and each response data, resulting in a human-machine cognitive efficiency model with causal cognitive coupling in typical scenarios. Section 3, the causal cognition analysis model of the human-machine collaborative system, integrates the multivariate data causal relationship model and the human-machine cognitive efficiency model in typical scenarios to establish a unified model framework. Based on the Bayesian network model based on the causal cognitive feature variables of human-machine and the Bayesian network model based on the causal cognitive feature variables of the autonomous driving system, it is extended into a comprehensive causal cognitive network. Considering the common variables of the two sections, directed edges are established to connect the common variables in the two models to represent the causal relationship between them. Data features of the human-machine collaborative system in various typical scenarios are extracted, and the support vector machine algorithm is used to associate the extracted features with the measurement index of human-machine cognitive effectiveness. The conditional probabilities between each node and its child nodes are determined using the data training results. At the same time, the conditional probability tables of nodes involving common variables are revised to ensure that the variables shared in the two models have consistent parameters when calculating the conditional probabilities, and the scope of application of the model is continuously expanded and updated.

4. The driving situation reasoning method based on metadata-driven and causal analysis theory according to claim 1, characterized in that: The steps in the third step are as follows: The specific steps in step one of the third step are as follows: Step 1: Summarizing and Semantic Analysis of Driving Context Elements. Based on the composition of the driving context, the elements of the driving context are summarized into the following categories: 1) Driving mode: Driving mode refers to the way and behavior used to control the driving of a car, such as following the car and changing lanes; 2) Driving intention: Driving intention refers to the driver's intention and purpose during the driving process. It indicates that the driver hopes to achieve a certain goal through specific actions or movements. When the driver turns on the turn signal and approaches the intersection, it can be inferred that his driving intention is to prepare to turn. 3) Driving behavior: Driving behavior refers to the movement of a vehicle under the control of a driver or an automated driving system, including acceleration, braking, and steering; 4) Traffic participants and traffic signs; 5) Road conditions and weather conditions; First, a convolutional neural network model is used to pre-train entity elements in the driving scenario to identify and classify various entity elements. Second, a support vector machine model is used to train a relation extraction model to identify the relationship between entity elements and rule elements in the driving scenario, such as the rule that vehicles cannot cross the stop line when the traffic light is red. A pre-trained semantic role labeling model is used to label the semantic roles in the sentence. Finally, a semantic interpretation model is used to convert the text or speech information into a semantic representation that the machine can understand. Section Two: Attribute Constraint Specifications. In specific driving scenarios, the driving scenario elements, their attributes, and the constraints of each element are specified. The constraints of each element include traffic rules, road conditions, vehicle status, and driver behavior. Taking the situation where a vehicle approaches a traffic light intersection along a straight lane when the traffic light is red, and the vehicle slows down, as an example, the possible driving behaviors corresponding to the vehicle slowing down are low-speed driving or stopping. Since the vehicle is in the straight lane, the driving behaviors are only normal driving or stopping to wait for the traffic light. Driving is restricted by the straight traffic light, and a red light prohibits vehicles from passing. Therefore, the purpose of slowing down the vehicle is to stop and wait for the traffic light to change. Step 3: Logical completeness analysis. First, each constraint is transformed into a logical expression or propositional logic form to ensure that each constraint can be accurately expressed as a logical statement, and logical symbols are used to represent the relationship. Secondly, check whether the constructed logical expression is complete, that is, whether it covers all possible situations and constraints, and ensure that the logical expression can cover all possibilities in the driving situation; check whether the relationship between the elements in the logical expression is correct, and ensure that the logical relationship accurately reflects the actual relationship between the elements in the driving situation. Finally, verify the accuracy and completeness of the logical expression by testing actual driving situation cases. If the logical expression can correctly predict and describe the elements and constraints in the driving situation, then the logical resolution can be considered complete. The specific steps in step two of the third step are as follows: Step 1: Transient and Long-Term Memory Structure. To construct an extended semantic network for driving situations, this step adopts a memory model that mimics the brain. Transient memory is responsible for temporarily storing reasoning information, while long-term memory stores all received facts and high-probability inferences. Long-term memory, as a large extended semantic network, stores factual evidence obtained from input text and information with high probability after reasoning. Because the traffic rules generated by long-term memory and traffic experience trained on a large amount of driving data are difficult to retrieve, they are indexed and retrieved through metadata. Transient memory is a small, extended semantic network that extracts entities and relationships of driving context elements from the input text, constructs them simply, and extracts related entities and relationships with the probability of occurrence from long-term memory. For the needs of reasoning, reverse edges are added to the relationships in transient memory to represent another voice. After the transient memory is constructed, the text information is reasoned, and the reasoning results are sent to other sub-modules. The transient memory network is automatically deleted. If the relationship obtained by the transient memory reasoning is based on facts, then the relationship and related entities are directly stored in long-term memory. If it is a deduced conclusion, it will be stored in long-term memory if the probability is greater than 80%; otherwise, it will not be stored in long-term memory. Step 2: Memorizing the definition of a network; After semantic analysis of the current driving situation, the driving situation is represented in sentence form S={S1,S2,…,S…} n The presentation extracts the subject headword or object headword from a series of sentences into V' = {v k |k=1,2,…,n}, extract the relational terms as E'={e d (v i ,v j )|d=1,2,…,n};G'={V',E'} constitutes an extended semantic network containing S, where V' is a vertex and E' is an edge; Step 3: Memorize the definitions of vertices and edges; The vertices in the extended semantic network include the entity name, identification code, and related attributes of the sentence. The entity is the subject and object of the sentence, representing the conceptual entity in the driving context, such as car, lane line, and pedestrian. Taking car as the representative, car belongs to a category. In the multi-vehicle driving context, in order to distinguish the vehicles, each vehicle needs to be assigned a specific identification code for differentiation and identification. The vehicle type, speed, and distance information are recorded in the related attributes. For convenient querying and retrieval, the entity name, entity name ID, identification code ID, and attribute ID information are stored in the metadata baseline table, metadata mapping table, and domain mapping table. In extended semantic networks, edges represent partial order relations between two entities. Key properties of partial order relations include: 1) Relation Name: Use the relational conjunction in the sentence as the relation name; 2) Factual basis indicator: Used to indicate whether the edges of the semantic web can be used as factual basis for the reasoning model; 3) Probability of Relationship Occurrence: By setting a certain probability for the relationship, the authenticity of the sentence can be measured. When this probability is higher than the set threshold, the edge of relationship 2 will be retained; otherwise, it will be deleted. 4) The occurrence and duration of the relationship: The occurrence and duration of the relationship will have a significant impact on causal reasoning and need to be stored to determine the order in which the relationship occurs; 5) Unary and binary relations: In extended semantic networks, edges represent all binary relations, which include direction and purpose. For sentences with only subject-verb structure, such as "the car stopped", in the case of no object, it is assumed that there is a virtual object, which is represented by the subject itself. That is, the relation starts from the subject and returns to the subject. 6) Multiple relationships between two vertices: There may be multiple relationships between any two elements in a driving scenario. In this case, there can be multiple edges between two fixed points. Similarly, the relation name, relation name ID, and relation attribute ID are stored in the metadata base table, metadata mapping table, and domain mapping table; Section 4: Adaptive Bayesian Driving Context Network. The set of entity events in the extended semantic network based on driving context is E = [E i Based on the metadata baseline table, query and extract event E sequentially. i And select all those containing E i Two-node Bayesian confidence networks are added to the Bayesian confidence network set B. Entity events are grouped according to the relevance of factual evidence, and their corresponding two-node Bayesian confidence network sets are merged. Two-node Bayesian confidence networks that are related to factual evidence are selected to reduce the complexity of the inference model. Finally, the selected two-node Bayesian confidence networks are combined into a whole Bayesian confidence network based on common nodes, and the conditional probabilities are updated.

5. The driving situation reasoning method based on metadata-driven and causal analysis theory according to claim 1, characterized in that: The steps in the fourth step are as follows: The specific steps in step one of the fourth step are as follows: Step 1: Design of causal correlation indicators; To confirm the effectiveness of the causal analysis model for the human-machine collaborative system, it is necessary to design relevant indicators to evaluate the model's causal analysis efficiency. In the car-following scenario, the causal relationship analysis model of the human-machine cooperative system concludes that there is a causal relationship between the speed and distance of the vehicle in front and the operating state of the human-machine co-driving system of the vehicle itself. In this scenario, let the evaluation factor set be A = [the force applied by the driver to the brake pedal, the force applied by the driver to the accelerator pedal, the braking signal of the autonomous driving system, and the acceleration signal of the autonomous driving system], and use the dangerous collision time TTC under the car-following condition as the evaluation. In lane-changing scenarios, the causal relationship analysis model of the human-machine collaborative system concludes that there is a causal relationship between the speed and distance of the vehicle in front in the original lane, the speed and distance of the vehicles in front and behind in the target lane, and the operating state of the human-machine co-driving system of this vehicle. In this scenario, let the evaluation factor set be A = [force applied by the driver to the brake pedal, force applied by the driver to the accelerator pedal, steering torque applied by the driver to the steering wheel, braking signal of the automatic driving system, acceleration signal of the automatic driving system, steering signal of the automatic driving system], and the dangerous collision time TTC in lane-changing conditions is used as the evaluation. Step 2: Construction of the fuzzy comprehensive evaluation system; First, the weights W of each evaluation factor are determined using principal component analysis and factor analysis. Then, the evaluation factor A in the human-machine collaborative causal cognition analysis model is treated as a fuzzy mapping to the comment set V, and a fuzzy evaluation matrix R is established. Based on fuzzy theory, the weights of the evaluation factors and the fuzzy evaluation matrix are calculated to obtain the driver's situational cognition comprehensive evaluation B, and the evaluation result is obtained according to the maximum membership principle. B = W·R (19); The specific steps in step two of the fourth step are as follows: Step 1: Vehicle Dynamics Internal Model Construction. A linear two-degree-of-freedom vehicle dynamics model is proposed based on the following assumptions: only the lateral motion along the y-axis and the yaw angle motion around the z-axis are considered; the steering wheel angle is equated to the front wheel angle, and the deformation and interference of the suspension and steering systems are ignored; the effect of the self-centering torque is ignored, and it is assumed that each tire has a linear elasticity F. Y =k*α, where the lateral stiffness k is a constant for a given tire; Linear two-degree-of-freedom vehicle dynamics model: Where M is the mass of the car, δ f For the front wheel steering angle, C f C r Let l represent the lateral stiffness of the front and rear wheels, l1 be the distance from the center of gravity to the front axle, l2 be the distance from the center of gravity to the rear axle, and Ω be the lateral stiffness. z V is the yaw rate of the vehicle. x V y The velocity of the vehicle along the x-axis and y-axis; Section 2: Derivation of Model Prediction Equations. To predict the future state of the driving scenario, model predictive control is used to predict the future state of the vehicle. The prediction equations of the controller are as follows: Where, x k x represents the state or variable of the system at time step k; k+1 A represents the state vector at the next time step k+1; d The state transition matrix describes how the system state changes within a time step; u k The control input vector represents the control input or operation of the system at time step k; B d The control input matrix describes the effect of the control input within one time step; D d This is the direct action matrix, representing the effect of control inputs directly on state variables, without going through the state transition matrix; y k C is the output vector, representing the system's output or measurement at time step k; d The output matrix describes how the state vector is mapped to the output vector; D d,dis Let be the disturbance vector, representing the disturbance or interference to the system at time step k; Formula (22) can be rearranged into formula (23): X=FX0+ΦU+E (23) Where X is the state vector, representing the state or variable of the system at the current time step; F is the state transition matrix, describing how the system state changes within a time step; X0 is the initial state vector, representing the state or variable of the system at the initial time step; Φ is the control input matrix, describing the effect of the control input within a time step; U is the control input vector, representing the control input or operation of the system at the current time step; and E is the disturbance vector, representing the disturbance or interference of the system at the current time step. The variables and coefficient matrix in formula (23) are as follows: Step 3: Spatiotemporal evolution of the inference model. This step integrates the vehicle dynamics model with the model prediction equations. By inputting the vehicle parameters in the current state, the prediction results of the vehicle state at future moments can be obtained. The specific steps in step three of the fourth step are as follows: Step 1: Metadata-driven spatiotemporal data integration. During driving, the metadata-driven engine calls up the current sensor data, vehicle status data, vehicle-to-vehicle communication data, and driving scenario reasoning model for subsequent calculations. Step 2: Target Intent Analysis. An adaptive Bayesian driving context network is constructed. The vehicle's attitude prediction for future moments is obtained through joint calculation using the vehicle dynamics model and model predictive control. The predicted vehicle state is input into the transient memory module for inference. By comparing it with long-term memory data, reasonable inference results are retained and unreasonable inference results are ignored. The vehicle state in the reasonable inference results is combined with the driving context for analysis to obtain the driving intent of the human-machine collaborative system in the current driving context. Step 3: Driving risk assessment, establishing a driving risk field based on driving risk field theory: Among them, E s_j Let E be the resultant vector of the driving risk field at position j of the vehicle. v_j Let E be the resultant vector of the kinetic field strength at position j of the vehicle. R_j Let E be the resultant vector of the static energy field at position j of the vehicle. D_j Let F be the resultant vector of the field strength at position j of the vehicle; j Let M be the resultant vector of the driving risk field at position j of the vehicle, i.e., the driving risk. j Let v be the equivalent mass of vehicle j. j Let R be the speed of vehicle j. j Let be the road factor at vehicle j, k2 be a given constant, and θ be the road factor at vehicle j. j Let D be the angle between the direction of the vehicle's velocity j and the direction of the electric field. ri For driver risk factors; The scenario data and driving data from the current moment to the predicted time are put into the driving risk field for calculation to obtain the driving risk at the current moment and within the predicted time, thereby judging whether the decision of the human-machine collaborative system is reasonable; The specific steps in step four of the fourth procedure are as follows: Step 1: Updating the metadata list. As driving scenarios continue to expand, data content, data fields, and data types are constantly being updated from various data sources. The metadata list should be updated along with the data updates to facilitate subsequent data retrieval and storage. Step 2: Updating the causal cognition logic. The existing causal cognition model is trained based on previously acquired driving data and is difficult to cover the complete driving situation. As driving data is collected and stored, the causal cognition model will be trained together with the original data after a certain driving mileage interval, so that the causal cognition logic is constantly updated. Step 3: Updating the reasoning knowledge base. The expansion of driving scenarios causes the long-term memory extended semantic network to continuously add or delete nodes and edges, and continuously correct the probabilities. At the same time, the reasoning rule base is expanded. The reasoning knowledge base is updated through the continuous memory reconstruction process. Step 4: Integration of Contextual Evolution Assessment Results. This step integrates the reasoning results based on spatiotemporal patterns, risk assessment results, and driving parameters of the human-machine collaborative system, and inputs them into the decision-making module to complete subsequent decision-making and planning operations.

Images