Hm-TRW and hagenn structure search-based collision risk prediction method and system for emergency rescue autonomous driving vehicle

Abstract

Disclosed in the present invention are an HM-TRW and HAGENN structure search-based collision risk prediction method and system for an emergency rescue autonomous driving vehicle. The method comprises: abstracting the relationship between an emergency rescue vehicle and surrounding moving objects into a dynamic heterogeneous graph, using HM-TRW to capture the importance and dynamics of each surrounding moving object and various interaction relationships, and fusing same with original features of each surrounding moving object, and inputting same into a HAGENN to perform collision risk prediction. The method which embeds the dynamic heterogeneous graph is applied to a hierarchical attention-based neural network to further learn the heterogeneous characteristics and the dynamic change rules of the surrounding moving objects; and during attention calculation of different hierarchies, a positioning space is used to determine an application position of attention and a parameterized space is used for searching an attention function, and a multi-stage differential search is introduced to accelerate the search process. The present invention can more comprehensively and accurately predict the collision risks of emergency rescue autonomous driving vehicles during operation processes.

Description

Collision risk prediction method and system for emergency rescue autonomous driving vehicles based on HM-TRW and HAGENN structure search Technical Field

[0001] The present invention belongs to the field of autonomous driving collision risk prediction, and specifically relates to a method and system for predicting collision risk of emergency rescue autonomous driving vehicles based on high-order memory-guided temporal random walk (HM-TRW) and hierarchical attention graph embedding neural network (HAGENN) structure search. Background Art

[0002] During an emergency, autonomous rescue vehicle must reach its target location promptly and safely to efficiently complete the mission. During the journey, various uncontrollable factors and unexpected situations may arise, delaying the rescue mission. Therefore, autonomous emergency rescue vehicles must be able to make informed decisions based on the actual situation. Predicting collision risk is an essential step in this decision-making process. Collision risk prediction first utilizes vehicle sensors and communication technology to collect data on surrounding moving objects and predict their future trajectories. By leveraging data from surrounding moving objects and the vehicle itself (the autonomous emergency rescue vehicle), the autonomous vehicle's collision risk can be predicted, providing a basis for trajectory planning and maneuvering decisions. Ultimately, the chassis uses these outputted decision-making parameters to drive the vehicle safely and efficiently.

[0003] Existing methods for predicting collision risk for autonomous emergency rescue vehicles primarily rely on machine learning and deep learning. However, existing models struggle to capture the complex, heterogeneous characteristics and dynamic relationships found in real-world vehicle environments, limiting prediction accuracy. Furthermore, some deep learning models suffer from computational inefficiencies when processing large amounts of real-time data, hindering real-time prediction.

[0004] Summary of the Invention

[0005] In view of the shortcomings in the existing technology, the present invention provides a collision risk prediction method and system for emergency rescue autonomous driving vehicles based on HM-TRW and HAGENN structure search.

[0006] The present invention achieves the above technical objectives through the following technical means.

[0007] Collision risk prediction method for emergency rescue autonomous driving vehicles based on HM-TRW and HAGENN structure search:

[0008] Dataset construction: In real-vehicle experiments, data acquisition equipment is used to collect vehicle data and data on surrounding moving objects, which are then pre-processed and stored in a database.

[0009] Collision risk prediction model construction: A dynamic heterogeneous graph is constructed using collected vehicle data and data on surrounding moving objects. A high-order memory-guided temporal random walk algorithm learns the heterogeneous characteristics and dynamic change patterns of the dynamic heterogeneous graph. The resulting node representations of each surrounding moving object are then fed into a hierarchical attention graph embedding neural network to predict collision risk.

[0010] Model training: Input the training set and validation set in the database into the collision risk prediction model respectively to optimize the model parameters;

[0011] Model testing: Use the test set in the database to test the trained collision risk prediction model, and evaluate and analyze the model prediction performance based on the test results;

[0012] Collision risk prediction: Real-time data collection of emergency rescue autonomous driving vehicles and surrounding moving objects is input into the collision risk prediction model after testing to predict collision risks.

[0013] Furthermore, the learning of the heterogeneous characteristics of the dynamic heterogeneous graph includes:

[0014] (1) Transfer vector

[0015] Set the initial high-order memory queue to be empty, and the type transfer vector accesses each type of surrounding motion objects with equal probability. When accessing the surrounding motion object v j When its type is transferred to vector Update by pressing:

[0016] Where, Represents the surrounding moving objects v j The type of Q represents a first-in-first-out queue, and Norm() represents the return norm of the vector;

[0017] (2) Type conversion

[0018] according to The probability distribution of determines the type of the next surrounding moving object to be visited, and a search mechanism with a search factor α∈[0,1] is used to solve the type trap problem, as follows:

[0019] Where, Indicates the surrounding motion object initially accessed by the type transfer vector, Pr(h n+1 ) indicates that the type of the next surrounding motion object to be accessed is h n+1 probability;

[0020] (3) High-level memory recording

[0021] After visiting the surrounding moving objects v jAfter that, the transfer vector Store it in the first-in-first-out queue Q' for the next step:

[0022] Among them, Put is a queue operator, which means that when the first-in-first-out queue is full, the first transfer vector is popped out and placed at the end of the queue.

[0023] Furthermore, the dynamic change law of the learning dynamic heterogeneous graph is specifically as follows:

[0024] For emergency rescue vehicles i , use h n+1 Indicates the type of surrounding moving objects, so:

[0025] in, Indicates emergency rescue vehicle v i The set of surrounding moving objects at the future timestamp, t′ represents the timestamp of the previous random walk, e t is the edge set of edge type;

[0026] Take an exponentially decaying distribution from the set Select the next motion object to visit:

[0027] Among them, the timestamp t of the k-th random walk in the future k ∈t, t represents the timestamp of random walk; Pr(v n+1 ) indicates that the next moving object to be accessed is v n+1 The probability of v k represents the surrounding moving objects visited by the k-th random walk; the discount rate δ∈[0,1].

[0028] Furthermore, the node representation of each surrounding moving object is a fusion of the output of the high-order memory-guided temporal random walk algorithm and the original features of each surrounding moving object:

[0029] in, They are the surrounding moving objects v j The transfer vector, original features and one-hot vector of is the latent embedding of all moving objects, represents a set of real numbers, N is the number of nodes, D is the feature dimension, W f and W represents the surrounding moving objects v j Learnable parameters that are not shared with other moving objects, and x vjThey represent the original features of surrounding moving objects, the identified embedded features and the final node representation respectively.

[0030] Furthermore, the hierarchical attention graph is embedded in the neural network to predict the collision risk by aggregating different information through node-level attention, edge-level attention and time-level attention;

[0031] For node-level attention, when the timestamp is t, for the interaction relationship type r, the emergency rescue vehicle v i and its surrounding moving objects v j The importance of Calculated by the following formula:

[0032] Among them, σ is the activation function, x i 、x j They are emergency rescue vehicles v i and surrounding moving objects v j The input representation is, Is a linear transformation matrix, || represents connection, Represents the emergency rescue vehicle v of interaction relationship type r at timestamp t i All surrounding moving objects, a r is the weight vector, is a r The transpose of x k represents the kth moving object around the vehicle;

[0033] Emergency rescue vehicle v with interaction relationship type r at timestamp t i Node embedding Expressed as:

[0034] For edge-level attention, an attention mechanism is used to learn the importance of different types of interaction relationships. And computed by a multilayer perceptron:

[0035] Among them, w T is the edge-level attention vector, U el and b el is the single layer parameter of the multilayer perceptron, R is the set of edge types;

[0036] Emergency rescue vehicles v considering the importance of different interaction relationship types i Fusion embedding Expressed as:

[0037] For time-level attention, the fused embeddings of emergency rescue vehicles at all timestamps are aggregated and packaged as T represents the number of historical timestamps used to predict collision risk; the query-key-value vector of the fused embedding is calculated:

[0038] P=G i ·U P

[0039] K=G i ·U K

[0040] V=G i ·U V

[0041] Among them, P, K, V represent query, key and value vectors respectively, and U P 、U K 、U V Indicates that G i Converted into corresponding matrices of query, key, and value vectors, D is the feature dimension, represents the set of real numbers;

[0042] Use the softmax function to calculate the time-level attention:

[0043] Among them, Z i represents temporal attention, is a mask matrix, D′ is the dimension of the query-key-value vector;

[0044] Z i T As the final fused embedding, calculate the collision risk:

[0045] Y=softmax(W2·ReLU(W1·Z i T +b1)+b2)

[0046] Where softmax(·) is the output activation function, W1 and W2 are the weight matrices of the hierarchical attention graph embedding neural network, ReLU(·) is the activation function, and b1 and b2 represent bias terms.

[0047] Furthermore, the attention localization space is used to sparsely process the attention calculated by the hierarchical attention map embedding neural network when predicting collision risk:

[0048] Through the matrix A LO Select the type of surrounding motion objects, interaction relationship type, and number of timestamps you are interested in. The specific calculation is as follows:

[0049] When the timestamp is t, through the matrix Determine whether to focus on surrounding moving objects with interaction type r Represents the emergency rescue vehicle v of interaction relationship type r at timestamp t′ i All surrounding moving objects;

[0050] By using attention to locate the space, the time complexity is:

[0051] in, express The number of non-zero values ​​in , T represents the number of historical timestamps used to predict collision risk, and denote the number of interaction relationships of type r at timestamps t' and t, respectively. O(·) denotes the time complexity, and |R| denotes the number of edge types.

[0052] Furthermore, the attention parameterization space is used to sparsely process the attention calculated by the hierarchical attention graph embedding neural network when predicting collision risk:

[0053] Parameterize the space A using attention Pa Search attention function, the expression is as follows:

[0054] A Pa =A N ×A R

[0055] Among them, A N ={1,…,K N} T×|H| is the node mapping function F N The parameterized matrix of (·), A R ={1,…,K R} 2T×|R| is the edge mapping matrix F R The parameterization matrix of (·), K N and K R are two hyperparameters, |H| represents the number of node types.

[0056] Furthermore, multi-stage differential search is used to reduce the parameter search complexity in the positioning space and parameter space:

[0057] a) Space constraints

[0058] The following two constraints are introduced to reduce the search range and limit the complexity: first, emergency rescue vehicles can only receive information from surrounding moving objects in historical time, and second, through To constrain the number of surrounding moving objects and the number of interaction relationships used for collision risk prediction at each timestamp, is a hyperparameter, 1≦t≦T;

[0059] b) Supernet construction: Using a supernet, we transform the parameter search in the localization space and parameterization space into a neural architecture search problem. Specifically, we represent the choice of operation as a probability distribution:

[0060] Where x is the input, is the output, |A| represents the number of operations, β i Represents the mapping function F corresponding to the i-th operation i (·) mixed weight;

[0061] By adopting a supernet, we jointly optimize the mixing weight β and all parameters in the mapping function in a differentiable way:

[0062] Among them, η w and η β Represent the learning rates of structure weights and model weights respectively, and Represent the loss functions of the training set and the validation set respectively, w represents the structural weight, and β represents the model weight;

[0063] c) Multi-stage supernet training: To stabilize the training of the supernet, the training process is divided into three stages: motion object parameterization, interaction relationship parameterization, and attention localization space search. Each stage focuses on a different parameterization space.

[0064] Furthermore, the dynamic heterogeneous graph is constructed as follows: emergency rescue vehicles and their surrounding moving objects are represented as nodes, and various interactive relationships between them are represented as edges. The nodes and edges change dynamically over time to form a dynamic heterogeneous graph, which is expressed as follows:

[0065] G t =(v t ,e t ,u t )

[0066] Among them, v t is a node set of node type h, e t is the set of edges of edge type r, Represents the feature set of all moving objects, h∈H, r∈R, H and R are the node type set and edge type set respectively, represents a set of real numbers, N is the number of nodes, and D is the feature dimension.

[0067] A collision risk prediction system for emergency rescue autonomous driving vehicles based on HM-TRW and HAGENN structure search, including:

[0068] Data collection equipment, including on-board sensors, roadside equipment, and communication technology, used to collect data from the vehicle and surrounding moving objects;

[0069] Data preprocessing module, used to clean, normalize, extract features, reduce data dimension and divide data sets into collected data;

[0070] A prediction model comprising a high-order memory-guided temporal random walk algorithm, a hierarchical attention graph embedding neural network, and an optimal parameter search module comprising an attention localization space, an attention parameterization space, and a multi-stage differential search module;

[0071] A visualization module for displaying the predicted collision risk.

[0072] The beneficial effects of the present invention are:

[0073] (1) This application constructs a dynamic heterogeneous graph of the interaction between emergency rescue vehicles and surrounding moving objects, which can better learn different types of moving objects and the complex dynamic relationships between them. In addition, the dynamic heterogeneous graph can also capture the static and dynamic features of surrounding moving objects at the same time, thereby providing more comprehensive and accurate collision risk prediction results. Specifically, it learns the position changes of each moving object at different timestamps based on the time series data of the vehicle and surrounding moving objects. Finally, the dynamic heterogeneous graph has strong reliability and generalization capabilities, can automatically adapt to various complex traffic environments, and can handle a variety of dynamic interaction relationships.

[0074] (2) This application uses a high-order memory-guided temporal random walk algorithm and a hierarchical attention graph embedding neural network to further learn the importance of surrounding moving objects and their interactions with emergency rescue vehicles. The high-order memory-guided temporal random walk algorithm can make full use of the historical data of surrounding moving objects and non-decreasing time constraints to consider moving objects that have a greater impact on emergency rescue vehicles, thereby more accurately and efficiently predicting collision risks. At the same time, the hierarchical attention graph embedding neural network uses hierarchical attention layers to capture the importance of each surrounding moving object to the emergency rescue vehicle and the importance of various interactions between the two, further improving the accuracy of the prediction. At the same time, the future collision risk is calculated by integrating important features under historical timestamps.

[0075] (3) This application improves the computational efficiency of the model by constructing an attention localization and parameterization space, and adopts a multi-stage differentiable search algorithm to further accelerate the computation of the model. In actual collision risk prediction, emergency rescue vehicles need to consider the spatiotemporal characteristics of all surrounding moving objects and calculate the importance of various interaction relationships to focus on the interaction features that have a greater impact on the collision risk, so the computational cost is very high. In order to achieve efficient and accurate collision risk prediction, it is necessary to adopt a lighter and more efficient model architecture. The attention localization space and parameterization space can flexibly determine the application location and calculation function of attention, while the multi-stage differentiable search algorithm can screen out invalid model architectures by adopting heuristic constraints and use a single neural architecture search algorithm to determine the optimal architecture. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] FIG1 is a framework diagram of the HM-TRW and HAGENN structure search model according to the present invention;

[0077] FIG2 is a flow chart of a method for predicting collision risk of an emergency rescue autonomous driving vehicle based on HM-TRW and HAGENN structure search according to the present invention;

[0078] FIG3 is a flow chart of model training and verification according to the present invention;

[0079] FIG4 is an example diagram of the interaction between an emergency rescue vehicle and other moving objects in the original traffic scene of the present invention;

[0080] FIG5 is a dynamic heterogeneous graph of the interaction between the emergency rescue autonomous driving vehicle and surrounding moving objects in the present invention;

[0081] FIG6 is a diagram illustrating an example scenario of collision risk prediction for an emergency rescue autonomous driving vehicle according to the present invention;

[0082] FIG7 is a diagram showing the collision risk prediction results of the emergency rescue autonomous driving vehicle according to the present invention. DETAILED DESCRIPTION

[0083] In order to make the purpose and technical solution of this application clearer and easier to understand, the application is further explained below in conjunction with the accompanying drawings, but the protection of this application is not limited thereto.

[0084] 1 and 2 , the emergency rescue autonomous driving vehicle collision risk prediction system based on HM-TRW and HAGENN structural search described in this application includes data acquisition equipment, a data preprocessing module, a prediction model and a visualization module.

[0085] The data collection equipment includes onboard sensors (lidar, accelerometers, speed sensors, steering angle sensors, GPS, cameras, etc.) used in real-vehicle experiments, roadside equipment (cameras and speed radar, etc.), and communication technologies (vehicle-to-vehicle communication and vehicle-to-infrastructure communication) to collect data from the vehicle and surrounding moving objects. The collected data primarily includes the vehicle's speed, acceleration, steering angle, yaw rate, pedal force, and energy consumption; the surrounding moving object data primarily includes the position, speed, acceleration, and movement trajectory of vehicles surrounding the emergency rescue autonomous vehicle and vulnerable traffic groups.

[0086] The data preprocessing module is mainly used to clean, normalize, extract features, reduce the dimension of the collected raw data, and partition the data set so that the prediction model can learn better; the preprocessed data is stored in the database. The specific steps of preprocessing are as follows:

[0087] (1) Data cleaning: remove duplicate data, noisy data, and irrelevant data, fill in missing values, and delete outliers.

[0088] (2) Data normalization: Convert vehicle operation and pedestrian movement data of different dimensions into a unified format to facilitate subsequent processing and analysis.

[0089] (3) Feature extraction: Select important features through feature importance analysis for collision risk prediction.

[0090] (4) Data dimensionality reduction: Principal component analysis is used to reduce the dimensionality of high-dimensional data, thereby reducing computational complexity and memory usage while retaining the main feature information and improving the efficiency and accuracy of collision risk prediction.

[0091] (5) Dataset Division: The data in the database is divided into a training set, a validation set, and a test set. The training set is used to train the model; the validation set is used to optimize the model's prediction performance, calculate the prediction error, and continuously adjust the model parameters to achieve a specific accuracy in collision risk prediction; the test set is used to evaluate and analyze the model's accuracy and generalization ability. The former is evaluated through the accuracy index and the AUC curve (area under the curve), while the latter is used to evaluate the stability of the model's prediction by inputting data from different traffic scenarios (such as signalized intersections, unsignalized intersections, roundabouts, merging areas, etc.). The training set, validation set, and test set are divided into 70%, 20%, and 10% respectively.

[0092] The prediction model consists of a random walk algorithm, a hierarchical attention graph embedded neural network and an optimal parameter search module. The random walk algorithm adopts a high-order memory and non-decreasing time constraint strategy to capture the importance and dynamics of surrounding moving objects. The hierarchical attention graph embedded neural network includes three levels: node-level attention, edge-type attention and time-level attention. The first two levels respectively learn the importance of each surrounding moving object to the emergency rescue vehicle and the importance of various interactive relationships between the two. The last level aggregates the important features learned under historical timestamps to calculate the collision risk. The optimal parameter search module includes an attention positioning space and an attention parameterization space, and also includes a multi-stage differential structure search module to speed up the calculation of attention in the hierarchical attention graph embedding and improve the efficiency of collision risk prediction.

[0093] The visualization module is used to display the predicted collision risk so that the driver can take appropriate risk avoidance measures in a timely manner.

[0094] Furthermore, a collision risk prediction method for emergency rescue autonomous vehicles based on high-order memory-guided temporal random walks and hierarchical attention graph embedding neural network structure search mainly relies on high-order memory-guided temporal random walks and hierarchical attention graph embedding neural networks to achieve efficient and accurate collision risk prediction. The construction process of this model is as follows:

[0095] (1) Dynamic Heterogeneous Graph (DyHG) Construction: All moving objects (including emergency rescue vehicles and their surrounding moving objects) are represented as nodes, and various interactive relationships between them (such as approach trends, competitive relationships, and collaborative relationships) are represented as edges (see Figure 4). Nodes and edges change dynamically over time, forming a dynamic heterogeneous graph (see Figure 5). Its expression is as follows:

[0096] G t =(v t ,e t ,u t )

[0097] Among them, v t is a set of nodes of type h∈H (including surrounding motion object types), e t is the set of edges of type r∈R, Represents the feature set of all moving objects (such as speed, acceleration, motion trajectory, etc.), H and R are the node type set and edge type set respectively, |H| and |R| represent the number of node types and the number of edge types, represents a set of real numbers, N is the number of nodes, and D is the feature dimension.

[0098] (2) High-order memory guided temporal random walk (HM-TRW): When an emergency rescue vehicle is performing collision risk prediction, the importance of the surrounding moving objects varies, and the different interaction relationships with the emergency rescue vehicle also vary in importance. Therefore, this heterogeneous characteristic needs to be considered. At the same time, the surrounding moving objects are constantly changing during the driving process of the emergency rescue vehicle, and the interaction relationships with the emergency rescue vehicle also change accordingly. This will greatly increase the complexity of the dynamic heterogeneous graph structure, so this dynamic characteristic also needs to be considered. Therefore, the high-order memory guided temporal random walk algorithm is introduced.

[0099] Specifically, the heterogeneous characteristics are first learned through high-order memory guidance, where the high-order memory is a first-in-first-out queue that stores different types of surrounding moving objects. The specific steps are as follows:

[0100] Step 1: Transfer vector. Set the initial high-order memory queue to empty, and the type transfer vector accesses each type of surrounding moving objects with equal probability. When accessing the surrounding moving object v j When its type is transferred to vector Update by pressing:

[0101] Where, Represents the surrounding moving objects v j The type of Q represents a first-in-first-out queue, and Norm() represents the return norm of the vector.

[0102] Step 2: Type conversion. According to step 1 The probability distribution of determines the type of the next surrounding moving object to be visited, and a search mechanism with a search factor α∈[0,1] is used to solve the type trap problem, as follows:

[0103] Where, Indicates the surrounding motion object initially accessed by the type transfer vector, Pr(h n+1 ) indicates that the type of the next surrounding motion object to be accessed is h n+1 probability.

[0104] Step 3: High-level memory recording. After visiting the surrounding moving objects v j After that, the transfer vector Store in the first-in-first-out queue for the next step:

[0105]

[0106] Among them, Put is a queue operator, which means that when the first-in-first-out queue is full, the first transfer vector is popped out and placed at the end of the queue.

[0107] Secondly, the dynamic change law of the dynamic heterogeneous graph is learned through non-decreasing time constraints. i (Self-vehicle), use h n+1 Indicates the type of surrounding moving objects, so:

[0108] in, Indicates emergency rescue vehicle v i The set of surrounding moving objects at the future timestamp, t′ represents the timestamp of the previous random walk, Represents the surrounding moving objects v j type.

[0109] Take an exponentially decaying distribution from the set Select the next motion object to visit:

[0110] Among them, t k ∈t, t represents the timestamp, t k represents the timestamp of the kth random walk in the future; Pr(v n+1 ) indicates that the next moving object to be accessed is v n+1 The probability of v k represents the surrounding moving objects visited by the k-th random walk; the discount rate δ∈[0,1] is used to correct the time probability distribution.

[0111] Finally, the output of the high-order memory-guided temporal random walk algorithm is fused with the original features of each surrounding moving object to obtain their respective node representations:

[0112] in, and Represent the original features of the surrounding moving objects, the recognition embedded features and the final node representation, They are the surrounding moving objects v j The transfer vector, original features and one-hot vector of is the potential embedding of all moving objects, W f and W represents the surrounding moving objects v j Learnable parameters that are not shared with other moving objects, Represents the surrounding moving objects v embedded in the neural network with input hierarchical attention map j Node representation.

[0113] (3) Hierarchical Attention Graph Embedding Neural Network (HAGENN): The node representation of each surrounding moving object obtained by the high-order memory-guided temporal random walk algorithm is then fed into the Hierarchical Attention Graph Embedding Neural Network to predict collision risk. It further captures the importance of surrounding moving objects and their interactions, as well as their temporal evolution trends, to improve prediction efficiency and accuracy. Specifically, different information is aggregated through node-level attention, edge-level attention, and temporal-level attention.

[0114] For node-level attention, when the timestamp is t, for the interaction relationship type r, the emergency rescue vehicle v i and its surrounding moving objects v j The importance between can be calculated by the following formula:

[0115] Among them, σ is the activation function, x i 、x j They are emergency rescue vehicles v i , surrounding moving objects v j The input representation is, Is a linear transformation matrix, || represents connection, Represents the emergency rescue vehicle v of interaction relationship type r at timestamp t i All surrounding moving objects, a r is a weight vector that parameterizes the attention function for the interaction type r, is a r The transpose of x k represents the kth moving object around the vehicle. Thus, we can get the emergency rescue vehicle v with interaction relationship type r at timestamp t i Node embedding:

[0116] For edge-level attention, an attention mechanism is used to learn the importance of different types of interaction relationships and calculate them through a multi-layer perceptron:

[0117] Among them, σ is the activation function, w T is the edge-level attention vector, U el and b el is a single layer parameter of the multilayer perceptron, and the importance of different types of interaction relationships is considered for the emergency rescue vehicles v i The fusion embedding of can be expressed as:

[0118] For time-level attention, the fused embeddings of emergency rescue vehicles at all timestamps are aggregated and packaged as Represents the number of historical timestamps used to predict collision risk. Then calculate the query-key-value vector of the fused embedding:

[0119] P=G i ·U P

[0120] K=G i ·U K

[0121] V=G i ·U V

[0122] Among them, P, K, V represent query, key and value vectors respectively, and U P 、U K 、U V Indicates that G i Convert to corresponding matrices of query, key, and value vectors.

[0123] Use the softmax function to calculate the time-level attention:

[0124] Among them, Z i represents temporal attention, is a mask matrix, and D′ is the dimension of the query-key-value vector. i T As the final fused embedding, the collision risk can be calculated:

[0125] Y=softmax(W2·ReLU(W1·Z i T +b1)+b2)

[0126] Where softmax(·) is the output activation function, W1 and W2 are the weight matrices of the hierarchical attention graph embedding neural network, ReLU(·) is the activation function, and b1 and b2 represent bias terms.

[0127] (4) Attention Localization and Parameterized Space: When predicting collision risk, a hierarchical attention graph embedded neural network needs to calculate the attention between different moving objects, different interaction relationships, and different time stamps. This computational cost is high, which leads to low prediction efficiency and makes it impossible to predict collision risk in a timely manner. To address this, attention localization and parameterized space are proposed to sparsely process attention to achieve a more efficient architecture.

[0128] a) Attention localization

[0129] Before adopting the attention localization space, the time complexity of hierarchical attention map embedding is as follows:

[0130] Where T represents the number of historical timestamps used to predict collision risk, and denote the number of interaction relationships of type r at timestamps t' and t, respectively, and O(·) denotes the time complexity.

[0131] Through the matrix You can select the type of surrounding motion objects, interaction relationship type, and number of timestamps you want to focus on. The specific calculation is as follows:

[0132] Furthermore, when the timestamp is t, through the matrix Determine whether to focus on surrounding moving objects with interaction type r (Emergency rescue vehicle v with interaction relationship type r at timestamp t′ i all surrounding moving objects), It can be said that Completely determines where the attention function is applied.

[0133] By using attention to locate the space, the time complexity is greatly reduced:

[0134] in, express By controlling the total number of non-zero values ​​in , the time complexity of hierarchical attention graph embedding can be reduced to a level that is independent of both T and |R|.

[0135] b) Parameterized space

[0136] To further reduce the number of parameters, a parameterized space is proposed to search for the calculation method of the attention function. The expression of the parameterized space is as follows:

[0137] A Pa =A N ×A R

[0138] Among them, A N ={1,…,K N} T×|H| is the node mapping function F N The parameterized matrix of (·); A R ={1,…,K R} 2T×|R| is the edge mapping matrix F R The parameterization matrix of (·); K N and K R are two hyper parameters. The mapping functions of surrounding motion objects and interaction relationships are respectively from A N and A R Select from the drop-down list.

[0139] By using the above parameterized space, shared parameters applicable to similar traffic scenarios and rescue tasks can be adaptively searched and learned. In addition, the use of parameterized space can also reduce the number of learnable parameters. The number of learnable parameters of the original hierarchical attention map embedded neural network can be expressed as O(T|H|+|R|). By adopting the parameterized space, this number can be reduced to O(K N +K R ). When K N and K R When the constraint is a constant, the number of learnable parameters also becomes a constant.

[0140] (5) Multi-stage differential search: In order to further reduce the parameter search complexity in the positioning space and parameter space and achieve efficient collision risk prediction, a heuristic constraint is proposed to remove invalid structures in the search space, and a one-shot neural structure search algorithm is used to speed up the search process.

[0141] a) Spatial constraints: Two constraints are introduced to reduce the search range and limit the complexity: first, emergency rescue vehicles can only receive information from surrounding moving objects in historical time; second, through 1≦t≦T is used to constrain the number of surrounding moving objects and the number of interaction relationships used for collision risk prediction at each timestamp, where is a hyperparameter.

[0142] b) Supernet Construction: Using a supernet, we transform the parameter search in the localization space and parameterization space into a neural architecture search problem. Specifically, we represent the choice of operation as a probability distribution:

[0143] Where x is the input, is the output, |A| represents the number of operations, β i Represents the mapping function F corresponding to the i-th operation i Here, the choice of operation represents the application of attention function or the selection of motion objects and interaction relationships.

[0144] By adopting a supernet, we jointly optimize the mixing weight β and all parameters in the mapping function in a differentiable way:

[0145] Among them, η w and η β Represent the learning rates of structure weights and model weights respectively, and Represent the loss functions of the training set and the validation set respectively, w represents the structural weight, and β represents the model weight.

[0146] c) Multi-stage supernet training: To stabilize the training of the supernet, the training process is divided into three stages: motion object parameterization, interaction relationship parameterization, and attention localization space search. Each stage focuses on a different parameterization space.

[0147] As shown in Figure 3, before actual collision risk prediction is performed, the model must be trained and validated, with model parameters continuously adjusted and optimized to achieve optimal prediction performance. The model's prediction performance is then evaluated and analyzed using test data. Finally, the prediction model is applied to actual traffic scenarios, and the collision risk prediction results are visualized.

[0148] Furthermore, the model training and validation process is as follows:

[0149] (1) Data input: The feature vectors of the vehicle data and surrounding moving object data are extracted from the divided training set as the input of the model.

[0150] (2) Model training: A high-order memory-guided temporal random walk algorithm and a hierarchical attention graph embedding neural network learn the importance of different moving objects and their interactions. The latter predicts the collision risk distribution at the current timestamp based on historical data.

[0151] (3) Model evaluation: Calculate the difference between the predicted collision risk and the actual collision risk.

[0152] (4) Parameter optimization: Based on the model evaluation results, the optimizer is used to optimize and update the model parameters to reduce the prediction error.

[0153] (5) Iteration: Repeat the above steps (2) to (4), use different training sample groups to iteratively train the model, and update the model parameters at the end of each iteration.

[0154] (6) Model validation: During the training process, the validation set is used periodically to evaluate the model performance, monitor the generalization ability and prediction accuracy of the model, and adjust the model hyperparameters based on the validation results to obtain the best model performance.

[0155] Figures 7(a), (b), (c), and (d) illustrate the collision risk prediction results for autonomous emergency rescue vehicles described in this application. It should be noted that this diagram serves only as an example interface for predicting collision risk for autonomous emergency rescue vehicles, contains only essential functionality, and may be improved upon based on specific needs in the future. It is not a constraint on this application.

[0156] Referring to Figure 7(a), the interface consists of five parts. Frame 1 displays the collision risk prediction interface. Other functional interfaces, such as driving model switching and emergency rescue information, are not described here and may be further improved in the future. Frame 2 displays the vehicle's current operating status, including starting, driving, braking, and stopping. Frame 3 displays the current remaining battery, signal strength, and time. Frame 4 displays the four display interface buttons of the collision risk prediction system: "Real-time Road Scene," "Vehicle Operation Data," "Collision Risk Prediction," and "Safe Driving Recommendations." Specific display information can be viewed by selecting different function buttons. Frame 5 displays the specific information corresponding to the function button selected in Frame 4.

[0157] In detail, the "Real-time Road Scene" interface (refer to Figure 7(a)) can fully display the road environment in which the emergency rescue vehicle is located at the current moment and the specific positions of the surrounding moving objects, so that the driver can perceive and understand the surrounding environmental information in a specific and comprehensive manner; the "Vehicle Operation Data" interface (refer to Figure 7(b)) displays the vehicle's current motion status and driving route. This information can guide the driver's behavioral operations and is crucial for collision risk prediction; the "Collision Risk Prediction" interface (refer to Figure 7(c)) displays the surrounding high-collision risk objects and potential collision risk objects; the "Safe Driving Recommendations" interface (refer to Figure 7(d)) ​​provides the driver with a safe driving plan by analyzing the predicted collision risk and reminds the driver in the form of voice broadcast.

[0158] The graph embedding used in the collision risk prediction system for emergency rescue autonomous driving vehicles based on HM-TRW and HAGENN structural search in this application is an algorithm for learning complex relationships between nodes. It encodes all nodes in the graph and maps them into equal-dimensional vectors that can be directly used by machine learning algorithms to achieve efficient and accurate predictions. In order to retain more effective information between nodes, this application further expands the graph embedding algorithm into a hierarchical attention graph embedding neural network algorithm, which improves the ability to capture structural heterogeneity and dynamics. In the collision risk prediction of emergency rescue vehicles, the hierarchical attention graph embedding neural network can predict the potential conflict relationship between emergency rescue vehicles and surrounding moving objects by learning the complex relationship between them, and then predict the collision risk. Therefore, this application is expected to provide emergency rescue vehicles with accurate and efficient collision risk prediction services, on this basis, it can further guide the driver's decision-making and help autonomous driving vehicles plan safer and more efficient driving paths, thereby better serving emergency rescue work.

[0159] The collision risk prediction method for emergency rescue autonomous driving vehicles based on HM-TRW and HAGENN structural search described in this application, on the basis of perceiving the surrounding environment of autonomous driving, combines the vehicle's own operation data to abstract the complex interaction relationship between the emergency rescue vehicle and the surrounding moving objects into a dynamic heterogeneous graph; then, the random walk algorithm and the hierarchical attention graph embedding algorithm are used to model the complex interaction relationship between them, and the importance of the surrounding moving objects and the various interaction relationships between them and the emergency rescue vehicle is learned; then, the vehicle's own data and the surrounding moving object data in the test set are input into the trained model to calculate the collision risk at the future timestamp; finally, according to actual needs, the predicted collision risk is visualized to assist the driver in taking risk avoidance operations safely and efficiently. This application can provide more efficient and accurate collision risk prediction for emergency rescue autonomous driving vehicles, ensure the efficiency and safety of emergency rescue, and help build a safe traffic environment.

[0160] The above embodiments are preferred implementations of the present application, but the present application is not limited to the described implementations. Any obvious modifications and substitutions that can be made by those skilled in the art to the present application are within the scope of protection of the present application.

Claims

1. A collision risk prediction method for autonomous emergency rescue vehicles based on HM-TRW and HAGENN structure search, characterized by: Dataset construction: In real-vehicle experiments, data acquisition equipment is used to collect vehicle data and data on surrounding moving objects, which are then pre-processed and stored in a database. Collision risk prediction model construction: A dynamic heterogeneous graph is constructed using collected vehicle data and data on surrounding moving objects. A high-order memory-guided temporal random walk algorithm learns the heterogeneous characteristics and dynamic change patterns of the dynamic heterogeneous graph. The resulting node representations of each surrounding moving object are then fed into a hierarchical attention graph embedding neural network to predict collision risk. Model training: Input the training set and validation set in the database into the collision risk prediction model respectively to optimize the model parameters; Model testing: Use the test set in the database to test the trained collision risk prediction model, and evaluate and analyze the model prediction performance based on the test results; Collision risk prediction: Real-time data collection of autonomous emergency rescue vehicles and surrounding moving objects is input into the tested collision risk prediction model to predict collision risks. The dynamic heterogeneous graph is constructed as follows: the emergency rescue vehicle and its surrounding moving objects are represented as nodes, and the various interactive relationships between them are represented as edges. The nodes and edges change dynamically over time to form a dynamic heterogeneous graph, which is expressed as follows: G t =(v t ,e t ,u t ) Among them, v t is a node set of node type h, e t is the set of edges of edge type r, Represents the feature set of all moving objects, h∈H, r∈R, H and R are the node type set and edge type set respectively, represents a set of real numbers, N is the number of nodes, and D is the feature dimension; The learning of the heterogeneous characteristics of the dynamic heterogeneous graph includes: (1) Transfer vector Set the initial high-order memory queue to be empty, and the type transfer vector accesses each type of surrounding motion objects with equal probability. When accessing the surrounding motion object v j When its type is transferred to vector Update by pressing: Where, φ(v j ) represents the surrounding moving object v j The type of Q represents a first-in-first-out queue, and Norm() represents the return norm of the vector; (2) Type conversion according to The probability distribution of determines the type of the next surrounding moving object to be visited, and a search mechanism with a search factor α∈[0,1] is used to solve the type trap problem, as follows: Where, Indicates the surrounding motion object initially accessed by the type transfer vector, Pr(h n+1 ) indicates that the type of the next surrounding motion object to be accessed is h n+1 probability; (3) High-level memory recording After visiting the surrounding moving objects v j After that, the transfer vector Store it in the first-in-first-out queue Q' for the next step: Among them, Put is a queue operator, which means that when the first-in-first-out queue is full, the first transfer vector is popped out and placed at the end of the queue; The dynamic change rules of the learning dynamic heterogeneous graph are specifically as follows: For emergency rescue vehicles i , use h n+1 Indicates the type of surrounding moving objects, so: in, Indicates emergency rescue vehicle v i The set of surrounding moving objects at the future timestamp, t′ represents the timestamp of the previous random walk, e t is the edge set of edge type; Take an exponentially decaying distribution from the set Select the next motion object to visit: Among them, the timestamp t of the k-th random walk in the future k ∈t, t represents the timestamp of random walk; Pr(v n+1 ) indicates that the next moving object to be accessed is v n+1 The probability of v k represents the surrounding moving objects visited by the k-th random walk; discount rate δ∈[0,1]; The node representation of each surrounding moving object is a fusion of the output of the high-order memory-guided temporal random walk algorithm and the original features of each surrounding moving object: in, They are the surrounding moving objects v j The transfer vector, original features and one-hot vector of is the latent embedding of all moving objects, represents a set of real numbers, N is the number of nodes, D is the feature dimension, W f and W represents the surrounding moving objects v j Learnable parameters that are not shared with other moving objects, and They represent the original features of surrounding moving objects, the identified embedded features and the final node representation respectively.

2. The method for predicting collision risk of an emergency rescue autonomous driving vehicle according to claim 1, characterized in that: The hierarchical attention graph embedded in the neural network predicts collision risk by aggregating different information through node-level attention, edge-level attention and time-level attention; For node-level attention, when the timestamp is t, for the interaction relationship type r, the emergency rescue vehicle v i and its surrounding moving objects v j The importance of Calculated by the following formula: Among them, σ is the activation function, x i 、x j They are emergency rescue vehicles v i and surrounding moving objects v j The input representation is, Is a linear transformation matrix, || represents connection, Represents the emergency rescue vehicle v of interaction relationship type r at timestamp t i All surrounding moving objects, a r is the weight vector, is a r The transpose of x k represents the kth moving object around the vehicle; Emergency rescue vehicle v with interaction relationship type r at timestamp t i Node embedding Expressed as: For edge-level attention, an attention mechanism is used to learn the importance of different types of interaction relationships. And computed by a multilayer perceptron: Among them, w T is the edge-level attention vector, U el and b el is the single layer parameter of the multilayer perceptron, R is the set of edge types; Emergency rescue vehicles v considering the importance of different interaction relationship types i Fusion embedding Expressed as: For time-level attention, the fused embeddings of emergency rescue vehicles at all timestamps are aggregated and packaged as T represents the number of historical timestamps used to predict collision risk; the query-key-value vector of the fused embedding is calculated: P=G i ·U P K=G i ·U K V=G i ·U V Among them, P, K, V represent query, key and value vectors respectively, and U P 、U K 、U V Indicates that G i Converted into corresponding matrices of query, key, and value vectors, D is the feature dimension, represents the set of real numbers; Use the softmax function to calculate the time-level attention: Among them, Z i represents temporal attention, is a mask matrix, D′ is the dimension of the query-key-value vector; Z i T As the final fused embedding, calculate the collision risk: <h2 style=";text-align:left;direction:ltr">Y=softmax(W2·ReLU(W1·Z<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> T <h2 style=";text-align:left;direction:ltr"> +b1)+b2) Where softmax(·) is the output activation function, W1 and W2 are the weight matrices of the hierarchical attention graph embedding neural network, ReLU(·) is the activation function, and b1 and b2 represent bias terms.

3. The method for predicting collision risk of an emergency rescue autonomous driving vehicle according to claim 2, characterized in that: The attention localization space is used to sparsely process the attention calculated by the hierarchical attention map embedding neural network when predicting collision risk: Through the matrix Select the type of surrounding motion objects, interaction relationship type, and number of timestamps you are interested in. The specific calculation is as follows: When the timestamp is t, through the matrix Determine whether to focus on surrounding moving objects with interaction type r Represents the emergency rescue vehicle v of interaction relationship type r at timestamp t′ i All surrounding moving objects; By using attention to locate the space, the time complexity is: in, express The number of non-zero values ​​in , T represents the number of historical timestamps used to predict collision risk, and denote the number of interaction relationships of type r at timestamps t' and t, respectively. O(·) denotes the time complexity, and |R| denotes the number of edge types.

4. The method for predicting collision risk of an emergency rescue autonomous driving vehicle according to claim 3, characterized in that: The attention parameterization space is used to sparsely process the attention calculated by the hierarchical attention map embedding neural network when predicting collision risk: Parameterize the space A using attention Pa Search attention function, the expression is as follows: Among them, A N ={1,…,K N } T×|H| is the node mapping function F N The parameterized matrix of (·), A R ={1,…,K R } 2T×|R| is the edge mapping matrix F R The parameterization matrix of (·), K N and K R are two hyperparameters, |H| represents the number of node types.

5. The method for predicting collision risk of an emergency rescue autonomous driving vehicle according to claim 4, characterized in that: Use multi-stage differential search to reduce the parameter search complexity in positioning space and parameter space: a) Space constraints The following two constraints are introduced to reduce the search range and limit the complexity: first, emergency rescue vehicles can only receive information from surrounding moving objects in historical time, and second, through To constrain the number of surrounding moving objects and the number of interaction relationships used for collision risk prediction at each timestamp, is a hyperparameter, 1≦t≦T; b) Supernet Construction Using a hypernet, we transform the parameter search in the localization space and parameterization space into a neural architecture search problem, specifically by expressing the choice of operation as a probability distribution: Where x is the input, is the output, |A| represents the number of operations, β i Represents the mapping function F corresponding to the i-th operation i (·) mixed weight; By adopting a supernet, we jointly optimize the mixing weight β and all parameters in the mapping function in a differentiable way: Among them, η w and η β Represent the learning rates of structure weights and model weights respectively, and Represent the loss functions of the training set and the validation set respectively, w represents the structural weight, and β represents the model weight; c) Multi-stage supernet training To stabilize the training of the supernet, the training process is divided into three stages: motion object parameterization, interaction relationship parameterization, and attention localization space search, each stage focusing on a different parameterization space.

6. A system for implementing the method for predicting collision risk of an emergency rescue autonomous driving vehicle according to any one of claims 1 to 5, characterized in that: include: Data collection equipment, including on-board sensors, roadside equipment, and communication technology, used to collect data from the vehicle and surrounding moving objects; Data preprocessing module, used to clean, normalize, extract features, reduce data dimension and divide data sets into collected data; A prediction model comprising a high-order memory-guided temporal random walk algorithm, a hierarchical attention graph embedding neural network, and an optimal parameter search module comprising an attention localization space, an attention parameterization space, and a multi-stage differential search module; A visualization module for displaying the predicted collision risk.

Images