A credible pedestrian intention prediction method based on multi-attention and evidence fusion

Abstract

This invention belongs to the technical field of intention prediction, and specifically relates to a credible pedestrian intention prediction method based on multi-attention and evidence fusion. The method comprises the following steps: obtaining a pedestrian anchor frame and ego-vehicle speed from the perspective of the ego-vehicle in front of the vehicle; constructing an embedding module to obtain anchor frame features and speed features; constructing a feature enhancement module based on a multi-head self-attention mechanism to obtain enhanced anchor frame features and enhanced speed features; constructing a feature fusion module based on an adaptive and learnable multi-head cross-attention module and a hybrid channel attention module to obtain fused features; performing temporal correlation extraction; performing decision-level fusion based on the D-S evidence theory, using Dirichlet distribution to form evidence and use it as the basis for judging pedestrian crossing intention; training the entire network; and using the trained entire network to obtain pedestrian crossing intention from the perspective of the ego-vehicle. This invention improves the accuracy and credibility of pedestrian crossing prediction.

Description

Technical Field

[0001] The present invention belongs to the technical field of intention prediction, and specifically relates to a credible pedestrian intention prediction method based on multi-attention and evidence fusion. Background Art

[0002] In recent years, autonomous driving systems have made unprecedented progress, fundamentally reshaping the future of transportation. For autonomous vehicles in urban environments, understanding pedestrian behavior and predicting their underlying intentions are crucial for human-vehicle interaction. Pedestrian intention prediction aims to predict whether a target pedestrian will cross the road, giving autonomous vehicles sufficient reaction time to make the correct decision, thereby reducing collisions and improving traffic safety.

[0003] However, in urban traffic scenarios, pedestrian behavior is quite irregular and can be easily influenced by multiple factors such as destination, social interaction, and traffic regulations. For example, pedestrians are more likely to stop crossing the road when the vehicle ahead is traveling too fast or too close to them. To this end, obtaining sufficient input data and capturing the potential interactions between pedestrians and vehicles plays an indispensable role in predicting pedestrian intentions. The data modalities commonly used for intention prediction can be divided into two types: visual data (e.g., local and global images) and non-visual data (e.g., anchor boxes, vehicle speed). Current methods mainly apply a multi-branch network structure, where each individual branch is responsible for extracting features from each data modality.

[0004] Although several methods have been proposed for predicting pedestrian crossing intentions, these methods suffer from several drawbacks. First, as the number of input modalities increases, the prediction model becomes more susceptible to data quality issues; missing or degraded modalities hinder the effectiveness of the prediction model. Furthermore, excessive input modalities can lead to significant information redundancy. In addition to predictive accuracy, quantifying the uncertainty of deep learning models is crucial for safety-sensitive autonomous driving applications. This reflects the model's confidence in its predictions. Existing technologies struggle to achieve effective feature fusion when the number of input modalities is small and do not adequately consider evidence fusion to enhance prediction confidence. Summary of the Invention

[0005] In view of the above deficiencies in the prior art, the purpose of the present invention is to provide a credible prediction method for pedestrian intention based on multi-attention and evidence fusion, which solves the problem of excessive complexity and unreliable prediction caused by too many input modalities in the existing algorithm, and improves the accuracy and credibility of pedestrian crossing prediction.

[0006] To achieve the above objectives, the present invention provides a method for credible prediction of pedestrian intention based on multi-attention and evidence fusion, comprising the following steps:

[0007] S1. Obtain the pedestrian anchor frame B and the vehicle speed V from the perspective of the vehicle in front of the vehicle;

[0008] S2, build an embedding module, input B and V into the embedding module to obtain the anchor frame feature P1 and speed feature P2;

[0009] S3. Construct a feature enhancement module based on the multi-head self-attention mechanism, input P1 and P2 into the feature enhancement module, capture the temporal correlation between different historical frames, and obtain the enhanced anchor frame feature S1 and the enhanced speed feature S2;

[0010] S4, build a feature fusion module based on the adaptive learnable multi-head cross attention module and the hybrid channel attention module, input S1 and S2 into the feature fusion module to obtain the fused feature S3;

[0011] S5, extract the temporal correlation of S1, S2 and S3 to obtain evidence of anchor frame branch, speed branch and feature fusion branch;

[0012] S6. Based on DS evidence theory, the Dirichlet distribution is used to model the category classification and form subjective opinions for each branch. The subjective opinions of the anchor frame branch, speed branch, and feature fusion branch are then fused at the decision level. The subjective opinions obtained from the decision-level fusion are then used to form evidence using the Dirichlet distribution and used as the basis for judging pedestrian crossing intentions.

[0013] S7. Use a multi-task variational loss function to train the entire network built in S2-S6, so that the predictions of the anchor branch, velocity branch, and feature fusion branch together form reasonable evidence;

[0014] S8. Use the trained network and the actual B and V as input to obtain the pedestrian crossing intention from the perspective of the vehicle.

[0015] As a preferred solution of the present invention, in S1, the pedestrian anchor frame B is a coordinate sequence of the upper left corner and lower right corner of the two-dimensional rectangular frame surrounding the pedestrian, and the vehicle speed V is the continuous vehicle speed data obtained by the vehicle-mounted sensor.

[0016] As a preferred solution of the present invention, in the S2, the embedding module includes a one-dimensional convolution with a kernel size of 1 and position coding. First, B and V are projected into a high-dimensional feature space through one-dimensional convolution, and then position coding is added to reflect the temporal relationship of the sequence to obtain P1 and P2.

[0017] As a preferred embodiment of the present invention, in the aforementioned S3, the feature enhancement module includes a multi-head self-attention mechanism, a residual network, a normalization and a feedforward network. The process of obtaining S1 and S2 using the feature enhancement module is as follows:

[0018] S3.1. Apply the multi-head self-attention mechanism to P1, set the key, value, and query of the self-attention to P1, calculate the attention weight matrix, capture the correlation between different frames, and achieve feature enhancement. After completing the self-attention operation, obtain R1, which represents the features processed by the multi-head self-attention mechanism.

[0019] S3.2, use the residual network and normalization operation on R1 to obtain , Represents the features after residual network and normalization operation;

[0020] S3.3, Input the feedforward network and compare its output with The fusion is performed through the residual network, and the normalization operation is performed again after fusion to obtain the enhanced anchor frame feature S1;

[0021] S3.4. For P2, perform the same processing as S3.1-S3.3 to obtain the enhanced speed feature S2.

[0022] As a preferred embodiment of the present invention, in the S4, the adaptive learnable multi-head cross attention module includes a multi-head cross attention mechanism, residual connection and normalization, a multi-layer perceptron, and a final residual connection and normalization arranged in sequence; first, the interaction between the modalities is enhanced by the multi-head cross attention mechanism, and its output is processed by the multi-layer perceptron after residual connection and normalization, and finally, after the final residual connection and normalization, refined modal features are obtained;

[0023] The hybrid channel attention module includes a maximum pooling layer, an average pooling layer, a splicing layer, and a multi-layer perceptron. First, the input passes through the maximum pooling layer and the average pooling layer in parallel. Then, the output of the maximum pooling layer is spliced ​​through the splicing layer, and the output of the average pooling layer is spliced ​​through the splicing layer. The output of the splicing layer is then input into the multi-layer perceptron to reduce redundant information and form an excitation signal. Finally, the excitation signal is used to calibrate the input features.

[0024] The multi-head cross attention module and the mixed channel attention module are connected in sequence to obtain the feature fusion module.

[0025] As a preferred solution of the present invention, in S4, the process of obtaining the fusion feature S3 based on the feature fusion module is as follows:

[0026] S4.1. Input S1 and S2 into the adaptive learnable multi-head cross attention module. For S1, in the multi-head cross attention mechanism, S1 is used as the query and S2 as the key and value. By calculating the attention weight matrix, the interaction of different modalities is enhanced. After completing the cross attention operation, we get , Represents the features processed by the multi-head cross attention mechanism;

[0027] S4.2, use residual connection to connect S1 and Connect and normalize to get , Represents the features after residual connection and normalization operation;

[0028] S4.3. Input the multilayer perceptron and compare its output with Connected by residual connection, and then normalized again to obtain , Represents the features after multi-layer perceptron, residual connection and normalization operations;

[0029] S4.4, for S2, in the multi-head cross attention mechanism, take S2 ​​as the query and S1 as the key and value, and process it in the same way as S4.1-S4.4 to get ;

[0030] S4.5. and Input mixed channel attention module, compressed in parallel through average pooling layer and maximum pooling layer and , eliminate redundant information in the time dimension;

[0031] S4.6. Connect the data stream output by the average pooling layer through the splicing layer to obtain the connected feature A corresponding to the average pooling layer avg , connect the data stream output by the maximum pooling layer through the splicing layer to obtain the connected feature A corresponding to the maximum pooling layer max ;

[0032] S4.7, A avg and A max Input the multi-layer perceptron based on the encoder-decoder architecture to learn the complex relationship between different features and obtain the corresponding learned features. and ;

[0033] S4.8, Split into two eigenvectors of equal length, the first half is recorded as The second half is recorded as ; Similarly, Split into two eigenvectors of equal length, the first half is recorded as The second half is recorded as ;

[0034] S4.9, 、 With learnable parameters The weights are summed and the excitation signal C1 is formed through the sigmoid function. 、 With learnable parameters Sum the weights and pass them through the sigmoid function to form the excitation signal C2;

[0035] S4.10, C1 and Multiply C2 by element Multiply element by element to complete the channel feature recalibration and obtain the corresponding calibrated feature vectors M1 and M2;

[0036] S4.11. Connect M1 and M2 along the channel dimension, and after dimensionality reduction through a multi-layer perceptron, obtain the fusion feature S3.

[0037] As a preferred embodiment of the present invention, in S5, the method for obtaining evidence of the anchor frame branch, the velocity branch, and the feature fusion branch is:

[0038] S5.1. Process S1, S2, and S3 through two fully connected layers, and set a tanh activation function between the two fully connected layers;

[0039] S5.2. For the corresponding outputs after passing through the two fully connected layers, the corresponding weight vectors B1, B2, and B3 are obtained through the Softmax activation function. B1, B2, and B3 are multiplied element-wise with S1, S2, and S3 respectively, and the scale of each time dimension is adjusted. The multiplication results are then compressed in the time dimension through adaptive average pooling to obtain the corresponding features T1, T2, and T3 after the compression of the time dimension.

[0040] S5.3. Pass T1, T2, and T3 through the fully connected layer with the activation function SoftPlus, and obtain the evidence e1, e2, and e3 corresponding to the anchor frame branch, speed branch, and feature fusion branch, respectively, expressed as 、 、 ,in 、 Evidence of pedestrians not crossing and crossing given by the anchor frame branch respectively; 、 Evidence for pedestrians not crossing and crossing given for the speed branch respectively; 、 The evidence of pedestrians not crossing and crossing given by the feature fusion branch respectively.

[0041] As a preferred solution of the present invention, in S6, the process of performing decision-level fusion based on DS evidence theory is as follows:

[0042] S6.1. Calculate en The corresponding Dirichlet distribution parameter , confidence quality , and the corresponding uncertainty u n , forming the subjective opinions of each branch ,in K n The corresponding distribution parameters of the positive and negative classes in the Dirichlet distribution, K n The confidence quality of the positive and negative classes in the corresponding Dirichlet distribution, n = 1, 2, 3, represents the anchor frame branch, speed branch and feature fusion branch respectively, and the calculation formula is:

[0043] ;

[0044] ;

[0045] ;

[0046] In the formula, k = 1 or 2, indicating the category;

[0047] S6.2. K1, K2, and K3 are integrated to form the overall subjective opinion K4, which is expressed as:

[0048] ;

[0049] Where, Represents the fusion symbol; the specific fusion process is:

[0050] S6.2.1. and , the subjective opinion K after the fusion of the two 12 Expressed as:

[0051] ;

[0052] Where, It's K 12 The confidence quality of the positive and negative classes in the corresponding Dirichlet distribution; It's K 12 The corresponding uncertainties are expressed as:

[0053] ;

[0054] ;

[0055] Where, ;

[0056] S6.2.2, based on S6.2.1, similarly, K 12 Combined with K3, we get ;in They represent the confidence quality of the positive and negative classes in the Dirichlet distribution corresponding to K4; u4 represents the uncertainty corresponding to K4;

[0057] S6.3. Calculate the corresponding evidence based on K4 , It represents the evidence of pedestrians not crossing and crossing given by the decision-level fusion branch, and then calculates the Dirichlet distribution parameter corresponding to e4 , are the distribution parameters of the positive and negative classes in the Dirichlet distribution corresponding to K4, expressed as:

[0058] ;

[0059] ;

[0060] S6.4. Utilization To judge the pedestrian's intention to cross the street, argmax represents the parameter value of the maximum point of a function.

[0061] As a preferred solution of the present invention, in S7, the multi-task variational loss function It is the sum of the loss function of each branch and the loss function of the decision-level fusion branch, expressed as:

[0062] ;

[0063] Where n=1, 2, 3, and 4 represent the anchor frame branch, velocity branch, feature fusion branch, and decision-level fusion branch, respectively; is the loss function of branch n; where:

[0064] ;

[0065] Where, is a hyperparameter, represents the random variable corresponding to branch n, which obeys the parameter Dirichlet distribution , , and express The expectation and variance of represents element-wise multiplication; represents the sample label, represents Dirichlet distribution and The KL divergence between Represents a vector consisting of all 1s.

[0066] As a preferred embodiment of the present invention, using Sure , where epochs is the number of training rounds and annealing_step is initialized to 10 as the annealing coefficient.

[0067] The algorithm involved in the present invention can be executed by an electronic device, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The above-mentioned algorithm calculation is realized by executing the software through the processor.

[0068] The beneficial effects of the present invention are:

[0069] This method uses a self-attention mechanism to extract correlations between pedestrians' historical time frames. The extracted features are then fed into a feature fusion module. Using cross-attention and channel-attention mechanisms, the pedestrian anchor frame branch and the velocity branch are fused to enhance complementarity between the different modalities, resulting in a fused branch. These three branches are then fed simultaneously into a prediction subnetwork to further extract temporal correlations, addressing the drawbacks of inadequate and incomplete feature extraction and utilization.

[0070] This invention dynamically evaluates the quality of different predictions and makes trustworthy decisions based on the associated uncertainties. Compared to other models, the proposed model achieves higher accuracy and performance. By leveraging a multi-task variational loss function, it ensures fair participation of anchor boxes, velocity, and fusion branches in the opinion formation phase, thereby reducing the uncertainty of pedestrian intention prediction and helping to mitigate potential driving risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 is a flow chart of the method of the present invention;

[0072] Figure 2 It is a schematic diagram of the process of obtaining prediction results of the present invention;

[0073] Figure 3 Schematic diagram of the architecture of the hybrid channel attention module in an embodiment of the present invention;

[0074] Figure 4 is a schematic diagram of the evidence fusion process in an embodiment of the present invention;

[0075] Figure 5 : is a comparison chart of experimental results of various branch predictions in the embodiment of the present invention, Figure 5 (a) is a comparison chart of the first experimental results; Figure 5 (b) is a comparison chart of the second experimental results;

[0076] Figure 6 is an uncertainty frequency distribution diagram in an embodiment of the present invention. DETAILED DESCRIPTION

[0077] The embodiments of the present invention are further described below with reference to the accompanying drawings:

[0078] like Figure 1 As shown in Figure 2, a method for credible pedestrian intention prediction based on multi-attention and evidence fusion includes the following steps:

[0079] S1. Obtain the pedestrian anchor frame B and the vehicle speed V from the perspective of the vehicle in front of the vehicle;

[0080] S2, build an embedding module, input B and V into the embedding module to obtain the anchor frame feature P1 and speed feature P2;

[0081] S3. Construct a feature enhancement module based on the multi-head self-attention mechanism, input P1 and P2 into the feature enhancement module, capture the temporal correlation between different historical frames, and obtain the enhanced anchor frame feature S1 and the enhanced speed feature S2;

[0082] S4, build a feature fusion module based on the adaptive learnable multi-head cross attention module and the hybrid channel attention module, input S1 and S2 into the feature fusion module to obtain the fused feature S3;

[0083] S5, extract the temporal correlation of S1, S2 and S3 to obtain evidence of anchor frame branch, speed branch and feature fusion branch;

[0084] S6. Based on DS evidence theory, the Dirichlet distribution is used to model the category classification and form subjective opinions for each branch. The subjective opinions of the anchor frame branch, speed branch, and feature fusion branch are then fused at the decision level. The subjective opinions obtained from the decision-level fusion are then used to form evidence using the Dirichlet distribution and used as the basis for judging pedestrian crossing intentions.

[0085] S7. Use a multi-task variational loss function to train the entire network built in S2-S6, so that the predictions of the anchor branch, velocity branch, and feature fusion branch together form reasonable evidence;

[0086] S8. Use the trained network and the actual B and V as input to obtain the pedestrian crossing intention from the perspective of the vehicle.

[0087] Dempster-Shafer evidence theory (DS evidence theory) is a mathematical framework for dealing with uncertainty and incomplete information. In the residual network and residual connection involved in this embodiment, an adaptive learnable parameter can be assigned to each end of the connection to make the model fit better. Based on the above steps, the process principle diagram for obtaining the prediction result is as follows Figure 2 shown.

[0088] In S1, the pedestrian anchor box B is the coordinate sequence of the upper left corner and lower right corner of the two-dimensional rectangular box surrounding the pedestrian, and the vehicle speed V is the continuous vehicle speed data obtained by the on-board sensor.

[0089] In S2, the embedding module contains a one-dimensional convolution with a kernel size of 1 and position encoding. First, B and V are projected into a high-dimensional feature space through one-dimensional convolution to obtain richer feature information. Then, position encoding is added to reflect the temporal relationship of the sequence to obtain P1 and P2.

[0090] In S3, the feature enhancement module includes a multi-head self-attention mechanism, a residual network, normalization, and a feedforward network. The process of obtaining S1 and S2 using the feature enhancement module is as follows:

[0091] S3.1. Apply the multi-head self-attention mechanism to P1, set the key, value, and query of the self-attention to P1, calculate the attention weight matrix, capture the correlation between different frames, and achieve feature enhancement. After completing the self-attention operation, obtain R1, which represents the features processed by the multi-head self-attention mechanism.

[0092] S3.2, use the residual network and normalization operation on R1 to obtain , Represents the features after residual network and normalization operation;

[0093] S3.3, Input the feedforward network and compare its output with The fusion is performed through the residual network, and the normalization operation is performed again after fusion to obtain the enhanced anchor frame feature S1;

[0094] S3.4. For P2, perform the same processing as S3.1-S3.3 to obtain the enhanced speed feature S2.

[0095] In S4, the adaptive and learnable multi-head cross-attention module includes a multi-head cross-attention mechanism, residual connection and normalization, a multi-layer perceptron, and a final residual connection and normalization. First, the multi-head cross-attention mechanism is used to enhance the interaction between the modalities. Its output is processed by the multi-layer perceptron after residual connection and normalization, and finally, after the final residual connection and normalization, refined modal features are obtained.

[0096] The hybrid channel attention module includes a maximum pooling layer, an average pooling layer, a splicing layer, and a multi-layer perceptron. First, the input passes through the maximum pooling layer and the average pooling layer in parallel. Then, the output of the maximum pooling layer is spliced ​​through the splicing layer, and the output of the average pooling layer is spliced ​​through the splicing layer. The output of the splicing layer is then input into the multi-layer perceptron to reduce redundant information and form an excitation signal. Finally, the excitation signal is used to calibrate the input features.

[0097] The multi-head cross attention module and the mixed channel attention module are connected in sequence to obtain the feature fusion module.

[0098] In S4, the process of obtaining the fusion feature S3 based on the feature fusion module is as follows:

[0099] S4.1. Input S1 and S2 into the adaptive learnable multi-head cross attention module. For S1, in the multi-head cross attention mechanism, S1 is used as the query and S2 as the key and value. By calculating the attention weight matrix, the interaction of different modalities is enhanced. After completing the cross attention operation, we get , Represents the features processed by the multi-head cross attention mechanism;

[0100] S4.2, use residual connection to connect S1 and Connect and normalize to get , Represents the features after residual connection and normalization operation;

[0101] S4.3. Input the multilayer perceptron and compare its output with Connected by residual connection, and then normalized again to obtain , Represents the features after multi-layer perceptron, residual connection and normalization operations;

[0102] S4.4, for S2, in the multi-head cross attention mechanism, take S2 ​​as the query and S1 as the key and value, and process it in the same way as S4.1-S4.4 to get ;

[0103] S4.5, such as Figure 3 As shown, and Input mixed channel attention module, taking into account the potential correlation of channel dimension, through average pooling layer and maximum pooling layer parallel compression and , eliminate redundant information in the time dimension to better aggregate channel information;

[0104] S4.6. Connect the data stream output by the average pooling layer through the splicing layer to obtain the connected feature A corresponding to the average pooling layer avg , connect the data stream output by the maximum pooling layer through the splicing layer to obtain the connected feature A corresponding to the maximum pooling layer max ;

[0105] S4.7, A avg and A maxInput the multi-layer perceptron based on the encoder-decoder architecture to learn the complex relationship between different features and obtain the corresponding learned features. and ; In the encoder, the channel features are compressed at a certain compression ratio (Z avg and Z max ), and then restore its shape in the decoder;

[0106] S4.8, Split into two eigenvectors of equal length, the first half is recorded as The second half is recorded as ; Similarly, Split into two eigenvectors of equal length, the first half is recorded as The second half is recorded as ; Ensure that information from both average pooling and max pooling is utilized;

[0107] S4.9, 、 With learnable parameters The weights are summed and the excitation signal C1 is formed through the sigmoid function. 、 With learnable parameters Sum the weights and pass them through the sigmoid function to form the excitation signal C2;

[0108] S4.10, C1 and Multiply C2 by element Multiply element by element to complete the channel feature recalibration and obtain the corresponding calibrated feature vectors M1 and M2;

[0109] S4.11. Connect M1 and M2 along the channel dimension, and after dimensionality reduction through a multi-layer perceptron, obtain the fusion feature S3.

[0110] Because the features obtained after different pooling operations have different degrees of importance. In order to ensure the fairness of the incentive signal, adaptive learnable parameters are introduced in both branches. 、 , enabling the network to better identify which information to emphasize and which to suppress.

[0111] In S5, the method for obtaining evidence for the anchor frame branch, velocity branch, and feature fusion branch is as follows:

[0112] S5.1. Process S1, S2, and S3 through two fully connected layers, and set a tanh activation function between the two fully connected layers to ensure that its value is in the range of -1 to 1;

[0113] S5.2. For the corresponding outputs after passing through the two fully connected layers, the corresponding weight vectors B1, B2, and B3 are obtained through the Softmax activation function. B1, B2, and B3 are multiplied element-wise with S1, S2, and S3 respectively, and the scale of each time dimension is adjusted. The multiplication results are then compressed in the time dimension through adaptive average pooling to obtain the corresponding features T1, T2, and T3 after the compression of the time dimension.

[0114] S5.3. Pass T1, T2, and T3 through the fully connected layer with the activation function SoftPlus, and obtain the evidence e1, e2, and e3 corresponding to the anchor frame branch, speed branch, and feature fusion branch, respectively, expressed as 、 、 ,in 、 Evidence of pedestrians not crossing and crossing given by the anchor frame branch respectively; 、 Evidence for pedestrians not crossing and crossing given for the speed branch respectively; 、 The evidence of pedestrians not crossing and crossing given by the feature fusion branch respectively.

[0115] In S6, the process of decision-level fusion based on DS evidence theory is as follows:

[0116] S6.1. Calculate e n The corresponding Dirichlet distribution parameter , confidence quality , and the corresponding uncertainty u n , forming the subjective opinions of each branch ,in K n The corresponding distribution parameters of the positive and negative classes in the Dirichlet distribution, K n The confidence quality of the positive and negative classes in the corresponding Dirichlet distribution, n = 1, 2, 3, represents the anchor frame branch, speed branch and feature fusion branch respectively, and the calculation formula is:

[0117] ;

[0118] ;

[0119] ;

[0120] In the formula, k = 1 or 2, indicating the category;

[0121] S6.2. K1, K2, and K3 are integrated to form the overall subjective opinion K4, which is expressed as:

[0122] ;

[0123] Where, Represents the fusion symbol; the specific fusion process is:

[0124] S6.2.1. and , the subjective opinion K after the fusion of the two 12 Expressed as:

[0125] ;

[0126] Where, It's K 12 The confidence quality of the positive and negative classes in the corresponding Dirichlet distribution; It's K 12 The corresponding uncertainties are expressed as:

[0127] ;

[0128] ;

[0129] In the formula, the intermediate variable ;

[0130] S6.2.2, based on S6.2.1, similarly, K 12 Combined with K3, we get ;in They represent the confidence quality of the positive and negative classes in the Dirichlet distribution corresponding to K4; u4 represents the uncertainty corresponding to K4;

[0131] S6.3. Calculate the corresponding evidence based on K4 , It represents the evidence of pedestrians not crossing and crossing given by the decision-level fusion branch, and then calculates the Dirichlet distribution parameter corresponding to e4 , are the distribution parameters of the positive and negative classes in the Dirichlet distribution corresponding to K4, expressed as:

[0132] ;

[0133] ;

[0134] S6.4. Utilization To judge the pedestrian's intention to cross the street, argmax represents the parameter value of the maximum point of a function.

[0135] The evidence fusion process in the above process is as follows: Figure 4 shown.

[0136] In S7, multi-task variational loss function It is the sum of the loss function of each branch and the loss function of the decision-level fusion branch, expressed as:

[0137] ;

[0138] Where n=1, 2, 3, and 4 represent the anchor frame branch, velocity branch, feature fusion branch, and decision-level fusion branch, respectively; is the loss function of branch n; where:

[0139] ;

[0140] Where, is a hyperparameter, represents the random variable corresponding to branch n, which obeys the parameter Dirichlet distribution , , and express The expectation and variance of represents element-wise multiplication; represents the sample label, represents Dirichlet distribution and The KL divergence between Represents a vector consisting of all 1s. KL divergence, full name Kullback-Leibler divergence, is a method to measure the difference between two probability distributions.

[0141] use Sure , where epochs is the number of training rounds and annealing_step is initialized to 10 as the annealing coefficient.

[0142] The verification process is:

[0143] The pedestrian intention prediction model proposed in this example (i.e., the entire constructed network) was trained and tested on the PIE dataset. For each pedestrian sample, the input history length was set to 8 frames, and predictions were made on whether the pedestrian had the intention to cross the street. During network training, the batch size was set to 16. The number of attention heads was set to 8. The dropout setting for all modules was set to 0.1. The learning rate was 5e. -4 , trained for 400 epochs. The Adam optimizer was used, and EarlyStop was applied to prevent overfitting. Training was terminated if the validation loss stopped decreasing after 15 epochs. The feature dimension was set to 128, and the number of layers in both the feature enhancement and feature fusion modules was set to 3.

[0144] Table 1 Performance comparison between different pedestrian intention prediction models

[0145]

[0146] Table 1 shows a performance comparison between different pedestrian intention prediction models. Among the performance evaluation metrics, ACC reflects the percentage of correctly predicted crossing and negative samples, AUC reflects the area under the receiver operating characteristic (ROC) curve, precision (P) reflects the ratio of correctly classified crossing samples to the total number of samples classified as crossing, recall (R) reflects the ratio of actual crossing samples that were predicted as crossing, and F1 reflects the harmonic mean of precision (P) and recall (R). Compared with other models, the model proposed in this embodiment achieves improved performance across all metrics.

[0147] like Figure 5 As shown, for Figure 5 (a) and Figure 5 In (b), the feature fusion branch (Pred_f) makes an incorrect prediction with high uncertainty, i.e., it is relatively unreliable. However, the other two branches, the anchor box branch (Pred_b) and the velocity branch (Pred_v), make correct predictions. The final prediction is proven to be correct. Compare Figure 5 (a) and Figure 5 In (b), note that Figure 5 The uncertainty of the three branches in (b) is greater than Figure 5 The uncertainty in (a) is reduced. Nevertheless, the model is still correct and reduces the uncertainty. Figure 5 In (a), the model tends to believe the velocity branch (Pred_v) with lower uncertainty, and the DS evidence theory further reduces the uncertainty.

[0148] like Figure 6 As shown, in this embodiment, the uncertainty of predicting whether to cross the road in most pedestrian movement scenarios is reduced to below 0.2. When the autonomous vehicle faces pedestrians with high dynamics and less constraints from maps and traffic rules, the planning and control module enables it to make corresponding decisions more firmly, thereby ensuring the traffic safety of pedestrians, reducing the number of traffic accidents, and minimizing casualties.

Claims

1. A credible pedestrian intention prediction method based on multi-attention and evidence fusion, characterized by The following steps are involved: S1. Obtain the pedestrian anchor frame B and the vehicle speed V from the perspective of the vehicle in front of the vehicle; S2, build an embedding module, input B and V into the embedding module to obtain the anchor frame feature P1 and speed feature P2; S3. Construct a feature enhancement module based on the multi-head self-attention mechanism, input P1 and P2 into the feature enhancement module, capture the temporal correlation between different historical frames, and obtain the enhanced anchor frame feature S1 and the enhanced speed feature S2; S4, build a feature fusion module based on the adaptive learnable multi-head cross attention module and the hybrid channel attention module, input S1 and S2 into the feature fusion module to obtain the fused feature S3; S5, extract the temporal correlation of S1, S2 and S3 to obtain evidence of anchor frame branch, speed branch and feature fusion branch; S6. Based on DS evidence theory, the Dirichlet distribution is used to model the category classification and form subjective opinions for each branch. The subjective opinions of the anchor frame branch, speed branch, and feature fusion branch are then fused at the decision level. The subjective opinions obtained from the decision-level fusion are then used to form evidence using the Dirichlet distribution and used as the basis for judging pedestrian crossing intentions. S7. Use a multi-task variational loss function to train the entire network built in S2-S6, so that the predictions of the anchor branch, velocity branch, and feature fusion branch together form reasonable evidence; S8. Using the trained network, the actual B and V are used as input to obtain the pedestrian's intention to cross the street from the perspective of the vehicle. In the above-mentioned S4, the adaptive learnable multi-head cross attention module includes a multi-head cross attention mechanism, residual connection and normalization, a multi-layer perceptron, and a final residual connection and normalization, which are arranged in sequence. First, the interaction between the modalities is enhanced by the multi-head cross attention mechanism, and its output is processed by the multi-layer perceptron after residual connection and normalization. Finally, after the final residual connection and normalization, refined modal features are obtained. The hybrid channel attention module includes a maximum pooling layer, an average pooling layer, a splicing layer, and a multi-layer perceptron. First, the input passes through the maximum pooling layer and the average pooling layer in parallel. Then, the output of the maximum pooling layer is spliced ​​through the splicing layer, and the output of the average pooling layer is spliced ​​through the splicing layer. The output of the splicing layer is then input into the multi-layer perceptron to reduce redundant information and form an excitation signal. Finally, the excitation signal is used to calibrate the input features. Connect the multi-head cross attention module and the mixed channel attention module in sequence to obtain the feature fusion module; In S5, the method for obtaining evidence of the anchor frame branch, the velocity branch, and the feature fusion branch is as follows: S5.

1. Process S1, S2, and S3 through two fully connected layers, and set a tanh activation function between the two fully connected layers; S5.

2. For the corresponding outputs after passing through the two fully connected layers, the corresponding weight vectors B1, B2, and B3 are obtained through the Softmax activation function. B1, B2, and B3 are multiplied element-wise with S1, S2, and S3 respectively, and the scale of each time dimension is adjusted. The multiplication results are then compressed in the time dimension through adaptive average pooling to obtain the corresponding features T1, T2, and T3 after the compression of the time dimension. S5.

3. Pass T1, T2, and T3 through the fully connected layer with the activation function SoftPlus, and obtain the evidence e1, e2, and e3 corresponding to the anchor frame branch, speed branch, and feature fusion branch, respectively, expressed as 、 、 ,in 、 Evidence of pedestrians not crossing and crossing given by the anchor frame branch respectively; 、 Evidence for pedestrians not crossing and crossing given for the speed branch respectively; 、 The evidence of pedestrians not crossing and crossing given by the feature fusion branch respectively.

2. The method for credible pedestrian intention prediction based on multi-attention and evidence fusion according to claim 1, characterized in that: In the above S1, the pedestrian anchor frame B is the coordinate sequence of the upper left corner and lower right corner of the two-dimensional rectangular frame surrounding the pedestrian, and the vehicle speed V is the continuous vehicle speed data obtained by the vehicle sensor.

3. The method for credible pedestrian intention prediction based on multi-attention and evidence fusion according to claim 1, characterized in that: In the above-mentioned S2, the embedding module includes a one-dimensional convolution with a kernel size of 1 and position encoding. First, B and V are projected into a high-dimensional feature space through one-dimensional convolution, and then position encoding is added to reflect the temporal relationship of the sequence to obtain P1 and P2.

4. The method for credible pedestrian intention prediction based on multi-attention and evidence fusion according to claim 1, characterized in that: In the above S3, the feature enhancement module includes a multi-head self-attention mechanism, a residual network, normalization, and a feedforward network. The process of obtaining S1 and S2 using the feature enhancement module is as follows: S3.

1. Apply the multi-head self-attention mechanism to P1, set the key, value, and query of the self-attention to P1, calculate the attention weight matrix, capture the correlation between different frames, and achieve feature enhancement. After completing the self-attention operation, obtain R1, which represents the features processed by the multi-head self-attention mechanism. S3.2, use the residual network and normalization operation on R1 to obtain , Represents the features after residual network and normalization operation; S3.3, Input the feedforward network and compare its output with The fusion is performed through the residual network, and the normalization operation is performed again after fusion to obtain the enhanced anchor frame feature S1; S3.

4. For P2, perform the same processing as S3.1-S3.3 to obtain the enhanced speed feature S2.

5. The method for credible pedestrian intention prediction based on multi-attention and evidence fusion according to claim 1 is characterized by: In the above S4, the process of obtaining the fusion feature S3 based on the feature fusion module is as follows: S4.

1. Input S1 and S2 into the adaptive learnable multi-head cross attention module. For S1, in the multi-head cross attention mechanism, S1 is used as the query and S2 as the key and value. By calculating the attention weight matrix, the interaction of different modalities is enhanced. After completing the cross attention operation, we get , Represents the features processed by the multi-head cross attention mechanism; S4.2, use residual connection to connect S1 and Connect and normalize to get , Represents the features after residual connection and normalization operation; S4.

3. Input the multilayer perceptron and compare its output with Connected by residual connection, and then normalized again to obtain , Represents the features after multi-layer perceptron, residual connection and normalization operations; S4.4, for S2, in the multi-head cross attention mechanism, take S2 ​​as the query and S1 as the key and value, and process it in the same way as S4.1-S4.4 to get ; S4.

5. and Input mixed channel attention module, compressed in parallel through average pooling layer and maximum pooling layer and , eliminate redundant information in the time dimension; S4.

6. Connect the data stream output by the average pooling layer through the splicing layer to obtain the connected feature A corresponding to the average pooling layer avg , connect the data stream output by the maximum pooling layer through the splicing layer to obtain the connected feature A corresponding to the maximum pooling layer max ; S4.7, A avg and A max Input the multi-layer perceptron based on the encoder-decoder architecture to learn the complex relationship between different features and obtain the corresponding learned features. and ; S4.8, Split into two eigenvectors of equal length, the first half is recorded as The second half is recorded as ; Similarly, Split into two eigenvectors of equal length, the first half is recorded as The second half is recorded as ; S4.9, 、 With learnable parameters The weights are summed and the excitation signal C1 is formed through the sigmoid function. 、 With learnable parameters Sum the weights and pass them through the sigmoid function to form the excitation signal C2; S4.10, C1 and Multiply C2 by element Multiply element by element to complete the channel feature recalibration and obtain the corresponding calibrated feature vectors M1 and M2; S4.

11. Connect M1 and M2 along the channel dimension, and after dimensionality reduction through a multi-layer perceptron, obtain the fusion feature S3.

6. The method for credible pedestrian intention prediction based on multi-attention and evidence fusion according to claim 1, characterized in that: In S6, the process of decision-level fusion based on DS evidence theory is as follows: S6.

1. Calculate e n The corresponding Dirichlet distribution parameter , confidence quality , and the corresponding uncertainty u n , forming the subjective opinions of each branch ,in K n The corresponding distribution parameters of the positive and negative classes in the Dirichlet distribution, K n The confidence quality of the positive and negative classes in the corresponding Dirichlet distribution, n = 1, 2, 3, represents the anchor frame branch, speed branch and feature fusion branch respectively, and the calculation formula is: ； ； ； In the formula, k = 1 or 2, indicating the category; S6.

2. K1, K2, and K3 are integrated to form the overall subjective opinion K4, which is expressed as: ； Where, Represents the fusion symbol; the specific fusion process is: S6.2.

1. and , the subjective opinion K after the fusion of the two 12 Expressed as: ； Where, It's K 12 The confidence quality of the positive and negative classes in the corresponding Dirichlet distribution; It's K 12 The corresponding uncertainties are expressed as: ； ； Where, ; S6.2.2, based on S6.2.1, similarly, K 12 Combined with K3, we get ;in They represent the confidence quality of the positive and negative classes in the Dirichlet distribution corresponding to K4; u4 represents the uncertainty corresponding to K4; S6.

3. Calculate the corresponding evidence based on K4 , It represents the evidence of pedestrians not crossing and crossing given by the decision-level fusion branch, and then calculates the Dirichlet distribution parameter corresponding to e4 , are the distribution parameters of the positive and negative classes in the Dirichlet distribution corresponding to K4, expressed as: ； ； S6.

4. Utilization To judge the pedestrian's intention to cross the street, argmax represents the parameter value of the maximum point of a function.

7. The method for credible pedestrian intention prediction based on multi-attention and evidence fusion according to claim 6, characterized in that: In S7, the multi-task variational loss function It is the sum of the loss function of each branch and the loss function of the decision-level fusion branch, expressed as: ； Where n=1, 2, 3, and 4 represent the anchor frame branch, velocity branch, feature fusion branch, and decision-level fusion branch, respectively; is the loss function of branch n; where: ； Where, is a hyperparameter, represents the random variable corresponding to branch n, which obeys the parameter Dirichlet distribution , , and express The expectation and variance of represents element-wise multiplication; represents the sample label, represents Dirichlet distribution and The KL divergence between Represents a vector consisting of all 1s.

8. The method for credible pedestrian intention prediction based on multi-attention and evidence fusion according to claim 7, characterized in that: use Sure , where epochs is the number of training rounds and annealing_step is initialized to 10 as the annealing coefficient.

Images