Test method for pedestrian trajectory prediction and emergency braking in automatic driving platform

Abstract

The invention belongs to the field of data processing, and discloses a test method for pedestrian trajectory prediction and emergency braking in an automatic driving platform. Comprising the steps of segmenting a real road environment in advance, obtaining grades of small road segments according to calculation of segmentation coefficients, carrying out resource allocation on the small segments according to the grades, and collecting multi-source heterogeneous data; importance optimization is introduced into a Transform model, possible prediction trajectories of individuals are generated, graph data are constructed by using the possible prediction trajectories, an Autoencoder anomaly detection mechanism is introduced, high-risk trajectories are screened out, and high-risk scenes are output; combining the road geometric model with the high-risk scene to form a scene library, loading the scene in a simulation platform, carrying out automatic driving test, carrying out virtual-real data cross validation in combination with real vehicle test data, and obtaining performance evaluation of an algorithm; pedestrian track prediction and emergency braking in the automatic driving process are achieved.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and more specifically, to a testing method for pedestrian trajectory prediction and emergency braking in an autonomous driving platform. Background Art

[0002] The rapid development of autonomous driving technology has brought revolutionary changes to the way people travel. Pedestrian trajectory prediction and emergency braking, as two key technologies, play a vital role in ensuring the safe and reliable operation of autonomous vehicles.

[0003] During the test, pedestrian behavior is highly uncertain and dynamic, especially in complex traffic scenarios (such as sudden crossing, group avoidance, etc.). Traditional prediction methods are difficult to accurately capture pedestrian intentions and interactive behaviors. In real road environments, high-risk scenarios (such as accidents or near-accidents) are often sparse and difficult to accurately identify, and traditional methods may ignore potential risks. The complexity of the real road environment leads to high data collection and processing costs, and uneven resource allocation may lead to insufficient data in key areas.

[0004] In view of this, the present invention proposes a testing method for pedestrian trajectory prediction and emergency braking in an autonomous driving platform to solve the above problems. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned purpose, the present invention provides the following technical solution: A test method for pedestrian trajectory prediction and emergency braking in an autonomous driving platform, comprising: Step S1: pre-segmenting a real road environment and obtaining the level of a road segment according to the calculation of a segmentation coefficient, allocating resources to the segment according to the level and collecting multi-source heterogeneous data; Step S2: Importance optimization is introduced into the Transformer model to generate possible prediction trajectories of individuals. The possible prediction trajectories are used to construct graph data. The GNN model obtains the interaction behavior trajectories after individual association by processing the graph data. The Autoencoder anomaly detection mechanism is introduced to screen out high-risk trajectories and output high-risk scenarios. Step S3: Combine the road geometry model with the high-risk scenarios to form a scenario library, load the scenarios into the simulation platform and conduct autonomous driving tests, and perform cross-validation of virtual and real data in combination with real vehicle test data to obtain performance evaluation of the algorithm; Step S4: Arrange the test process and results on a visual interface.

[0006] Preferably, the method of pre-segmenting the real road environment and obtaining the level of the road segments according to the calculation of the segmentation coefficient, allocating resources to the segments according to the level and collecting multi-source heterogeneous data includes: Multi-source heterogeneous data include pedestrian trajectory data, traffic flow data, environmental data and scene data. Scene data include accident record data and approximate accident data. Divide the real road into small segments in advance and calculate the segmentation coefficient for each small segment ,in, is the total number of influencing factors, It is the first In the cycle time The normalized value of the influencing factors ranges from , Yes The mapping function of It is In the cycle time Dynamic weights of influencing factors; A first limit threshold and a second limit threshold are preset, and the first limit threshold is greater than the second limit threshold. If the segmentation coefficient is greater than the first limit threshold, it is determined to be a high-risk section and the collection equipment is deployed at a high density. If the segmentation coefficient is within the interval between the first limit threshold and the second limit threshold, it is determined to be a medium-risk section and the collection equipment is deployed at a medium density. If the segmentation coefficient is less than the second limit threshold, it is determined to be a low-risk section and the collection equipment is deployed at a low density. Calculate the The division coefficient of the cycle time is determined and the level is judged. The collection devices are dynamically deployed in small segments of a cycle time, and multi-source heterogeneous data are collected in real time through the collection devices.

[0007] Preferably, the high-risk scenarios and acquisition methods include: Clean and interpolate the collected multi-source heterogeneous data; For each small segment, extract multi-source heterogeneous data within G_U cycles and obtain the complete input vector through feature embedding processing; Use the Transformer model to predict the possible behavior of individual pedestrians and generate the possible predicted trajectory of the individual; Use the GNN model to associate the interaction relationships between pedestrians, correct the possible predicted trajectories of individuals, and generate the associated interaction behavior trajectories; The Autoencoder anomaly detection mechanism is introduced to remove interactive behavior trajectories with low probability of occurrence, retain high-risk trajectories with high probability of occurrence, and output high-risk scenarios.

[0008] Preferably, the method of obtaining a complete input vector through feature embedding processing includes: Extract each pedestrian’s historical trajectory data, environmental context data, and pedestrian’s intention data to form raw data; Clean, smooth and standardize the raw data; The data of each modality in the original data is embedded independently, including: for the historical trajectory data of pedestrians, MLP or RNN is used to extract the temporal features of the trajectory sequence; for the environmental context data, MLP is used to embed the environmental data; for the pedestrian intention data, MLP is used to embed the intention data and the embedded features are normalized; Calculate the mean and variance of each modal embedding feature, adjust the activation function of the embedding layer to make it close to the standard normal distribution, calculate the correlation matrix of each modal embedding feature, and use PCA for dimensionality reduction if the correlation exceeds the predefined judgment threshold; The embedded features of historical trajectory data are used as the main features, and the embedded features of environmental context data and intent data are used as auxiliary features. The Cross-Attention mechanism is used to capture the interaction between modalities. The features after interaction are weightedly fused to obtain a complete input vector.

[0009] Preferably, the method of using the Transformer model to predict the possible behavior of individual pedestrians and generate the possible predicted trajectory of the individual pedestrians includes: The input of the Transformer encoder in the Transformer model is set as the input vector, and the output is the encoded feature, recorded as the encoded feature. The input of the conditional variational autoencoder in the Transformer model is the encoded feature, the embedded feature of the environmental context data, and the embedded feature of the intent data. The output is the parameters of the latent distribution. The Gaussian expression of the latent distribution is obtained according to the parameters, and the latent variables are sampled from the latent distribution to generate F_O possible future trajectories. In the process of sampling latent variables from the latent distribution, importance optimization is introduced, that is, calculating the importance weight of each latent variable. , is the encoding feature, is the embedding feature of the intent data, For the Trajectory The probability of generating is the prior distribution of the latent variable, Indicates potential variables, is the index of the trajectory, is the index of the latent variable; Arrange the importance weight of each latent variable in descending order, select the first A_O latent variables, record them as possible prediction trajectories, and then output the possible prediction trajectories; Set trajectory constraints, including diversity constraints, physical constraints and intention constraints. The diversity constraint is to calculate the minimum Euclidean distance between any two possible predicted trajectories at all time steps and maximize the minimum Euclidean distance. The physical constraint is that the acceleration and speed of the pedestrian are both less than the maximum acceleration and maximum speed. The intention constraint is to minimize the Euclidean distance between the trajectory and the target position.

[0010] Preferably, the method of using the GNN model to associate the interaction relationship between pedestrians, correcting the possible predicted trajectory of individuals, and generating the associated interaction behavior trajectory includes: Extract the distance between pedestrians and the distance between pedestrians and vehicles or roadblocks from multi-source data, record them as interaction data, and combine the interaction data with the possible predicted trajectory of individuals as the input data of the GNN model; Construct a blank graph, represent each pedestrian as a node in the graph, and the node features include the individual's possible predicted trajectory, environmental context data, and intention data. The distance between pedestrians is represented as an edge in the graph, and the edge features include distance information. The distance between pedestrians and vehicles or roadblocks is also represented as an edge, and the edge features include the position and speed of the vehicle or roadblock to form graph data. Use graph neural network GNN to process graph data. GNN updates the features of each node through multi-layer graph convolution or graph attention mechanism. During the updating process, social force model constraints are introduced to output the updated node features. The updated node features are mapped back to the trajectory space, and the interaction behavior trajectory of each pedestrian is output.

[0011] Preferably, the method of introducing the Autoencoder anomaly detection mechanism, removing the interactive behavior trajectories with low probability of occurrence, retaining the high-risk trajectories with high probability of occurrence, and outputting the high-risk scenarios includes: Use the Autoencoder time series anomaly detection model, with the input being the interaction behavior trajectory of each pedestrian and the output being the anomaly score of each interaction behavior trajectory; Based on the abnormal threshold and trajectory characteristics, high-risk types are defined as sudden crossing, direction change and group avoidance. The abnormal threshold and trajectory characteristics of each high-risk type are set. The trajectory characteristic of sudden crossing is that the pedestrian trajectory crosses the lane line, the trajectory characteristic of direction change is that the speed direction of the pedestrian trajectory changes suddenly, and the trajectory characteristic of group avoidance is that the trajectories of more than one pedestrian change direction at the same time. For each pedestrian's interactive behavior trajectory, the difference between the anomaly score and the anomaly threshold is calculated, and the trajectory features are combined to convert it into a probability value using the Sigmoid activation function, and the probability value is recorded as a high-risk probability; A risk threshold is set. When the high-risk probability exceeds the risk threshold, the interactive behavior trajectory is marked as a high-risk behavior. The environmental context data at the time of the high-risk behavior is extracted and combined with it to form a high-risk scenario, and the high-risk scenario is output.

[0012] Preferably, the road geometry model is combined with high-risk scenarios to form a scenario library, the scenarios are loaded in the simulation platform and the autonomous driving test is performed, virtual and real data are cross-validated in combination with real vehicle test data, and the performance evaluation of the algorithm is obtained, and the method includes: Based on high-precision maps and sensor data, a road geometry model is constructed. Weather conditions are added using weather simulation tools. Traffic flow is generated through SUMO and imported into CARLA to simulate the road traffic environment. Import high-risk scenarios into a simulated road traffic environment to form a scenario library, and store the scenario library in a standard format; Load the scenario library in the simulation platform, load the test vehicle in the scenario library, deploy the autonomous driving algorithm on the test vehicle and run it; The test data was recorded and cross-validated with real and virtual data. The performance evaluation of the algorithm included trajectory error and performance index. The trajectory error was the absolute difference between the interactive behavior trajectory of each pedestrian and the high-risk scenario predicted by the autonomous driving algorithm on the test vehicle and the scenario library. The performance index was obtained by accumulating the braking distance, stopping distance, and offset of the vehicle. Outputs a performance evaluation of the algorithm.

[0013] Preferably, the method of arranging the test process and results on a visual interface includes: Display the operation of the simulation platform on a visual interface; Set up R_P test vehicles in parallel, run the R_P test vehicles together in the scenario library, and perform tests over S_F time periods; The performance evaluation of the algorithm on R_P test vehicles in S_F time periods is plotted in a graph, and the graph is arranged on a visualization interface for intuitive observation by users.

[0014] Preferably, the visualization interface includes a terminal page of a computer or a display interface of a mobile device, and performs interface interaction with the user.

[0015] Technical effects and advantages of the test method for pedestrian trajectory prediction and emergency braking in the autonomous driving platform of the present invention: 1. By collecting pedestrian trajectories, traffic flows, environment and scene data (including accident and near-accident data), multimodal data fusion is achieved. Through feature embedding and Cross-Attention mechanism, the interactive relationship between modalities is captured, and the integrity and expression ability of the input vector are enhanced. The Transformer model is combined with the conditional variational autoencoder to generate multiple possible future trajectories, and the high-likelihood trajectories are screened through importance optimization to improve the prediction accuracy. Trajectory constraints (diversity, physics and intention constraints) are introduced to ensure that the predicted trajectory conforms to the physical laws and behavioral logic of the real scene.

[0016] 2. Use the GNN model to construct graph data, combine the constraints of the social force model, capture the interaction between pedestrians, correct individual trajectories, and improve the robustness of the prediction results. Through graph convolution or graph attention mechanism, the update ability of node features is enhanced to adapt to dynamic interactions in complex scenarios. Introduce the Autoencoder anomaly detection mechanism, combine the characteristics of high-risk behavior types (such as sudden crossing, change of direction, group avoidance), remove low-probability trajectories, and retain high-risk trajectories. Quantify the degree of risk by calculating the anomaly score and high-risk probability to ensure the accuracy and practicality of high-risk scenarios.

[0017] 3. Through dynamic calculation of the segmentation coefficient, the road sections are divided into high, medium and low risk levels, and the density of collection equipment is adjusted according to the level to optimize the data collection efficiency. Dynamic deployment of collection equipment to adapt to changes in the road environment improves the real-time and targeted nature of data collection. Combining high-precision maps, weather simulation and traffic flow simulation, a realistic simulation environment is built, and high-risk scenarios are loaded through the scenario library to improve the comprehensiveness of the test. Through cross-validation of virtual and real data, combined with trajectory error and performance index (braking distance, stopping distance, offset), the performance of the autonomous driving algorithm is comprehensively evaluated.

[0018] This solution solves the key problems in pedestrian trajectory prediction and emergency braking testing through multi-model collaboration, dynamic resource allocation, high-risk scenario screening and virtual-reality combined testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a structural schematic diagram of a testing method for pedestrian trajectory prediction and emergency braking in an automatic driving platform of the present invention; Figure 2 A schematic diagram of the steps of a testing method for pedestrian trajectory prediction and emergency braking in the autonomous driving platform of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Example 1

[0021] See also Figure 1 and Figure 2 As shown, the test method for pedestrian trajectory prediction and emergency braking in the autonomous driving platform described in this embodiment includes: During the test, pedestrian behavior is highly uncertain and dynamic, especially in complex traffic scenarios (such as sudden crossing, group avoidance, etc.). Traditional prediction methods find it difficult to accurately capture pedestrian intentions and interactive behaviors. In real road environments, high-risk scenarios (such as accidents or near-accidents) are often sparse and difficult to accurately identify, and traditional methods may ignore potential risks.

[0022] Step S1: pre-segment the real road environment and obtain the level of the road segment according to the calculation of the segmentation coefficient, allocate resources to the segment according to the level and collect multi-source heterogeneous data; The method of pre-segmenting the real road environment and obtaining the level of the road segment according to the calculation of the segmentation coefficient, allocating resources to the segment according to the level and collecting multi-source heterogeneous data includes: Multi-source heterogeneous data include pedestrian trajectory data (such as pedestrian location, speed, acceleration, and sudden appearance data), traffic flow data (such as vehicle density, speed distribution data), environmental data (such as weather, light, road conditions and other natural environmental impact data that affect vehicle driving) and scene data. Scene data includes accident record data (such as vehicle rollover, pedestrian collision and other accident record data) and near-accident data (such as pedestrians suddenly crossing, vehicle emergency braking, sharp turns and other events that did not cause accidents). In the real road environment, the setting of the segmentation coefficient is a key issue. Its purpose is to divide the road into different areas based on experience or needs according to factors such as road complexity, accident probability, and traffic flow, so as to reasonably allocate data collection resources and focus according to the segmentation coefficient. The setting of the segmentation coefficient requires comprehensive consideration of multiple influencing factors, and the real road is divided into small sections in advance. The segmentation coefficient is calculated for each small section. ,in, is the total number of influencing factors, It is In the cycle time The normalized value of the influencing factors ranges from , Yes The mapping function is used to capture the nonlinear effects of factors. It is In the cycle time The dynamic weight of the influencing factors satisfies , reflecting the changes in the importance of factors in different scenarios; the segmentation coefficient is a comprehensive indicator used to measure the complexity and risk level of a certain section of road. The higher the value, the more complex or risky the section is, requiring finer-grained segmentation and higher resource investment (such as sensor density and data collection frequency).

[0023] Assume there are 5 influencing factors: road curvature , accident frequency , Traffic flow density , approximate accident frequency , Environmental complexity , Used to capture the nonlinear effects of various factors on segmentation.

[0024] For example, the accident frequency The risk is small at low frequencies, but increases dramatically at high frequencies. Therefore, an exponential function is used to calculate the accident frequency. and approximate accident frequency Deformation design, such as , ; The Sigmoid function is suitable for scenarios where the influence changes smoothly within a certain range, so it can be used to calculate the road curvature. ; The power function is suitable for scenarios where the impact grows exponentially, so it can be used to calculate traffic flow density. ; The piecewise function is suitable for scenarios where the impact has different performance in different intervals, such as environmental complexity, so it can be used to calculate environmental complexity. , where environmental complexity You can use machine learning models (such as random forests, gradient boosted trees, neural networks) to directly obtain the numerical value of environmental complexity. The input of the model can be the collected environmental data. Dynamic Weight Dynamic adjustments are made according to real-time scenarios to adapt to changes in traffic flow, weather, etc. The specific method is based on adjustments based on machine learning. For example, weights are dynamically updated using online learning algorithms (such as reinforcement learning). The goal of reinforcement learning is set to maximize the coverage of high-risk sections. By dynamically adjusting weights, the segmentation coefficient can adapt to real-time scenario changes. For example, on rainy days, the weight of environmental complexity increases, making the segmentation coefficient pay more attention to the impact of weather and road conditions. During peak traffic hours, the weight of traffic flow density increases, making the segmentation coefficient pay more attention to the risks of congested sections. Dynamic weight adjustment improves the flexibility and real-time performance of the model, and can better cope with dynamically changing road environments, such as sudden weather changes and traffic accidents.

[0025] The segmentation coefficient can more accurately identify high-risk sections (such as accident-prone sections and congested sections), thereby guiding more fine-grained road segmentation and resource allocation. It can also adapt to environmental changes and improve the pertinence of data collection and the effectiveness of monitoring. The combination of nonlinear models and dynamic weights makes the segmentation coefficient more robust and generalizable, and is suitable for different road types (such as urban roads, rural roads, and highways).

[0026] A first limit threshold and a second limit threshold are preset, and the first limit threshold is greater than the second limit threshold. If the segmentation coefficient is greater than the first limit threshold, it is determined to be a high-risk section and the collection equipment is deployed at a high density. If the segmentation coefficient is within the interval between the first limit threshold and the second limit threshold, it is determined to be a medium-risk section and the collection equipment is deployed at a medium density. If the segmentation coefficient is less than the second limit threshold, it is determined to be a low-risk section and the collection equipment is deployed at a low density. Calculate the The division coefficient of the cycle time is determined and the level is judged. The collection devices are dynamically deployed in small segments of a cycle time, and multi-source heterogeneous data are collected in real time through the collection devices.

[0027] In the above description, it is assumed that the first limit threshold is 0.3 and the second limit threshold is 0.6, and there are 30 available acquisition devices. For high-risk sections, 15 can be deployed to monitor the road environment in all directions, 10 can be deployed to medium-risk sections to observe key locations, and 5 can be deployed to low-risk sections. The acquisition devices are used to obtain multi-source heterogeneous data, such as lidar, camera, millimeter-wave radar, GPS and other sensors. Lidar collects point cloud data for target detection and trajectory tracking, camera collects image data for target recognition and scene segmentation, millimeter-wave radar collects target distance, speed and angle, and GPS / IMU collects vehicle positioning and motion status. These sensors can be worn on mobile devices (such as drones or robot dogs) to facilitate rapid mobile scheduling, realize adaptive adjustment of resources and maximize efficiency.

[0028] Step S2: Importance optimization is introduced into the Transformer model to generate possible prediction trajectories of individuals. The possible prediction trajectories are used to construct graph data. The GNN model obtains the interaction behavior trajectories after individual association by processing the graph data. The Autoencoder anomaly detection mechanism is introduced to screen out high-risk trajectories and output high-risk scenarios. High-risk scenarios and acquisition methods include: The collected multi-source heterogeneous data are cleaned and preprocessed by interpolation; cleaning is used to remove noise and outliers in the multi-source heterogeneous data, and median filtering is used for processing; interpolation is used to interpolate missing data, and linear interpolation or spline interpolation methods can be used.

[0029] Input data: For each small segment, extract multi-source heterogeneous data within G_U cycles, and obtain the complete input vector through feature embedding processing; The complete input vector is obtained through feature embedding processing, which includes: Extract each pedestrian’s historical trajectory data (including position, speed, and acceleration over a historical period of time), environmental context data (such as road type, obstacle distribution, and weather conditions), and pedestrian intention data (such as analyzing the pedestrian’s head orientation or gestures through image data to infer possible movement directions) to form raw data; Clean, smooth and standardize the raw data to reduce the impact of noise and outliers; For example, for the historical trajectory data of pedestrians, use Kalman filtering or moving average filtering to smooth the position, velocity and acceleration data, reduce sensor noise, use statistical methods (such as Z score) or density-based anomaly detection (such as DBSCAN) to identify and remove outliers, and standardize the trajectory data (zero mean, unit variance); for environmental context data, perform one-hot encoding or label encoding on discrete variables (such as road type, weather conditions), standardize continuous variables such as obstacle distribution, and use interpolation (such as linear interpolation) or mean / median to fill in missing environmental data; for pedestrian intention data, use pre-trained target detection models (such as YOLOv5 or Faster R-CNN) to extract pedestrian head orientation and gesture features, post-process the extracted features (such as smoothing or threshold filtering) to reduce detection errors, and standardize the intention features.

[0030] The data of each modality in the original data is embedded independently to generate high-quality feature representations, including: for the historical trajectory data of pedestrians, use MLP or RNN (such as LSTM, GRU) to extract time series features from the trajectory sequence; for environmental context data, use MLP to embed environmental data (such as road type, obstacle distribution, weather); for pedestrian intention data, use MLP to embed intention data (such as head orientation, gestures); normalize the embedded features; calculate the mean and variance of each modality embedding feature, adjust the activation function of the embedding layer (such as using ReLU or Tanh) to make it close to the standard normal distribution, calculate the correlation matrix of each modality embedding feature, check whether there is redundancy, and if the correlation exceeds the predefined judgment threshold (such as (correlation|r|> judgment threshold 0.8)), use PCA to reduce the dimension (the embedding layer structure can also be adjusted); Feature distribution analysis ensures the numerical stability of embedded features. Correlation analysis reduces feature redundancy.

[0031] The embedded features of historical trajectory data are used as the main features, and the embedded features of environmental context data and intent data are used as auxiliary features. The Cross-Attention mechanism is used to capture the interaction between modalities, and the features after interaction are weighted fused to obtain a complete input vector. The fusion based on Cross-Attention optimizes the feature splicing method to avoid information redundancy or dimensionality explosion caused by direct splicing.

[0032] Defining the Cross-Attention Mechanism , calculate the interaction characteristics between the trajectory and the environment ,Similarly, the interactive features of trajectory and intention are calculated , get the input vector after weighted fusion ; in, is the query vector, is the key vector, is a vector of values, is the dimension of the key vector, is the embedded feature of historical trajectory data, is the embedded feature of the environmental context data, , and is a learnable weight matrix, and are learnable weights, normalized by softmax.

[0033] Use the Transformer model to predict the (multiple) possible behaviors of individual pedestrians and generate possible predicted trajectories of the individuals; Use the Transformer model to predict the possible behavior of individual pedestrians and generate the possible predicted trajectory of the individual. The methods include: The input of the Transformer encoder in the Transformer model is set as the input vector. The Transformer uses a multi-head self-attention mechanism to focus on important information at different time steps in the trajectory, such as sudden changes in speed or acceleration. Capture the temporal dependencies in the trajectory data. The output is the encoded feature, recorded as the encoded feature, which is used to represent the potential pattern of the pedestrian trajectory. The input of the encoder (Encoder) in the conditional variational autoencoder (CVAE) in the Transformer model is the encoded feature, the embedded feature of the environmental context data, and the embedded feature of the intention data. The output is the parameters of the potential distribution. According to the parameters, the Gaussian expression of the potential distribution is obtained, that is, the Gaussian distribution, which can also be said to be the expression of the potential distribution. The latent variables are sampled from the latent distribution, and the latent variables, encoded features, embedded features of the environmental context data, and embedded features of the intention data are input into the decoder (Decoder) together. The trajectory sequence is initialized using MLP, and the future trajectory is iteratively generated using the Transformer decoder, thereby generating F_O possible future trajectories. In the process of sampling latent variables from the latent distribution, importance optimization is introduced, that is, the importance weight of each latent variable is calculated , is the encoding feature, is the embedding feature of the intent data, For the Trajectory The probability of generating is the prior distribution of the latent variable, Indicates potential variables, is the index of the trajectory, is the index of the latent variable; Arrange the importance weight of each latent variable in descending order, select the first A_O latent variables, record them as possible prediction trajectories, and then output the possible prediction trajectories; Reduce the number of sampling times and improve computational efficiency through importance sampling. Use simulation tools (such as CARLA and SUMO) to verify the generated trajectory. Compare the similarity between the simulated trajectory and the real trajectory. Set trajectory constraints, including diversity constraints, physical constraints, and intention constraints. The diversity constraint is to calculate the minimum Euclidean distance between any two possible predicted trajectories at all time steps, maximize the minimum Euclidean distance, and thus increase diversity. The physical constraint is that the acceleration and speed of the pedestrian are both less than the maximum acceleration and maximum speed. The intention constraint is to minimize the Euclidean distance between the trajectory and the target position. Encourage the trajectory to move in the target direction. The target position is to determine the destination that the pedestrian wants to reach.

[0034] CVAE first maps the encoded features to the latent space to generate a distribution of latent variables; then samples from the latent distribution, combines environmental context and intention data, and generates multiple possible future trajectories. In order to enhance the diversity of trajectories, diversity constraints are introduced to encourage the generated trajectories to cover different behavior modes, such as going straight, crossing, or changing direction. In order to ensure the authenticity of the trajectory, behavioral constraints are introduced, such as limiting the acceleration of the trajectory to a reasonable range, or ensuring that the trajectory moves in the target direction. The output can specifically be the generation of multiple possible future trajectories for each pedestrian, such as 5 trajectories, each of which includes the position coordinates for a period of time in the future, such as the position change in the next 5 seconds.

[0035] In summary, the advantages of the above process are enhanced diversity (CVAE and other methods ensure that the generated trajectories cover different behavior patterns), improved accuracy (Transformer decoder, behavior constraints and intent embedding improve prediction accuracy), improved computational efficiency (importance sampling reduces the number of sampling times and optimizes computing performance), and behavioral constraints (physical constraints and intent constraints ensure that the trajectories conform to the laws of reality and enhance practicality).

[0036] Use the GNN model to associate the interaction relationships between pedestrians, correct the possible predicted trajectories of individuals, and generate the associated interaction behavior trajectories; The GNN model is used to associate the interaction relationship between pedestrians, correct the possible predicted trajectory of individuals, and generate the associated interaction behavior trajectory. The method includes: Extract the distance between pedestrians and the distance between pedestrians and vehicles or roadblocks from multi-source data, record them as interaction data, and combine the interaction data with the possible predicted trajectory of individuals as the input data of the GNN model; Construct a blank graph, represent each pedestrian as a node in the graph, and the node features include the individual's possible predicted trajectory, environmental context data, and intention data. The distance between pedestrians is represented as an edge in the graph, and the edge features include distance information. The distance between pedestrians and vehicles or roadblocks is also represented as an edge, and the edge features include the position and speed of the vehicle or roadblock to form graph data. Use graph neural network GNN to process graph data and capture the interaction between pedestrians. GNN updates the features of each node through multi-layer graph convolution or graph attention mechanism. In the updating process, social force model constraints are introduced. In order to ensure that the corrected trajectory conforms to the laws of physics, GNN needs to consider the attraction and repulsion between pedestrians. For example, when two pedestrians are too close, avoidance behavior may occur. GNN also considers the interaction between pedestrians and obstacles. For example, when pedestrians approach vehicles, they may change their direction of movement. Consider the interaction between pedestrians and environmental forces, such as avoiding the overlap of trajectories with obstacles. GNN outputs updated node features; Map the updated node features back to the trajectory space, correct the individual trajectory prediction results, generate the trajectory after interaction, record it as the interaction behavior trajectory (behavior features after interaction), and output the interaction behavior trajectory of each pedestrian, for example, 5 types of trajectories, each of which includes the position coordinates in the future period of time, reflecting the impact of group interaction; The social force model is a model used to simulate and analyze crowd behavior. It has been widely used in crowd simulation, traffic flow analysis, architectural design and other fields.

[0037] The Autoencoder anomaly detection mechanism is introduced to remove interactive behavior trajectories with low probability of occurrence, retain high-risk trajectories with high probability of occurrence, and output high-risk scenarios.

[0038] The Autoencoder anomaly detection mechanism is introduced to remove the interactive behavior trajectories with low probability of occurrence, retain the high-risk trajectories with high probability of occurrence, and output the high-risk scenarios. The methods include: The Autoencoder time series anomaly detection model is used to analyze the trajectory after the interaction and identify high-risk behaviors. The input is the interaction behavior trajectory of each pedestrian, and the output is the anomaly score of each interaction behavior trajectory. The specific principle is to compress the trajectory into low-dimensional features and then try to reconstruct the original trajectory; by comparing the difference between the reconstructed trajectory and the original trajectory, the anomaly score is calculated. The higher the anomaly score, the more likely the trajectory contains abnormal behaviors, such as sudden crossing or group avoidance; Based on the abnormal threshold and trajectory characteristics, high-risk types are defined as sudden crossing, direction change and group avoidance. The abnormal threshold and trajectory characteristics of each high-risk type are set. The trajectory characteristic of sudden crossing is that the pedestrian trajectory crosses the lane line, the trajectory characteristic of direction change is that the speed direction of the pedestrian trajectory changes suddenly, and the trajectory characteristic of group avoidance is that the trajectories of more than one pedestrian change direction at the same time. The abnormal threshold of each high-risk type can be determined through historical data statistics, such as analyzing approximate accident data.

[0039] For each pedestrian's interactive behavior trajectory, the difference between the anomaly score and the anomaly threshold is calculated, and the trajectory features are combined to convert it into a probability value using the Sigmoid activation function, and the probability value is recorded as a high-risk probability; for example, .

[0040] A risk threshold is set. When the high-risk probability exceeds the risk threshold (for example, 0.5), the interaction behavior trajectory is marked as a high-risk behavior. The environmental context data at the time of the high-risk behavior is extracted and combined with it to form a high-risk scenario. The high-risk scenario is output. For example, the high-risk scenario includes the pedestrian's post-interaction trajectory (such as location coordinates within the next 5 seconds), high-risk behavior type (such as sudden crossing or group avoidance), and environmental context data (such as road type, obstacle distribution, and weather conditions).

[0041] In the above process, first, the Transformer model is trained separately to optimize the accuracy and diversity of individual trajectory prediction. Then, the GNN model is trained separately to optimize the correction effect of the trajectory after interaction. Finally, the Transformer, GNN and anomaly detection models are jointly trained to optimize the overall performance.

[0042] The optimization goals are to ensure accurate prediction of individual trajectories, cover a variety of behavior patterns, ensure that the trajectories after interaction reflect the rules of group interaction, avoid unreasonable trajectories, ensure accurate detection of high-risk behaviors, and reduce false positives and negatives.

[0043] The data is divided into training set, validation set and test set, for example, 80% for training, 10% for validation, and 10% for testing. The training set is set according to the input and output of the model, the validation set is used to adjust the model parameters, and the test set is used to evaluate the model performance.

[0044] Use the gradient descent method to gradually optimize the model to ensure model convergence. During the training process, regularly check the performance of the model on the validation set, adjust the learning speed or other parameters to avoid overfitting. In addition, when there are fewer samples of high-risk behaviors, data enhancement or weighting methods can be used to increase the model's attention to high-risk behaviors.

[0045] Evaluation indicators can be the following: 1. Trajectory prediction accuracy: Evaluate the average distance error between the predicted trajectory and the true trajectory, such as the average deviation between the predicted position and the actual position. Evaluate the final position error, such as the deviation between the end point of the predicted trajectory and the actual end point.

[0046] 2. High-risk behavior detection performance: Evaluate the precision of high-risk behavior detection, that is, the proportion of correctly identified high-risk behaviors to all identified high-risk behaviors. Evaluate the recall rate, that is, the proportion of correctly identified high-risk behaviors to all actual high-risk behaviors. Combine the precision and recall rates to calculate comprehensive performance indicators, such as the F1 score.

[0047] Step S3: Combine the road geometry model with the high-risk scenarios to form a scenario library, load the scenarios into the simulation platform and conduct autonomous driving tests, and perform cross-validation of virtual and real data with real vehicle test data to obtain performance evaluation of the algorithm.

[0048] Combine the road geometry model with high-risk scenarios to form a scenario library, load the scenarios in the simulation platform and conduct autonomous driving tests, and perform cross-validation of virtual and real data with real vehicle test data to obtain performance evaluation of the algorithm. The methods include: Based on high-precision maps and sensor data, a road geometry model is constructed (generally including lane lines, intersections and obstacle distribution models, obstacles can be vehicles and roadblocks). High-precision maps and sensor data can be acquired in advance or locally extracted from multi-source heterogeneous data. Weather simulation tools are used to add weather conditions, such as rain, snow, fog, haze, night and other severe conditions. Traffic flow is generated through SUMO and imported into CARLA to simulate the road traffic environment, thus achieving a real urban traffic environment simulation. Import high-risk scenarios into the simulated road traffic environment to form a scenario library, and store the scenario library in a standard format; such as OpenSCENARIO, support docking with the simulation platform, regularly update the scenario library, and supplement newly discovered boundary conditions.

[0049] Load the scenario library in the simulation platform (such as Carla, SUMO), load the test vehicle in the scenario library, and deploy and run the autonomous driving algorithm on the test vehicle; the autonomous driving algorithm is also the pedestrian trajectory prediction and emergency braking algorithm. Ensure that the test vehicle is exactly the same as the actual vehicle.

[0050] The test data was recorded and cross-validated with real and virtual data. The performance evaluation of the algorithm included trajectory error and performance index. The trajectory error was the absolute difference between the interactive behavior trajectory of each pedestrian and the high-risk scenario predicted by the autonomous driving algorithm on the test vehicle and the scenario library. The performance index was obtained by accumulating the braking distance, stopping distance, and offset of the vehicle. Among them, the trajectory error is to subtract the interactive behavior trajectory of each pedestrian predicted by the autonomous driving algorithm from that in the scene library, and then calculate the absolute value. The absolute difference of the high-risk scenario refers to the absolute difference between the number predicted by the autonomous driving algorithm on the test vehicle and the number in the scene library. The braking distance of the vehicle is the length of the road from the pedestrian when the vehicle starts to avoid the pedestrian and starts braking. The stopping distance is the distance between the pedestrian and the vehicle at the last stop. The offset is the relative offset position of the vehicle after braking to the start of braking.

[0051] Outputs a performance evaluation of the algorithm.

[0052] Step S4: Arrange the test process and results on a visual interface; Methods for arranging the test process and results on a visual interface include: Display the operation of the simulation platform on a visual interface; Set up R_P test vehicles in parallel, run the R_P test vehicles together in the scenario library, and perform tests over S_F time periods; The performance evaluation of the algorithm on R_P test vehicles in S_F time periods is plotted in a graph, and the graph is arranged on a visualization interface for intuitive observation by users.

[0053] The visual interface includes the terminal page of the computer or the display interface of the mobile device, and interacts with the user.

[0054] The user clicks, queries, downloads and other interactive operations on the test report through the computer page (computer display screen, where the computer refers to a smart device, and the mobile device can be a mobile phone or tablet). Example 2

[0055] See also Figure 1 As shown, the part not described in detail in this embodiment is described in Example 1, and a test system for pedestrian trajectory prediction and emergency braking in an autonomous driving platform is provided, including: Data acquisition and reconstruction module: used to calculate the segmentation coefficient based on the pre-segmented real road, obtain the level of the small segment and perform resource allocation, and collect multi-source heterogeneous data of the small segment; High-risk scenario screening module: Importance optimization is introduced into the Transformer model to generate possible prediction trajectories of individuals. The possible prediction trajectories are used to construct graph data. The GNN model obtains the interaction behavior trajectories after individual association by processing graph data, introduces the Autoencoder anomaly detection mechanism, screens out high-risk trajectories, and outputs high-risk scenarios. Algorithm evaluation module: Combines road geometry models with high-risk scenarios to form a scenario library, loads scenarios into the simulation platform and conducts autonomous driving tests, and combines real vehicle test data to perform cross-validation of virtual and real data to obtain algorithm performance evaluation; Visualization module: arranges the test process and results on a visual interface.

[0056] Through multi-model collaboration, the combination of Transformer, GNN and Autoencoder solves individual prediction, interaction modeling and anomaly detection problems respectively, forming a complete prediction and screening process. Road classification based on segmentation coefficients and dynamic equipment deployment optimize data collection efficiency and reduce costs.

[0057] Through anomaly detection and high-risk behavior characteristics, high-risk scenarios are accurately screened, improving the pertinence of the test. Cross-validation of simulation and real vehicle testing ensures the reliability and practicality of the test results. The application of trajectory constraints, importance optimization and social force models improves the physical rationality and behavioral logic of the prediction results. Example 3

[0058] This embodiment discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the operation mode of the test method for pedestrian trajectory prediction and emergency braking in the autonomous driving platform provided above is implemented.

[0059] Since the electronic device introduced in this embodiment is an electronic device used to implement the test method for pedestrian trajectory prediction and emergency braking in the autonomous driving platform in the embodiment of this application, based on the test method for pedestrian trajectory prediction and emergency braking in the autonomous driving platform introduced in the embodiment of this application, a person skilled in the art can understand the specific implementation of the electronic device of this embodiment and its various variations, so how the electronic device implements the method in the embodiment of this application will not be described in detail here. As long as the electronic device used by a person skilled in the art to implement the test method for pedestrian trajectory prediction and emergency braking in the autonomous driving platform in the embodiment of this application, it belongs to the scope of protection of this application.

[0060] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters and thresholds in the formula are set by technicians in this field according to actual conditions.

[0061] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technical users in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A test method for pedestrian trajectory prediction and emergency braking in an autonomous driving platform, characterized in that: include: Step S1: pre-segment the real road environment and obtain the level of the road segment according to the calculation of the segmentation coefficient, allocate resources to the segment according to the level and collect multi-source heterogeneous data; Step S2: Importance optimization is introduced into the Transformer model to generate possible prediction trajectories of individuals. The possible prediction trajectories are used to construct graph data. The GNN model obtains the interaction behavior trajectories after individual association by processing the graph data. The Autoencoder anomaly detection mechanism is introduced to screen out high-risk trajectories and output high-risk scenarios. Step S3: Combine the road geometry model with the high-risk scenarios to form a scenario library, load the scenarios into the simulation platform and conduct autonomous driving tests, and perform cross-validation of virtual and real data in combination with real vehicle test data to obtain performance evaluation of the algorithm; Step S4: Arrange the test process and results on a visual interface.

2. The method for testing pedestrian trajectory prediction and emergency braking in an automatic driving platform according to claim 1, characterized in that: The method of pre-segmenting the real road environment and obtaining the level of the road segment according to the calculation of the segmentation coefficient, allocating resources to the segment according to the level and collecting multi-source heterogeneous data includes: Multi-source heterogeneous data include pedestrian trajectory data, traffic flow data, environmental data and scene data. Scene data include accident record data and approximate accident data. Divide the real road into small segments in advance and calculate the segmentation coefficient for each small segment ,in, is the total number of influencing factors, It is the first In the cycle time The normalized value of the influencing factors ranges from , Yes The mapping function of It is In the cycle time Dynamic weights of influencing factors; A first limit threshold and a second limit threshold are preset, and the first limit threshold is greater than the second limit threshold. If the segmentation coefficient is greater than the first limit threshold, it is determined to be a high-risk section and the collection equipment is deployed at a high density. If the segmentation coefficient is within the interval between the first limit threshold and the second limit threshold, it is determined to be a medium-risk section and the collection equipment is deployed at a medium density. If the segmentation coefficient is less than the second limit threshold, it is determined to be a low-risk section and the collection equipment is deployed at a low density. Calculate the The division coefficient of the cycle time is determined and the level is judged. The collection devices are dynamically deployed in small segments of a cycle time, and multi-source heterogeneous data are collected in real time through the collection devices.

3. The method for testing pedestrian trajectory prediction and emergency braking in an automatic driving platform according to claim 2, characterized in that: The high-risk scenarios and acquisition methods include: Clean and interpolate the collected multi-source heterogeneous data; For each small segment, extract multi-source heterogeneous data within G_U cycles and obtain the complete input vector through feature embedding processing; Use the Transformer model to predict the possible behavior of individual pedestrians and generate the possible predicted trajectory of the individual; Use the GNN model to associate the interaction relationships between pedestrians, correct the possible predicted trajectories of individuals, and generate the associated interaction behavior trajectories; The Autoencoder anomaly detection mechanism is introduced to remove interactive behavior trajectories with low probability of occurrence, retain high-risk trajectories with high probability of occurrence, and output high-risk scenarios.

4. The method for testing pedestrian trajectory prediction and emergency braking in an automatic driving platform according to claim 3, characterized in that: The method of obtaining a complete input vector by feature embedding processing includes: Extract each pedestrian’s historical trajectory data, environmental context data, and pedestrian’s intention data to form raw data; Clean, smooth and standardize the raw data; The data of each modality in the original data is embedded independently, including: for the historical trajectory data of pedestrians, MLP or RNN is used to extract the temporal features of the trajectory sequence; for the environmental context data, MLP is used to embed the environmental data; for the pedestrian intention data, MLP is used to embed the intention data and the embedded features are normalized; Calculate the mean and variance of each modal embedding feature, adjust the activation function of the embedding layer to make it close to the standard normal distribution, calculate the correlation matrix of each modal embedding feature, and use PCA for dimensionality reduction if the correlation exceeds the predefined judgment threshold; The embedded features of historical trajectory data are used as the main features, and the embedded features of environmental context data and intent data are used as auxiliary features. The Cross-Attention mechanism is used to capture the interaction between modalities. The features after interaction are weightedly fused to obtain a complete input vector.

5. The method for testing pedestrian trajectory prediction and emergency braking in an automatic driving platform according to claim 4, characterized in that: The method of using the Transformer model to predict the possible behavior of individual pedestrians and generate possible predicted trajectories of the individual pedestrians includes: The input of the Transformer encoder in the Transformer model is set as the input vector, and the output is the encoded feature, recorded as the encoded feature. The input of the conditional variational autoencoder in the Transformer model is the encoded feature, the embedded feature of the environmental context data, and the embedded feature of the intent data. The output is the parameters of the latent distribution. The Gaussian expression of the latent distribution is obtained according to the parameters, and the latent variables are sampled from the latent distribution to generate F_O possible future trajectories. In the process of sampling latent variables from the latent distribution, importance optimization is introduced, that is, calculating the importance weight of each latent variable. , is the encoding feature, is the embedding feature of the intent data, For the Trajectory The probability of generating is the prior distribution of the latent variable, Indicates potential variables, is the index of the trajectory, is the index of the latent variable; Arrange the importance weight of each latent variable in descending order, select the first A_O latent variables, record them as possible prediction trajectories, and then output the possible prediction trajectories; Set trajectory constraints, including diversity constraints, physical constraints and intention constraints. The diversity constraint is to calculate the minimum Euclidean distance between any two possible predicted trajectories at all time steps and maximize the minimum Euclidean distance. The physical constraint is that the acceleration and speed of the pedestrian are both less than the maximum acceleration and maximum speed. The intention constraint is to minimize the Euclidean distance between the trajectory and the target position.

6. The method for testing pedestrian trajectory prediction and emergency braking in an automatic driving platform according to claim 5, characterized in that: The method of using the GNN model to associate the interaction relationship between pedestrians, correcting the possible predicted trajectory of individuals, and generating the associated interaction behavior trajectory includes: Extract the distance between pedestrians and the distance between pedestrians and vehicles or roadblocks from multi-source data, record them as interaction data, and combine the interaction data with the possible predicted trajectory of individuals as the input data of the GNN model; Construct a blank graph, represent each pedestrian as a node in the graph, and the node features include the individual's possible predicted trajectory, environmental context data, and intention data. The distance between pedestrians is represented as an edge in the graph, and the edge features include distance information. The distance between pedestrians and vehicles or roadblocks is also represented as an edge, and the edge features include the position and speed of the vehicle or roadblock to form graph data. Use graph neural network GNN to process graph data. GNN updates the features of each node through multi-layer graph convolution or graph attention mechanism. During the updating process, social force model constraints are introduced to output the updated node features. The updated node features are mapped back to the trajectory space, and the interaction behavior trajectory of each pedestrian is output.

7. The method for testing pedestrian trajectory prediction and emergency braking in an automatic driving platform according to claim 6, characterized in that: The method of introducing the Autoencoder anomaly detection mechanism, removing the interactive behavior trajectories with low probability of occurrence, retaining the high-risk trajectories with high probability of occurrence, and outputting the high-risk scenarios includes: Use the Autoencoder time series anomaly detection model, with the input being the interaction behavior trajectory of each pedestrian and the output being the anomaly score of each interaction behavior trajectory; Based on the abnormal threshold and trajectory characteristics, high-risk types are defined as sudden crossing, direction change and group avoidance. The abnormal threshold and trajectory characteristics of each high-risk type are set. The trajectory characteristic of sudden crossing is that the pedestrian trajectory crosses the lane line, the trajectory characteristic of direction change is that the speed direction of the pedestrian trajectory changes suddenly, and the trajectory characteristic of group avoidance is that the trajectories of more than one pedestrian change direction at the same time. For each pedestrian's interactive behavior trajectory, the difference between the anomaly score and the anomaly threshold is calculated, and the trajectory features are combined to convert it into a probability value using the Sigmoid activation function, and the probability value is recorded as a high-risk probability; A risk threshold is set. When the high-risk probability exceeds the risk threshold, the interactive behavior trajectory is marked as a high-risk behavior. The environmental context data at the time of the high-risk behavior is extracted and combined with it to form a high-risk scenario, and the high-risk scenario is output.

8. The method for testing pedestrian trajectory prediction and emergency braking in an automatic driving platform according to claim 7, characterized in that: The method of combining the road geometry model with the high-risk scenario to form a scenario library, loading the scenario in the simulation platform and performing an autonomous driving test, and performing virtual and real data cross-validation in combination with real vehicle test data to obtain an algorithm performance evaluation includes: Based on high-precision maps and sensor data, a road geometry model is constructed. Weather conditions are added using weather simulation tools. Traffic flow is generated through SUMO and imported into CARLA to simulate the road traffic environment. Import high-risk scenarios into a simulated road traffic environment to form a scenario library, and store the scenario library in a standard format; Load the scenario library in the simulation platform, load the test vehicle in the scenario library, deploy the autonomous driving algorithm on the test vehicle and run it; The test data was recorded and cross-validated with real and virtual data. The performance evaluation of the algorithm included trajectory error and performance index. The trajectory error was the absolute difference between the interactive behavior trajectory of each pedestrian and the high-risk scenario predicted by the autonomous driving algorithm on the test vehicle and the scenario library. The performance index was obtained by accumulating the braking distance, stopping distance, and offset of the vehicle. Outputs a performance evaluation of the algorithm.

9. The method for testing pedestrian trajectory prediction and emergency braking in an automatic driving platform according to claim 8, characterized in that: The method for arranging the test process and results on a visual interface comprises: Display the operation of the simulation platform on a visual interface; Set up R_P test vehicles in parallel, run the R_P test vehicles together in the scenario library, and perform tests over S_F time periods; The performance evaluation of the algorithm on R_P test vehicles in S_F time periods is plotted in a graph, and the graph is arranged on a visualization interface for intuitive observation by users.

10. The method for testing pedestrian trajectory prediction and emergency braking in an automatic driving platform according to claim 9, characterized in that: The visualization interface includes a terminal page of a computer or a display interface of a mobile device, and performs interface interaction with the user.

Images