Human-vehicle behavior trajectory analysis method and system for urban safety

Abstract

The present invention discloses a method and system for analyzing the trajectory of human and vehicle behavior for urban safety, belonging to the field of transportation. The method for analyzing the trajectory of human and vehicle behavior for urban safety comprises the following steps: obtaining the position, speed, acceleration, and heading angle data of the vehicle itself, capturing surrounding vehicles and pedestrians, and detecting the relative speed and distance data between the vehicle itself and surrounding vehicles and pedestrians; establishing a unified time reference, constructing a coordinate system with the vehicle itself as the origin, and using an algorithm to associate targets in continuous frames. Compared with the existing technology, the present invention has the following beneficial effects: the present invention identifies abnormal traffic behavior by real-time acquisition and analysis of the motion trajectory data of human and vehicle, and divides the surrounding area into three levels of safety situation maps, providing a danger prediction function for the driver; in response to different road friction coefficients, the relevant parameters are adjusted so that the three-level safety situation map division is applicable to different environments; in response to the characteristic mode of overtaking and emergency stop, dynamic processing is performed to ensure driving safety.

Description

Technical Field

[0001] The present invention belongs to the field of transportation, and in particular relates to a method and system for analyzing human and vehicle behavior trajectories for urban safety. Background Art

[0002] Traffic accidents are a frequent safety issue in modern society, and their mechanisms are significantly linked to abnormal traffic behavior. Research has shown that dangerous behaviors such as sudden braking, speeding, illegal driving on the wrong side of the road, and pedestrian jaywalking and ignoring traffic lights are key factors in traffic accidents.

[0003] If we can collect and analyze the movement trajectory data of people and vehicles in real time, identify abnormal traffic behaviors, and trigger a graded warning mechanism before an accident occurs, it will effectively improve the effectiveness of road safety protection and provide an innovative technical path for traffic accident prevention. Summary of the Invention

[0004] Based on this, it is necessary to provide a human and vehicle behavior trajectory analysis method and system for urban safety to address the above problems.

[0005] The embodiment of the present invention is implemented as follows: a method for analyzing the trajectory of human and vehicle behavior for urban safety includes the following steps:

[0006] Obtain the vehicle's location (latitude and longitude), speed, acceleration, and heading angle data, capture surrounding vehicles and pedestrians, and detect the relative speed and distance data between the vehicle and surrounding vehicles and pedestrians;

[0007] Establish a unified time base and construct a coordinate system with the ego vehicle as the origin (e.g., the center of the ego vehicle's front bumper). Use algorithms (e.g., Kalman filtering, Hungarian algorithm, and DeepSORT) to correlate targets in consecutive frames and generate the movement trajectories of surrounding vehicles and pedestrians.

[0008] Input the movement trajectories of the vehicle, surrounding vehicles, and pedestrians into a machine learning model (such as an LSTM-Transformer hybrid model) to predict the future movement trajectories of the vehicle, surrounding vehicles, and pedestrians.

[0009] Based on the future movement trajectories of the own vehicle, surrounding vehicles and pedestrians, a polar coordinate risk field model is constructed. The Monte Carlo method is used to calculate the collision probability of each grid cell, and a three-level safety situation map is output: safe zone (P<0.01), warning zone (0.01≤P<0.1), and danger zone (P≥0.1).

[0010] In one embodiment, the present invention provides a method for analyzing human and vehicle behavior trajectories for urban safety. The steps of establishing a unified time base, constructing a coordinate system with the vehicle as the origin (e.g., the center of the vehicle's front bumper), and using algorithms (e.g., Kalman filtering, Hungarian algorithm, and DeepSORT) to associate targets in consecutive frames to generate the movement trajectories of surrounding vehicles and pedestrians specifically include:

[0011] Use the Precision Time Protocol (PTP) to provide a unified clock source for sensors (sensors such as cameras and radars, which obtain the positions of the vehicle, surrounding vehicles, and pedestrians). For example, a camera has a frame rate of 30Hz, with one frame every 33ms, and a radar scan period of 50ms, all aligned to the same reference through timestamps).

[0012] Calculate the extrinsic parameter matrix of each sensor (camera, radar) relative to the vehicle, convert global coordinates (such as longitude and latitude) into local coordinates through UTM projection, and convert all target positions into a coordinate system with the vehicle as the origin;

[0013] The detection targets at different times are associated as the movement trajectory of the same object, and the movement trajectories of surrounding vehicles and pedestrians are generated.

[0014] In one embodiment, the present invention provides a method for analyzing human and vehicle behavior trajectories for urban safety. The step of inputting the movement trajectories of the vehicle, surrounding vehicles, and pedestrians into a machine learning model (such as an LSTM-Transformer hybrid model) and predicting the future movement trajectories of the vehicle, surrounding vehicles, and pedestrians specifically includes:

[0015] Input the movement trajectories of the vehicle, surrounding vehicles, and pedestrians into the input encoding layer of the machine learning model;

[0016] The LSTM timing module in the spatiotemporal fusion layer of the machine learning model independently runs a bidirectional LSTM for each target, capturing temporal behavior patterns (e.g., the egocar's LSTM learns the driver's operating habits). Temporal behavior patterns include sudden braking and lane changes. The Transformer interaction module in the spatiotemporal fusion layer uses the LSTM output vector of each target as input and uses a multi-head attention mechanism to model spatial relationships between targets (e.g., the correlation between the turn signal of the vehicle in front of you and a pedestrian crossing the road).

[0017] Through the multimodal prediction layer of the machine learning model, multiple possible trajectories of all targets in the next 3 seconds are output.

[0018] In one embodiment, the present invention provides a method for analyzing human and vehicle behavior trajectories for urban safety. The method constructs a polar coordinate risk field model based on the future trajectories of the own vehicle, surrounding vehicles, and pedestrians, uses the Monte Carlo method to calculate the collision probability of each grid cell, and outputs a three-level safety situation map: safe zone (P < 0.01), warning zone (0.01 ≤ P < 0.1), and danger zone (P ≥ 0.1). The steps specifically include:

[0019] Based on the future trajectories of the ego vehicle, surrounding vehicles, and pedestrians, the predicted acceleration, braking distance, and turning radius limits are adjusted for different road friction coefficients (e.g., μ = 0.7 for normal roads and μ = 0.1 for icy surfaces). A polar coordinate system is established with the ego vehicle as the center, and grid cells are divided (e.g., 0.5m radial spacing and 5° angular spacing) to construct a polar coordinate system risk field model.

[0020] A Monte Carlo method is used to generate a set of trajectory samples that conform to dynamic constraints. In low-friction scenarios, the probability of uncontrolled trajectories is corrected by increasing the sampling weight of slip trajectories (for example, when μ = 0.1 on ice, a 30% sampling probability is assigned to slip trajectories). Low-friction scenarios include ice surfaces.

[0021] Traverse all trajectory samples, count the number of spatial overlaps between the ego vehicle and surrounding target trajectories in each grid cell, calculate the collision probability, and output a three-level safety situation map: safe zone (P < 0.01), warning zone (0.01 ≤ P < 0.1), and danger zone (P ≥ 0.1).

[0022] In one embodiment, the present invention provides a method for analyzing human and vehicle behavior trajectories for urban safety. The method constructs a polar coordinate risk field model based on the future trajectories of the own vehicle, surrounding vehicles, and pedestrians, uses the Monte Carlo method to calculate the collision probability of each grid cell, and outputs a three-level safety situation map: safe zone (P < 0.01), warning zone (0.01 ≤ P < 0.1), and danger zone (P ≥ 0.1). After the steps, the method further includes:

[0023] Real-time analysis of the preceding vehicle's acceleration rate of change, throttle depth, and traffic light countdown identifies characteristic patterns of overtaking and sudden stops, predicting risks 1.5 seconds in advance. A dynamic Bayesian network is established to quantify the probability of sudden stops, integrating the road friction coefficient to correct prediction reliability.

[0024] An emergency response mode is introduced into the Monte Carlo method. When the preceding vehicle's sudden stop is detected, the time step is automatically shortened to 0.2 seconds, and 10 times more dense sampling is performed within a 5-meter area to accurately capture short-term high-risk areas.

[0025] Build a dynamic response based on the remaining collision time. In the prediction stage (outside 1.5 seconds), a warning will flash; in the critical stage (1.5 seconds-0.8 seconds), the seat belt will be automatically tightened and pre-pressurized for braking; in the emergency stage (within 0.8 seconds), automatic emergency braking (AEB) will be triggered and the electronic stability system will be activated, and a detour path will be generated simultaneously to guide the direction of avoidance.

[0026] In one embodiment, the present invention provides a human and vehicle behavior trajectory analysis system for urban safety, including:

[0027] The data acquisition unit is used to obtain the vehicle's position (latitude and longitude), speed, acceleration, and heading angle data, capture surrounding vehicles and pedestrians, and detect the relative speed and distance data between the vehicle and surrounding vehicles and pedestrians;

[0028] The motion trajectory acquisition unit is used to establish a unified time base, construct a coordinate system with the ego vehicle as the origin (such as the center of the ego vehicle's front bumper), and use algorithms (such as Kalman filtering, Hungarian algorithm, and DeepSORT) to associate targets in consecutive frames to generate the motion trajectories of surrounding vehicles and pedestrians.

[0029] The trajectory prediction unit is used to input the trajectory of the vehicle, surrounding vehicles, and pedestrians into a machine learning model (such as an LSTM-Transformer hybrid model) to predict the future trajectory of the vehicle, surrounding vehicles, and pedestrians;

[0030] The safety judgment unit is used to construct a polar coordinate risk field model based on the future movement trajectories of the own vehicle, surrounding vehicles and pedestrians, use the Monte Carlo method to calculate the collision probability of each grid cell, and output a three-level safety situation map: safe zone (P < 0.01), warning zone (0.01 ≤ P < 0.1), and danger zone (P ≥ 0.1).

[0031] In one embodiment, the present invention provides a human and vehicle behavior trajectory analysis system for urban safety, wherein the action trajectory acquisition unit includes:

[0032] The time synchronization subunit uses the Precision Time Protocol (PTP) to provide a unified clock source for sensors (such as cameras and radars, which acquire the positions of the vehicle, surrounding vehicles, and pedestrians). For example, a camera has a frame rate of 30Hz, with one frame every 33ms, and a radar has a scan period of 50ms, all aligned to the same reference through timestamps).

[0033] The spatial synchronization subunit is used to calculate the external parameter matrix of each sensor (camera, radar) relative to the ego vehicle, convert the global coordinates (such as longitude and latitude) into local coordinates through UTM projection, and convert all target positions into a coordinate system with the ego vehicle as the origin;

[0034] The motion trajectory generation subunit is used to associate the detection targets at different times into the motion trajectory of the same object and generate the motion trajectories of surrounding vehicles and pedestrians.

[0035] In one embodiment, the present invention provides a human and vehicle behavior trajectory analysis system for urban safety, wherein the action trajectory prediction unit includes:

[0036] The data input subunit is used to input the movement trajectories of the vehicle, surrounding vehicles, and pedestrians into the input encoding layer of the machine learning model;

[0037] The data processing subunit is used to independently run a bidirectional LSTM for each target through the LSTM timing module in the spatiotemporal fusion layer of the machine learning model to capture temporal behavior patterns (for example, the egocar LSTM learns the driver's operating habits). Temporal behavior patterns include sudden braking and lane changes. The Transformer interaction module in the spatiotemporal fusion layer uses the LSTM output vector of each target as input and uses a multi-head attention mechanism to model the spatial relationship between targets (for example, the correlation between the turn signal of the vehicle in front of the right and the pedestrian crossing the road).

[0038] The data output subunit is used to output multiple possible trajectories of all targets in the next 3 seconds through the multimodal prediction layer of the machine learning model.

[0039] In one embodiment, the present invention provides a human and vehicle behavior trajectory analysis system for urban safety, wherein the safety judgment unit includes:

[0040] The grid unit is divided into sub-units. This is used to adjust the acceleration, braking distance, and turning radius limits of the trajectory prediction based on the future movement trajectory of the ego vehicle, surrounding vehicles, and pedestrians, for different road friction coefficients (e.g., μ = 0.7 for normal roads and μ = 0.1 for ice surfaces). A polar coordinate system is established with the ego vehicle as the center, and the grid units are divided (e.g., 0.5m radial spacing and 5° angular spacing) to construct a polar coordinate system risk field model.

[0041] The trajectory sample set acquisition subunit uses the Monte Carlo method to generate trajectory sample sets that meet dynamic constraints. In low-friction scenarios, the probability of uncontrolled trajectories is corrected by increasing the sampling weight of slip trajectories (for example, when μ = 0.1 on ice, a 30% sampling probability is given to slip trajectories). Low-friction scenarios include ice surfaces.

[0042] The grid unit safety judgment subunit is used to traverse all trajectory samples, count the number of spatial overlaps between the ego vehicle and surrounding target trajectories in each grid unit, calculate the collision probability, and output a three-level safety situation map: safe zone (P < 0.01), warning zone (0.01 ≤ P < 0.1), and danger zone (P ≥ 0.1).

[0043] In one embodiment, the present invention provides a human and vehicle behavior trajectory analysis system for urban safety, further comprising:

[0044] The cut-over and emergency stop detection unit analyzes the acceleration change rate, throttle depth, and signal light countdown of the preceding vehicle in real time to identify characteristic patterns of cut-over and emergency stops, predicting risks 1.5 seconds in advance. It establishes a dynamic Bayesian network to quantify the probability of emergency stops and integrates the road friction coefficient to correct the prediction reliability.

[0045] The emergency response unit is used to introduce an emergency response mode into the Monte Carlo method. When the sudden stop feature of the preceding vehicle is detected, the time step is automatically shortened to 0.2 seconds, and the area within 5 meters is sampled 10 times more densely to accurately capture short-term high-risk areas;

[0046] The dynamic processing unit is used to build a dynamic response based on the remaining collision time. In the prediction stage (outside 1.5 seconds), a flashing warning is issued; in the critical stage (1.5 seconds to 0.8 seconds), the seat belt is automatically tightened and pre-pressurized for braking. In the emergency stage (within 0.8 seconds), automatic emergency braking (AEB) is triggered and the electronic stability system is activated, and a detour path is simultaneously generated to guide the direction of avoidance.

[0047] Compared with the existing technology, the beneficial effects of the present invention are: the present invention collects and analyzes the movement trajectory data of people and vehicles in real time, identifies abnormal traffic behaviors, divides the surrounding area into three levels of safety situation maps, and provides drivers with danger prediction functions; in the face of different road friction coefficients, adjusts relevant parameters to make the three-level safety situation map division applicable to different environments; in the face of overtaking-emergency stop feature modes, dynamic processing is performed to ensure driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flow chart of the first part of the method for analyzing pedestrian and vehicle behavior trajectories for urban safety provided by an embodiment of the present invention.

[0049] Figure 2 A schematic diagram of the process of generating movement trajectories of surrounding vehicles and pedestrians provided by an embodiment of the present invention.

[0050] Figure 3 A schematic diagram of a process for predicting the future movement trajectories of a vehicle, surrounding vehicles, and pedestrians provided in an embodiment of the present invention.

[0051] Figure 4 A schematic diagram of a process for providing security instructions according to an embodiment of the present invention.

[0052] Figure 5 This is a flow chart of the second part of the method for analyzing pedestrian and vehicle behavior trajectories for urban safety provided by an embodiment of the present invention.

[0053] Figure 6This is a schematic diagram of the first part of the human and vehicle behavior trajectory analysis system for urban safety provided by an embodiment of the present invention.

[0054] Figure 7 A schematic diagram of a motion trajectory acquisition unit provided in an embodiment of the present invention.

[0055] Figure 8 A schematic diagram of a motion trajectory prediction unit provided in an embodiment of the present invention.

[0056] Figure 9 A schematic diagram of a security determination unit provided in an embodiment of the present invention.

[0057] Figure 10 This is a schematic diagram of the second part of the human and vehicle behavior trajectory analysis system for urban safety provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0059] In one embodiment, Figure 1 As shown in FIG, the human-vehicle behavior trajectory analysis method for urban safety includes the following steps:

[0060] Step S1: Acquire the vehicle's position (latitude and longitude), speed, acceleration, and heading angle data, capture surrounding vehicles and pedestrians, and detect the relative speed and distance data between the vehicle and surrounding vehicles and pedestrians;

[0061] Step S2: Establish a unified time base and construct a coordinate system with the ego vehicle as the origin (e.g., the center of the ego vehicle's front bumper). Use algorithms (e.g., Kalman filtering, Hungarian algorithm, and DeepSORT) to correlate targets in consecutive frames and generate the movement trajectories of surrounding vehicles and pedestrians.

[0062] Step S3: Input the movement trajectories of the vehicle, surrounding vehicles, and pedestrians into a machine learning model (e.g., an LSTM-Transformer hybrid model) to predict the future movement trajectories of the vehicle, surrounding vehicles, and pedestrians.

[0063] In step S4, a polar coordinate risk field model is constructed based on the future movement trajectories of the own vehicle, surrounding vehicles, and pedestrians. The Monte Carlo method is used to calculate the collision probability of each grid cell, and a three-level safety situation map is output: safe zone (P < 0.01), warning zone (0.01 ≤ P < 0.1), and danger zone (P ≥ 0.1).

[0064] The Beidou satellite positioning system directly outputs longitude and latitude coordinates, and the wheel speedometer measures wheel speed and converts it into vehicle speed; the inertial measurement unit (IMU) has a built-in three-axis accelerometer and gyroscope, which directly outputs the vehicle's three-axis acceleration and angular velocity; the electronic compass or IMU's built-in magnetometer directly outputs the vehicle's azimuth relative to the geographic North Pole, obtaining relevant data about the vehicle itself; the camera outputs the coordinates of the detected pedestrian and vehicle bounding boxes in real time through the embedded image processing chip; the millimeter wave radar directly outputs the target distance, relative speed and azimuth through the time difference and frequency shift between the transmission and reception of electromagnetic waves; the laser radar (LiDAR) generates raw point cloud data through the time of flight (ToF) method, which includes the three-dimensional coordinates of surrounding obstacles; the ultrasonic radar directly outputs the distance to obstacles within 0.2-5 meters based on the sound wave reflection time, and detects the relative speed and distance data of the vehicle itself and surrounding vehicles and pedestrians. The data is unified in time and space, and a dynamic coordinate system is constructed with the vehicle as the origin. This generates continuous trajectories of surrounding vehicles and pedestrians. These trajectories are then fed into a machine learning model to predict the future trajectories of the vehicle, surrounding vehicles, and pedestrians. Based on these future trajectories, the safety of the area surrounding the vehicle is determined. The surrounding grids are divided into safe zones, warning zones, and danger zones based on collision probability, providing the driver with a hazard prediction function. A safe zone is defined as P < 0.01, a warning zone is defined as 0.01 ≤ P < 0.1, and a danger zone is defined as P ≥ 0.1, where P is the collision probability.

[0065] The 0.01 threshold in the safety zone corresponds to the ASIL D requirement for random hardware failure probability in automotive electronic systems in ISO 26262 (automotive functional safety standard). It also matches the statistical significance boundary of 10,000 samples (95% confidence level) in Monte Carlo simulation, avoiding false alarms caused by low-probability, long-tail distributions.

[0066] The 0.1 threshold upper limit of the warning zone is derived from the nonlinear response characteristics of human drivers to risk perception. Research shows that when the probability of collision exceeds 10%, the driver's stress reaction time is shortened by 30% (MIT 2019 experimental data). This threshold can trigger non-invasive interventions such as pre-braking.

[0067] Danger zone 0.1 critical value: Based on the vehicle's dynamic limits, when the vehicle is traveling at 60 km / h, a 10% collision probability is equivalent to a remaining collision avoidance time of ≤ 1.2 seconds (considering a maximum deceleration of 0.3g), which creates time redundancy with the minimum activation time of the AEB (automatic emergency braking) system (0.5-0.8 seconds).

[0068] In one embodiment, Figure 2As shown, step S2, establishing a unified time reference, constructing a coordinate system with the vehicle as the origin (such as the center of the vehicle's front bumper), using algorithms (such as Kalman filtering, Hungarian algorithm, and DeepSORT) to associate targets in consecutive frames, and generating movement trajectories of surrounding vehicles and pedestrians, specifically includes:

[0069] Step S21: Use the Precision Time Protocol (PTP) to provide a unified clock source for sensors (sensors such as cameras and radars, which acquire the positions of the vehicle, surrounding vehicles, and pedestrians, for example: a camera with a frame rate of 30 Hz, one frame every 33 ms, and a radar with a scanning period of 50 ms, are aligned to the same reference through timestamps).

[0070] Step S22: Calculate the extrinsic parameter matrix of each sensor (camera, radar) relative to the ego vehicle, convert the global coordinates (such as longitude and latitude) into local coordinates using UTM projection, and convert all target positions into a coordinate system with the ego vehicle as the origin;

[0071] Step S23 , associating the detected targets at different times into the movement trajectory of the same object, and generating the movement trajectory of surrounding vehicles and pedestrians.

[0072] Step S23 involves first using the Kalman filter to predict the target's motion state in consecutive frames (updating the position and velocity covariance matrices), compensating for positioning errors caused by sensor noise and occlusion. Next, the Hungarian algorithm is used to optimally match the detected frame in the current frame with the predicted frame (using a cost matrix constructed based on Mahalanobis distance or Intersection over Union), addressing the cross-frame object ID association problem. Finally, DeepSORT's deep learning features (such as the appearance embedding vector extracted by the ReID network) are introduced to fuse motion similarity and appearance similarity (weighted by cosine distance) during the data association phase, effectively addressing object occlusion and morphological changes in dense scenes. These three methods work together: the Kalman filter provides a dynamic prediction benchmark, the Hungarian algorithm performs coarse-grained matching, and DeepSORT achieves fine-grained tracking through feature enhancement, ultimately outputting a stable and continuous trajectory sequence.

[0073] In one embodiment, Figure 3 As shown, step S3, inputting the movement trajectories of the vehicle, surrounding vehicles, and pedestrians into a machine learning model (such as an LSTM-Transformer hybrid model) to predict the future movement trajectories of the vehicle, surrounding vehicles, and pedestrians, specifically includes:

[0074] Step S31, inputting the movement trajectories of the vehicle, surrounding vehicles and pedestrians into the input coding layer of the machine learning model;

[0075] In step S32, the LSTM timing module in the spatiotemporal fusion layer of the machine learning model independently runs a bidirectional LSTM for each target to capture temporal behavior patterns (e.g., the egocar LSTM learns the driver's operating habits). Temporal behavior patterns include sudden braking and lane changes. The Transformer interaction module in the spatiotemporal fusion layer uses the LSTM output vector of each target as input and uses a multi-head attention mechanism to model the spatial relationship between targets (e.g., the correlation between the turn signal of the vehicle in front of the right and the pedestrian crossing the road).

[0076] Step S33: Output multiple possible trajectories of all targets within the next 3 seconds through the multimodal prediction layer of the machine learning model.

[0077] The LSTM-Transformer hybrid model was chosen here because it takes into account the complementary nature of temporal dependencies and spatial interactions. The LSTM, through its recurrent structure (e.g., bidirectional design), independently models the long-range temporal characteristics of each target (e.g., the inertial delay of a vehicle braking suddenly or the gait cycle of a pedestrian), effectively capturing individual dynamic patterns. The Transformer, on the other hand, uses multi-head attention (e.g., encoding relative positions across targets) to globally analyze complex spatial relationships (e.g., chain avoidance caused by adjacent vehicles changing lanes), addressing the synchronous interactions that the LSTM's serial processing often overlooks. This combination preserves the temporal continuity of individual behaviors while explicitly quantifying the game-playing relationships of group motion. This balances independence and collaboration in multi-target trajectory prediction tasks, improving the accuracy of modeling complex traffic scenarios.

[0078] In one embodiment, Figure 4 As shown, step S4 constructs a polar coordinate risk field model based on the future movement trajectories of the own vehicle, surrounding vehicles, and pedestrians, calculates the collision probability of each grid cell using the Monte Carlo method, and outputs a three-level safety situation map: safe zone (P<0.01), warning zone (0.01≤P<0.1), and danger zone (P≥0.1). The steps specifically include:

[0079] Step S41: Based on the future trajectories of the ego vehicle, surrounding vehicles, and pedestrians, and for different road friction coefficients (e.g., μ = 0.7 for normal roads and μ = 0.1 for icy surfaces), the acceleration, braking distance, and turning radius limits of the trajectory prediction are adjusted. A polar coordinate system is established with the ego vehicle as the center, and grid cells are divided (e.g., with 0.5m radial spacing and 5° angular spacing) to construct a polar coordinate system risk field model.

[0080] Step S42: Generate a trajectory sample set that meets the dynamic constraints using the Monte Carlo method. In low-friction scenarios, the probability of uncontrolled trajectories is corrected by increasing the sampling weight of slip trajectories (e.g., when μ = 0.1 on ice, a 30% sampling probability is assigned to slip trajectories). Low-friction scenarios include ice surfaces.

[0081] In step S43, all trajectory samples are traversed, the number of spatial overlaps between the ego vehicle and surrounding target trajectories in each grid cell is counted, the collision probability is calculated, and a three-level safety situation map is output: safe zone (P < 0.01), warning zone (0.01 ≤ P < 0.1), and danger zone (P ≥ 0.1).

[0082] In low-friction scenarios (e.g., icy surfaces with μ = 0.1), the sampling weighting of slip trajectories is adjusted based on the principle of a positive correlation between dynamic constraints and loss of control probability. First, a threshold is set based on the road friction coefficient μ (e.g., μ < 0.3 indicates a low-friction scenario). A friction coefficient-slip probability mapping function is then used (e.g., a 30% weight is assigned when μ = 0.1) to dynamically increase the sampling ratio of loss-of-control trajectories such as sideslip and drift. During Monte Carlo sampling, the tire force saturation threshold is calculated based on the μ value. For trajectories exceeding the grip limit (e.g., lateral acceleration exceeding μg), their sampling weight is adjusted inversely proportional to the μ value (e.g., when μ = 0.1, the weight of the originally filtered slip trajectories is increased to 30%). Furthermore, longitudinal acceleration and steering curvature parameters are constrained, ensuring that sampling conforms to physical laws while enhancing risk exposure in low-adhesion scenarios. By adjusting relevant parameters for varying road friction coefficients, the three-level safety situation map is adapted to diverse environments, such as the extensive icy conditions of Northeast China during winter.

[0083] In one embodiment, Figure 5 As shown, step S4, based on the future movement trajectories of the own vehicle, surrounding vehicles and pedestrians, constructs a polar coordinate system risk field model, calculates the collision probability of each grid cell using the Monte Carlo method, and outputs a three-level safety situation map: safe zone (P < 0.01), warning zone (0.01 ≤ P < 0.1), and danger zone (P ≥ 0.1). After this step, the following steps are also included:

[0084] Step S5: Real-time analysis of the preceding vehicle's acceleration rate of change, throttle depth, and traffic light countdown to identify overtaking and sudden stop characteristic patterns, predicting risks 1.5 seconds in advance, establishing a dynamic Bayesian network to quantify sudden stop probability, and integrating the road friction coefficient to correct prediction reliability.

[0085] Step S6: Introducing an emergency response mode into the Monte Carlo method. When the preceding vehicle's sudden stop feature is detected, the time step is automatically shortened to 0.2 seconds, and 10 times dense sampling is performed on the area within 5 meters to accurately capture short-term high-risk areas.

[0086] Step S7 constructs a dynamic response based on the remaining collision time. In the prediction phase (outside 1.5 seconds), a warning light flashes; in the critical phase (1.5 seconds to 0.8 seconds), the seat belt is automatically tightened and pre-pressurized for braking; in the emergency phase (within 0.8 seconds), automatic emergency braking (AEB) is triggered and the electronic stability system is activated, and a detour path is simultaneously generated to guide the direction of avoidance.

[0087] In conventional traffic scenarios, the driver of the vehicle ahead may suddenly accelerate to rush through an intersection when the yellow light appears. However, due to a misjudgment or the signal light switching too quickly, the vehicle cannot safely stop before the stop line, which triggers emergency braking and easily causes traffic accidents. This application identifies the characteristic pattern of the vehicle ahead rushing and stopping suddenly, predicts the risk, and the relevant numerical reasons for the design are:

[0088] 1.5-second prediction threshold: Based on the average human driver reaction time (approximately 1.2 seconds) and the system perception-decision delay (approximately 0.3 seconds), this ensures time redundancy between risk warning and driver action, while also covering the physical limit of braking distance (approximately 12 meters) at typical urban speeds (e.g., 30 km / h).

[0089] ‌0.2-second time step‌: This matches the response delay of the emergency braking system (e.g., hydraulic pressure buildup takes 0.1-0.3 seconds). High-frequency sampling is used to capture sudden acceleration changes (e.g., a collision risk within 0.8 seconds caused by a sudden stop of the preceding vehicle), thus avoiding the smoothing distortion of short-term dynamics caused by the traditional 1-second time step.

[0090] ‌5-meter dense sampling area‌: corresponds to the maximum braking distance and sensor blind spot coverage at low vehicle speeds (<20 km / h). 10x sampling weighting enhances the collision probability resolution of near-field trajectories (e.g., confidence level >95% under Poisson distribution) while avoiding computational overhead caused by global sampling.

[0091] ‌0.8-second emergency stage threshold‌: Based on the AEB (Automatic Emergency Braking) system's minimum activation time (0.5-second mechanical response + 0.3-second path planning), combined with the required deceleration of 2 m / s² on icy surfaces (μ=0.1), ensuring that the vehicle can be stopped within the remaining distance.

[0092] In one embodiment, Figure 6 As shown in the figure, the pedestrian and vehicle behavior trajectory analysis system for urban safety includes:

[0093] Data acquisition unit 1, used to obtain the vehicle's position (latitude and longitude), speed, acceleration, and heading angle data, capture surrounding vehicles and pedestrians, and detect the relative speed and distance data between the vehicle and surrounding vehicles and pedestrians;

[0094] The motion trajectory acquisition unit 2 is used to establish a unified time reference, construct a coordinate system with the ego vehicle as the origin (such as the center of the ego vehicle's front bumper), and use algorithms (such as Kalman filtering, Hungarian algorithm, and DeepSORT) to associate targets in consecutive frames to generate the motion trajectories of surrounding vehicles and pedestrians.

[0095] The trajectory prediction unit 3 is used to input the trajectory of the vehicle, surrounding vehicles and pedestrians into a machine learning model (such as an LSTM-Transformer hybrid model) to predict the future trajectory of the vehicle, surrounding vehicles and pedestrians;

[0096] Safety judgment unit 4 is used to construct a polar coordinate risk field model based on the future movement trajectories of the own vehicle, surrounding vehicles and pedestrians, use the Monte Carlo method to calculate the collision probability of each grid cell, and output a three-level safety situation map: safe zone (P < 0.01), warning zone (0.01 ≤ P < 0.1), and danger zone (P ≥ 0.1).

[0097] Data acquisition unit 1‌ acquires real-time raw data from the vehicle and surrounding targets through sensors, providing input for subsequent analysis. The motion trajectory acquisition unit 2‌ converts discrete data into continuous trajectories based on spatiotemporal synchronization and target tracking algorithms, establishing a dynamic environment model. The motion trajectory prediction unit 3‌ uses trajectory data to train a time series prediction model, inferring future motion trends and providing a predictive basis for risk assessment. The safety judgment unit 4‌ combines the predicted trajectory with physical rules to construct a risk quantification model, outputting a layered safety map and forming a closed-loop decision-making process.

[0098] In one embodiment, Figure 7 As shown, the motion trajectory acquisition unit 2 includes:

[0099] The time synchronization subunit 21 uses the Precision Time Protocol (PTP) to provide a unified clock source for sensors (sensors such as cameras and radars, which obtain the positions of the vehicle, surrounding vehicles, and pedestrians. For example, a camera has a frame rate of 30Hz, with one frame every 33ms, and a radar scan period of 50ms, and they are aligned to the same reference through timestamps).

[0100] The spatial synchronization subunit 22 is used to calculate the extrinsic parameter matrix of each sensor (camera, radar) relative to the ego vehicle, convert the global coordinates (such as longitude and latitude) into local coordinates through UTM projection, and convert all target positions into a coordinate system with the ego vehicle as the origin;

[0101] The motion trajectory generating subunit 23 is used to associate the detection targets at different times into the motion trajectory of the same object, and generate the motion trajectories of surrounding vehicles and pedestrians.

[0102] The time synchronization subunit 21 uses the PTP protocol to unify the clock sources of all sensors, eliminating time misalignment between devices (for example, aligning camera clocks at 33ms / frame with radar clocks at 50ms / cycle), ensuring temporal consistency of multi-source data. The spatial synchronization subunit 22 uses sensor extrinsics and coordinate transformations (such as UTM projection) to uniformly map global coordinates and sensor-detected targets to a local coordinate system with the vehicle as the origin, addressing spatial reference discrepancies. The trajectory generation subunit 23 leverages the spatiotemporal alignment data output by the previous two steps and uses target association algorithms (such as Kalman filtering, Hungarian algorithm, and DeepSORT) to match the same object across frames, ultimately generating continuous and stable motion trajectories of surrounding targets. These three components are interconnected in a step-by-step manner: time synchronization is a prerequisite for spatial alignment, which in turn forms the foundation for trajectory modeling. Together, they achieve spatiotemporal consistency in environmental perception.

[0103] In one embodiment, Figure 8 As shown, the action trajectory prediction unit 3 includes:

[0104] A data input subunit 31 is used to input the movement trajectories of the vehicle, surrounding vehicles, and pedestrians into the input encoding layer of the machine learning model;

[0105] The data processing subunit 32 is configured to independently run a bidirectional LSTM for each target through the LSTM timing module in the spatiotemporal fusion layer of the machine learning model to capture temporal behavior patterns (e.g., the egocar LSTM learns the driver's operating habits). Temporal behavior patterns include sudden braking and lane changes. The Transformer interaction module in the spatiotemporal fusion layer uses the LSTM output vector of each target as input and models the spatial relationship between targets through a multi-head attention mechanism (e.g., the correlation between the turn signal of the vehicle in front of the right and the pedestrian crossing the road).

[0106] The data output subunit 33 is used to output multiple possible trajectories of all targets in the next 3 seconds through the multimodal prediction layer of the machine learning model.

[0107] The data input subunit 31 encodes the spatiotemporally aligned trajectory data into a time series vector that can be processed by the model, providing structured input for subsequent feature extraction. The data processing subunit 32 independently models the temporal behavior of each target through a bidirectional LSTM (such as the dynamic patterns of a vehicle braking suddenly or a pedestrian crossing the road). The Transformer multi-head attention mechanism then captures the spatial interaction relationship across targets (such as the correlation between vehicle lane changes and pedestrian movement), achieving a deep fusion of temporal features and spatial associations. The data output subunit 33 decodes and generates the probability distribution of future trajectories based on the fused multimodal features.

[0108] In one embodiment, Figure 9 As shown, the safety judgment unit 4 includes:

[0109] The grid cell division sub-unit 41 is used to adjust the acceleration, braking distance, and turning radius limits of the trajectory prediction based on the future movement trajectories of the ego vehicle, surrounding vehicles, and pedestrians, for different road friction coefficients (e.g., μ = 0.7 for normal roads and μ = 0.1 for ice surfaces). A polar coordinate system is established with the ego vehicle as the center, and grid cells are divided (e.g., with 0.5m radial spacing and 5° angular spacing) to construct a polar coordinate system risk field model.

[0110] The trajectory sample set acquisition subunit 42 generates a trajectory sample set that meets the dynamic constraints using the Monte Carlo method. In low-friction coefficient scenarios, the probability of the uncontrolled trajectory is corrected by increasing the sampling weight of the slip trajectory (e.g., when μ = 0.1 on ice, the slip trajectory is given a 30% sampling probability). Low-friction coefficient scenarios include ice surfaces.

[0111] The grid unit safety judgment subunit 43 is used to traverse all trajectory samples, count the number of spatial overlaps between the ego vehicle and surrounding target trajectories in each grid unit, calculate the collision probability, and output a three-level safety situation map: safe zone (P < 0.01), warning zone (0.01 ≤ P < 0.1), and danger zone (P ≥ 0.1).

[0112] The grid unit division subunit 41 dynamically adjusts trajectory dynamic constraints based on the road friction coefficient (such as reducing the braking distance limit on icy surfaces) and constructs a polar coordinate grid centered on the vehicle, providing a spatial quantification framework for the risk field. The trajectory sample set acquisition subunit 42 uses the Monte Carlo method to generate a trajectory sample set that meets the current road conditions (such as focusing on slip trajectory sampling on icy surfaces) based on the physical constraints of the grid division (such as the turning radius limit), and incorporates probability corrections into trajectory generation. The grid unit safety judgment subunit 43 traverses all trajectory samples, counts the frequency of spatial overlap within the polar coordinate grid, and outputs a collision probability map.

[0113] In one embodiment, Figure 10 As shown in the figure, the pedestrian and vehicle behavior trajectory analysis system for urban safety also includes:

[0114] Cutting-over and emergency stop detection unit 5 is used to analyze the acceleration change rate, throttle depth and signal light countdown of the leading vehicle in real time, identify the characteristic pattern of cutting-over and emergency stop, predict the risk 1.5 seconds in advance, establish a dynamic Bayesian network to quantify the probability of emergency stop, and integrate the road friction coefficient to correct the prediction reliability;

[0115] Emergency Response Unit 6, used to introduce an emergency response mode into the Monte Carlo method. When an emergency stop feature of the preceding vehicle is detected, the time step is automatically shortened to 0.2 seconds, and 10 times denser sampling is performed within a 5-meter area to accurately capture short-term high-risk areas;

[0116] Dynamic processing unit 7 is used to build a dynamic response based on the remaining collision time. In the prediction stage (outside 1.5 seconds), a flashing warning is performed; in the critical stage (1.5 seconds to 0.8 seconds), the seat belt is automatically tightened and pre-pressurized for braking. In the emergency stage (within 0.8 seconds), automatic emergency braking (AEB) is triggered and the electronic stability system is activated, and a detour path is simultaneously generated to guide the direction of avoidance.

[0117] The overtaking emergency stop detection unit 5‌ integrates vehicle dynamics and traffic light timing (such as sudden acceleration changes of the preceding vehicle and yellow light countdown) through a dynamic Bayesian network to quantify the probability of an emergency stop risk in real time. It then modifies the prediction confidence based on the road friction coefficient (e.g., μ = 0.1 for ice), providing a risk trigger signal for subsequent responses. Based on the detection results, the emergency response unit 6‌ dynamically switches sampling modes in Monte Carlo trajectory prediction (e.g., a high-frequency step length of 0.2 seconds and a 10x sampling density in a 5-meter area), focusing computing power on high-risk spatiotemporal ranges to generate refined obstacle avoidance boundaries. The dynamic processing unit 7‌ combines the remaining collision time with the trajectory prediction results to activate the human-machine co-driving strategy in stages. In the prediction stage, visual warnings are used to retain the driver's initiative. In the critical stage, pre-pressurized braking and seatbelt tightening are used to reduce collision damage. In the emergency stage, AEB and detour path planning are triggered to achieve active avoidance. The three form a closed loop of risk perception-prediction reinforcement-graded control, and achieve a smooth transition from probabilistic prediction to physical execution through progressive response in the time dimension (1.5 seconds → 0.8 seconds → instantaneous) and dynamic focusing in the spatial dimension (global → high-risk area).

[0118] It should be understood that, although the various steps in the flow chart of each embodiment of the present invention are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in order, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0119] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0120] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

[0121] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0122] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A human-vehicle behavior trajectory analysis method for urban safety, characterized by: The human-vehicle behavior trajectory analysis method for urban safety includes the following steps: Obtain the vehicle's position, speed, acceleration, and heading angle data, capture surrounding vehicles and pedestrians, and detect the relative speed and distance data between the vehicle and surrounding vehicles and pedestrians; Establish a unified time base, construct a coordinate system with the vehicle as the origin, use an algorithm to associate targets in consecutive frames, and generate the movement trajectories of surrounding vehicles and pedestrians; Input the movement trajectories of the vehicle, surrounding vehicles, and pedestrians into the machine learning model to predict the future movement trajectories of the vehicle, surrounding vehicles, and pedestrians; Based on the future movement trajectories of the vehicle, surrounding vehicles, and pedestrians, a polar coordinate risk field model is constructed. The Monte Carlo method is used to calculate the collision probability of each grid cell, and a three-level safety situation map is output: safe zone, warning zone, and danger zone. Real-time analysis of the preceding vehicle's acceleration rate of change, throttle depth, and traffic light countdown identifies characteristic patterns of overtaking and sudden stops, predicting risks 1.5 seconds in advance. A dynamic Bayesian network is established to quantify the probability of sudden stops, integrating the road friction coefficient to correct prediction reliability. An emergency response mode is introduced into the Monte Carlo method. When the preceding vehicle's sudden stop is detected, the time step is automatically shortened to 0.2 seconds, and 10 times more dense sampling is performed within a 5-meter area to accurately capture short-term high-risk areas. Build a dynamic response based on the remaining collision time. In the prediction phase, a flashing warning is issued; in the critical phase, the seat belt is automatically tightened and pre-pressurized braking is controlled; in the emergency phase, automatic emergency braking is triggered and the electronic stability system is activated, and a detour path is simultaneously generated to guide the direction of avoidance. The polar coordinate risk field model is constructed based on the future movement trajectories of the own vehicle, surrounding vehicles and pedestrians, and the collision probability of each grid cell is calculated using the Monte Carlo method. The three-level safety situation map is output: safe zone, warning zone, and danger zone. The steps specifically include: Based on the future trajectories of the ego vehicle, surrounding vehicles, and pedestrians, the acceleration, braking distance, and turning radius limits of the trajectory prediction are adjusted for different road friction coefficients. A polar coordinate system is established with the ego vehicle as the center, and grid cells are divided to construct a polar coordinate system risk field model. A Monte Carlo method is used to generate a set of trajectory samples that meet dynamic constraints. In low-friction scenarios, the probability of uncontrolled trajectories is corrected by increasing the sampling weight of slip trajectories. Low-friction scenarios include ice surfaces. Traverse all trajectory samples, count the number of times the vehicle and surrounding target trajectories overlap in each grid cell, calculate the collision probability, and output a three-level safety situation map: safe zone, warning zone, and danger zone.

2. The method for analyzing human and vehicle behavior trajectories for urban safety according to claim 1 is characterized in that: The steps of establishing a unified time reference, constructing a coordinate system with the vehicle as the origin, and using an algorithm to associate targets in consecutive frames to generate movement trajectories of surrounding vehicles and pedestrians specifically include: Use the Precision Time Protocol to provide a unified clock source for sensors; Calculate the extrinsic parameter matrix of each sensor relative to the ego vehicle, convert the global coordinates into local coordinates through UTM projection, and transform all target positions into a coordinate system with the ego vehicle as the origin; The detection targets at different times are associated as the movement trajectory of the same object, and the movement trajectories of surrounding vehicles and pedestrians are generated.

3. The method for analyzing human and vehicle behavior trajectories for urban safety according to claim 1 is characterized in that: The step of inputting the movement trajectories of the vehicle, surrounding vehicles, and pedestrians into the machine learning model to predict the future movement trajectories of the vehicle, surrounding vehicles, and pedestrians specifically includes: Input the movement trajectories of the vehicle, surrounding vehicles, and pedestrians into the input encoding layer of the machine learning model; The LSTM timing module in the spatiotemporal fusion layer of the machine learning model independently runs a bidirectional LSTM for each target to capture temporal behavior patterns, such as sudden braking and lane changes. The Transformer interaction module in the spatiotemporal fusion layer uses the LSTM output vector of each target as input and models the spatial relationship between targets using a multi-head attention mechanism. Through the multimodal prediction layer of the machine learning model, multiple possible trajectories of all targets in the next 3 seconds are output.

4. The pedestrian and vehicle behavior trajectory analysis system for urban safety is characterized by: The human and vehicle behavior trajectory analysis system for urban safety includes: The data acquisition unit is used to obtain the position, speed, acceleration, and heading angle data of the vehicle, capture surrounding vehicles and pedestrians, and detect the relative speed and distance data between the vehicle and surrounding vehicles and pedestrians; The motion trajectory acquisition unit is used to establish a unified time base, construct a coordinate system with the vehicle as the origin, use an algorithm to associate targets in consecutive frames, and generate the motion trajectories of surrounding vehicles and pedestrians; The motion trajectory prediction unit is used to input the motion trajectories of the vehicle, surrounding vehicles, and pedestrians into the machine learning model to predict the future motion trajectories of the vehicle, surrounding vehicles, and pedestrians; The safety judgment unit is used to construct a polar coordinate risk field model based on the future movement trajectories of the own vehicle, surrounding vehicles, and pedestrians. It uses the Monte Carlo method to calculate the collision probability of each grid cell and output a three-level safety situation map: safe zone, warning zone, and danger zone. The cut-over and emergency stop detection unit analyzes the acceleration change rate, throttle depth, and signal light countdown of the preceding vehicle in real time to identify characteristic patterns of cut-over and emergency stops, predicting risks 1.5 seconds in advance. It establishes a dynamic Bayesian network to quantify the probability of emergency stops and integrates the road friction coefficient to correct the prediction reliability. The emergency response unit is used to introduce an emergency response mode into the Monte Carlo method. When the preceding vehicle's sudden stop feature is detected, the time step is automatically shortened to 0.2 seconds, and the area within 5 meters is sampled 10 times more densely to accurately capture short-term high-risk areas; The dynamic processing unit is used to build a dynamic response based on the remaining collision time. In the pre-judgment stage, a flashing warning is provided; in the critical stage, the seat belts are automatically tightened and pre-pressurized braking is controlled; in the emergency stage, automatic emergency braking is triggered and the electronic stability system is activated, and a detour path is simultaneously generated to guide the direction of avoidance. The safety judgment unit includes: The grid unit is divided into sub-units to adjust the acceleration, braking distance, and turning radius limits of the trajectory prediction based on the future movement trajectory of the vehicle, surrounding vehicles, and pedestrians, according to different road friction coefficients. A polar coordinate system is established with the vehicle as the center, and the grid units are divided to construct a polar coordinate system risk field model. The trajectory sample set acquisition subunit is used to generate a trajectory sample set that meets dynamic constraints using the Monte Carlo method. In low-friction coefficient scenarios, the probability of uncontrolled trajectories is corrected by increasing the sampling weight of slip trajectories. Low-friction coefficient scenarios include ice surfaces. The grid unit safety judgment sub-unit is used to traverse all trajectory samples, count the number of spatial overlaps between the ego vehicle and surrounding target trajectories in each grid unit, calculate the collision probability, and output a three-level safety situation map: safe zone, warning zone, and danger zone.

5. The human and vehicle behavior trajectory analysis system for urban safety according to claim 4 is characterized in that: The motion trajectory acquisition unit includes: The time synchronization subunit is used to provide a unified clock source for sensors using the precise time protocol; The spatial synchronization subunit is used to calculate the external parameter matrix of each sensor relative to the ego vehicle, convert the global coordinates into local coordinates through UTM projection, and convert all target positions into a coordinate system with the ego vehicle as the origin; The motion trajectory generation subunit is used to associate the detection targets at different times into the motion trajectory of the same object and generate the motion trajectories of surrounding vehicles and pedestrians.

6. The human and vehicle behavior trajectory analysis system for urban safety according to claim 4 is characterized in that: The action trajectory prediction unit includes: The data input subunit is used to input the movement trajectories of the vehicle, surrounding vehicles, and pedestrians into the input encoding layer of the machine learning model; The data processing subunit is used to independently run a bidirectional LSTM for each target through the LSTM timing module in the spatiotemporal fusion layer of the machine learning model to capture temporal behavior patterns, such as sudden braking and lane changes. The LSTM output vector of each target is used as input through the Transformer interaction module in the spatiotemporal fusion layer to model the spatial relationship between targets using a multi-head attention mechanism. The data output subunit is used to output multiple possible trajectories of all targets in the next 3 seconds through the multimodal prediction layer of the machine learning model.

Images