A vehicle surrounding traffic participant trajectory prediction method, device, equipment and medium

Abstract

The present disclosure provides a vehicle surrounding traffic participant trajectory prediction method and device, electronic equipment and storage medium, the method comprising: acquiring continuous image frames of the vehicle surrounding environment, and collecting the category information and trajectory sequence of each vehicle surrounding traffic participant in the continuous image frames of the vehicle surrounding environment; splicing the category feature vector and the trajectory feature vector of each vehicle surrounding traffic participant to obtain the composite feature vector of each vehicle surrounding traffic participant; wherein the category feature vector is generated by encoding the category information of each vehicle surrounding traffic participant, and the trajectory feature vector is generated by mapping the trajectory sequence of each vehicle surrounding traffic participant; according to the composite feature vector, the interaction feature of each vehicle surrounding traffic participant is constructed through the attention mechanism, and the future trajectory of each vehicle surrounding traffic participant is predicted based on the interaction feature. Accordingly, the present disclosure introduces category semantic prior, so that the model can adaptively identify the influence difference between different vehicle surrounding traffic participants, and improve the accuracy of trajectory prediction.

Description

Technical Field

[0001] This disclosure relates to the field of autonomous driving technology, and in particular to a method, apparatus, electronic device, and storage medium for predicting the trajectories of traffic participants around a vehicle. Background Technology

[0002] In autonomous driving technologies, accurately perceiving the vehicle's surroundings and predicting collision risks are the core of ensuring driving safety. The ability to issue warnings to the driver or control system before potential dangers occur, thus avoiding or mitigating accident losses, directly determines the reliability of the autonomous driving system.

[0003] However, existing trajectory prediction methods typically treat each traffic participant (such as vehicles and pedestrians) as an isolated individual, extrapolating solely from their historical trajectories, ignoring the multi-agent interactions prevalent in real-world traffic scenarios (such as vehicles yielding to pedestrians and following other vehicles). Furthermore, existing methods often employ fixed weights or simple fusion strategies to process multi-source information, making it difficult to adaptively capture the dynamic relationships between different participants (e.g., a large truck has a far greater impact on surrounding smaller vehicles than another small car, while a suddenly running child deserves more attention than a stationary roadside obstacle). This lack of context-aware modeling limits the accuracy and generalization ability of trajectory prediction. Summary of the Invention

[0004] In view of this, the present disclosure provides a method for predicting the trajectory of traffic participants around a vehicle, the method comprising: Acquire continuous image frames of the vehicle surrounding environment, and collect category information and trajectory sequences of each traffic participant around the vehicle in the continuous image frames of the vehicle surrounding environment; The category feature vector and trajectory feature vector of each traffic participant around the vehicle are concatenated to obtain a composite feature vector of each traffic participant around the vehicle; wherein, the category feature vector is generated by encoding the category information of each traffic participant around the vehicle, and the trajectory feature vector is generated by mapping the trajectory sequence of each traffic participant around the vehicle. Based on the composite feature vector, the interaction features of each vehicle-around traffic participant are constructed through an attention mechanism, and the future trajectory of each vehicle-around traffic participant is predicted based on the interaction features.

[0005] Optionally, the step of acquiring continuous image frames of the vehicle surrounding environment and collecting category information and trajectory sequences of each traffic participant around the vehicle in the continuous image frames includes: Acquire continuous image frames of the vehicle's surrounding environment captured by the vehicle-mounted camera; The continuous image frames of the vehicle surrounding environment are input into the feature extraction model, and the feature extraction model generates the location information and category information of each traffic participant around the vehicle from the top view of each image frame; Based on the location and category information, each vehicle-around traffic participant is tracked in the continuous image frames of the vehicle-around environment to obtain the trajectory sequence of each vehicle-around traffic participant.

[0006] Optionally, the location information is defined by a two-dimensional bounding box. The step of tracking each traffic participant around the vehicle in consecutive image frames of the vehicle's surrounding environment based on the location information and category information to obtain a trajectory sequence for each traffic participant includes: The two-dimensional bounding box and category information of each traffic participant around the vehicle are input into the multi-target tracking algorithm so that the multi-target tracking algorithm can track each traffic participant around the vehicle in continuous image frames of the vehicle surrounding environment to obtain the tracking result. If the speed of the vehicle-mounted camera exceeds a preset threshold, the tracking results are optimized based on the appearance characteristics of each traffic participant around the vehicle to obtain a trajectory sequence for each traffic participant around the vehicle. Otherwise, the trajectory sequence of each traffic participant around the vehicle is obtained directly based on the tracking results.

[0007] Optionally, the step of concatenating the category feature vector and trajectory feature vector of each vehicle-around traffic participant to obtain a composite feature vector for each vehicle-around traffic participant includes: The category information of each traffic participant around the vehicle is input into a semantic embedding network, so that the semantic embedding network maps the category information to generate a category feature vector for each traffic participant around the vehicle. The trajectory sequence of each traffic participant around the vehicle is input into a temporal coding network so that the temporal coding network encodes the trajectory sequence to generate a trajectory feature vector for each traffic participant around the vehicle. The category feature vector and the trajectory feature vector are concatenated and then input into the feature fusion network, so that the feature fusion network can extract features from the concatenated vector to generate a composite feature vector.

[0008] Optionally, the step of constructing interaction features of each vehicle-around traffic participant using an attention mechanism based on the composite feature vector, and predicting the future trajectory of each vehicle-around traffic participant based on the interaction features, includes: The composite feature vector of each traffic participant around the vehicle is weighted by three pre-learned weight parameters to generate three types of vectors for each traffic participant around the vehicle: query, key, and value. Based on the three types of vectors, calculate the attention weight of the target vehicle's surrounding traffic participants to each other vehicle's surrounding traffic participants; The interaction features of the target vehicle-around traffic participant are generated by weighting and summing the value vectors of each other vehicle-around traffic participant according to the attention weights. Based on the interaction features and the trajectory feature vector, predict the future trajectory of the traffic participants around the target vehicle.

[0009] Optionally, predicting the future trajectory of the traffic participant around the target vehicle based on the interaction features and the trajectory feature vector includes: The trajectory feature vectors of the traffic participants around the target vehicle are concatenated with the interaction features and then input into the feature fusion network, so that the feature fusion network encodes the concatenated features to generate fused features. The fused features are input into the decoder so that the decoder models the temporal regularity of the fused features and outputs the continuous position coordinates of the traffic participants around the target vehicle within a preset time period. Based on the continuous position coordinates, the future trajectories of traffic participants around the target vehicle are determined.

[0010] Optionally, the future trajectory includes continuous location points, and after predicting the future trajectory of each vehicle-around traffic participant based on the interaction features, the method further includes: The motion state of each traffic participant is obtained by performing numerical difference based on the coordinates and timestamps of continuous position points of each traffic participant around the vehicle. The motion state includes position, velocity and acceleration. Based on the current motion state of the vehicle and the motion state of each traffic participant, predict the time interval between the upcoming collision between the vehicle and each traffic participant around the vehicle. If the time interval between an impending collision between a vehicle and a traffic participant around the target vehicle falls within the threshold level range corresponding to the category information of the traffic participant around the target vehicle, a warning signal matching the threshold level range is triggered.

[0011] This disclosure also provides a device for predicting the trajectory of traffic participants around a vehicle, the device comprising: The acquisition unit is used to acquire continuous image frames of the vehicle surrounding environment, and to acquire the category information and trajectory sequence of each traffic participant around the vehicle in the continuous image frames of the vehicle surrounding environment; The splicing unit is used to splice the category feature vector and trajectory feature vector of each vehicle-around traffic participant to obtain a composite feature vector of each vehicle-around traffic participant; wherein, the category feature vector is generated by encoding the category information of each vehicle-around traffic participant, and the trajectory feature vector is generated by mapping the trajectory sequence of each vehicle-around traffic participant. The prediction unit is used to construct the interaction features of each traffic participant around the vehicle through an attention mechanism based on the composite feature vector, and predict the future trajectory of each traffic participant around the vehicle based on the interaction features.

[0012] This disclosure also provides an electronic device, including a communication interface, a processor, a memory, and a bus, wherein the communication interface, the processor, and the memory are interconnected via the bus; The memory stores machine-readable instructions, and the processor executes the above method by invoking the machine-readable instructions.

[0013] This disclosure also provides a machine-readable storage medium storing machine-readable instructions that, when called and executed by a processor, implement the above-described method.

[0014] Therefore, in the technical solution disclosed herein, the category information and trajectory sequence of each traffic participant (such as vehicles, pedestrians, etc.) around the vehicle are first extracted from continuous image frames; then, the category information is encoded into category feature vectors and the trajectory sequence is mapped into trajectory feature vectors, and the two types of vectors are concatenated to form a composite feature vector that integrates semantic attributes and motion state; finally, an attention mechanism is used to interactively model the composite feature vectors of all traffic participants around the vehicle to generate interactive features that reflect contextual dependencies, and these features are used to predict the future trajectory of each traffic participant around the vehicle.

[0015] Through the above methods, this disclosure explicitly models the complex interactive behaviors among traffic participants (such as vehicles avoiding pedestrians, following vehicles, and pedestrians avoiding obstacles), making the predicted trajectory more consistent with real traffic dynamics. On the other hand, by introducing category semantic priors, the model can adaptively identify the differences in influence among different traffic participants around the vehicle (for example, large vehicles have a greater impact on the surrounding area, and children's behavior is more uncertain), thereby improving the accuracy and robustness of trajectory prediction and laying a solid foundation for highly reliable collision risk prediction and proactive safety warning. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating an exemplary embodiment of a method for predicting the trajectories of traffic participants around a vehicle; Figure 2 This is a schematic diagram illustrating a continuous image frame processing flow for a vehicle surrounding environment, as shown in an exemplary embodiment. Figure 3 This is a schematic diagram illustrating a collision risk assessment and graded early warning system as an exemplary embodiment; Figure 4 This is an exemplary embodiment illustrating the hardware structure of an electronic device; Figure 5 This is a block diagram illustrating an exemplary embodiment of a vehicle perimeter traffic participant trajectory prediction device. Detailed Implementation

[0018] To enable those skilled in the art to better understand the technical solutions in this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this disclosure.

[0019] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this disclosure in other embodiments. In some other embodiments, the methods may include more or fewer steps than those described in this disclosure. Furthermore, a single step described in this disclosure may be broken down into multiple steps in other embodiments; and multiple steps described in this disclosure may be combined into a single step in other embodiments.

[0020] The present disclosure will now be described through specific embodiments and in conjunction with specific application scenarios.

[0021] Please see Figure 1 , Figure 1 This is a flowchart illustrating an exemplary embodiment of a method for predicting the trajectories of traffic participants around vehicles. The method may perform the following steps: Step 102: Acquire continuous image frames of the vehicle surrounding environment, and collect the category information and trajectory sequence of each traffic participant around the vehicle in the continuous image frames of the vehicle surrounding environment.

[0022] For example, the vehicle controller (hereinafter referred to as the controller) uses a surround-view vision system consisting of six cameras deployed at the front, rear, left front, left rear, right front, and right rear of the vehicle to continuously acquire continuous image frames of the 360-degree environment around the vehicle at a fixed frequency (e.g., 10 frames per second). Subsequently, the controller processes these images to identify various traffic participants, including vehicles and pedestrians in front, and outputs their categories (e.g., "vehicles," "pedestrians," "non-motorized vehicles") and the position of that category in continuous time steps in the vehicle coordinate system (usually represented by two-dimensional or three-dimensional coordinates), thereby forming the category information and corresponding historical trajectory sequence of each traffic participant around the vehicle.

[0023] Among them, "category information" refers to the semantic type of traffic participants around the vehicle determined by the controller based on image recognition, used to distinguish objects with different behavioral patterns, including pedestrians, vehicles, non-motorized vehicles, animals, guardrails, roadblocks, etc. "Track sequence" refers to the sequence of two-dimensional or three-dimensional spatial positions of the traffic participant around the vehicle in the vehicle coordinate system over a number of time steps in the past (e.g., within the last 2 seconds, with an interval of 0.1 seconds), usually generated from the detection results of multiple frames, used to characterize its motion state relative to the vehicle.

[0024] Step 104: Concatenate the category feature vector and trajectory feature vector of each traffic participant around the vehicle to obtain a composite feature vector of each traffic participant around the vehicle; wherein, the category feature vector is generated by encoding the category information of each traffic participant around the vehicle, and the trajectory feature vector is generated by mapping the trajectory sequence of each traffic participant around the vehicle.

[0025] For example, the controller encodes the features of each identified traffic participant around the vehicle (such as a non-motorized vehicle or a pedestrian): taking the traffic participant around the vehicle in the category of "non-motorized vehicle" as an example, the controller calls the category encoder in the multimodal interactive encoder to encode the discrete category label of "non-motorized vehicle" through a learnable embedding layer to generate a 32-dimensional category feature vector; at the same time, the controller calls the trajectory encoder in the multimodal interactive encoder to map the trajectory sequence of 20 position points sampled at 0.1-second intervals in the past 2 seconds in the vehicle coordinate system to generate a 64-dimensional trajectory feature vector; then, the two vectors are concatenated along the channel axis in the feature dimension to form a 96-dimensional composite feature vector, which is used to characterize the semantic attributes and motion state of the traffic participant around the vehicle in the category of "non-motorized vehicle".

[0026] Here, "location point" can be a two-dimensional coordinate (x, y) from a top-down view or a three-dimensional coordinate (x, y, z) in the vehicle's coordinate system. "Category feature vector" refers to the semantic embedding generated by the category encoder based on category information, used to distinguish the behavioral priors of different traffic participants around the vehicle (e.g., pedestrians are more likely to change direction abruptly, and non-motorized vehicles often travel along the shoulder). "Trajectory feature vector" refers to the vector representation output by the trajectory encoder after temporal modeling of historical trajectory sequences, capable of capturing dynamic features such as speed, acceleration, and turning trends of traffic participants around the vehicle. "Concatenation" refers to connecting vectors along the feature dimension, a common method for fusing multimodal features, providing a unified input for subsequent attention mechanisms to construct interaction relationships.

[0027] Step 106: Based on the composite feature vector, construct the interaction features of each traffic participant around the vehicle through an attention mechanism, and predict the future trajectory of each traffic participant around the vehicle based on the interaction features.

[0028] For example, the controller inputs the composite feature vectors (e.g., 96-dimensional) of all traffic participants around the vehicle (such as a vehicle ahead, a pedestrian on the left, and a roadblock in the distance) into a multi-head self-attention module. The multi-head self-attention module calculates the attention weights between each pair of traffic participants. For instance, it finds a high interaction weight between the "vehicle directly ahead" and the "pedestrian suddenly crossing from the left," while the interaction weight between the "vehicle directly ahead" and the "roadblock in the distance" is close to zero. Then, the controller weights and aggregates the composite feature vectors of the "pedestrian suddenly crossing from the left" and the "roadblock in the distance" based on these interaction weights, generating an interaction feature vector for the "vehicle directly ahead" that incorporates the influence of the surrounding context. Similarly, the interaction feature vectors for the "pedestrian suddenly crossing from the left" and the "roadblock in the distance" can be obtained. Finally, the controller inputs the interaction feature vector of each traffic participant around the vehicle into a trajectory decoder, which outputs 30 predicted location points for each participant within the next second, spaced at 0.1-second intervals, forming the future trajectory.

[0029] The "future trajectory" is a parameterized curve composed of smoothed, continuous position points. This trajectory is differentiable, supporting subsequent calculations of instantaneous velocity, acceleration, and collision risk indicators. The "composite feature vector" is a joint representation formed by concatenating the category feature vector and the trajectory feature vector. It integrates the semantic type and historical motion state of traffic participants (such as temporal dynamic characteristics like uniform straight-line motion, deceleration, braking, and sharp turns). This vector serves as input to the attention mechanism, ensuring that subsequent interaction modeling possesses semantic-motion joint representation capabilities. The "trajectory decoder" employs a multilayer perceptron structure to perform backpropagation joint optimization on the input composite feature vector. It can learn the coupling relationship between category semantics and motion patterns. For example, when the "pedestrian" category is combined with a "low-speed + high lateral offset" trajectory, it strengthens the discriminative response to sudden crossing behavior.

[0030] In one embodiment, the step of acquiring continuous image frames of the vehicle surrounding environment and collecting category information and trajectory sequences of each traffic participant in the continuous image frames includes: acquiring the continuous image frames of the vehicle surrounding environment captured by an onboard camera; inputting the continuous image frames of the vehicle surrounding environment into a feature extraction model, and generating position information and category information of each traffic participant in the vehicle surrounding environment from a top-down perspective for each image frame; and tracking each traffic participant in the continuous image frames of the vehicle surrounding environment based on the position information and category information to obtain a trajectory sequence of each traffic participant.

[0031] For example, the controller continuously captures consecutive image frames of the 360-degree environment around the vehicle at a fixed frequency (e.g., 10 frames per second) using deployed vehicle-mounted cameras. These images are then input into a feature extraction model based on a deep neural network (e.g., BEVFormer). This model fuses the multi-view images and converts them into a bird's-eye view with a unified top-down perspective, outputting the location and category label (e.g., "vehicle," "pedestrian," "non-motorized vehicle") for each traffic participant around the vehicle. The controller then invokes a multi-target tracking module to perform cross-frame association for each traffic participant around the vehicle based on the positional similarity and motion continuity of targets of the same category in the bird's-eye view within consecutive frames. This ultimately forms a trajectory sequence of the traffic participants around the vehicle within the vehicle's coordinate system over the past 2 seconds at 0.1-second intervals.

[0032] Please see Figure 2 , Figure 2 This is a schematic diagram illustrating a continuous image frame processing flow for a vehicle surrounding environment, as shown in an exemplary embodiment. Figure 2 As shown, the controller acquires continuous image frames of the vehicle's surrounding environment via onboard cameras. These cameras are distributed at different locations on the vehicle, including forward-looking, left-front-looking, left-rear-looking, right-front-looking, right-rear-looking, and rear-looking directions, ensuring omnidirectional coverage. The controller then inputs the acquired multi-view images into a feature extraction model (e.g., BEVFormer). This model fuses image information from different perspectives and converts it into a unified top-down view representation, i.e., a bird's-eye view. In the generated bird's-eye view, the location and category label of each traffic participant are accurately marked. The vehicle is located near the origin (0, 0), surrounded by three traffic participants: a "vehicle" to the left rear with its bottom left corner coordinates (-14, 8) and top right corner coordinates (-10, 12); a "non-motorized vehicle" to the right rear with its bottom left corner coordinates (-6, 7) and top right corner coordinates (-3, 9); and a "pedestrian" to the left front with its bottom left corner coordinates (-11, 27) and top right corner coordinates (-7, 33).

[0033] The "position of traffic participants around the vehicle" can be represented by bounding boxes, or by parameterization of the center point coordinates (x, y) combined with the orientation angle and dimensions (length, width), or by geometric structures expressed as key points (such as the four corners of a vehicle, the footholds of a pedestrian). The "feature extraction model" is a deep learning model that can map multi-view images to a unified semantic space (such as a top-down view). Typical implementations include Transformer-based BEV perception networks or a fusion structure of geometric projection and neural networks, whose output includes the target's position and semantic category in the vehicle's coordinate system. "Tracking" refers to connecting the detection results of the same traffic participant in consecutive image frames into a trajectory sequence through data association algorithms, i.e., a spatial location list arranged in chronological order, representing the movement history of the traffic participants around the vehicle.

[0034] In this embodiment, the controller generates a bird's-eye view using image data acquired by a multi-view vehicle-mounted camera. This not only avoids reliance on additional radar hardware and reduces costs, but also effectively avoids target occlusion issues through a top-down perspective. This method improves the accuracy of target recognition and tracking while reducing computational load.

[0035] Before inputting the vehicle surrounding environment images into the feature extraction model, the model needs to undergo offline pre-training. The training process is as follows: First, a large number of vehicle surrounding environment images and the 3D position data of each traffic participant in the images are acquired (which can be collected by sensors such as LiDAR or multi-view stereo vision). Then, based on the 3D position data, the original images are geometrically projected to a top-down view, and the ground truth 2D bounding boxes of each traffic participant are labeled in this view. Next, the original vehicle surrounding environment images are input into the feature extraction model to be trained, and the model outputs the predicted 2D bounding boxes and predicted category information in the top-down view. Finally, by calculating the localization and classification errors between the predicted bounding boxes and the labeled ground truth values, the model parameters are iteratively optimized until the error converges to below a preset threshold.

[0036] It should be noted that the images used for training can come from the same or different onboard cameras as the deployed vehicle, supporting multi-source, centralized offline training; after the model is trained, it can run solely on visual input, without the need for expensive sensors such as radar to be installed on the vehicle, thereby reducing hardware costs while ensuring perception accuracy.

[0037] In one embodiment, the location information is defined by a two-dimensional bounding box. The step of tracking each vehicle-around traffic participant in continuous image frames of the vehicle-around environment based on the location information and category information to obtain a trajectory sequence for each participant includes: inputting the two-dimensional bounding box and category information of each participant into a multi-target tracking algorithm, so that the multi-target tracking algorithm tracks each participant in continuous image frames of the vehicle-around environment to obtain a tracking result; if the movement speed of the vehicle-mounted camera exceeds a preset threshold, optimizing the tracking result based on the appearance characteristics of each participant to obtain a trajectory sequence for each participant; otherwise, directly obtaining the trajectory sequence for each participant based on the tracking result.

[0038] For example, the controller continuously captures consecutive image frames of the 360-degree environment around the vehicle using a deployed vehicle-mounted camera at a fixed frequency (e.g., 10 frames per second). The controller then inputs these consecutive image frames into a feature extraction model, which generates two-dimensional bounding boxes for each traffic participant around the vehicle in each image frame (e.g., the bounding box coordinates for a vehicle directly in front are [(100, 50), (150, 80)]), with a category label of "vehicle". Next, the controller inputs these bounding boxes and category information into the ByteTrack multi-object tracking algorithm, which tracks the two-dimensional bounding boxes to generate initial tracking trajectories for the traffic participants around the vehicle. If the camera's speed exceeds a preset threshold (e.g., 50 km / h) during vehicle movement, the ReID (Re-Identification) module is further used to extract the appearance features (e.g., color, shape) of the target traffic participants around the vehicle, and the tracking results are optimized to ensure accurate tracking even at high speeds; otherwise, the output of the ByteTrack multi-object tracking algorithm is used directly as the final trajectory sequence.

[0039] The "two-dimensional bounding box" refers to the rectangular area enclosed by the horizontal and vertical coordinates of the upper left and lower right corners in a bird's-eye view, representing the spatial occupancy of each traffic participant around a vehicle in a single frame image. "Multi-target tracking algorithms" aim to associate the same target in different frames to form a continuous trajectory, including DeepSORT and ByteTrack. "Appearance features" refer to the unique visual attributes of a target (such as color and shape), which can be used to improve tracking accuracy, especially in fast-moving situations. "Motion speed of the vehicle-mounted camera" does not directly measure the camera's own speed, but rather the vehicle speed obtained through the vehicle's CAN bus; this speed reflects the overall intensity of motion of the camera's imaging plane.

[0040] In one embodiment, concatenating the category feature vector and trajectory feature vector of each traffic participant around a vehicle to obtain a composite feature vector for each participant includes: inputting the category information of each participant into a semantic embedding network to map the category information and generate a category feature vector for each participant; inputting the trajectory sequence of each participant into a temporal coding network to encode the trajectory sequence and generate a trajectory feature vector for each participant; and concatenating the category feature vector and the trajectory feature vector and inputting the concatenated vector into a feature fusion network to extract features from the concatenated vector and generate a composite feature vector.

[0041] For example, the controller processes a vehicle-around traffic participant identified as a "non-motorized vehicle": First, its category label "non-motorized vehicle" is input into a 32-dimensional semantic embedding network (such as a learnable embedding layer), outputting the corresponding category feature vector. Simultaneously, the controller inputs a trajectory sequence consisting of 20 location points (x, y) sampled at 0.1-second intervals over the past 2 seconds in the vehicle coordinate system into a temporal coding network composed of two layers of LSTM (Long Short-Term Memory), outputting a 64-dimensional trajectory feature vector. Then, the controller concatenates these two vectors into a 96-dimensional concatenated vector, which is then fed into a feature fusion network containing two fully connected layers and a ReLU activation function. After nonlinear transformation, the final 96-dimensional composite feature vector is output for subsequent interaction modeling.

[0042] Among them, the "semantic embedding network" is a lightweight learnable mapping module implemented using an embedding layer. This embedding layer is optimized through end-to-end training, making the cosine distance between semantically similar categories in the embedding space smaller, thus preserving the topological structure between categories. Its output is the category feature vector, which is used to represent the behavioral priors of different participants. The "temporal coding network" refers to a neural network structure used to model the dynamic characteristics of trajectory sequences. Its output trajectory feature vector can effectively capture motion trends such as speed and acceleration. The "feature fusion network" refers to a network that further refines the spliced ​​multimodal features. It is usually composed of fully connected layers or lightweight residual modules, aiming to enhance the feature expression ability and eliminate redundant information, thereby generating more discriminative composite feature vectors.

[0043] In one embodiment, the step of constructing interaction features of each vehicle-surrounding traffic participant using an attention mechanism based on the composite feature vector, and predicting the future trajectory of each vehicle-surrounding traffic participant based on the interaction features, includes: weighting the composite feature vector of each vehicle-surrounding traffic participant with three pre-learned weight parameters to generate query, key, and value vectors for each vehicle-surrounding traffic participant; calculating the attention weight of the target vehicle-surrounding traffic participant for each other vehicle-surrounding traffic participant based on the three types of vectors; weighting and summing the value vectors of each other vehicle-surrounding traffic participant based on the attention weights to generate the interaction features of the target vehicle-surrounding traffic participant; and predicting the future trajectory of the target vehicle-surrounding traffic participant based on the interaction features and the trajectory feature vector.

[0044] For example, the controller obtains the current time. After generating the composite feature vectors of each vehicle-week traffic participant (such as the 96-dimensional composite feature vectors of a pedestrian, a vehicle, and a non-motorized vehicle), the composite feature vectors of each vehicle-week traffic participant are then... With three sets of learnable weight matrices respectively , , Multiply to generate the corresponding query vector. Key vector Sum value vector Subsequently, the controller targets traffic participants around the "vehicle". Calculate its query vector Key vectors of other vehicle-centric traffic participants dot product And by using the attention weight calculation formula, the target is obtained. For pedestrians, non-motorized vehicles and other objects attention weights , indicating the target For the target The level of attention. Then, the controller determines the level of attention based on attention weights. Value vectors of other traffic participants around the vehicle A weighted summation is performed to generate a target that incorporates the effects of surrounding interactions. Interactive features Finally, the controller will target... Interactive features and target Trajectory feature vector After concatenation and encoding, the encoded result is input into the trajectory decoder to obtain the target. Target The future trajectory is formed from 10 predicted location points at 0.1-second intervals within the next second. .

[0045] In this context, query vector, key vector, and value vector are standard terms in attention mechanisms. The query vector characterizes the attentional tendency of traffic participants around the target vehicle towards other objects in the environment, representing "Whom do I want to pay attention to?". The key vector characterizes the attentional attributes of other traffic participants, representing "Can I be noticed?". The value vector carries the actual dynamic information of other traffic participants, representing "If I am noticed, what impact will I have?", and is used for the weighted fusion of subsequent interaction features.

[0046] Attention weight The calculation formula is: In the formula, The dimension is a vector dimension, used for stable gradient training.

[0047] Target The calculation formula is: "Pre-learned weight parameters" refer to three sets of learnable projection matrices optimized through backpropagation during model training, used to generate query vectors, key vectors, and value vectors, respectively. These parameters are loaded and used during inference. This design enables the model to achieve differentiated attention capabilities in a unified embedding space based on the characteristics of different traffic participants around the vehicle, thus supporting scene-adaptive weight allocation. The "attention mechanism" employs a scaled dot product self-attention structure, capable of dynamically modeling the strength of mutual influence between different participants. The "trajectory decoder" is typically a lightweight neural network, such as an MLP (Multilayer Perceptron) or a small LSTM.

[0048] It should be noted that, in addition to direct concatenation, other fusion strategies can be used to combine interactive features and trajectory feature vectors. For example, within the feature fusion network, the trajectory feature vector can be used as the backbone, and interactive features can be injected through gating or conditional bias; or a residual structure can be used to superimpose interactive features onto the trajectory features before fusion encoding. These methods can be flexibly selected based on the model architecture and real-time requirements to further improve the accuracy and robustness of trajectory prediction.

[0049] In one embodiment, predicting the future trajectory of the target vehicle-around traffic participant based on the interaction features and the trajectory feature vector includes: concatenating the trajectory feature vector of the target vehicle-around traffic participant with the interaction features and inputting the concatenated features into a feature fusion network, so that the feature fusion network encodes the concatenated features to generate fused features; inputting the fused features into a decoder, so that the decoder models the temporal regularity of the fused features and outputs the continuous position coordinates of the target vehicle-around traffic participant within a preset time period; and determining the future trajectory of the target vehicle-around traffic participant based on the continuous position coordinates.

[0050] For example, after completing interaction modeling, the controller obtains the target. The interaction features (64-dimensional) and trajectory feature vector (64-dimensional) of the "non-motorized vehicle" are then analyzed. The controller concatenates these two vectors along the feature dimensions to form a 128-dimensional joint feature vector. This joint feature vector is then input into a feature fusion network composed of an MLP (Multi-Level Processing) algorithm. This network performs non-linear encoding on the joint feature vector to generate the target... Fusion characteristics Next, the controller will fuse this feature. The data is fed into a two-layer LSTM decoder, which fuses the features. As the initial hidden state, (x, y) position coordinates are generated progressively at fixed time steps (e.g., 0.1 seconds), one every 0.1 seconds for the next second, for a total of 10 points. These consecutive coordinate points form the target in chronological order. The future trajectory of a "non-motorized vehicle" in the vehicle's coordinate system It is used for subsequent collision risk assessment or path planning.

[0051] Among them, the target Fusion characteristics The calculation formula is: Target future trajectory The calculation formula is: "Interactive features" refer to context-enhanced features generated by aggregating the influence of other traffic participants through attention mechanisms, reflecting the target's interactive behavior in a specific traffic scenario (such as avoiding pedestrians, following the vehicle in front, etc.). "Feature fusion networks" include MLPs, residual connections, or small cross-attention modules, which encode joint features, eliminate redundancy, and generate more discriminative "fused features." "Decoders" are temporal generation models used to map fused features to future spatiotemporal trajectories, including recurrent neural networks (such as LSTM, GRU), Transformer decoders, or multilayer RNNs. "Continuous position coordinates" refer to a sequence of planar coordinates in the vehicle's coordinate system, where each coordinate point contains x and y components and a corresponding timestamp.

[0052] In one embodiment shown, the future trajectory includes continuous location points. After predicting the future trajectory of each traffic participant based on the interaction features, the method further includes: performing numerical difference based on the coordinates and timestamps of the continuous location points of each traffic participant to obtain the motion state of each traffic participant, the motion state including position, velocity, and acceleration; predicting the time interval between the vehicle and each traffic participant that a collision is about to occur based on the current motion state of the vehicle and the motion state of each traffic participant; and triggering a warning signal matching the threshold level range if the time interval between the vehicle and the target traffic participant that a collision is about to occur falls within the threshold level range corresponding to the category information of the target traffic participant.

[0053] For example, see Figure 3 , Figure 3 This is a schematic diagram illustrating a collision risk assessment and graded early warning system, as an exemplary embodiment. Figure 3 As shown, after acquiring the future trajectories (composed of multiple consecutive location points) of each traffic participant around the vehicle, the controller performs numerical difference processing on the trajectory sequence of each target, calculating its velocity and acceleration at each time step, thereby constructing complete motion state information. Subsequently, the controller combines the vehicle's real-time motion state (such as position, velocity, and heading angle) and uses kinematic algorithms to calculate the time interval between the vehicle and each traffic participant around the vehicle that is about to collide. Then, the controller compares this time interval with... Figure 3The target category is compared with the preset risk threshold table shown: taking the target vehicle's surrounding traffic participant "pedestrian" as an example, if the time interval between the impending collision between the "pedestrian" and the vehicle is 1.5s, corresponding to the "high risk" range (<2s), it is judged as high risk and a red warning is triggered, such as issuing a buzzer; if the time interval between the impending collision between the "pedestrian" and the vehicle is 3s, corresponding to the "medium risk" range (2-4s), it is judged as medium risk and a yellow warning is triggered, such as a flashing warning light; if the time interval between the impending collision between the "pedestrian" and the vehicle is 5s, not falling into any risk range, no warning is triggered.

[0054] The "current vehicle motion status" is obtained in real time by the vehicle's CAN bus, including the vehicle's position. Heading angle Speed Longitudinal velocity lateral velocity Longitudinal acceleration All parameters are strictly aligned with the trajectory sequence of traffic participants around the vehicle using hardware timestamps to ensure spatiotemporal consistency. For each target, at consecutive time points... and Above, the positions are respectively and Then the average velocity vector , The calculation formulas are as follows: Speed The calculation formula is: In some embodiments, after acquiring continuous location points of the future trajectories of traffic participants around the target vehicle, the controller does not directly calculate the velocity using two adjacent location points. Instead, it uses a sliding window to perform linear regression fitting on the most recent number of location points (e.g., 5 location points, with a time span of 0.4 seconds) to obtain a smoother velocity estimate. Simultaneously, a moving average filter is applied to the velocity sequence to further suppress jitter, thereby obtaining a stable acceleration value. This method effectively reduces the impact of position jumps caused by detection noise or occlusion on motion state estimation, making subsequent collision risk assessment more reliable and avoiding false alarms.

[0055] The "time interval before an impending collision," or TTC (Time to Collision), is calculated by the controller based on the motion states of both parties. After obtaining the future trajectories of traffic participants around the target vehicle, the controller sets the relative position vectors of these participants to the vehicle. The relative velocity vector is The relative speed is obtained by subtracting the vehicle speed from the speeds of traffic participants around the target vehicle. The controller first calculates the approach speed along the relative position direction: The formula for calculating the collision time TTC is: The controller calculates the approach speed along the relative position direction. Then, substitute it into the formula for calculating collision time TTC to obtain the collision time TTC.

[0056] Corresponding to the embodiments of the above-described method for predicting the trajectory of traffic participants around vehicles, this disclosure also provides an embodiment of a device for predicting the trajectory of traffic participants around vehicles.

[0057] Please see Figure 4 , Figure 4 This is an exemplary embodiment illustrating the hardware structure of an electronic device. At the hardware level, the device includes a processor 402, an internal bus 404, a network interface 406, memory 408, and non-volatile memory 410, and may also include other necessary hardware. One or more embodiments of this disclosure can be implemented in software, for example, the processor 402 reads the corresponding computer program from the non-volatile memory 410 into memory 408 and then runs it. Of course, besides software implementation, one or more embodiments of this disclosure do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution entity of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0058] Please see Figure 5 , Figure 5 This is a block diagram illustrating an exemplary embodiment of a vehicle-around traffic participant trajectory prediction device 500. This vehicle-around traffic participant trajectory prediction device 500 can be applied to, for example... Figure 4 The illustrated electronic device is used to implement the technical solution of this disclosure. The device includes: The acquisition unit 502 is used to acquire continuous image frames of the vehicle surrounding environment, and to acquire the category information and trajectory sequence of each traffic participant around the vehicle in the continuous image frames of the vehicle surrounding environment; The splicing unit 504 is used to splice the category feature vector and trajectory feature vector of each vehicle-around traffic participant to obtain a composite feature vector of each vehicle-around traffic participant; wherein, the category feature vector is generated by encoding the category information of each vehicle-around traffic participant, and the trajectory feature vector is generated by mapping the trajectory sequence of each vehicle-around traffic participant. The first prediction unit 506 is used to construct the interaction features of each vehicle-around traffic participant through an attention mechanism based on the composite feature vector, and predict the future trajectory of each vehicle-around traffic participant based on the interaction features.

[0059] In some embodiments, acquiring continuous image frames of the vehicle surrounding environment and collecting category information and trajectory sequences of each traffic participant around the vehicle in the continuous image frames includes: Acquire continuous image frames of the vehicle's surrounding environment captured by the vehicle-mounted camera; The continuous image frames of the vehicle surrounding environment are input into the feature extraction model, and the feature extraction model generates the location information and category information of each traffic participant around the vehicle from the top view of each image frame; Based on the location and category information, each vehicle-around traffic participant is tracked in the continuous image frames of the vehicle-around environment to obtain the trajectory sequence of each vehicle-around traffic participant.

[0060] In some embodiments, the location information is defined by a two-dimensional bounding box, and the tracking of each vehicle-around traffic participant in continuous image frames of the vehicle-around environment based on the location information and category information to obtain a trajectory sequence for each vehicle-around traffic participant includes: The two-dimensional bounding box and category information of each traffic participant around the vehicle are input into the multi-target tracking algorithm so that the multi-target tracking algorithm can track each traffic participant around the vehicle in continuous image frames of the vehicle surrounding environment to obtain the tracking result. If the speed of the vehicle-mounted camera exceeds a preset threshold, the tracking results are optimized based on the appearance characteristics of each traffic participant around the vehicle to obtain a trajectory sequence for each traffic participant around the vehicle. Otherwise, the trajectory sequence of each traffic participant around the vehicle is obtained directly based on the tracking results.

[0061] In some embodiments, concatenating the category feature vector and trajectory feature vector of each vehicle-around traffic participant to obtain a composite feature vector for each vehicle-around traffic participant includes: The category information of each traffic participant around the vehicle is input into a semantic embedding network, so that the semantic embedding network maps the category information to generate a category feature vector for each traffic participant around the vehicle. The trajectory sequence of each traffic participant around the vehicle is input into a temporal coding network so that the temporal coding network encodes the trajectory sequence to generate a trajectory feature vector for each traffic participant around the vehicle. The category feature vector and the trajectory feature vector are concatenated and then input into the feature fusion network, so that the feature fusion network can extract features from the concatenated vector to generate a composite feature vector.

[0062] In some embodiments, the step of constructing interaction features of each vehicle-around traffic participant using an attention mechanism based on the composite feature vector, and predicting the future trajectory of each vehicle-around traffic participant based on the interaction features, includes: The composite feature vector of each traffic participant around the vehicle is weighted by three pre-learned weight parameters to generate three types of vectors for each traffic participant around the vehicle: query, key, and value. Based on the three types of vectors, calculate the attention weight of the target vehicle's surrounding traffic participants to each other vehicle's surrounding traffic participants; The interaction features of the target vehicle-around traffic participant are generated by weighting and summing the value vectors of each other vehicle-around traffic participant according to the attention weights. Based on the interaction features and the trajectory feature vector, predict the future trajectory of the traffic participants around the target vehicle.

[0063] In some embodiments, predicting the future trajectory of the target vehicle-around traffic participant based on the interaction features and the trajectory feature vector includes: The trajectory feature vectors of the traffic participants around the target vehicle are concatenated with the interaction features and then input into the feature fusion network, so that the feature fusion network encodes the concatenated features to generate fused features. The fused features are input into the decoder so that the decoder models the temporal regularity of the fused features and outputs the continuous position coordinates of the traffic participants around the target vehicle within a preset time period. Based on the continuous position coordinates, the future trajectories of traffic participants around the target vehicle are determined.

[0064] In some embodiments, the future trajectory includes consecutive location points, and after predicting the future trajectory of each vehicle-around traffic participant based on the interaction features, the device further includes: The differential unit 508 is used to perform numerical difference based on the coordinates and timestamps of the continuous position points of each traffic participant around the vehicle to obtain the motion state of each traffic participant, which includes position, velocity and acceleration. The second prediction unit 510 is used to predict the time interval between the vehicle and each traffic participant around the vehicle based on the current motion state of the vehicle and the motion state of each traffic participant. The triggering unit 512 is used to trigger a warning signal that matches the threshold level range if the time interval between the impending collision between the vehicle and the target vehicle's surrounding traffic participants falls within the threshold level range corresponding to the category information of the target vehicle's surrounding traffic participants.

[0065] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0066] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0067] The systems, devices, modules, or units described in the above embodiments can be implemented by a computer entity or by a product with a certain function. A typical implementation device is a computer, which can be a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0068] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0069] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0070] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0071] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant regions, and corresponding operation portals shall be provided for users to choose to authorize or refuse.

[0072] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0073] The foregoing has described specific embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0074] The terminology used in one or more embodiments of this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this disclosure. The singular forms “a,” “the,” and “the” as used in one or more embodiments of this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0075] It should be understood that although the terms first, second, third, etc., may be used to describe various information in one or more embodiments of this disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of one or more embodiments of this disclosure, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0076] The above description is merely a preferred embodiment of one or more embodiments of this disclosure and is not intended to limit the scope of one or more embodiments of this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this disclosure should be included within the scope of protection of one or more embodiments of this disclosure.

Claims

1. A method for predicting the trajectory of traffic participants around a vehicle, characterized in that, The method includes: Acquire continuous image frames of the vehicle surrounding environment, and collect category information and trajectory sequences of each traffic participant around the vehicle in the continuous image frames of the vehicle surrounding environment; The category feature vector and trajectory feature vector of each traffic participant around the vehicle are concatenated to obtain a composite feature vector of each traffic participant around the vehicle; wherein, the category feature vector is generated by encoding the category information of each traffic participant around the vehicle, and the trajectory feature vector is generated by mapping the trajectory sequence of each traffic participant around the vehicle. Based on the composite feature vector, the interaction features of each vehicle-around traffic participant are constructed through an attention mechanism, and the future trajectory of each vehicle-around traffic participant is predicted based on the interaction features.

2. The method according to claim 1, characterized in that, The process of acquiring continuous image frames of the vehicle surrounding environment, and collecting category information and trajectory sequences of each traffic participant around the vehicle in the continuous image frames, includes: Acquire continuous image frames of the vehicle's surrounding environment captured by the vehicle-mounted camera; The continuous image frames of the vehicle surrounding environment are input into the feature extraction model, and the feature extraction model generates the location information and category information of each traffic participant around the vehicle from the top view of each image frame; Based on the location and category information, each vehicle-around traffic participant is tracked in the continuous image frames of the vehicle-around environment to obtain the trajectory sequence of each vehicle-around traffic participant.

3. The method according to claim 2, characterized in that, The location information is defined by a two-dimensional bounding box. Based on the location information and category information, each traffic participant around a vehicle is tracked in consecutive image frames of the vehicle's surrounding environment to obtain a trajectory sequence for each traffic participant, including: The two-dimensional bounding box and category information of each traffic participant around the vehicle are input into the multi-target tracking algorithm so that the multi-target tracking algorithm can track each traffic participant around the vehicle in continuous image frames of the vehicle surrounding environment to obtain the tracking result. If the speed of the vehicle-mounted camera exceeds a preset threshold, the tracking results are optimized based on the appearance characteristics of each traffic participant around the vehicle to obtain a trajectory sequence for each traffic participant around the vehicle. Otherwise, the trajectory sequence of each traffic participant around the vehicle is obtained directly based on the tracking results.

4. The method according to claim 1, characterized in that, The step of concatenating the category feature vector and trajectory feature vector of each traffic participant around a vehicle to obtain a composite feature vector for each traffic participant around a vehicle includes: The category information of each traffic participant around the vehicle is input into a semantic embedding network, so that the semantic embedding network maps the category information to generate a category feature vector for each traffic participant around the vehicle. The trajectory sequence of each traffic participant around the vehicle is input into a temporal coding network so that the temporal coding network encodes the trajectory sequence to generate a trajectory feature vector for each traffic participant around the vehicle. The category feature vector and the trajectory feature vector are concatenated and then input into the feature fusion network, so that the feature fusion network can extract features from the concatenated vector to generate a composite feature vector.

5. The method according to claim 1, characterized in that, The step of constructing interaction features of each traffic participant around a vehicle using an attention mechanism based on the composite feature vector, and predicting the future trajectory of each traffic participant around a vehicle based on the interaction features, includes: The composite feature vector of each traffic participant around the vehicle is weighted by three pre-learned weight parameters to generate three types of vectors for each traffic participant around the vehicle: query, key, and value. Based on the three types of vectors, calculate the attention weight of the target vehicle's surrounding traffic participants to each other vehicle's surrounding traffic participants; The interaction features of the target vehicle-around traffic participant are generated by weighting and summing the value vectors of each other vehicle-around traffic participant according to the attention weights. Based on the interaction features and the trajectory feature vector, predict the future trajectory of the traffic participants around the target vehicle.

6. The method according to claim 5, characterized in that, The step of predicting the future trajectory of the target vehicle's surrounding traffic participants based on the interaction features and the trajectory feature vector includes: The trajectory feature vectors of the traffic participants around the target vehicle are concatenated with the interaction features and then input into the feature fusion network, so that the feature fusion network encodes the concatenated features to generate fused features. The fused features are input into the decoder so that the decoder models the temporal regularity of the fused features and outputs the continuous position coordinates of the traffic participants around the target vehicle within a preset time period. Based on the continuous position coordinates, the future trajectories of traffic participants around the target vehicle are determined.

7. The method according to claim 1, characterized in that, The future trajectory includes continuous location points, and after predicting the future trajectory of each vehicle-around traffic participant based on the interaction features, the method further includes: The motion state of each traffic participant is obtained by performing numerical difference based on the coordinates and timestamps of continuous position points of each traffic participant around the vehicle. The motion state includes position, velocity and acceleration. Based on the current motion state of the vehicle and the motion state of each traffic participant, predict the time interval between the upcoming collision between the vehicle and each traffic participant around the vehicle. If the time interval between an impending collision between a vehicle and a traffic participant around the target vehicle falls within the threshold level range corresponding to the category information of the traffic participant around the target vehicle, a warning signal matching the threshold level range is triggered.

8. A device for predicting the trajectory of traffic participants around a vehicle, characterized in that, The device includes: The acquisition unit is used to acquire continuous image frames of the vehicle surrounding environment, and to acquire the category information and trajectory sequence of each traffic participant around the vehicle in the continuous image frames of the vehicle surrounding environment; The splicing unit is used to splice the category feature vector and trajectory feature vector of each vehicle-around traffic participant to obtain a composite feature vector of each vehicle-around traffic participant; wherein, the category feature vector is generated by encoding the category information of each vehicle-around traffic participant, and the trajectory feature vector is generated by mapping the trajectory sequence of each vehicle-around traffic participant. The prediction unit is used to construct the interaction features of each traffic participant around the vehicle through an attention mechanism based on the composite feature vector, and predict the future trajectory of each traffic participant around the vehicle based on the interaction features.

9. An electronic device, characterized in that, It includes a communication interface, a processor, a memory, and a bus, wherein the communication interface, the processor, and the memory are interconnected via the bus; The memory stores machine-readable instructions, and the processor executes the method according to any one of claims 1 to 7 by invoking the machine-readable instructions.

10. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores machine-readable instructions, which, when invoked and executed by a processor, implement the method described in any one of claims 1 to 7.

Images