Motion trend prediction method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a motion trend prediction method and device, electronic equipment and a storage medium. A target model is determined; acquiring four-dimensional millimeter wave radar data for the target object; constructing a classification and behavior prediction target list for the target object based on the four-dimensional millimeter wave radar data; inputting the classification and behavior prediction target list into the target model, determining a time sequence feature sequence used for representing the target model through the target model, and determining a motion motivation of the target model based on the time sequence feature sequence; and controlling the target model, outputting the motion trend of the target object based on the motion motivator, realizing introduction of depth dynamic image reading AI, and performing depth behavior analysis and high-precision motion trend prediction on mass radar data through the depth dynamic image reading AI. And the decision accuracy, real-time performance and safety of the automatic driving system in a complex dynamic environment are greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of motion trend prediction, in particular to a motion trend prediction method, a motion trend prediction device, an electronic device and a computer readable storage medium. BACKGROUND

[0002] With the rapid development of autonomous driving technology, vehicle-mounted sensors, as the "eyes" of vehicles to perceive the surrounding environment, their performance becomes crucial. The stable working ability of traditional millimeter wave radars in bad weather and light conditions makes them indispensable sensors.

[0003] Four-dimensional (4D) millimeter wave radars with accurate height perception capabilities can provide the distance, speed, azimuth angle and elevation angle of the target, thereby constructing rich three-dimensional point cloud data and significantly improving the understanding of the environment, especially in distinguishing "on-road" and "in-air" targets.

[0004] However, although four-dimensional millimeter wave radars can provide accurate three-dimensional point cluster data, current autonomous driving systems have significant bottlenecks in real-time and efficient processing of massive data and deep understanding of target behavior and intent, hindering their potential in complex traffic scenarios. SUMMARY

[0005] Embodiments of the present application provide a motion trend prediction method, device, electronic device and computer readable storage medium to overcome the above problems or at least partially solve the above problems.

[0006] Embodiments of the present application disclose a motion trend prediction method, comprising: determining a target model; acquiring four-dimensional millimeter wave radar data for a target object; constructing a classification and behavior prediction target list for the target object based on the four-dimensional millimeter wave radar data; inputting the classification and behavior prediction target list into the target model, determining a time sequence feature sequence for representing the target model through the target model, and determining a motion motive of the target model based on the time sequence feature sequence; controlling the target model, and outputting a motion trend of the target object based on the motion motive.

[0007] Optionally, the step of constructing a classification and behavior prediction target list for the target object based on the four-dimensional millimeter wave radar data comprises: determining the original perception information of potential reflection points from the four-dimensional millimeter wave radar data to generate an original perception point list; Traverse the original sensing point list, perform confidence assessment and filtering operations on the potential reflection points in the original sensing point list to determine the target reflection point, and use the target reflection point to generate a three-dimensional high-confidence sensing point list; Traverse the list of three-dimensional high-confidence sensing points, transform and calibrate the target reflection points in the list of three-dimensional high-confidence sensing points from the radar coordinate system to the vehicle coordinate system, and generate a calibrated three-dimensional high-confidence point cloud; Points belonging to the same physical target are identified from the calibrated 3D high-confidence point cloud, and an aggregation operation is performed on the points belonging to the same physical target to generate a 3D point cluster list; the 3D point cluster list includes the 3D position information, velocity information and size information of the target object; Traverse the list of three-dimensional point clusters, extract the geometric and kinematic properties of the point clusters representing the target object, and generate a list of target geometric motion properties; Based on the target geometric motion attribute list, determine the category information of the target object represented by each point cluster, and generate a classification target list based on the category information; Acquire historical data trajectory information, associate each target object in the classification target list with the historical data trajectory information, and generate a target list containing multiple frames of historical data; Based on the target list containing multiple frames of historical data, preliminary behavior prediction information of the target objects is determined; A unique identifier is determined for the point cluster representing the target object, and a classification and behavior prediction target list is generated based on the unique identifier, the three-dimensional position information, the velocity information, the size information, the category information, the preliminary behavior prediction information, and the historical data trajectory information.

[0008] Optionally, the step of determining the temporal feature sequence used to characterize the target model through the target model includes: Based on the classification and behavior prediction target list, historical perception data of the target model in consecutive frames is extracted, and the historical perception data is used to generate the original target historical data stream; Perform time window truncation and normalization operations on the generated original target historical data stream to generate standardized time series input; The target model is used to extract high-dimensional abstract features of the target object in the time dimension based on the standardized temporal input, and the high-dimensional abstract features are used to generate a target temporal feature tensor; The target model is used to perform a dimensionality reduction operation on the target temporal feature tensor, transforming the target temporal feature tensor into a temporal feature sequence.

[0009] Optionally, the step of determining the motion motivation of the target model based on the temporal feature sequence includes: The potential motivational features of the target object are determined based on the temporal feature sequence; Acquire the temporal characteristics of the environmental interaction between the target object and other objects surrounding the target object; Based on the potential motivational features and the environmental interaction timing features, comprehensive motion motivation judgment data is generated. The generated comprehensive motion motivation judgment data is used to determine the motion motivation.

[0010] Optionally, the step of outputting the motion trend of the target object based on the motion motivation includes: Determine the confidence level of the stated motivation for movement; When the confidence level is determined to be greater than or equal to a preset threshold, the motion trend corresponding to the motion motivation is determined and output.

[0011] Optionally, it also includes: When the confidence level is determined to be lower than a preset threshold, the motion motivation is marked as invalid data; Delete the invalid data.

[0012] Optionally, it also includes: When the confidence level is determined to be lower than a preset threshold, the motion motivation is marked as low-confidence data; Cross-validation is performed between the low-confidence data and other motion motivations with confidence levels greater than or equal to a preset threshold to generate validation results for the motion motivations. When the verification result meets the preset conditions, the motion trend corresponding to the motion motivation is determined and output.

[0013] This invention also discloses a motion trend prediction device, comprising: The target model determination module is used to determine the target model; The four-dimensional millimeter-wave radar data acquisition module is used to acquire four-dimensional millimeter-wave radar data for targeting objects. The classification and behavior prediction target list construction module is used to construct a classification and behavior prediction target list for the target object based on the four-dimensional millimeter-wave radar data; The motion motivation determination module is used to input the classification and behavior prediction target list into the target model, determine the temporal feature sequence used to characterize the target model through the target model, and determine the motion motivation of the target model based on the temporal feature sequence; The motion trend output module is used to control the target model and output the motion trend of the target object based on the motion motivation.

[0014] This invention also discloses an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes a program stored in the memory, it implements the method described in the embodiments of the present invention.

[0015] This invention also discloses a computer-readable storage medium storing instructions that, when executed by one or more processors, cause the processors to perform the methods described in this invention.

[0016] The embodiments of the present invention have the following advantages: This invention, through the following embodiments, involves: determining a target model; acquiring four-dimensional millimeter-wave radar data for the target object; constructing a classification and behavior prediction target list for the target object based on the four-dimensional millimeter-wave radar data; inputting the classification and behavior prediction target list into the target model; determining a temporal feature sequence to characterize the target model through the target model; determining the motion motivation of the target model based on the temporal feature sequence; controlling the target model to output the motion trend of the target object based on the motion motivation; and introducing depth dynamic image reading AI. This allows for depth behavior analysis and high-precision motion trend prediction of massive amounts of radar data using depth dynamic image reading AI, significantly improving the decision-making accuracy, real-time performance, and safety of autonomous driving systems in complex dynamic environments. Attached Figure Description

[0017] Figure 1 This is a flowchart of the steps of a motion trend prediction method provided in an embodiment of the present invention; Figure 2 This is a structural block diagram of a motion trend prediction device provided in an embodiment of the present invention; Figure 3 This is a hardware structure block diagram of an electronic device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of a computer-readable medium provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] With the rapid development of autonomous driving technology, the performance of onboard sensors, which serve as the "eyes" of vehicles to perceive their surroundings, has become crucial. The stable operation of traditional millimeter-wave radar in adverse weather and lighting conditions makes it an indispensable sensor.

[0020] Four-dimensional (4D) millimeter-wave radar with precise altitude sensing capabilities can provide the target's range, velocity, azimuth, and elevation angles, thereby constructing rich three-dimensional point cloud data, significantly improving the ability to understand the environment, especially achieving a breakthrough in distinguishing between "road" and "air" targets.

[0021] However, although four-dimensional millimeter-wave radar can acquire precise point cluster data containing the horizontal direction, height, distance, and velocity of a target object, current autonomous driving systems still face significant challenges in efficiently utilizing this data. The main problem is: First, the amount of data generated by four-dimensional millimeter-wave radar far exceeds that of traditional radar, especially in complex scenarios where the number and density of point clusters increase dramatically. Existing data processing frameworks often struggle to efficiently extract features, recognize patterns, and understand semantics from such massive amounts of data in the highly demanding real-time environment of autonomous driving. This means that there is a significant processing bottleneck between the raw, precise point clusters and the high-level decision-making information available at higher levels.

[0022] Secondly, even with precise point clusters, transforming them into a deep understanding of target behavior and intent remains a challenge. Current processing methods may focus more on the geometric and kinematic properties of the target, making it difficult to fully utilize this precise three-dimensional information to infer the motivation behind the target's actions, predict its future trajectory and curves, or determine the target's authenticity. This lack of processing depth and efficiency prevents the full potential of four-dimensional millimeter-wave radar from being realized, making it difficult for the system to make optimal and safest decisions in complex, dynamic, and uncertain traffic environments. Therefore, how to efficiently and deeply process and understand four-dimensional millimeter-wave radar data is a key technical challenge that urgently needs to be solved in the field of autonomous driving.

[0023] Reference Figure 1 The diagram illustrates a flowchart of a motion trend prediction method provided in an embodiment of the present invention, which may specifically include the following steps: Step 101, Determine the target model; Target Model: Refers to Deep Dynamic Image Reading AI (DDIRAI). DDIRAI is not simply about "looking" at images or videos; its key feature is: 1. AI Technology Field: It is clearly stated that DDIRAI belongs to the category of artificial intelligence, which means that it utilizes advanced technologies such as machine learning and deep learning.

[0024] 2. Understanding dynamic changes through images: "Through its images": This indicates that it primarily takes visual data (images, video streams) as input.

[0025] "Dynamic changes of the target": This emphasizes that it deals with information in the time dimension, that is, the changes in the target's movement, posture, state, etc. over time.

[0026] 3. Core competency: Analyzing, interpreting, and judging motivations / reasons: This goes far beyond traditional "recognition" or "tracking." DDIRAI can not only know that a target is moving, but also analyze why it is moving and the motivations or underlying reasons behind the action. This is a relatively advanced reasoning ability that requires understanding physical laws, behavioral patterns, and situational context.

[0027] For example, it might not only recognize that a person is running, but also determine whether they are "chasing something" or "running away." Or it might determine whether a car is slowing down because it is "waiting at a red light" or "avoiding an obstacle."

[0028] 4. Measure the trend and curve of change: Based on its understanding of motivation and cause, DDIRAI is able to predict the future dynamics of a target, including its trajectory (“path and curve”) and state changes.

[0029] This is crucial for applications that require anticipation and planning, such as autonomous driving.

[0030] 5. Determine the authenticity or deception of the target: This is an important capability, meaning that DDIRAI can identify false targets, camouflaged targets, or false alarms in the environment. This may involve in-depth analysis of subtle features of reflected signals, consistency of motion patterns, or matching with the background to distinguish real entities from illusions.

[0031] In radar perception, this helps distinguish real obstacles from false targets caused by multipath reflections, radar sidelobes, etc.

[0032] Purpose: Establish the core of the analysis: Clarify which agent or algorithm will be used in subsequent steps to perform high-level target behavior analysis and prediction.

[0033] Specialized division of labor: Separating the processing of raw radar data (point cloud generation, preliminary classification) from higher-level behavioral understanding and prediction to achieve modular and specialized processing.

[0034] Beneficial effects: Improve processing efficiency: Encapsulate complex target behavior analysis tasks in specialized models, making the responsibilities of the entire perception chain clearer and easier to optimize.

[0035] Improve prediction accuracy: By learning from massive amounts of real data, pre-trained target models can more accurately capture the dynamic patterns of targets, laying the foundation for subsequent accurate predictions.

[0036] Step 102: Acquire four-dimensional millimeter-wave radar data for targeting the object; 4D Millimeter-Wave Radar Data: This refers to the raw or pre-processed data directly output from a 4D millimeter-wave radar sensor. This data includes the target's position (range, azimuth, and elevation) in three-dimensional space, as well as its relative velocity. It is richer than traditional radar data, especially in containing precise altitude (elevation) information.

[0037] Purpose: Data source supply: Provides the most basic and comprehensive radar input for the entire system to perform environmental perception. This is the foundation for all subsequent high-level processing.

[0038] All-weather perception: Leveraging the advantages of millimeter-wave radar in adverse weather and low-light conditions, it ensures the acquisition of reliable perception data in various environments.

[0039] Beneficial effects: Acquire rich raw information: Provide the system with more detailed and accurate raw environmental data than traditional radar, especially critical altitude information.

[0040] All-weather operation capability: Ensure that the autonomous driving system can effectively perceive the environment under various weather and lighting conditions, thereby improving the robustness of the system.

[0041] Step 103: Construct a target list for classification and behavior prediction of the target object based on the four-dimensional millimeter-wave radar data; Classification and Behavior Prediction Target List: This structured dataset contains each individual target entity identified by the radar system (e.g., a car, a pedestrian, a traffic cone). Each target in the list has its current precise perception attributes (such as 3D position, velocity, and size) as well as the results of preliminary processing and classification (such as the target's specific category and basic motion pattern information). This list is a high-level representation of the raw radar data after initial intelligent processing.

[0042] Purpose: Data abstraction and structuring: Clustering, segmenting and transforming raw, scattered four-dimensional radar data (such as point clouds) into meaningful, structured target entity information to facilitate subsequent high-level processing.

[0043] Preliminary semanticization: The target is initially classified into object categories, giving the data a certain semantic meaning.

[0044] Provide input for the target model: Generate a list of preliminary classification and motion attributes that the target model (the model in step 101) can directly process.

[0045] Beneficial effects: Improve data processing efficiency: Abstracting massive amounts of raw radar data into a finite and structured list of targets significantly reduces the complexity of subsequent analysis.

[0046] Improved target recognition accuracy: Through clustering and classification, the system can distinguish and identify different physical targets, overcoming the discreteness of radar point clouds.

[0047] Laying the foundation for behavioral understanding: It provides inputs containing key information such as target identity, location, and motion state for subsequent target models, and is a prerequisite for motivation judgment and trend prediction.

[0048] Step 104: Input the classification and behavior prediction target list into the target model, determine the temporal feature sequence used to characterize the target model through the target model, and determine the motion motivation of the target model based on the temporal feature sequence; Temporal Feature Sequence: This refers to the multi-dimensional feature data stream that the target model extracts from the input "classification and behavior prediction target list" for each target, and that varies over time. This includes historical information such as the target's position, velocity, size, category, and interactions with other targets over a past period. This information is organized into a continuous sequence that reflects the dynamic evolution of the target.

[0049] Movement motivation refers to the underlying reasons or intentions that drive the current or future movement of a target. This is not a simple physical quantity, but rather a semantic understanding of the target's behavior. Examples include "lane change intention," "stopping and waiting," "accelerating to overtake," and "avoiding pedestrians."

[0050] Purpose: Deep Behavior Analysis: Leveraging the powerful learning and reasoning capabilities of target models, this feature mines the target's movement patterns and interaction patterns from historical time-series data, and infers the underlying intentions behind its behavior.

[0051] From "what" to "why": Moving beyond simply identifying target attributes to understanding the reasons behind target behavior is a crucial step in achieving intelligent driving decision-making.

[0052] Beneficial effects: Achieving a higher level of scene understanding: The vehicle no longer just knows "there is a car moving in front", but understands "the vehicle in front is slowing down and may be preparing to stop and give way".

[0053] Improving prediction accuracy: Understanding the motivation behind movement is fundamental to more accurately predicting the future movement trend of a target. For example, knowing whether a vehicle's deceleration is for changing lanes or stopping will greatly affect the predicted trajectory.

[0054] Enhancing the robustness of decision-making: Judging motivation helps distinguish between normal and abnormal behavior, improving the ability of autonomous driving systems to cope with emergencies.

[0055] Step 105: Control the target model to output the motion trend of the target object based on the motion motivation.

[0056] Controlling the target model: refers to the system managing and scheduling the operation of the target model so that it generates specific prediction results based on the determined motion motivation.

[0057] Movement Trend: Refers to the possible trajectory, velocity changes, and state evolution of a target object over a future period of time. This is typically represented as a series of predicted future positions, velocity vectors, or probability distributions, rather than a single numerical value.

[0058] Purpose: Generate actionable predictions: Transform the understanding of motion motivation into specific, quantifiable predictions of future motion, providing direct input for decision-making and planning in autonomous driving.

[0059] Empowering proactive decision-making: The system can plan routes and adjust speeds in advance based on predictions of future trends, rather than passively responding to events that have already occurred.

[0060] Beneficial effects: Achieving proactive driving: Autonomous vehicles can anticipate the behavior of other road users, just like human drivers, and thus make smoother and safer driving decisions.

[0061] Improve driving efficiency and comfort: Predictive planning can avoid sudden braking and acceleration, enhance the passenger experience, and optimize energy consumption.

[0062] Enhancing the safety of autonomous driving: Accurately predicting the future movements of other vehicles or pedestrians is key to avoiding collisions and handling complex traffic scenarios, especially in high-risk areas such as intersections and merging into main roads.

[0063] Improve the decision-making cycle: Organically combine perception, understanding, and prediction to form a complete intelligent decision-making closed loop.

[0064] This invention, through the following embodiments, involves: determining a target model; acquiring four-dimensional millimeter-wave radar data for the target object; constructing a classification and behavior prediction target list for the target object based on the four-dimensional millimeter-wave radar data; inputting the classification and behavior prediction target list into the target model; determining a temporal feature sequence to characterize the target model through the target model; determining the motion motivation of the target model based on the temporal feature sequence; controlling the target model to output the motion trend of the target object based on the motion motivation; and introducing depth dynamic image reading AI. This allows for depth behavior analysis and high-precision motion trend prediction of massive amounts of radar data using depth dynamic image reading AI, significantly improving the decision-making accuracy, real-time performance, and safety of autonomous driving systems in complex dynamic environments.

[0065] Based on the above embodiments, modified embodiments of the above embodiments are proposed. It should be noted that, in order to keep the description brief, only the differences from the above embodiments are described in the modified embodiments.

[0066] In an optional embodiment of the present invention, the step of constructing a target list for classification and behavior prediction of the target object based on the four-dimensional millimeter-wave radar data includes: The original sensing information of potential reflection points is determined from the four-dimensional millimeter-wave radar data, and an original sensing point list is generated; Traverse the original sensing point list, perform confidence assessment and filtering operations on the potential reflection points in the original sensing point list to determine the target reflection point, and use the target reflection point to generate a three-dimensional high-confidence sensing point list; Traverse the list of three-dimensional high-confidence sensing points, transform and calibrate the target reflection points in the list of three-dimensional high-confidence sensing points from the radar coordinate system to the vehicle coordinate system, and generate a calibrated three-dimensional high-confidence point cloud; Points belonging to the same physical target are identified from the calibrated 3D high-confidence point cloud, and an aggregation operation is performed on the points belonging to the same physical target to generate a 3D point cluster list; the 3D point cluster list includes the 3D position information, velocity information and size information of the target object; Traverse the list of three-dimensional point clusters, extract the geometric and kinematic properties of the point clusters representing the target object, and generate a list of target geometric motion properties; Based on the target geometric motion attribute list, determine the category information of the target object represented by each point cluster, and generate a classification target list based on the category information; Acquire historical data trajectory information, associate each target object in the classification target list with the historical data trajectory information, and generate a target list containing multiple frames of historical data; Based on the target list containing multiple frames of historical data, preliminary behavior prediction information of the target objects is determined; A unique identifier is determined for the point cluster representing the target object, and a classification and behavior prediction target list is generated based on the unique identifier, the three-dimensional position information, the velocity information, the size information, the category information, the preliminary behavior prediction information, and the historical data trajectory information.

[0067] In a specific implementation, embodiments of the present invention can determine the original sensing information of potential reflection points from the four-dimensional millimeter-wave radar data and generate an original sensing point list. Four-dimensional millimeter-wave radar data refers to the raw or low-level processed echo signal data received by the radar, which includes information such as frequency and phase. This information can ultimately be used to determine the target's range, velocity, azimuth, and elevation angle (the fourth dimension), serving as the raw sensing information.

[0068] Potential reflection point: refers to the peak point of the target reflection signal that may be identified through preliminary signal processing (such as CFAR detection).

[0069] CFAR (Constant False Alarm Rate) detection is a radar signal processing technique that dynamically adjusts the detection threshold to adapt to changes in background noise and clutter, thereby maintaining a constant false alarm probability in various environments and effectively identifying targets.

[0070] Original sensing point list: Contains basic attributes of each potential reflection point, such as preliminary estimates of their distance, velocity, azimuth, and elevation angles in the radar coordinate system.

[0071] Purpose: This is the starting point for data processing, transforming the raw radar signal into a series of discrete, unfiltered measurements of potential targets.

[0072] Beneficial effects: It lays the foundation for all subsequent sensing processing, converting continuous radar echoes into discrete data points that can be computed and processed.

[0073] In a specific implementation, the embodiments of the present invention can traverse the original sensing point list, perform confidence assessment and filtering operations on the potential reflection points in the original sensing point list to determine the target reflection point, and use the target reflection point to generate a three-dimensional high-confidence sensing point list; Confidence assessment and filtering: This embodiment of the invention can perform reliability assessment on each potential reflection point, typically based on its signal strength (SNR), duration, and difference from known noise patterns. Potential reflection points with confidence levels below a preset threshold are removed, and they are considered noise or false alarms.

[0074] Target reflection point: A reliable data point that is considered to be effectively reflected by a real target after confidence assessment.

[0075] The 3D high-confidence sensing point list contains all verified, high-quality radar reflection points, each with precise 3D position (calculated from range, azimuth, and elevation angles) and velocity information.

[0076] Objective: To eliminate noise and false detections to ensure that the data processed subsequently is reliable and meaningful.

[0077] Beneficial effects: Significantly improved the purity and reliability of point clouds, reduced misjudgments caused by erroneous data, and laid a solid foundation for subsequent accurate perception.

[0078] In a specific implementation, the embodiments of the present invention can traverse the list of three-dimensional high-confidence sensing points, transform and calibrate the target reflection points in the list of three-dimensional high-confidence sensing points from the radar coordinate system to the vehicle coordinate system, and generate a calibrated three-dimensional high-confidence point cloud. Radar coordinate system: A coordinate system with the radar sensor itself as the origin, usually with the X-axis pointing forward and the Y-axis pointing to the side.

[0079] Vehicle coordinate system: A coordinate system with a fixed point on the vehicle (such as the center of the vehicle body) as the origin, which is related to the vehicle's direction of motion and attitude.

[0080] Transformation and calibration: The point coordinates in the radar coordinate system are mapped to the vehicle coordinate system through geometric transformation (rotation and translation), and measurement deviations caused by radar installation errors or changes in vehicle attitude (pitch, roll, yaw) are corrected.

[0081] Calibrated 3D high-confidence point cloud: A high-quality set of points after coordinate system 1 and error correction, whose coordinates can accurately reflect the position of the points in the real space around the vehicle.

[0082] Objective: To integrate radar perception data with data from the vehicle itself and other sensors (such as cameras and lidar) within a unified reference framework, facilitating overall system understanding and fusion.

[0083] Beneficial effects: Improved spatial accuracy and consistency of perception data, providing reliable input for multi-sensor fusion and decision-making such as path planning and obstacle avoidance in autonomous driving.

[0084] In a specific implementation, embodiments of the present invention can determine points belonging to the same physical target from the calibrated three-dimensional high-confidence point cloud, and perform an aggregation operation on the points belonging to the same physical target to generate a three-dimensional point cluster list; the generated three-dimensional point cluster list includes the three-dimensional position information, velocity information and size information of the target object; Points belonging to the same physical target: refers to radar reflection points originating from the same physical object (such as a vehicle, a pedestrian, or a road sign).

[0085] Aggregation operation: Using clustering algorithms (such as DBSCAN or other AI-based semantic / instance segmentation methods) to group discrete points belonging to the same physical target together to form a "cluster".

[0086] 3D point cluster list: A collection containing all identified individual physical targets. Each point cluster is no longer a single point, but a set of points, along with preliminary calculations of the target's 3D position, velocity, and size (e.g., estimated via the point cluster bounding box).

[0087] Objective: To organize discrete point data into meaningful, independent physical entities, moving from "seeing points" to "seeing objects".

[0088] Beneficial effects: It greatly simplifies data complexity, enabling the system to process higher-level "objects" rather than raw points; at the same time, it provides structured input for subsequent target recognition and behavior prediction.

[0089] In a specific implementation, embodiments of the present invention can traverse the three-dimensional point cluster list, extract the geometric and kinematic properties of the point clusters representing the target object, and generate a target geometric motion property list. Geometric and kinematic properties include geometric properties and kinematic properties: Geometric attributes: mainly refer to the shape and size information of the target in space (such as length, width, height, center point coordinates, orientation angle, etc.), as well as its accurate position in three-dimensional space (height information can be obtained through the Z value distribution inside the point cluster).

[0090] Kinematic properties: These mainly refer to the target's motion state (such as its average velocity vector, acceleration, angular velocity, etc.), which are estimated by comprehensively considering the velocity information of all points within the point cluster.

[0091] List of target geometric motion attributes: Contains precise geometric information (position, size) and kinematic information (velocity, acceleration) for each identified target object.

[0092] Objective: To further refine and quantify the core measurable characteristics of the physical target represented by each point cluster.

[0093] Beneficial effects: It provides highly accurate and comprehensive target physical attributes, which serve as a direct basis for subsequent target classification and behavior prediction.

[0094] In a specific implementation, embodiments of the present invention can determine the category information of the target object represented by each point cluster based on the target geometric motion attribute list, and generate a classification target list based on the category information; Category information: refers to the specific category label of the target object, such as "pedestrian", "bicycle", "car", "truck", "obstacle", etc.

[0095] Classification Target List: Contains the geometric and kinematic attributes of all identified targets, and adds a specific object category for each target.

[0096] Objective: To infer the semantic meaning of a target from its geometric and kinematic features, i.e., "What is it?".

[0097] Beneficial effects: It represents a key step from physical perception to semantic understanding, enabling autonomous driving systems to "recognize" different traffic participants and environmental elements, thereby allowing them to adopt more targeted driving strategies.

[0098] In a specific implementation, the embodiments of the present invention can obtain historical data trajectory information, associate each target object in the classification target list with the historical data trajectory information, and generate a target list containing multiple frames of historical data; Historical data trajectory information: refers to the data sequence of changes in attributes such as position, velocity, size, and category of the same target that the system has continuously tracked and recorded over multiple consecutive time frames in the past.

[0099] In this embodiment of the invention, the target detected in the current frame can be matched with the corresponding target in the past frame by a target tracking algorithm (such as Kalman filtering, Hungarian algorithm, etc.) to ensure the temporal continuity and consistency of the data.

[0100] A target list containing multiple frames of historical data: It includes the attributes of each identified target at the current moment, along with a complete history of the target's motion and attributes from its appearance to the present (or within a set time period in the past).

[0101] Objective: To introduce time-based information to construct a complete dynamic history of the target, providing a foundation for in-depth behavioral analysis and trend prediction.

[0102] Beneficial effects: It enables the system to understand the dynamic evolution of the target object, rather than just its instantaneous state; this is key to predicting the target's future behavior and judging its motivation.

[0103] In a specific implementation, embodiments of the present invention can determine preliminary behavior prediction information of the target object based on the target list containing multiple frames of historical data; Preliminary behavior prediction information: This refers to a preliminary judgment based on the target object's category and historical motion data, regarding the simple actions or trends the target object may take in the short term. This may include "stationary," "uniform linear motion," "deceleration," "acceleration," "turning," etc., and is a rough version of more advanced behavior prediction.

[0104] Objective: To add preliminary behavioral predictions based on historical data for each target before the final target list is generated.

[0105] Beneficial effects: It provides a good starting point for subsequent more complex "target models" (such as DDIRAI), reduces the complexity of their processing, and can support rapid response to some extent.

[0106] In a specific implementation, embodiments of the present invention may also determine a unique identifier for the point cluster representing the target object, and generate a classification and behavior prediction target list based on the unique identifier, the three-dimensional position information, the velocity information, the size information, the category information, the preliminary behavior prediction information, and the historical data trajectory information; Object Unique Identifier: A unique ID assigned to each identified and tracked object, used to continuously reference and track the object throughout the perception and decision-making system.

[0107] Classification and Behavior Prediction Target List: The final output of the entire process. This is a highly structured dataset containing rich semantic and predictive information, where each entry represents an independent target understood and predicted by the system.

[0108] Objective: To integrate all processed, analyzed, and predicted information to form a complete perception result that can be directly used by high-level decision-making modules.

[0109] Beneficial effects: It achieves a complete transformation from raw radar signals to advanced, decision-making information. This list serves as a direct input for autonomous driving systems to perform path planning, obstacle avoidance, behavior prediction, and control, ensuring that the system can efficiently and accurately understand complex dynamic environments, thereby making safe and intelligent driving decisions.

[0110] In an optional embodiment of the present invention, the step of determining the temporal feature sequence for characterizing the target model using the target model includes: Based on the classification and behavior prediction target list, historical perception data of the target model in consecutive frames is extracted, and the historical perception data is used to generate the original target historical data stream; Perform time window truncation and normalization operations on the generated original target historical data stream to generate standardized time series input; The target model is used to extract high-dimensional abstract features of the target object in the time dimension based on the standardized temporal input, and the high-dimensional abstract features are used to generate a target temporal feature tensor; The target model is used to perform a dimensionality reduction operation on the target temporal feature tensor, transforming the target temporal feature tensor into a temporal feature sequence.

[0111] In a specific implementation, embodiments of the present invention can extract historical perception data of the target model in consecutive frames based on the classification and behavior prediction target list, and use the historical perception data to generate an original target historical data stream. Historical sensing data in consecutive frames: refers to the structured information collected by the system for the same target at multiple consecutive time points (or radar frames).

[0112] Original target historical data stream: The attributes (position, velocity, etc.) of the same target collected at different time points are arranged in chronological order to form an original sequence of data.

[0113] Objective: To collect and organize information on the dynamic changes of each target over a period of time, which is the basis for conducting time series analysis.

[0114] Beneficial effects: It leaps from instantaneous data points to sequential data with temporal context, laying the foundation for understanding the dynamic behavior of targets and allowing the system to "see" the movement process of targets, rather than just the state at a certain moment.

[0115] In a specific implementation, embodiments of the present invention can perform time window truncation and normalization operations on the generated original target historical data stream to generate standardized time-series input. Time window extraction: Extracting a specific length of data (e.g., the most recent 3 seconds or the most recent 10 frames) from the complete historical data stream for analysis. This is because excessively long historical data may contain irrelevant old information, while data that is too short is insufficient to capture trends.

[0116] Normalization: Scaling raw numerical features (such as position, velocity, size) of different dimensions (different units) or ranges to a uniform range (e.g., between 0 and 1), or making them have zero mean and unit variance.

[0117] Standardized time series input: Time series data that has been truncated and normalized and can be directly input into deep learning models for training and inference. It has a uniform format and stable values.

[0118] Purpose: Optimize input format: Transform raw, potentially irregular data into a uniform and stable format that is easier for deep learning models to process.

[0119] Improving model performance: Normalization can prevent certain features with large numerical ranges from dominating the model training process, which helps the model converge faster, improves training efficiency and prediction accuracy.

[0120] Beneficial effects: Improved training and inference efficiency of deep learning models, and enhanced the generalization ability and robustness of the models, enabling them to better adapt to data changes in different scenarios.

[0121] In a specific implementation, embodiments of the present invention can use the target model to extract high-dimensional abstract features of the target object in the time dimension based on the standardized temporal input, and use the high-dimensional abstract features to generate a target temporal feature tensor. In a practical implementation, the target model can be a deep learning network model, which includes an encoder part. This encoder (such as a recurrent neural network, a Transformer encoder, etc.) is designed to process sequential data.

[0122] High-dimensional abstract features: Encoders can transform raw, easily understood features (such as position and velocity) into more abstract and expressive numerical representations through multiple layers of nonlinear transformations. These abstract features can capture complex patterns and correlations in the raw data that are not easily observed directly.

[0123] Target temporal feature tensor: These high-dimensional abstract features output by the encoder are usually represented in the form of tensors (multidimensional arrays). Different dimensions of the tensor may represent time steps, feature dimensions, etc. It is a compact but information-rich numerical representation that can represent the dynamic behavior of the target.

[0124] Objective: Deep learning models, through their powerful feature extraction capabilities, automatically learn and extract the most useful latent patterns that represent the dynamic changes of a target from complex time-series data.

[0125] Beneficial effects: It realizes the automatic transformation from raw perceptual data to high-level semantic features, avoiding the complexity of traditional manual feature engineering; the features learned by the model are more expressive, providing a more solid foundation for subsequent motivation judgment and trend prediction.

[0126] In a specific implementation, embodiments of the present invention can use the target model to perform a dimensionality reduction operation on the target temporal feature tensor, transforming the target temporal feature tensor into a temporal feature sequence. Dimensionality reduction: The high-dimensional abstract feature tensor generated in the previous step is processed through specific layers inside the deep learning model (such as fully connected layers and pooling layers) to reduce its dimensionality while retaining the most important information.

[0127] Temporal feature sequence: The result after dimensionality reduction is still a sequence, but the feature dimension at each time step may be lower, or the entire sequence may be compressed into a single feature vector representing the entire temporal series. This "sequence" may refer to a final, compact single feature vector representing the entire historical time series, or a simplified version of the sequence.

[0128] Purpose: Refined features: Further remove redundant information and retain the most essential information that best represents the dynamic changes of the target.

[0129] Adapting to subsequent tasks: Transforming high-dimensional and complex features into a more concise format that is better suited for input into subsequent behavior analysis and prediction modules (such as another decoder or classifier).

[0130] Beneficial effects: Improved processing efficiency of subsequent models, reduced computational resource consumption, and ensured the integrity of core information; helped prevent overfitting and made the model more focused on the most important behavioral patterns.

[0131] For example, suppose there is an autonomous vehicle driving and its four-dimensional millimeter-wave radar system is continuously sensing the environment.

[0132] Target object: A car driving in front (Target ID: CAR_001) Assumption: The time series feature sequence needs to contain the target's state data every 0.5 seconds over the past 3 seconds.

[0133] Scenario: The target car CAR_001 suddenly began to decelerate and turn on its turn signal to the right within the past 3 seconds, starting from normal driving.

[0134] 1. Generate the original target historical data stream: T-3.0s Frame: ID:CAR_001, Position: (50.0, 1.5, 0.8)m, Speed: (20.0, 0.0, 0.0)m / s, Dimensions: (4.5, 1.8, 1.5)m, Category: Car, Preliminary Prediction: Uniform Linear Motion; T-2.5s frame: ID:CAR_001, Position: (40.0, 1.5, 0.8)m, Speed: (20.0, 0.0, 0.0)m / s, Size: (4.5, 1.8, 1.5)m, Category: Car, Preliminary prediction: Uniform linear motion; T-2.0s Frame: ID:CAR_001, Position: (30.0, 1.6, 0.8)m, Speed: (18.0, 0.1, 0.0)m / s, Size: (4.5, 1.8, 1.5)m, Category: Car, Preliminary Prediction: Deceleration (Radar may have initially detected deceleration); T-1.5s frame: ID:CAR_001, Position: (22.0, 1.8, 0.8)m, Speed: (15.0, 0.2, 0.0)m / s, Size: (4.5, 1.8, 1.5)m, Category: Car, Preliminary prediction: Deceleration; T-1.0s Frame: ID:CAR_001, Position: (16.0, 2.2, 0.8)m, Speed: (12.0, 0.3, 0.0)m / s, Size: (4.5, 1.8, 1.5)m, Category: Car, Preliminary Prediction: Decelerating and turning right (radar may have initially detected lateral speed); T-0.5s frame: ID:CAR_001, Position: (11.0, 2.8, 0.8)m, Speed: (9.0, 0.4, 0.0)m / s, Size: (4.5, 1.8, 1.5)m, Category: Car, Preliminary prediction: Slowing down and turning right; T-0.0s frame (current frame): ID:CAR_001, Position: (7.0, 3.5, 0.8)m, Speed: (6.0, 0.5, 0.0)m / s, Size: (4.5, 1.8, 1.5)m, Category: Car, Preliminary prediction: Slow down and turn right; 2. Generate standardized timing input: Time window capture: Capture 7 frames of data from T-3.0s to T-0.0s.

[0135] Normalization: All position, velocity, and size data are scaled to the range of 0 to 1. Category information may be encoded as one-hot vectors.

[0136] For example, the original velocity (20.0, 0.0, 0.0) might be normalized to (0.8, 0.5, 0.5) (assuming the maximum velocity is 25 m / s).

[0137] Result: A 7-frame x N feature dimension matrix is ​​obtained, where N is the number of normalized features (such as normalized X, Y, Z, Vx, Vy, Vz, length, width, height, class encoding, etc.).

[0138] 3. Extract high-dimensional abstract features to generate the target temporal feature tensor: Input: The normalized temporal input matrix of 7 frames x N feature dimensions generated in the previous step.

[0139] Target model (encoder): This is a pre-trained deep neural network (e.g., an LSTM network or a Transformer encoder). It receives this matrix as input.

[0140] Execution: The encoder processes the input sequence frame by frame and learns the temporal dependencies between frames. It captures the continuous rates of change of velocity and position, as well as the intrinsic relationship between these changes and the target size and category.

[0141] Result: The encoder outputs a high-dimensional target temporal feature tensor, for example, a 1 x D_hidden dimensional vector (if the encoder encodes the entire sequence as a fixed-dimensional vector), where D_hidden can be hundreds or even thousands of dimensions. This tensor is a condensed, abstract representation of the car's dynamic behavior over the past 3 seconds, containing complex patterns such as "it has been decelerating and moving to the right."

[0142] 4. Dimensionality reduction operation, transforming it into a time-series feature sequence: Input: The high-dimensional abstract feature tensor generated in the previous step (1 x D_hidden dimension).

[0143] Target Model (Dimensionality Reduction Layer / Fully Connected Layer): The target model typically contains one or more fully connected layers to further process the high-dimensional features output by the encoder.

[0144] Execution: These layers further refine and compress features to ensure that the final output "temporal feature sequence" is both representative of the dynamics within the entire 3 seconds and sufficiently concise.

[0145] Result: A more compact temporal feature sequence is ultimately generated. This "sequence" here might be a single, refined feature vector (e.g., 1 x M dimensional, where M is much smaller than D_hidden) representing the entire 3-second dynamic behavior, perfectly encapsulating all the temporal information of the car from normal driving to decelerating and turning right. This refined vector will be used as input to the subsequent motion motivation determination module.

[0146] Through the above steps, the system transforms the raw, scattered radar perception data into a highly abstract, condensed, and semantically rich temporal feature sequence. This sequence is a powerful basis for subsequent judgment of the target's movement motivation (e.g., "It is slowing down and turning right, its intention may be to turn at the intersection ahead") and prediction of its future movement trend.

[0147] In an optional embodiment of the present invention, the step of determining the motion motivation of the target model based on the temporal feature sequence includes: The potential motivational features of the target object are determined based on the temporal feature sequence; Acquire the temporal characteristics of the environmental interaction between the target object and other objects surrounding the target object; Based on the potential motivational features and the environmental interaction timing features, comprehensive motion motivation judgment data is generated. The generated comprehensive motion motivation judgment data is used to determine the motion motivation.

[0148] In a specific implementation, embodiments of the present invention can determine the potential motivational features of the target object based on the temporal feature sequence; Temporal Feature Sequence: This is the output from the previous step. It is a high-dimensional numerical sequence that has been compressed and abstracted, fully summarizing all the dynamic changes and behavioral patterns of a target object over a period of time. It can be understood as a "condensed summary" of the target's historical behavior.

[0149] Latent motivational features: These are more abstract numerical features that the target model (usually a deep learning model) learns and extracts directly from temporal feature sequences. They represent the model's initial understanding of the target's inherent motion tendencies or patterns, which may suggest a specific intention (e.g., a pattern of continuous deceleration may suggest a stopping intention, and a specific lateral movement pattern may suggest a lane-changing intention). They are the "raw cues" for motivational inference.

[0150] Objective: To automatically identify and extract the features that best represent the target's inherent behavioral trends and potential intentions from its own historical movement data.

[0151] Beneficial effects: This step leverages the powerful capabilities of deep learning to automatically capture complex behavioral patterns that are difficult for the human eye to perceive, providing a first layer of semantic understanding and data foundation for subsequent motivation judgments, and avoiding the tedious and potentially inaccurate manual feature engineering in traditional methods.

[0152] In a specific implementation, embodiments of the present invention can acquire the environmental interaction timing characteristics of the target object and other objects surrounding the target object; Temporal characteristics of environmental interaction: These are numerical sequences describing how a target object interacts with its surrounding environment (including other vehicles, pedestrians, lane lines, traffic lights, intersections, etc.) over time. Examples include changes in the relative distance and speed between the target and vehicles ahead, changes in the lateral offset between the target and lane lines, and changes in the speed and position of the target as it approaches an intersection or traffic light.

[0153] Objective: To provide crucial contextual information for judging motivation. A target's motivation for movement is rarely isolated; it is often influenced by the surrounding environment and the behavior of other traffic participants. Understanding these complex interactions is essential for accurately inferring motivation.

[0154] Beneficial effects: It significantly improves the accuracy and plausibility of motivation inference because it considers the real-world traffic context in which the target is situated. This enables AI to understand the "story" unfolding in the environment, rather than just isolated individual movements, thereby generating more realistic predictions and enhancing the robustness of autonomous driving systems.

[0155] In a specific implementation, embodiments of the present invention can generate comprehensive motion motivation judgment data based on the potential motivation characteristics and the environmental interaction timing characteristics; Comprehensive Motivation Judgment Data: This is a numerical representation obtained by fusing and refining the acquired intrinsic behavioral cues (potential motivational characteristics) of the target with previously acquired external environmental contextual information (environmental interaction temporal characteristics). It is the most comprehensive, richest, and most accurate input data for the target's potential motivation, providing a unified decision-making basis for the final motivation judgment.

[0156] Objective: To deeply integrate and consolidate all relevant information from different dimensions (the target itself and the external environment) to form a complete and unambiguous basis for judging motivation.

[0157] Beneficial effects: It enables a more comprehensive and in-depth understanding of target behavior. By integrating internal and external factors, the system can resolve potential ambiguities, more confidently and accurately assess the true intentions of targets, thereby reducing false positives and false negatives in motivational judgments and improving the reliability of decision-making.

[0158] In a specific implementation, embodiments of the present invention may use the aforementioned generated comprehensive motion motivation judgment data to determine motion motivation; Motion Motivation: The target model (usually the last layer of a deep learning network, such as a classification layer) receives comprehensive motion motivation judgment data as input and outputs the most likely semantic motivation label for the target object (e.g., "preparing to turn left", "stopping at a red light", "attempting to overtake", "avoiding an obstacle in front"), usually accompanied by a confidence score indicating the reliability of the judgment.

[0159] Objective: To transform the results of in-depth analysis of target behavior into specific semantic motivational information that can be directly used by high-level decision-making systems.

[0160] Beneficial effects: Predictive driving has been achieved: Understanding the "why" of a target enables autonomous vehicles to more accurately predict "what it will do next," allowing them to plan ahead, drive smoothly, and avoid reactive responses.

[0161] Improved driving safety: It can identify the dangerous intentions of other road users (such as sudden lane changes or emergency braking) earlier and more accurately, thus giving more time to react safely.

[0162] Enhanced natural human-machine interaction: Enables autonomous driving systems to understand and predict the behavior patterns of surrounding vehicles and pedestrians, just like human drivers, thereby improving the system's intelligence level.

[0163] In an optional embodiment of the present invention, the step of outputting the motion trend of the target object based on the motion motivation includes: Determine the confidence level of the stated motivation for movement; When the confidence level is determined to be greater than or equal to a preset threshold, the motion trend corresponding to the motion motivation is determined and output.

[0164] Optionally, it also includes: When the confidence level is determined to be lower than a preset threshold, the motion motivation is marked as invalid data; Delete the invalid data.

[0165] Optionally, it includes: When the confidence level is determined to be lower than a preset threshold, the motion motivation is marked as low-confidence data; Cross-validation is performed between the low-confidence data and other motion motivations with confidence levels greater than or equal to a preset threshold to generate validation results for the motion motivations. When the verification result meets the preset conditions, the motion trend corresponding to the motion motivation is determined and output.

[0166] In a specific implementation, the embodiments of the present invention can determine the confidence level of the motion motivation; Motivation: Refers to the underlying reasons or intentions (e.g., "changing lanes", "stopping", "avoiding") that are inferred from the previous step (e.g., "determining the motivation of the target model based on the time-series feature sequence).

[0167] Confidence level: This is a numerical value, typically between 0 and 1 (or 0% to 100%), representing the system's degree of certainty or reliability regarding the determined motivation for a movement. For example, a system might determine that a car's motivation is "changing lanes," giving a confidence level of 0.95 (95%), indicating that it is very certain; if it were 0.55 (55%), it would indicate that it is less certain.

[0168] Objective: To assess the quality of inferred motion motivation. In autonomous driving, the safety of decision-making is paramount; the system needs to know how reliable its judgment of a target's intent is in order to take appropriate action.

[0169] Beneficial effects: Risk management: Provides a basis for subsequent decision-making, ensuring that only high-confidence motives are used to directly drive vehicle behavior, thereby reducing the risk of accidents caused by misjudgment.

[0170] Enhance robustness: Allow the system to adopt more conservative or safer coping strategies when the motivation judgment is uncertain, thereby improving the overall fault tolerance of the system.

[0171] In a specific implementation, embodiments of the present invention can determine and output the motion trend corresponding to the motion motivation when the confidence level is determined to be greater than or equal to a preset threshold. Preset threshold: A pre-defined lower confidence level (e.g., 0.8 or 80%). A motivation is considered reliable only when the system's confidence in the motivation reaches or exceeds this threshold.

[0172] Motion trend: This refers to the specific trajectory, speed changes, and attitude evolution of a target object over a period of time in the future. This is usually not a single prediction point, but rather a series of future states that change over time, potentially including multiple possible trajectories (with probabilities), or predictions with a range of uncertainty. For example, predicting the position, speed, and orientation of a car every 0.5 seconds over the next 3 seconds.

[0173] Objective: When the motivational judgment of the target reaches a sufficiently reliable level, this semantic motivation is transformed into a quantifiable and specific prediction of future motion, providing direct input to the path planning and control modules of autonomous vehicles.

[0174] Beneficial effects: Achieving accurate predictions: Based on a deep understanding of motivations, the system can generate more accurate and realistic future motion trajectories, avoiding blind predictions. For example, if it is determined that the motivation of the vehicle ahead is "to turn right," its motion trend will be predicted as a rightward curving trajectory and deceleration, rather than straight-line travel.

[0175] Improved driving smoothness and efficiency: Vehicles can anticipate the behavior of other road users earlier, thus making smooth acceleration, deceleration, or lane change decisions in advance, improving ride comfort and fuel efficiency.

[0176] Enabling Active Obstacle Avoidance: When a potential dangerous motive is detected (such as sudden braking or loss of control), the system can immediately predict its dangerous trend and plan an avoidance path in advance, significantly improving the safety of autonomous driving.

[0177] In an optional embodiment of the present invention, when the confidence level is determined to be lower than a preset threshold, the system will take a series of processing measures, because this indicates that the reliability of a certain detection or recognition result of the radar is insufficient and not enough to be directly used for autonomous driving decisions.

[0178] Specifically, when the confidence level is lower than a preset threshold, the following typically occurs: 1. Data points or targets are filtered / discarded; Note: This is the most direct and common processing method. If the confidence level of a single radar point (when building a point cloud) or a complete cluster of points (when identifying a target) is lower than a preset threshold, it is likely to be marked as invalid by the system and not included in subsequent processing or directly removed from the sensing data.

[0179] Objective: To prevent low-quality, uncertain, or potentially noisy or false alarm data from contaminating subsequent decision-making processes. This is similar to filtering out spoiled ingredients before cooking.

[0180] Beneficial effects: Ensures that the data passed to the planning and control modules is reliable, avoids erroneous decisions due to uncertain data, and thus improves the security and robustness of the system.

[0181] 2. Reduce priority or use as supplementary information; Note: Sometimes, low-confidence data is not discarded entirely, but rather given lower priority. This means it will not be used as the primary basis for decision-making, but may serve as supplementary information for cross-validation with other high-confidence data.

[0182] Objective: To make the most of all available perceptual information, especially in complex or ambiguous scenarios, without introducing excessive risks.

[0183] Beneficial effects: Improves the system's environmental awareness coverage and early warning capabilities for potential threats, ensuring that even if the information is uncertain, it at least attracts the system's attention.

[0184] 3. Trigger multi-sensor fusion; Note: When the confidence level of a radar detection is low, the system will proactively request other sensors (such as cameras and lidar) to focus on observing and verifying the same area. If other sensors can provide high-confidence cross-validation results, the target may be reconfirmed; otherwise, it will be ignored.

[0185] Objective: To leverage the complementarity of different sensors to overcome the limitations of a single sensor and improve the accuracy and reliability of overall sensing.

[0186] Beneficial effects: Significantly enhances the system's perception capabilities and ability to handle uncertainties in complex environments, which is key to the robustness of autonomous driving.

[0187] 4. Request more data or change the perception strategy; Note: In some advanced systems, if the perception confidence level in a critical area remains low, the system may dynamically adjust the radar's operating mode (e.g., increase transmission power, change waveform, increase frame rate, or scan the area more frequently) or change the vehicle's driving strategy (e.g., reduce vehicle speed or increase safe distance) to obtain clearer and more reliable data.

[0188] Objective: To proactively adapt to environmental challenges and ensure safe operation even in uncertain environments.

[0189] Beneficial effects: Enhances the system's adaptability and risk avoidance capabilities in dynamically changing environments.

[0190] 5. Report uncertainties to the high-level decision-making module; Note: Even if the data is filtered or downgraded, the system may still report this "low confidence" situation to a higher-level decision-making and planning module. Based on this uncertainty, the decision-making module will adopt a more conservative driving strategy.

[0191] Objective: To ensure that decision-makers are fully aware of the limitations or uncertainties currently perceived.

[0192] Beneficial effects: Enables autonomous driving systems to make more cautious decisions when faced with perceptual uncertainty, for example, to adopt a safer, more conservative strategy when information is ambiguous.

[0193] When the confidence level is lower than a preset threshold, the system can avoid blindly using these data to make decisions. It can also take measures such as filtering, downgrading, verification, or adjusting strategies according to the specific situation to ensure the safety and reliability of autonomous driving.

[0194] To enable those skilled in the art to better understand the embodiments of the present invention, the following example will be used to illustrate the embodiments of the present invention.

[0195] 1. Input data: A high-level structured target list; The input to DDIRAI is no longer the raw radar signal or point cloud, but rather the "target list for classification and behavior prediction" output from step 104c (target type classification and behavior prediction (preliminary)). This list contains rich, structured information about each identified target, such as: 1.1 Target ID: A unique identifier used for tracking.

[0196] 1.2 Precise 3D position: X, Y, Z coordinates.

[0197] 1.3 Precise velocity: Vx, Vy, Vz vectors.

[0198] 1.4 Precise dimensions: length, width, and height.

[0199] 1.5 Clear target categories: such as "cars", "pedestrians", "trucks", etc.

[0200] 1.6 Preliminary behavioral prediction information: (If there is a preliminary prediction in step 104c).

[0201] 1.7 Historical Trajectory Data: Over time, DDIRAI will receive a list containing multi-frame historical data of these targets.

[0202] 2. DDIRAI's core processing logic; After receiving these structured lists of targets, DDIRAI will utilize its proprietary algorithms and deep learning models to focus on time series analysis, behavioral pattern recognition, and causal inference: 2.1 Data Integration and Time Series Modeling: DDIRAI first integrates historical target data (position, velocity, size, category, etc.) from multiple consecutive frames.

[0203] It uses temporal deep learning models (such as recurrent neural networks (RNN), long short-term memory networks (LSTM), gated recurrent units (GRU), or Transformer variants), which are particularly good at processing sequential data and capturing temporal dependencies and patterns.

[0204] The data obtained includes: the complete historical motion trajectory, velocity change curve, attitude change, and other temporal characteristic sequences of each target.

[0205] 2.2 Behavioral Pattern Recognition and Motivation Judgment (“Why”): The deep learning models in DDIRAI have been trained using massive amounts of real-world driving scenario data (including motion patterns, interaction behaviors, and corresponding human driving decisions or true intention labels for various targets).

[0206] The model analyzes the current target's motion patterns (e.g., whether it is decelerating, accelerating, turning, or braking suddenly), as well as its relative position, speed, and interaction with other surrounding targets.

[0207] By recognizing these complex patterns, DDIRAI is able to infer the motivation or reason behind the target action.

[0208] Example: A vehicle ahead is slowing down and its brake lights are on, while approaching a traffic light intersection → Determine the motive: preparing to stop and wait for the red light.

[0209] When a vehicle approaches an intersection, it signals its turn and veers off the target lane → Indicate motive: preparing to turn or change lanes.

[0210] A pedestrian on the crosswalk suddenly speeds up and looks at the road → Infer the motive: They intend to cross the road.

[0211] A cluster of points appears on the road ahead, whose Z-coordinate is consistently below the ground level and has a fixed shape. → Reason: This is a pothole or a broken road surface.

[0212] 2.3 Prediction of Movement Trends (“What Will Happen”): Based on an understanding of motivations and causes, as well as an analysis of historical movement trends, DDIRAI's predictive model infers the trajectory and curve of a target over a future period.

[0213] This includes predicting future position, velocity, and attitude, and even the probability that a target will arrive at a certain position at a certain point in time.

[0214] The data obtained includes: a sequence of predicted trajectory points for each target over the next X seconds (including predicted location and velocity), and the range of uncertainty in the prediction.

[0215] 2.4 Target Authenticity and Deception Judgment (Continuous Optimization): DDIRAI will continuously combine multi-frame data to conduct in-depth analysis of the target's stability, motion logic, and category consistency.

[0216] If a target's trajectory does not conform to physical laws, appears and disappears frequently, or its category identification result changes frequently in a short period of time, DDIRAI can determine it as a false target with higher confidence (such as multipath reflection, radar sidelobes, sensor transient noise, etc.) and remove it from the list of valid targets or reduce its priority.

[0217] The obtained data includes: a confidence score for the authenticity of each target.

[0218] 3. DDIRAI's output: the motion trend of the target object; DDIRAI's final output is a list of targets that has undergone highly intelligent analysis and prediction, with finer granularity and stronger predictive power: 3.1 A list of high-confidence targets, containing detailed prediction information: 3.1.1 Target ID: Same as above.

[0219] 3.1.2 Target Category: The final category after further verification and refinement by DDIRAI.

[0220] 3.1.3 Current status: Precise position, velocity, and size.

[0221] 3.1.4 Judging behavioral motives / intents: For example, "slowing down to prepare to stop", "about to change lanes", "intending to cross the road", etc.

[0222] 3.1.5 Predicting movement trends: 3.1.5.1 Trajectory point sequence in the next N seconds: For example, the predicted target position (Xt, Yt, Zt) and velocity (Vxt, Vyt, Vzt) every 0.1 seconds in the next 3 seconds.

[0223] 3.1.5.2 Uncertainty range / confidence level of predicted trajectory: Indicates the reliability of the prediction result.

[0224] 3.1.5.3 Critical event prediction: For example, predicting the time of collision with the vehicle, predicting the time when a target will enter a dangerous area, etc.

[0225] 3.1.5.4 Target authenticity determination: The final authenticity identifier.

[0226] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0227] Reference Figure 2 The diagram shows a structural block diagram of a motion trend prediction device provided in an embodiment of the present invention, which may specifically include the following modules: Target model determination module 201 is used to determine the target model; The four-dimensional millimeter-wave radar data acquisition module 202 is used to acquire four-dimensional millimeter-wave radar data for targeting objects. The classification and behavior prediction target list construction module 203 is used to construct a classification and behavior prediction target list for the target object based on the four-dimensional millimeter-wave radar data; The motion motivation determination module 204 is used to input the classification and behavior prediction target list into the target model, determine the temporal feature sequence used to characterize the target model through the target model, and determine the motion motivation of the target model based on the temporal feature sequence; The motion trend output module 205 is used to control the target model and output the motion trend of the target object based on the motion motivation.

[0228] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0229] In addition, embodiments of the present invention also provide an electronic device, such as... Figure 3 As shown, it includes a processor 301, a communication interface 302, a memory 303, and a communication bus 304, wherein the processor 301, the communication interface 302, and the memory 303 communicate with each other through the communication bus 304. Memory 303 is used to store computer programs; When the processor 301 executes the program stored in the memory 303, it implements any of the motion trend prediction methods described in the above embodiments: The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0230] The communication interface is used for communication between the aforementioned terminal and other devices.

[0231] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0232] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0233] like Figure 4 As shown, in another embodiment of the present invention, a computer-readable storage medium 401 is also provided, which stores instructions that, when executed on a computer, cause the computer to perform the motion trend prediction method described in the above embodiments.

[0234] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

[0235] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0236] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0237] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0238] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0239] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0240] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0241] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for predicting motion trends, characterized in that, include: Define the target model; Acquire four-dimensional millimeter-wave radar data for targeting objects; Based on the four-dimensional millimeter-wave radar data, a target list for classification and behavior prediction of the target object is constructed; The classification and behavior prediction target list is input into the target model, the target model is used to determine the temporal feature sequence used to characterize the target model, and the motion motivation of the target model is determined based on the temporal feature sequence; The target model is controlled to output the motion trend of the target object based on the motion motivation.

2. The method according to claim 1, characterized in that, The steps for constructing a target list for classification and behavior prediction of the target object based on the four-dimensional millimeter-wave radar data include: The original sensing information of potential reflection points is determined from the four-dimensional millimeter-wave radar data, and an original sensing point list is generated; Traverse the original sensing point list, perform confidence assessment and filtering operations on the potential reflection points in the original sensing point list to determine the target reflection point, and use the target reflection point to generate a three-dimensional high-confidence sensing point list; Traverse the list of three-dimensional high-confidence sensing points, transform and calibrate the target reflection points in the list of three-dimensional high-confidence sensing points from the radar coordinate system to the vehicle coordinate system, and generate a calibrated three-dimensional high-confidence point cloud; Points belonging to the same physical target are identified from the calibrated 3D high-confidence point cloud, and an aggregation operation is performed on the points belonging to the same physical target to generate a 3D point cluster list; the 3D point cluster list includes the 3D position information, velocity information and size information of the target object; Traverse the list of three-dimensional point clusters, extract the geometric and kinematic properties of the point clusters representing the target object, and generate a list of target geometric motion properties; Based on the target geometric motion attribute list, determine the category information of the target object represented by each point cluster, and generate a classification target list based on the category information; Acquire historical data trajectory information, associate each target object in the classification target list with the historical data trajectory information, and generate a target list containing multiple frames of historical data; Based on the target list containing multiple frames of historical data, preliminary behavior prediction information of the target objects is determined; A unique identifier is determined for the point cluster representing the target object, and a classification and behavior prediction target list is generated based on the unique identifier, the three-dimensional position information, the velocity information, the size information, the category information, the preliminary behavior prediction information, and the historical data trajectory information.

3. The method according to claim 2, characterized in that, The step of determining the temporal feature sequence used to characterize the target model through the target model includes: Based on the classification and behavior prediction target list, historical perception data of the target model in consecutive frames is extracted, and the historical perception data is used to generate the original target historical data stream; Perform time window truncation and normalization operations on the generated original target historical data stream to generate standardized time series input; The target model is used to extract high-dimensional abstract features of the target object in the time dimension based on the standardized temporal input, and the high-dimensional abstract features are used to generate a target temporal feature tensor; The target model is used to perform a dimensionality reduction operation on the target temporal feature tensor, transforming the target temporal feature tensor into a temporal feature sequence.

4. The method according to claim 3, characterized in that, The step of determining the motion motivation of the target model based on the temporal feature sequence includes: The potential motivational features of the target object are determined based on the temporal feature sequence; Acquire the temporal characteristics of the environmental interaction between the target object and other objects surrounding the target object; Based on the potential motivation characteristics and the environmental interaction timing characteristics, comprehensive motion motivation judgment data is generated. The generated comprehensive motion motivation judgment data is used to determine the motion motivation.

5. The method according to claim 4, characterized in that, The step of outputting the motion trend of the target object based on the motion motivation includes: Determine the confidence level of the stated motivation for movement; When the confidence level is determined to be greater than or equal to a preset threshold, the motion trend corresponding to the motion motivation is determined and output.

6. The method according to claim 5, characterized in that, Also includes: When the confidence level is determined to be lower than a preset threshold, the motion motivation is marked as invalid data; Delete the invalid data.

7. The method according to claim 5, characterized in that, Also includes: When the confidence level is determined to be lower than a preset threshold, the motion motivation is marked as low-confidence data; Cross-validation is performed between the low-confidence data and other motion motivations with confidence levels greater than or equal to a preset threshold to generate validation results for the motion motivations. When the verification result meets the preset conditions, the motion trend corresponding to the motion motivation is determined and output.

8. A motion trend prediction device, characterized in that, include: The target model determination module is used to determine the target model; The four-dimensional millimeter-wave radar data acquisition module is used to acquire four-dimensional millimeter-wave radar data for targeting objects. The classification and behavior prediction target list construction module is used to construct a classification and behavior prediction target list for the target object based on the four-dimensional millimeter-wave radar data; The motion motivation determination module is used to input the classification and behavior prediction target list into the target model, determine the temporal feature sequence used to characterize the target model through the target model, and determine the motion motivation of the target model based on the temporal feature sequence; The motion trend output module is used to control the target model and output the motion trend of the target object based on the motion motivation.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes a program stored in the memory, it implements the method as described in any one of claims 1-7.

10. A computer-readable storage medium having instructions stored thereon that, when executed by one or more processors, cause the processors to perform the method as described in any one of claims 1-7.

Images