A Dynamic Object Tracking Prediction Method and System Based on Multimodal Motion Model

Abstract

This invention relates to the field of autonomous driving environmental perception technology. It proposes a dynamic object tracking and prediction method and system based on a multimodal motion model. The method involves acquiring dynamic object point cloud and semantic information data, generating candidate object information, selecting a new tracking object motion model, estimating the object state, matching and managing candidate objects, updating the object state, updating the object trajectory and switching object motion models, estimating the object state in future multi-frame frames, and generating a region of interest (ROI). This invention employs a dynamic object tracking and prediction method based on a multimodal motion model. By constructing a multimodal motion model system and a ROI feedback mechanism, it significantly improves the accuracy and robustness of dynamic object point cloud tracking. Unlike the coarse prediction of traditional single motion models, this invention flexibly selects matching motion models for state estimation based on the motion characteristics of different object categories, effectively overcoming the prediction bias problem of a unified model for objects with different dynamic characteristics.

Description

Technical Field

[0001] This invention relates to the field of environmental perception technology for autonomous driving, and specifically to a dynamic object tracking and prediction method and system based on a multimodal motion model. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Environmental perception systems are a core component of autonomous driving technology, responsible for real-time perception of the vehicle's surrounding environment to support subsequent decision-making and control. LiDAR, as a key environmental perception sensor, provides high-precision 3D point cloud data for detecting and tracking dynamic objects. In autonomous driving systems, accurate detection and robust tracking of dynamic objects in LiDAR point clouds are crucial for ensuring that path planning and decision-making systems can correctly predict the behavior of traffic participants, reasonably avoid obstacles, and optimize driving strategies. A robust dynamic object point cloud tracking method not only needs to accurately identify and track dynamic targets such as pedestrians and vehicles in real time, but should also incorporate motion prediction models to infer the possible trajectories and occurrence areas of dynamic objects in the next few frames, thus providing prior information to the dynamic object detection system and further improving the robustness and accuracy of detection.

[0004] Currently, mainstream dynamic object point cloud tracking methods are mainly divided into two categories: detection-by-detection (TBD) tracking and joint detection and tracking (JDT). Detection-by-detection methods first perform target detection on the point cloud data, then match the same target in consecutive frames using data association algorithms, and construct a complete trajectory using motion prediction models (such as Kalman filtering). Joint detection and tracking methods, on the other hand, perform target detection and tracking simultaneously within the same framework, typically relying on deep learning for global optimization and association. Because detection-by-detection methods decouple detection and tracking tasks, allowing detectors and trackers to be optimized separately, and making the data association process more transparent and controllable, this method remains the primary choice in most dynamic object point cloud tracking tasks to ensure higher accuracy and robustness.

[0005] However, existing detection-based tracking methods typically use a single motion model to predict all targets. In real-world scenarios, different types of objects have their own unique motion patterns, and using a uniform motion model may lead to prediction bias, which in turn affects tracking accuracy. Furthermore, existing methods fail to feed the tracking prediction results back to the detection system to form a closed-loop optimization, resulting in the detection and tracking processes being relatively independent and failing to fully utilize temporal information to improve overall performance. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a dynamic object tracking and prediction method and system based on a multimodal motion model, which improves the accuracy and robustness of dynamic object point cloud tracking.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] In a first aspect, the present invention provides a dynamic object tracking and prediction method based on a multimodal motion model.

[0009] A dynamic object tracking and prediction method based on a multimodal motion model includes the following steps:

[0010] Acquire dynamic object point clouds and semantic information from each frame of point cloud data;

[0011] Candidate object information is generated based on the point cloud and semantic information of dynamic objects;

[0012] The category of the candidate object is determined based on the semantic information in the candidate object information of the current frame, and the motion model is selected based on the category;

[0013] Based on the selected motion model, extended Kalman filtering is used to estimate the state of different categories of objects and predict the state vector and covariance matrix of the next frame of point cloud data.

[0014] The predicted state vector and covariance matrix are matched with the candidate objects received in the next frame to determine the matching relationship between the predicted state vector and the candidate objects in the point cloud data of the next frame.

[0015] Update the state vector and covariance matrix of the dynamic object;

[0016] The updated state vector is appended to the trajectory sequence;

[0017] Estimate the state vector for future frames based on the trajectory sequence;

[0018] The region of interest is generated based on the state vector estimation results of the dynamic object in future frames.

[0019] Secondly, the present invention provides a dynamic object tracking and prediction system based on a multimodal motion model.

[0020] A dynamic object tracking and prediction system based on a multimodal motion model includes:

[0021] The data acquisition unit is configured to acquire dynamic object point clouds and semantic information for each frame of point cloud data.

[0022] The candidate object determination unit is configured to generate candidate object information based on the dynamic object point cloud and semantic information.

[0023] The motion model selection unit is configured to: determine the category of candidate objects based on the semantic information in the candidate object information of the current frame, and select a motion model based on the category;

[0024] The state estimation unit is configured to: perform state estimation on different categories of objects using extended Kalman filtering based on the selected motion model, and predict the state vector and covariance matrix of the next frame of point cloud data;

[0025] The matching relationship determination unit is configured to: match the predicted state vector and covariance matrix with the candidate objects received in the next frame, and determine the matching relationship between the predicted state vector and the candidate objects in the point cloud data of the next frame.

[0026] The state update unit is configured to update the state vector and covariance matrix of the dynamic object.

[0027] The trajectory sequence update unit is configured to append the updated state vector to the trajectory sequence.

[0028] The future state vector estimation unit is configured to estimate the state vectors for future multiple frames based on the trajectory sequence;

[0029] The region of interest generation unit is configured to generate a region of interest based on the state vector estimation result of a dynamic object in a future frame.

[0030] Compared with the prior art, the beneficial effects of the present invention are:

[0031] 1. This invention adopts a dynamic object tracking and prediction method based on a multimodal motion model. By constructing a multimodal motion model system and a region of interest feedback mechanism, the accuracy and robustness of dynamic object point cloud tracking are significantly improved. Unlike the coarse prediction of traditional single motion models, this invention flexibly selects matching motion models for state estimation based on the motion characteristics of different types of objects, effectively overcoming the prediction bias problem of a unified model for objects with different dynamic characteristics.

[0032] 2. This invention dynamically adjusts the region of interest based on the tracking prediction results and feeds the generated region of interest information back to the dynamic object detection system, thereby enabling the detection system to reduce missed detections and false detections and achieving bidirectional collaborative optimization of detection and tracking.

[0033] 3. This invention adapts to sudden changes in object motion patterns in real time through a multi-model switching mechanism, and enhances the ability to continuously capture complex motion trajectories by combining trajectory updates and future multi-frame prediction algorithms, thereby demonstrating stronger environmental adaptability in real-world scenarios such as mixed traffic and dense occlusion.

[0034] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0035] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0036] Figure 1 This is a flowchart illustrating the dynamic object tracking and prediction method based on a multimodal motion model provided in Embodiment 1 of the present invention.

[0037] Figure 2 A schematic diagram illustrating the data acquisition method provided in Embodiment 1 of the present invention;

[0038] Figure 3 This is a schematic diagram illustrating the selection of a new tracking object motion model provided in Embodiment 1 of the present invention;

[0039] Figure 4 This is a schematic diagram of the selection of motion model for vehicle-type objects provided in Embodiment 1 of the present invention;

[0040] Figure 5 This is a schematic diagram of candidate object matching and management provided in Embodiment 1 of the present invention;

[0041] Figure 6 This is a schematic diagram of the generated sector and rectangular regions of interest provided in Embodiment 1 of the present invention;

[0042] Figure 7 This is a schematic diagram of a dynamic object tracking and prediction system based on a multimodal motion model provided in Embodiment 2 of the present invention. Detailed Implementation

[0043] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0044] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0045] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0046] Example 1:

[0047] This implementation proposes a dynamic object tracking and prediction method based on a multimodal motion model, such as... Figure 1 As shown, the process includes the following:

[0048] S1: Acquire dynamic object point cloud and semantic information data.

[0049] More specifically, data output from a dynamic object point cloud detection system is received at a fixed frequency, such as... Figure 2 As shown, each frame of data includes a dynamic object point cloud and its semantic information. The data format of each point in the point cloud is as follows:

[0050] (1);

[0051] in, This indicates the first point cloud in that frame. One point; This indicates the coordinates of the point in the world coordinate system; This indicates that the point belongs to the frame number 1. A point in a dynamic object; This indicates that the point belongs to a dynamic object, including categories such as pedestrians, cyclists, vehicles, and other objects.

[0052] S2: Candidate object information generation.

[0053] More specifically, after receiving the dynamic object point cloud and semantic information for each frame, this method processes the point cloud data of the same dynamic object and generates candidate object information; wherein, point cloud data belonging to the same dynamic object refers to point cloud data with the same semantic label in the frame. A set of points. Dynamic objects are defined as follows:

[0054] (2);

[0055] in, Indicates the first A dynamic object; This indicates the first [object] in the dynamic object. One point, These represent the first, second, and third points in the dynamic object, respectively.

[0056] The centroid and axis-aligned bounding boxes of all dynamic objects in the current frame are calculated using the following formula:

[0057] (3);

[0058] in, Represents the centroid of a dynamic object. ; This indicates the smallest corner point of the axis-aligned bounding box; Indicates the maximum corner point of the axis-aligned bounding box; , , This indicates the length, width, and height of the axis-aligned bounding box. This represents the total number of points in a dynamic object. For the first The coordinates of the points.

[0059] Finally, the candidate object information was obtained:

[0060] (4);

[0061] in, Represents the semantic information of dynamic objects.

[0062] S3: Selection of a new motion model for the tracked object.

[0063] More specifically, after obtaining candidate object information, this method determines the category of the object based on its semantic information and selects the corresponding motion model, such as... Figure 3 As shown. Pedestrian objects are modeled using a random walk model; cyclist objects are modeled using a monorail bicycle model; for vehicle objects, the method first calculates their point cloud density:

[0064] (5);

[0065] in, This indicates the total number of points in the dynamic object.

[0066] When point cloud density Less than or equal to the set threshold When the point cloud density is high, a constant acceleration model is used; when the point cloud density is low, a constant acceleration model is used. Greater than the set threshold When the object is turning at a constant speed, a uniform velocity model is used; for other objects, a constant velocity model is used, such as... Figure 4 As shown.

[0067] S4: Object state estimation.

[0068] More specifically, after selecting the motion model for the tracked object, this invention uses an extended Kalman filter to estimate the state of different categories of objects. The state equation and observation equation of the extended Kalman filter are as follows:

[0069] (6);

[0070] in, Indicates the first The state vector of the frame point cloud; Indicates the first The state vector of the frame point cloud; Represents the state transition function; Indicates the first Process noise in frame point clouds; The covariance matrix represents the process noise; Represents the measurement vector of the point cloud in the current frame; Represents a measurement function; Indicates the first Measurement noise of frame point clouds; Represents the covariance matrix of the measurement noise; where the subscripts are... Indicates the point cloud of the current frame, index This indicates the point cloud from the previous frame.

[0071] For different types of dynamic objects, this invention employs corresponding state vectors, state transition functions, and process noise modeling:

[0072] (1) The state vector, state transition function and process noise of the human dynamic object using the random walk model are as follows:

[0073] (7);

[0074] at this time, Indicates the first The centroid coordinates of the human-shaped object in the frame point cloud; Indicates the first Velocity in the corresponding direction of the frame point cloud; Indicates the first The centroid coordinates of the human-shaped object in the frame point cloud; Indicates the first Velocity in the corresponding direction of the frame point cloud; Indicates the inter-frame time interval; These represent position noise in three directions, respectively. These represent the velocity uncertainties in the three directions, respectively.

[0075] (2) The dynamic object of the rider class using the monorail bicycle model has the following state vector, state transition function and process noise:

[0076] (8);

[0077] at this time, Indicates the first The centroid coordinates of the cyclist-type object in the frame point cloud; Indicates the first The XY axis velocities of the cyclist-type object in the frame point cloud; Indicates the first The heading angle of a cyclist-like object in a frame point cloud; Indicates the first The front wheel angle of a rider-type object in a frame point cloud;

[0078] Indicates the first The centroid coordinates of the cyclist-type object in the frame point cloud; Indicates the first The XY axis velocities of the cyclist-type object in the frame point cloud; Indicates the first The heading angle of a cyclist-like object in a frame point cloud; Indicates the first The front wheel angle of a rider-type object in a frame point cloud; The wheelbase of the bicycle model is represented by the following approximate calculation formula: ;in, This indicates the X and Y coordinates of the largest corner point of the bounding box of the dynamic object. This indicates the X and Y coordinates of the smallest corner point of the bounding box of the dynamic object. Indicates the inter-frame time interval; These represent position noise in three directions, respectively. Indicates speed uncertainty; Indicates the uncertainty of the heading angle; This indicates the uncertainty of the front wheel steering angle.

[0079] (3) For vehicle-type dynamic objects using the constant acceleration model, the state vector, state transition function, and process noise are as follows:

[0080] (9);

[0081] at this time, Indicates the first Centroid coordinates of vehicle-type objects in the frame point cloud; Indicates the first Velocity in the corresponding direction of the frame point cloud; Indicates the first Acceleration in the corresponding direction of the frame point cloud; Indicates the first Centroid coordinates of vehicle-type objects in the frame point cloud; Indicates the first Velocity in the corresponding direction of the frame point cloud; Indicates the first Acceleration in the corresponding direction of the frame point cloud; Indicates the inter-frame time interval; These represent position noise in three directions, respectively. These represent the velocity uncertainties in the three directions, respectively. These represent the acceleration uncertainties in the three directions, respectively.

[0082] (4) For vehicle-type dynamic objects using the uniform speed steering model, the state vector, state transition function, and process noise are as follows:

[0083] (10);

[0084] at this time, Indicates the first Centroid coordinates of vehicle-type objects in the frame point cloud; Indicates the first The XY axis velocities of vehicle-type objects in the frame point cloud; Indicates the first The heading angle of vehicle-type objects in the frame point cloud; Indicates the first Angular velocity of vehicle-type objects in frame point cloud; Indicates the first Centroid coordinates of vehicle-type objects in the frame point cloud; Indicates the first The XY axis velocities of vehicle-type objects in the frame point cloud; Indicates the first The heading angle of vehicle-type objects in the frame point cloud; Indicates the first Angular velocity of vehicle-type objects in frame point cloud; Indicates the inter-frame time interval; Indicates positional noise; This indicates the uncertainty in the centroid's XY axis velocity; Indicates the uncertainty of the heading angle; This indicates the uncertainty of angular velocity.

[0085] (5) For other dynamic objects using the constant velocity model, the state vector, state transition function, and process noise are as follows:

[0086] (11);

[0087] at this time, Indicates the first Centroid coordinates of other object classes in the frame point cloud; Indicates the first Velocity in the corresponding direction of the frame point cloud; Indicates the first Centroid coordinates of other object classes in the frame point cloud; Indicates the first The velocity of the point cloud in three corresponding directions; Indicates the inter-frame time interval; Represents the variance of acceleration noise; express The identity matrix.

[0088] Next, based on the motion pattern of the dynamic object, its state vector and covariance matrix for the next frame are predicted, as shown in the following formula:

[0089] (12);

[0090] at this time, Indicates based on Frame data pairs The predicted value of the frame state vector; express The state vector obtained from frame-time extended Kalman filter state update; Indicates based on Frame data pairs Predicted values ​​of the frame covariance matrix; express The Jacobian matrix of the frame state transition function; express The covariance matrix obtained from frame-time extended Kalman filter state update; The covariance matrix represents the process noise.

[0091] S5: Candidate object matching and management.

[0092] More specifically, after predicting the state vector and covariance matrix of the dynamic object in the next frame, it is necessary to match the predicted dynamic object with the candidate object received in the next frame.

[0093] For each predicted dynamic object state vector and each measured candidate object, the innovation quantity is calculated using the following formula:

[0094] (13);

[0095] at this time, Indicates the first One predicted state vector; Indicates the first One measurement candidate object; This represents the measurement vector of the k-th frame. ; Indicates the position of the centroid of the candidate object; Represents the measurement function. ; Represents the predicted state vector The position of the centroid in the middle.

[0096] The innovation covariance matrix of the predicted state vector is calculated using the following formula:

[0097] (14);

[0098] at this time, The Jacobian matrix represents the measurement function of the k-th frame; Indicates measurement noise; Indicates based on Frame data pairs Predicted values ​​of the frame covariance matrix.

[0099] The Mahalanobis distance between each pair of predicted state vectors and candidate objects is calculated using the following formula:

[0100] (15);

[0101] Next, construct the cost matrix. The formula is as follows:

[0102] (16);

[0103] At this point, the rows of the cost matrix correspond to the predicted state vector, and the columns correspond to the candidate targets; This indicates taking a larger value, representing a very large cost; This represents the threshold for the threshold test.

[0104] The cost matrix was processed using the Hungarian algorithm. A globally optimal allocation is performed to determine the matching relationship between the predicted state vector and the candidate targets. After matching the candidate objects, unmatched candidate objects and unmatched predicted objects are further managed, such as... Figure 5 As shown.

[0105] For unmatched candidate objects, if their point cloud count Exceeding the set threshold If the result is positive, it is considered a newly appearing object, and its candidate object information is used as the initial value; otherwise, it is determined to be a false detection by the dynamic object point cloud detection system.

[0106] For predicted objects that are not matched, then in the future The state is retained within the frame time interval, and the state vector of its future frames is calculated using step S8. If in the future... If no candidate target is matched within the frame interval, it is discarded.

[0107] S6: Object state update.

[0108] More specifically, after obtaining the matching relationship between the predicted state vector and the candidate target, the method updates the object's state vector.

[0109] First, calculate the amount of innovation. Innovation Covariance and Kalman gain The formula is as follows:

[0110] (17);

[0111] at this time, This represents the measurement vector of the k-th frame. ; Indicates the position of the centroid of the candidate object; Represents the measurement function. ; Represents the predicted state vector The position of the centroid in the middle; The Jacobian matrix represents the measurement function of the k-th frame; Indicates based on Frame data pairs Predicted values ​​of the frame covariance matrix; This represents the covariance matrix of the measurement noise.

[0112] Subsequently, the state vector and covariance matrix are updated using Kalman gain, as shown in the following formula:

[0113] (18)

[0114] at this time, express The state vector obtained from frame-extended Kalman filter state update; express The covariance matrix obtained from frame extended Kalman filter state update; Represents the identity matrix.

[0115] S7: Object trajectory update and object motion model switching.

[0116] More specifically, after obtaining the latest state of the object, this method updates the object's trajectory by updating the latest state vector. Add to trajectory sequence ;

[0117] For vehicle-type objects, this method uses the latest object information to calculate their point cloud density. And based on this, dynamically adjust the motion model, when the point cloud density Less than or equal to the set threshold When the point cloud density is high, switch to a constant acceleration model; when the point cloud density is low, switch to a constant acceleration model. Greater than the set threshold At that time, switch to the uniform speed steering model.

[0118] S8: Future multi-frame object state estimation.

[0119] More specifically, after updating the object trajectory, this method estimates the object state for multiple future frames.

[0120] For pedestrian dynamic objects using a random walk model, vehicle dynamic objects using a constant acceleration model, and other object dynamic objects using a constant velocity model, the future... The frame state estimation formula is as follows:

[0121] (19);

[0122] at this time, The predicted values ​​of the state vector for the next n frames. The predicted value of the covariance matrix for the next n frames; The Jacobian matrix represents the state transition function; The covariance matrix represents the process noise.

[0123] For dynamic objects of the rider class using a monorail bicycle model, an iterative method is used to estimate and calculate the future... The frame state vector is given by the following formula:

[0124] (20);

[0125] at this time, Indicates the first The centroid coordinates of the cyclist-type object in the frame point cloud; Indicates the first The XY axis velocities of the cyclist-type object in the frame point cloud; Indicates the first The heading angle of a cyclist-like object in a frame point cloud; Indicates the first The front wheel angle of a rider-type object in a frame point cloud; Indicates the first The centroid coordinates of the cyclist-type object in the frame point cloud; Indicates the first The XY axis velocities of the cyclist-type object in the frame point cloud; Indicates the first The heading angle of a cyclist-like object in a frame point cloud; Indicates the first The front wheel angle of a rider-type object in a frame point cloud; Indicates the inter-frame time interval; This indicates the wheelbase of the bicycle model.

[0126] For dynamic objects such as vehicles that use a uniform speed steering model, an iterative method is used to estimate the future direction. The frame state vector is given by the following formula:

[0127] (twenty one).

[0128] at this time, Indicates the first Centroid coordinates of vehicle-type objects in the frame point cloud; Indicates the first The XY axis velocities of vehicle-type objects in the frame point cloud; Indicates the first The heading angle of vehicle-type objects in the frame point cloud; Indicates the first Angular velocity of vehicle-type objects in frame point cloud. Indicates the first Angular velocity of vehicle-like objects in frame point cloud. Indicates the first Centroid coordinates of vehicle-type objects in the frame point cloud; Indicates the first The XY axis velocities of vehicle-type objects in the frame point cloud; Indicates the first The heading angle of vehicle-type objects in the frame point cloud; Indicates the time interval between frames;

[0129] S9: Region of Interest Generation.

[0130] More specifically, after estimating the state of objects in future frames, this method generates a region of interest based on the state vector of the dynamic object in future frames and feeds it back to the dynamic object point cloud detection system.

[0131] First, this method uses the latest object information to calculate its point cloud density. ;

[0132] Secondly, for dynamic objects using different motion models, this method generates different regions of interest, including fan-shaped and rectangular regions of interest. Schematic diagrams of the generated fan-shaped and rectangular regions of interest are shown below. Figure 6 As shown.

[0133] For pedestrian-type dynamic objects using a random walk model, cyclist-type dynamic objects using a monorail bicycle model, and vehicle-type dynamic objects using a uniform speed steering model, this method generates a fan-shaped region of interest. The center coordinates of the fan are... The formulas for the outer diameter, inner diameter, and central angle of a sector are as follows:

[0134] (twenty two);

[0135] (twenty three);

[0136] (twenty four);

[0137] at this time, , This indicates the length and width of the axis-aligned bounding box; Represents the set of integers; Indicates the first The frame state update obtains the centroid X-axis coordinate of the state vector; Indicates the first The frame state update obtains the centroid Y-axis coordinate of the state vector; Indicates the first The centroid X-axis coordinate of the frame prediction state vector; Indicates the first The centroid Y-axis coordinate of the frame prediction state vector; This is the adjustment coefficient.

[0138] For dynamic objects such as vehicles using a constant acceleration model and other object types using a constant velocity model, this method generates a rectangular region of interest, with the center coordinates of the rectangle being... The length of the rectangle Hekuan ;

[0139] Among them, the adjustment coefficient When point cloud density Less than or equal to the set threshold When, adjustment coefficient Take a value of 2.0~3.0, when the density Greater than the set threshold When, adjustment coefficient Use a value between 1.0 and 2.0.

[0140] Finally, the predicted region of interest information is sent to the dynamic object point cloud detection system.

[0141] Example 2:

[0142] like Figure 7 As shown, this implementation provides a dynamic object tracking and prediction system based on a multimodal motion model, including:

[0143] The data acquisition unit is configured to acquire dynamic object point clouds and semantic information for each frame of point cloud data.

[0144] The candidate object determination unit is configured to generate candidate object information based on the dynamic object point cloud and semantic information.

[0145] The motion model selection unit is configured to: determine the category of candidate objects based on the semantic information in the candidate object information of the current frame, and select a motion model based on the category;

[0146] The state estimation unit is configured to: perform state estimation on different categories of objects using extended Kalman filtering based on the selected motion model, and predict the state vector and covariance matrix of the next frame of point cloud data;

[0147] The matching relationship determination unit is configured to: match the predicted state vector and covariance matrix with the candidate objects received in the next frame, and determine the matching relationship between the predicted state vector and the candidate objects in the point cloud data of the next frame.

[0148] The state update unit is configured to update the state vector and covariance matrix of the dynamic object.

[0149] The trajectory sequence update unit is configured to append the updated state vector to the trajectory sequence.

[0150] The future state vector estimation unit is configured to estimate the state vectors for future multiple frames based on the trajectory sequence;

[0151] The region of interest generation unit is configured to generate a region of interest based on the state vector estimation result of a dynamic object in a future frame.

[0152] The specific working process of each of the above units is described in Example 1, and will not be repeated here.

[0153] It is understood that the aforementioned units can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. The aforementioned units are based on logical functional division. In practical applications, the function of one unit can be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the system may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0154] According to another embodiment of this application, the system described in this embodiment, and the method of embodiment 1 of this application, can be constructed and the method of embodiment 1 of this application implemented by running a computer program (including program code) capable of performing the steps involved in the corresponding method described in embodiment 1 on a general-purpose computing device including processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM). The computer program can be recorded on, for example, a computer-readable recording medium, loaded into the aforementioned computing device through the computer-readable recording medium, and run therein.

[0155] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A dynamic object tracking and prediction method based on a multimodal motion model, characterized in that, include: Acquire dynamic object point clouds and semantic information from each frame of point cloud data; Based on the dynamic object point cloud and semantic information, candidate object information is generated, wherein the candidate object information is... The candidate object category is determined based on the semantic information in the candidate object information of the current frame, and a motion model is selected according to the category; a random walk model is used for pedestrian objects; a monorail bicycle model is used for cyclist objects; and the point cloud density is calculated for vehicle objects. , and dynamically adjust the motion model accordingly, when the point cloud density Less than or equal to the set threshold When the point cloud density is high, switch to a constant acceleration model; when the point cloud density is low, switch to a constant acceleration model. Greater than the set threshold When the object is in constant velocity, switch to the uniform speed turning model; other objects use the constant speed model. Based on the selected motion model, extended Kalman filtering is used to estimate the state of different categories of objects and predict the state vector and covariance matrix of the next frame of point cloud data. The predicted state vector and covariance matrix are matched with the candidate objects received in the next frame to determine the matching relationship between the predicted state vector and the candidate objects in the point cloud data of the next frame. For unmatched candidate objects, if their point cloud count Exceeding the set threshold If so, it is considered a newly appearing object, and its candidate object information is used as the initial value; Otherwise, it is determined to be a false detection by the dynamic object point cloud detection system; For predicted objects that are not matched, then in the future The state is retained within the frame time interval, and the state vector of its future frames is calculated. If in the future... If no candidate target is matched within the frame interval, it is discarded. Update the state vector and covariance matrix of the dynamic object; The updated state vector is appended to the trajectory sequence; Estimate the state vector for future frames based on the trajectory sequence; Generate the region of interest based on the state vector estimation results of the dynamic object in future frames; Calculate its point cloud density using the latest object information; For pedestrian-type dynamic objects using a random walk model, cyclist-type dynamic objects using a monorail bicycle model, and vehicle-type dynamic objects using a uniform speed steering model, generate fan-shaped regions of interest; the center coordinates of the fan are... The outer diameter of the fan shape , inner diameter and central angle for: ； ； ； For dynamic objects such as vehicles using a constant acceleration model and other object types using a constant velocity model, a rectangular region of interest is generated, with the center coordinates of the rectangle being... ,in Represents the set of integers; This represents the centroid X-axis coordinate of the state vector obtained from the state update in the k-th frame; This represents the centroid Y-axis coordinate of the state vector obtained from the state update in the k-th frame; The centroid X-axis coordinate of the predicted state vector in the (k+n)th frame is represented. The rectangle represents the Y-axis coordinate of the centroid of the predicted state vector in frame k+n; the length of the rectangle is... Width The adjustment coefficient is When point cloud density Less than or equal to the set threshold When, adjustment coefficient Take a value of 2.0~3.0, when the density Greater than the set threshold When, adjustment coefficient Use a value between 1.0 and 2.

0.

2. The dynamic object tracking and prediction method based on a multimodal motion model as described in claim 1, characterized in that, The data format for each point in a dynamic object point cloud is as follows: ,in, This indicates the first point cloud in that frame. One point; This indicates the coordinates of the point in the world coordinate system; This indicates that the point belongs to the frame number 1. A point in a dynamic object; This indicates that the point belongs to a dynamic object.

3. A dynamic object tracking and prediction system based on a multimodal motion model, characterized in that, The dynamic object tracking and prediction method based on a multimodal motion model according to any one of claims 1-2 includes: The data acquisition unit is configured to acquire dynamic object point clouds and semantic information for each frame of point cloud data. The candidate object determination unit is configured to generate candidate object information based on the dynamic object point cloud and semantic information. The motion model selection unit is configured to: determine the category of candidate objects based on the semantic information in the candidate object information of the current frame, and select a motion model based on the category; The state estimation unit is configured to: perform state estimation on different categories of objects using extended Kalman filtering based on the selected motion model, and predict the state vector and covariance matrix of the next frame of point cloud data; The matching relationship determination unit is configured to: match the predicted state vector and covariance matrix with the candidate objects received in the next frame, and determine the matching relationship between the predicted state vector and the candidate objects in the point cloud data of the next frame. The state update unit is configured to update the state vector and covariance matrix of the dynamic object. The trajectory sequence update unit is configured to append the updated state vector to the trajectory sequence. The future state vector estimation unit is configured to estimate the state vectors for future multiple frames based on the trajectory sequence; The region of interest generation unit is configured to generate a region of interest based on the state vector estimation result of a dynamic object in a future frame.

Images