Target tracking method and target tracking apparatus

Abstract

The application relates to a target tracking method and a target tracking device. The method comprises the following steps: performing track prediction based on a three-dimensional point cloud at a previous moment and a motion track of a target object at the previous moment, obtaining a two-dimensional predicted track of the target object at a current moment, and determining a three-dimensional coordinate point associated with the two-dimensional predicted track at the current moment. A target coordinate point is determined from the associated three-dimensional coordinate point. A tracking state of the target coordinate point corresponding to the current moment is determined based on historical target coordinate points in a target coordinate point array, wherein the historical target coordinate points are target coordinate points associated with two-dimensional predicted tracks of the target object at historical moments in three-dimensional point clouds collected at the historical moments. The tracking state is used to represent that the target coordinate point is in a normal motion state or an abnormal motion state. A motion track of the target object is obtained according to the tracking state and the two-dimensional predicted track. The method can improve the tracking accuracy of target tracking.

Description

Technical Field

[0001] This application relates to the field of radar detection technology, and in particular to a target tracking method and a target tracking device. Background Technology

[0002] With the development of radar detection, millimeter-wave radar with a working band at the millimeter level has emerged. The working band of millimeter-wave radar is in the overlapping region of far-infrared and microwave, and is far away from visible light, so it is not easy to cause information leakage. In addition, millimeter-wave radar also has the advantages of high resolution and small equipment size.

[0003] In the traditional method of target tracking using millimeter-wave radar, after acquiring the three-dimensional point cloud of the target using millimeter-wave radar, the three-dimensional trajectory is directly predicted and tracked from the three-dimensional point cloud to obtain a relatively stable motion trajectory of the target.

[0004] However, in traditional methods, in some scenarios, such as a caregiver walking towards a lying patient or pouring water into a toilet (these scenarios are collectively referred to as close encounters between different targets), directly predicting and tracking the 3D trajectory of the target's 3D point cloud can lead to the 3D point cloud of the first target being associated with the trajectory of the second target. For example, part of the patient's 3D point cloud may be associated with the caregiver's trajectory, and the 3D point cloud of the water may be associated with the person's trajectory. In other words, the trajectories of different targets will affect each other, thereby affecting the accuracy of the target trajectory prediction and reducing the tracking precision of the target. Summary of the Invention

[0005] Therefore, it is necessary to provide a target tracking method, target tracking device, computer equipment, computer-readable storage medium, and computer program product that can improve the accuracy of trajectory prediction in response to the above-mentioned technical problems.

[0006] Firstly, this application provides a target tracking method. The method includes:

[0007] Based on the three-dimensional point cloud of the previous moment and the motion trajectory of the target object at the previous moment, trajectory prediction is performed to obtain the two-dimensional predicted trajectory of the target object at the current moment.

[0008] From the three-dimensional point cloud at the current moment, determine each three-dimensional coordinate point associated with the two-dimensional predicted trajectory;

[0009] Determine the target coordinate point from the associated three-dimensional coordinate points;

[0010] Based on the historical target coordinate points in the target coordinate point array, the tracking status of the target coordinate point corresponding to the current moment is determined. The historical target coordinate points are the target coordinate points in the three-dimensional point cloud collected at historical moments that are associated with the two-dimensional predicted trajectory of the target object at each historical moment. The tracking status is used to characterize whether the target coordinate point is in a normal motion state or an abnormal motion state.

[0011] Based on the tracking status and the two-dimensional predicted trajectory, the motion trajectory of the target object is obtained.

[0012] In one embodiment, the step of predicting the trajectory based on the three-dimensional point cloud of the previous moment and the motion trajectory of the target object at the previous moment to obtain the two-dimensional predicted trajectory of the target object at the current moment includes:

[0013] Obtain the historical 3D coordinates of the target object associated with its motion trajectory in the 3D point cloud at the previous moment;

[0014] Based on the historical three-dimensional coordinate points, a trajectory prediction is performed to obtain the two-dimensional predicted trajectory of the target object at the current moment.

[0015] In one embodiment, determining the target coordinate point from the associated three-dimensional coordinate points includes:

[0016] Obtain the height of each of the associated three-dimensional coordinate points;

[0017] Based on the height of each of the three-dimensional coordinate points, the multiple three-dimensional coordinate points are sorted from high to low to obtain a point cloud sequence;

[0018] The three-dimensional coordinate points that are first and second in the point cloud sequence are determined as the first sequence point and the second sequence point, respectively.

[0019] Based on the first height difference between the first sequence point and the second sequence point, the target coordinate point corresponding to the current moment is determined from the point cloud sequence.

[0020] In one embodiment, determining the target coordinate point corresponding to the current moment from the point cloud sequence based on the first height difference between the first sequence point and the second sequence point includes:

[0021] Determine the first height difference between the first sequence point and the second sequence point;

[0022] When the first height difference is less than the first height difference threshold, the first sequence point is used as the target coordinate point corresponding to the current time.

[0023] Alternatively, when the first height difference is greater than or equal to the first height difference threshold, the first sequence point is deleted from the point cloud sequence, and the step of determining the first and second three-dimensional coordinate points in the point cloud sequence as the first sequence point and the second sequence point is repeated until the first height difference between the first sequence point and the second sequence point is less than the first height difference threshold, and the first sequence point is used as the target coordinate point corresponding to the current time.

[0024] In one embodiment, determining the tracking status of the target coordinate point at the current moment based on historical target coordinate points in the target coordinate point array includes:

[0025] The historical target coordinate points are clustered based on their heights to obtain a target array, and the cluster centers of the target array are determined.

[0026] Starting from the cluster center, the target object is predicted based on the target coordinates at the current time to obtain the altitude trajectory of the target object.

[0027] Based on the altitude track and the target coordinates at the current time, determine the tracking status of the target coordinates at the current time.

[0028] In one embodiment, determining the tracking status of the target coordinate point at the current time based on the altitude track and the target coordinate point at the current time includes:

[0029] Determine the altitude of the target coordinate point corresponding to the current moment, and the second altitude difference between the altitude track and the altitude at the current moment;

[0030] When the second height difference is greater than or equal to the second height difference threshold, the number of times the foot is lost is increased by 1;

[0031] Based on the number of times the tracking fails, the tracking status of the target coordinate point at the current moment is determined.

[0032] In one embodiment, determining the tracking status of the target coordinate point at the current moment based on the number of tracking failures includes:

[0033] If the number of tracking errors is greater than or equal to a threshold, the tracking status of the target coordinate point at the current time is determined to be the first state; or...

[0034] If the number of tracking errors is less than the tracking error threshold, the tracking status of the target coordinate point at the current time is determined to be the second state.

[0035] The first state is used to characterize the target coordinate point as being in an abnormal motion state, and the second state is used to characterize the target coordinate point as being in a normal motion state.

[0036] In one embodiment, the method further includes: if the second height difference is not detected to be greater than or equal to the second height difference threshold within a preset time period, then the number of times the foot is lost is reset to zero.

[0037] In one embodiment, obtaining the motion trajectory of the target object based on the tracking state and the two-dimensional predicted trajectory includes:

[0038] If the tracking state is the first state, the two-dimensional predicted trajectory is interrupted;

[0039] After a preset time period, a new two-dimensional predicted trajectory is obtained by constructing a new trajectory based on the three-dimensional point cloud within the preset time period.

[0040] Based on the two-dimensional predicted trajectory and the new two-dimensional predicted trajectory, the motion trajectory of the target object is obtained;

[0041] Alternatively, when the tracking state is the second state, the motion trajectory of the target object is determined based on the two-dimensional predicted trajectory and the three-dimensional coordinate points associated with the two-dimensional predicted trajectory.

[0042] Secondly, this application also provides a target tracking device. The device includes:

[0043] The trajectory prediction module is used to predict the trajectory based on the three-dimensional point cloud of the previous moment and the motion trajectory of the target object at the previous moment, so as to obtain the two-dimensional predicted trajectory of the target object at the current moment.

[0044] The first determining module is used to determine each three-dimensional coordinate point associated with the two-dimensional predicted trajectory from the three-dimensional point cloud at the current moment.

[0045] The second determining module is used to determine the target coordinate point from the associated three-dimensional coordinate points;

[0046] The third determining module is used to determine the tracking status of the target coordinate point corresponding to the current time based on the historical target coordinate points in the target coordinate point array. The historical target coordinate points are target coordinate points in the three-dimensional point cloud collected at historical times that are associated with the two-dimensional predicted trajectory of the target object at each historical time. The tracking status is used to characterize whether the target coordinate point is in a normal motion state or an abnormal motion state.

[0047] The fourth determining module is used to obtain the motion trajectory of the target object based on the tracking status and the two-dimensional predicted trajectory.

[0048] In one embodiment, the trajectory prediction module is further configured to:

[0049] Obtain the historical 3D coordinates of the target object associated with its motion trajectory in the 3D point cloud at the previous moment;

[0050] Based on the historical three-dimensional coordinate points, a trajectory prediction is performed to obtain the two-dimensional predicted trajectory of the target object at the current moment.

[0051] In one embodiment, the second determining module is further configured to:

[0052] Obtain the height of each of the associated three-dimensional coordinate points;

[0053] Based on the height of each of the three-dimensional coordinate points, the multiple three-dimensional coordinate points are sorted from high to low to obtain a point cloud sequence;

[0054] The three-dimensional coordinate points that are first and second in the point cloud sequence are determined as the first sequence point and the second sequence point, respectively.

[0055] Based on the first height difference between the first sequence point and the second sequence point, the target coordinate point corresponding to the current moment is determined from the point cloud sequence.

[0056] In one embodiment, the second determining module is further configured to:

[0057] Determine the first height difference between the first sequence point and the second sequence point;

[0058] When the first height difference is less than the first height difference threshold, the first sequence point is used as the target coordinate point corresponding to the current time.

[0059] Alternatively, when the first height difference is greater than or equal to the first height difference threshold, the first sequence point is deleted from the point cloud sequence, and the step of determining the first and second three-dimensional coordinate points in the point cloud sequence as the first sequence point and the second sequence point is repeated until the first height difference between the first sequence point and the second sequence point is less than the first height difference threshold, and the first sequence point is used as the target coordinate point corresponding to the current time.

[0060] In one embodiment, the third determining module is further configured to:

[0061] The historical target coordinate points are clustered based on their heights to obtain a target array, and the cluster centers of the target array are determined.

[0062] Starting from the cluster center, the target object is predicted based on the target coordinates at the current time to obtain the altitude trajectory of the target object.

[0063] Based on the altitude track and the target coordinates at the current time, determine the tracking status of the target coordinates at the current time.

[0064] In one embodiment, the third determining module is further configured to:

[0065] Determine the altitude of the target coordinate point corresponding to the current moment, and the second altitude difference between the altitude track and the altitude at the current moment;

[0066] When the second height difference is greater than or equal to the second height difference threshold, the number of times the foot is lost is increased by 1;

[0067] Based on the number of times the tracking fails, the tracking status of the target coordinate point at the current moment is determined.

[0068] In one embodiment, the third determining module is further configured to:

[0069] If the number of tracking errors is greater than or equal to a threshold, the tracking status of the target coordinate point at the current time is determined to be the first state; or...

[0070] If the number of tracking errors is less than the tracking error threshold, the tracking status of the target coordinate point at the current time is determined to be the second state.

[0071] The first state is used to characterize the target coordinate point as being in an abnormal motion state, and the second state is used to characterize the target coordinate point as being in a normal motion state.

[0072] In one embodiment, the device further includes:

[0073] The zeroing module is used to reset the number of times the footsteps have been lost to zero if the second height difference is not detected to be greater than or equal to the second height difference threshold within a preset time period.

[0074] In one embodiment, the fourth determining module is further configured to:

[0075] If the tracking state is the first state, the two-dimensional predicted trajectory is interrupted;

[0076] After a preset time period, a new two-dimensional predicted trajectory is obtained by constructing a new trajectory based on the three-dimensional point cloud within the preset time period.

[0077] Based on the two-dimensional predicted trajectory and the new two-dimensional predicted trajectory, the motion trajectory of the target object is obtained;

[0078] Alternatively, when the tracking state is the second state, the motion trajectory of the target object is determined based on the two-dimensional predicted trajectory and the three-dimensional coordinate points associated with the two-dimensional predicted trajectory.

[0079] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0080] Based on the three-dimensional point cloud of the previous moment and the motion trajectory of the target object at the previous moment, trajectory prediction is performed to obtain the two-dimensional predicted trajectory of the target object at the current moment.

[0081] From the three-dimensional point cloud at the current moment, determine each three-dimensional coordinate point associated with the two-dimensional predicted trajectory;

[0082] Determine the target coordinate point from the associated three-dimensional coordinate points;

[0083] Based on the historical target coordinate points in the target coordinate point array, the tracking status of the target coordinate point corresponding to the current moment is determined. The historical target coordinate points are the target coordinate points in the three-dimensional point cloud collected at historical moments that are associated with the two-dimensional predicted trajectory of the target object at each historical moment. The tracking status is used to characterize whether the target coordinate point is in a normal motion state or an abnormal motion state.

[0084] Based on the tracking status and the two-dimensional predicted trajectory, the motion trajectory of the target object is obtained.

[0085] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0086] Based on the three-dimensional point cloud of the previous moment and the motion trajectory of the target object at the previous moment, trajectory prediction is performed to obtain the two-dimensional predicted trajectory of the target object at the current moment.

[0087] From the three-dimensional point cloud at the current moment, determine each three-dimensional coordinate point associated with the two-dimensional predicted trajectory;

[0088] Determine the target coordinate point from the associated three-dimensional coordinate points;

[0089] Based on the historical target coordinate points in the target coordinate point array, the tracking status of the target coordinate point corresponding to the current moment is determined. The historical target coordinate points are the target coordinate points in the three-dimensional point cloud collected at historical moments that are associated with the two-dimensional predicted trajectory of the target object at each historical moment. The tracking status is used to characterize whether the target coordinate point is in a normal motion state or an abnormal motion state.

[0090] Based on the tracking status and the two-dimensional predicted trajectory, the motion trajectory of the target object is obtained.

[0091] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0092] Based on the three-dimensional point cloud of the previous moment and the motion trajectory of the target object at the previous moment, trajectory prediction is performed to obtain the two-dimensional predicted trajectory of the target object at the current moment.

[0093] From the three-dimensional point cloud at the current moment, determine each three-dimensional coordinate point associated with the two-dimensional predicted trajectory;

[0094] Determine the target coordinate point from the associated three-dimensional coordinate points;

[0095] Based on the historical target coordinate points in the target coordinate point array, the tracking status of the target coordinate point corresponding to the current moment is determined. The historical target coordinate points are the target coordinate points in the three-dimensional point cloud collected at historical moments that are associated with the two-dimensional predicted trajectory of the target object at each historical moment. The tracking status is used to characterize whether the target coordinate point is in a normal motion state or an abnormal motion state.

[0096] Based on the tracking status and the two-dimensional predicted trajectory, the motion trajectory of the target object is obtained.

[0097] The aforementioned target tracking method, target tracking device, computer equipment, storage medium, and computer program product, based on the 3D point cloud of the previous moment and the target object's motion trajectory obtained through trajectory prediction at the previous moment, perform trajectory prediction to obtain the target object's 2D predicted trajectory at the current moment. Then, from the 3D point cloud of the current moment, determine the 3D coordinate points associated with the 2D predicted trajectory at the current moment, and determine the target coordinate point from the associated 3D point coordinates. Each moment can determine a corresponding target coordinate point, and the target coordinate points corresponding to historical moments are stored in a target coordinate point array. Secondly, based on the historical target coordinate points in the target coordinate point array, determine whether the tracking state of the target coordinate point corresponding to the current moment is in a normal motion state or an abnormal motion state. The historical target coordinate points are the target coordinate points in the 3D point cloud collected at historical moments that are associated with the target object's 2D predicted trajectory at each historical moment. Finally, based on the tracking state and the 2D predicted trajectory, obtain the target object's motion trajectory. Based on the aforementioned target tracking method, target tracking device, computer equipment, storage medium, and computer program product, the three-dimensional coordinate points monitored by the millimeter-wave radar are processed from both the horizontal plane and the vertical direction. The resulting two-dimensional predicted trajectory of the target object, obtained by predicting the trajectory of the three-dimensional point cloud, represents the target object's motion path. Next, trajectory prediction is performed on the target coordinate points among the three-dimensional coordinate points associated with the two-dimensional predicted trajectory to obtain the target object's altitude trajectory. The altitude trajectory determines whether the motion state of the target coordinate points is normal. When the motion state of the target coordinate points is abnormal, it indicates that the associated three-dimensional coordinate points contain the coordinates of other objects. In this case, the two-dimensional predicted trajectory can be adjusted accordingly based on the tracking status to avoid the influence of other objects on the trajectory prediction, thereby improving the accuracy of trajectory prediction and the tracking precision of the target. Attached Figure Description

[0098] Figure 1 This is a flowchart illustrating a target tracking method in one embodiment;

[0099] Figure 2 This is a flowchart illustrating the target tracking method in another embodiment;

[0100] Figure 3 This is a flowchart illustrating the target tracking method in another embodiment;

[0101] Figure 4 This is a flowchart illustrating the target tracking method in another embodiment;

[0102] Figure 5 This is a flowchart illustrating the target tracking method in another embodiment;

[0103] Figure 6This is a flowchart illustrating the target tracking method in another embodiment;

[0104] Figure 7 This is a flowchart illustrating the target tracking method in another embodiment;

[0105] Figure 8 This is a structural block diagram of a target tracking device in one embodiment;

[0106] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0107] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0108] In one embodiment, such as Figure 1 As shown, a target tracking method is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. The terminal can be various types of devices, such as a millimeter-wave radar. In this embodiment, the method includes the following steps:

[0109] Step 102: Based on the three-dimensional point cloud of the previous moment and the motion trajectory of the target object in the previous moment, perform trajectory prediction to obtain the two-dimensional predicted trajectory of the target object in the current moment.

[0110] In this embodiment, when tracking a target object using millimeter-wave radar, the millimeter-wave radar can detect the position of each point on the target object's surface relative to the radar. That is, the millimeter-wave radar can obtain measurement values ​​of each point on the target object's surface through detection. These measurement values ​​include the radial distance *r*, azimuth angle *a*, and elevation angle *ε* of any point on the target object relative to the millimeter-wave radar. These measurement values ​​can be directly acquired by the millimeter-wave radar, and each measurement value represents the coordinates of any point in a spherical coordinate system centered on the millimeter-wave radar. Based on the measurement values, the rectangular coordinates (i.e., three-dimensional coordinates) of any point in a rectangular coordinate system centered on the millimeter-wave radar can be obtained. The set of these three-dimensional coordinate points constitutes a three-dimensional point cloud. The target object is the tracking target of the millimeter-wave radar. The motion track is the optimal state value of the track at a given moment, determined based on the two-dimensional predicted track at that moment and the three-dimensional coordinates associated with the two-dimensional predicted track at that moment. The motion track accurately represents the motion state of the target object.

[0111] Specifically, the measurement values ​​of each point at the previous moment can first be obtained from the millimeter-wave radar, and these values ​​are converted into three-dimensional coordinates to obtain the three-dimensional point cloud at the previous moment. During processing, when other objects intersect with the target object at close range, it becomes impossible to distinguish the three-dimensional point clouds generated by other objects from the three-dimensional point cloud of the target object. Therefore, the three-dimensional point clouds at each moment may include the three-dimensional point clouds of other objects. Thus, based on the three-dimensional point cloud at the previous moment and the target object's trajectory at the previous moment, the three-dimensional point cloud associated with the trajectory at the previous moment can be determined, i.e., the target object's three-dimensional point cloud at the previous moment. Then, trajectory prediction is performed on the target object's three-dimensional point cloud at the previous moment to obtain the target object's two-dimensional predicted trajectory at the current moment.

[0112] In this context, any three-dimensional coordinate point in the three-dimensional point cloud can be represented as (x, y, z), where xoy plane represents the horizontal plane and z represents the height perpendicular to the horizontal plane. In the above trajectory prediction process, trajectory prediction is performed based on the horizontal plane coordinates (x, y) of each three-dimensional coordinate point at the previous moment to obtain the two-dimensional predicted trajectory of the target object.

[0113] Step 104: Determine the three-dimensional coordinate points associated with the two-dimensional predicted trajectory from the three-dimensional point cloud at the current moment.

[0114] In this embodiment, after obtaining the two-dimensional predicted trajectory of the target object at the current moment, the three-dimensional coordinate points associated with the two-dimensional predicted trajectory can be determined from the three-dimensional point cloud at the current moment. The association process is as follows: First, the two-dimensional coordinates of each point in the three-dimensional point cloud at the current moment are determined. Based on the two-dimensional predicted trajectory at the current moment and the two-dimensional coordinates of each point, the distance between each point in the three-dimensional point cloud at the current moment and the two-dimensional predicted trajectory is determined. When the distance is less than or equal to a threshold, the three-dimensional coordinate point is associated with the two-dimensional predicted trajectory. The threshold can be set by the operator based on the estimated size of the target object and the error of the trajectory prediction.

[0115] For example, a two-dimensional predicted trajectory at a certain moment can be represented by a two-dimensional coordinate point. The two-dimensional predicted trajectory is composed of two-dimensional coordinate points at various moments within a time period. Taking the two-dimensional predicted trajectory at the current moment as (0,3), the three-dimensional point cloud at the current moment includes three-dimensional coordinate points A (0,2.9,1.6), B (0.1,3,1.73), and C (0.2,3.2,1.64). Taking a threshold of 0.2 as an example, the two-dimensional coordinates of point A are (0,2.9), and the distance between point A and the two-dimensional predicted trajectory is 0.1; the two-dimensional coordinates of point B are (0.1,3), and the distance between point B and the two-dimensional predicted trajectory is 0.1; the two-dimensional coordinates of point C are (2,5), and the distance between point C and the two-dimensional predicted trajectory is... If the distances between point A and the two-dimensional predicted track and the distances between point B and the two-dimensional predicted track are both less than the threshold of 0.2, then points A and B are three-dimensional coordinate points associated with the two-dimensional predicted track. If the distance between point C and the two-dimensional predicted track is greater than the threshold, then point C is not associated with the two-dimensional predicted track.

[0116] In this embodiment of the application, each three-dimensional coordinate point associated with the two-dimensional predicted trajectory is labeled. At the same time, the labels of each three-dimensional coordinate point associated with the two-dimensional predicted trajectory are the same, while at different times, the labels of each three-dimensional coordinate point associated with the two-dimensional predicted trajectory are different.

[0117] Step 106: Determine the target coordinate point from the associated 3D coordinate points.

[0118] In this embodiment, the target coordinate point at the current moment can be determined based on the three-dimensional coordinate points associated with the two-dimensional predicted trajectory at the current moment. The associated three-dimensional coordinate points at any moment are denoised to remove error points. The reliable highest point of the denoised three-dimensional coordinate points is then used as the target highest point. The target coordinate point can be the highest point of the target object. For example, when the target object is a person, the top of the person's head is the highest point, so the top of the head is used as the target coordinate point. Alternatively, the target coordinate point can be the highest point of other objects. For example, if the target object is a patient sitting in a wheelchair and the other object is an upright caregiver, and the two objects intersect at a close distance at a certain moment, the caregiver's three-dimensional coordinate point will be associated with the patient's current two-dimensional predicted trajectory. Since the caregiver's highest point is higher than the patient's highest point, the caregiver's highest point (the top of the head) is the target highest point.

[0119] In other words, during target tracking, the distance between the target object and other objects is usually large. In this case, the associated 3D coordinate point is the 3D coordinate point of the target object, and the target coordinate point represents the highest point of the target object. When the target object intersects with other objects, the associated 3D coordinate point includes the 3D coordinate point of the target object and the 3D coordinate point of other objects. In this case, the target coordinate point may be the highest point of other objects.

[0120] Step 108: Based on the historical target coordinate points in the target coordinate point array, determine the tracking status of the target coordinate point corresponding to the current moment. The historical target coordinate points are the target coordinate points in the three-dimensional point cloud collected at historical moments that are associated with the two-dimensional predicted trajectory of the target object at each historical moment. The tracking status is used to characterize whether the target coordinate point is in a normal motion state or an abnormal motion state.

[0121] In this embodiment, during target tracking, target coordinates at various times can be obtained in real time. Historical target coordinates are stored in a target coordinate array, and the target coordinates at the current time are also stored in the target coordinate array. Then, based on the historical target coordinates in the target coordinate array, trajectory prediction is performed. Based on the altitude trajectory obtained from the trajectory prediction, the tracking status of the target coordinates at the current time can be determined as either normal or abnormal motion.

[0122] Secondly, during normal target tracking, other objects are far away from the target object and will not affect the tracking of the target object. At this time, for the same target object, the height of the target coordinate point at each moment will not change significantly (that is, the tracking state of the target coordinate point is normal motion state). However, when other objects intersect with the target object, the target coordinate point is the highest point of the target object at the moment before the intersection. At the moment of intersection, the target coordinate point becomes the highest point of the other object. In the above process, the height of the highest point of the other object is abruptly different from the height of the highest point of the target object before the intersection (that is, the tracking state of the target coordinate point is abnormal motion state).

[0123] In this embodiment, the tracking status of the target coordinate point at the current moment can be determined by judging whether the height of the highest point of the target object changes abruptly between adjacent moments. That is, when determining the tracking status of the target coordinate point at the current moment, only the data from the previous moment and the data from the current moment are needed. The target coordinate point array, which includes multiple historical target coordinate points, can be stored in the database. The memory of the database can be set by the staff. Since the amount of data required to determine the tracking status is small, a small amount of memory can be set for the database to reduce the resource occupation of the processing module by the storage space.

[0124] Step 110: Based on the tracking status and the two-dimensional predicted trajectory, obtain the motion trajectory of the target object.

[0125] In this embodiment, when the tracking state is in a normal motion state, the target coordinate points associated with the two-dimensional predicted trajectory are the three-dimensional coordinate points of the target object, and the tracking of the target object is not affected. When the tracking state is in an abnormal motion state, the target coordinate points associated with the two-dimensional predicted trajectory include the three-dimensional coordinate points of the target object and the three-dimensional coordinate points of other objects, and the tracking of other objects will affect the tracking of the target object. Therefore, the two-dimensional predicted trajectory can be adjusted accordingly based on the tracking state of the target coordinate points at the current moment to obtain the motion trajectory of the target object.

[0126] In the aforementioned target tracking method, the three-dimensional coordinate points monitored by the millimeter-wave radar are processed from both the horizontal plane and the vertical direction perpendicular to the horizontal plane. The two-dimensional predicted trajectory of the target object, obtained by predicting the trajectory of the three-dimensional point cloud, represents the target object's motion path. Next, trajectory prediction is performed on the target coordinate points among the three-dimensional coordinate points associated with the two-dimensional predicted trajectory to obtain the target object's altitude trajectory. Based on the altitude trajectory, the motion state of the target coordinate points can be determined. When the motion state of the target coordinate points is abnormal, it indicates that the associated three-dimensional coordinate points contain the coordinates of other objects. In this case, the two-dimensional predicted trajectory can be adjusted accordingly based on the tracking status to avoid the influence of other objects on the trajectory prediction, thereby improving the accuracy of trajectory prediction and target tracking.

[0127] In one embodiment, such as Figure 2 As shown, step 102 involves predicting the trajectory of the target object based on the 3D point cloud from the previous moment and the target object's trajectory from the previous moment, to obtain the target object's predicted 2D trajectory at the current moment, including:

[0128] Step 202: Obtain the historical 3D coordinates of the target object in the 3D point cloud of the previous moment, which are associated with the target object's motion trajectory in the previous moment.

[0129] In this embodiment, during the tracking process of the previous moment, it is necessary to determine the historical 3D coordinate points associated with the motion trajectory of the previous moment. The specific process of determining the historical 3D coordinate points is similar to the relevant description in the previous embodiments and will not be repeated here. After determining the historical 3D coordinate points, they can be cached and recorded. Then, at the current moment, the historical 3D coordinate points associated with the motion trajectory of the target object in the previous moment's 3D point cloud can be directly obtained from the cached records. The cached records only store the data of the previous moment. When the current moment becomes the previous moment, the data of the current moment will replace the data of the previous moment and be stored in the cached records.

[0130] Step 204: Based on historical 3D coordinate points, predict the trajectory to obtain the 2D predicted trajectory of the target object at the current moment.

[0131] In this embodiment, trajectory prediction is performed based on historical three-dimensional coordinate points associated with the motion trajectory at the previous moment, thereby obtaining the two-dimensional predicted trajectory of the target object at the current moment. This embodiment does not specifically limit the trajectory prediction method; any method that can perform trajectory prediction based on three-dimensional point clouds is applicable to this embodiment. For example, the Kalman filter method can be used for trajectory prediction.

[0132] In this embodiment, when performing trajectory prediction, the system cannot distinguish the three-dimensional coordinates of different target objects. Therefore, based on the motion trajectory of the previous moment, the system determines the historical three-dimensional coordinates associated with the motion trajectory and then performs trajectory prediction based on the associated historical three-dimensional coordinates. Compared with directly using all three-dimensional coordinates for trajectory prediction, the obtained two-dimensional predicted trajectory will be more accurate. This can avoid the influence of irrelevant three-dimensional coordinates on trajectory prediction and improve the accuracy of trajectory prediction.

[0133] In one embodiment, such as Figure 3 As shown, step 106, determining the target coordinate point from the associated 3D coordinate points, includes:

[0134] Step 302: Obtain the height of each associated 3D coordinate point.

[0135] In this embodiment, the coordinate system of the three-dimensional coordinate point is a coordinate system with the projection of the millimeter-wave radar onto the horizontal plane as the coordinate center o, the horizontal plane as xoy, and the direction perpendicular to the horizontal plane as the z-axis. That is, the height of the three-dimensional coordinate point is the coordinate of the three-dimensional coordinate point in the z-axis direction. Therefore, the z-coordinate (i.e., the height of the three-dimensional coordinate point) of each three-dimensional coordinate point can be obtained from the three-dimensional coordinate points associated with the two-dimensional predicted trajectory.

[0136] For example, consider the associated coordinate points A (1,1,1.6), B (2,1,1.7), and C (3,2,1.8), where the height of point A is 1.6, the height of point B is 1.7, and the height of point C is 1.8.

[0137] Step 304: Sort the multiple three-dimensional coordinate points from high to low according to their height to obtain a point cloud sequence.

[0138] In this embodiment of the application, after obtaining the height of each three-dimensional coordinate point, the three-dimensional coordinate points can be sorted in descending order of height to obtain a point cloud sequence. For example, taking the above example again, the height of point A is 1.6, the height of point B is 1.7, and the height of point C is 1.8. Sort them in descending order of height to obtain the point cloud sequence of point C (3,2,1.8), point B (2,1,1.7), and point A (1,1,1.6).

[0139] Step 306: The first and second three-dimensional coordinate points in the point cloud sequence are determined as the first sequence point and the second sequence point, respectively.

[0140] In this embodiment, the highest (first) 3D coordinate point in the point cloud sequence is designated as the first sequence point, and the second highest (second) 3D coordinate point is designated as the second sequence point. Using the example above, the point cloud sequence is point C (3,2,1.6), point B (2,1,1.7), and point A (1,1,1.8). Point C (3,2,1.8), being the first, is the first sequence point, and point B (2,1,1.7), being the second, is the second sequence point.

[0141] Step 308: Determine the target coordinate point corresponding to the current moment from the point cloud sequence based on the first height difference between the first sequence points and the second sequence points.

[0142] In this embodiment of the application, the z-coordinate of the first sequence point is subtracted from the z-coordinate of the second sequence point to obtain the first height difference. Then, based on the first height difference, the target coordinate point corresponding to the current moment is determined from the point cloud sequence.

[0143] In this context, the point cloud sequence consists of sorted and associated 3D coordinate points. The 3D coordinate points associated with the 2D predicted trajectory may be the 3D coordinate points of the target object, or include the 3D coordinate points of the target object and other objects. In the above cases, the target object and other objects are objects with a certain volume, and their 3D coordinate points should be continuous, that is, the height difference between the highest and second highest points in the 3D coordinate points is within a certain range. Therefore, when the first height difference is less than a preset value, the first sequence point is the highest point of the target object or other object (i.e., the highest point of the actual object), and only then does the first sequence point have value for trajectory prediction. If the first height difference is greater than or equal to the preset value, the first sequence point is not the highest point of the target object or any other object, but an error point introduced due to environmental factors such as noise.

[0144] In this embodiment, the three-dimensional coordinate points are sorted according to their height, and the target coordinate point is determined based on the first height difference between the highest and second highest points. Then, trajectory prediction can be performed based on the target coordinate point to obtain the height trajectory of the target object. The tracking status of the target coordinate point (whether the motion status is normal) can be determined based on the height trajectory. When the motion status of the target coordinate point is abnormal, it indicates that the three-dimensional coordinate points of other objects have been mixed in with the associated three-dimensional coordinate points. At this time, the two-dimensional predicted trajectory can be adjusted accordingly based on the tracking status to avoid the influence of other objects on the trajectory prediction, thereby improving the accuracy of the trajectory prediction.

[0145] In one embodiment, such as Figure 4 As shown, step 308, based on the first height difference between the first sequence points and the second sequence points, determines the target coordinate point corresponding to the current moment from the point cloud sequence, including:

[0146] Step 402: Determine the first height difference between the first sequence point and the second sequence point.

[0147] In this embodiment of the application, the height (z-coordinate) of the first sequence point is subtracted from the height (z-coordinate) of the second sequence point to obtain the first height difference between the first sequence point and the second sequence point.

[0148] Step 404: When the first height difference is less than the first height difference threshold, the first sequence point is taken as the target coordinate point corresponding to the current time.

[0149] In this embodiment of the application, the first height difference is compared with the first height difference threshold (which can be set by the staff). When the first height difference is less than the first height difference threshold, the first sequence point is the target coordinate point corresponding to the current time.

[0150] Step 406: When the first height difference is greater than or equal to the first height difference threshold, delete the first sequence point from the point cloud sequence. Repeat the step of determining the first and second three-dimensional coordinate points in the point cloud sequence as the first sequence point and the second sequence point until the first height difference between the first sequence point and the second sequence point is less than the first height difference threshold. Then, take the first sequence point as the target coordinate point corresponding to the current time.

[0151] In this embodiment, when the first height difference is greater than or equal to the first height difference threshold, the first sequence point in the point cloud sequence is deleted. At this time, the first and second three-dimensional coordinate points in the point cloud sequence change. The new first three-dimensional coordinate point is determined as the first sequence point, and the new second three-dimensional coordinate point is determined as the second sequence point. Then, the z-coordinate of the first sequence point and the z-coordinate of the second sequence point are subtracted again to obtain the new first height difference. It is determined whether the new first height difference is less than the first height difference threshold. If the new first height difference is greater than or equal to the first height difference threshold, the above operation is repeated until the first height difference is less than the first height difference threshold. The first sequence point at this time is taken as the target coordinate point corresponding to the current time.

[0152] For example, taking the point cloud sequence: point a(2,3,1.85), point b(4,4,1.7), point c(4,6,1.65), with a first height difference threshold of 0.1, firstly, the first sequence point is determined to be point a, and the second sequence point is determined to be point b. Then the first height difference H1 = 1.85 - 1.7 = 0.15. The first height difference H1 is greater than the first height difference threshold, so the first sequence point: point a is deleted from the point cloud sequence, resulting in a new point cloud sequence: point b(4,4,1.7), point c(4,6,1.65). At this time, the first sequence point is point b, and the second sequence point is point c. Then the first height difference H2 = 1.7 - 1.65 = 0.05. The first height difference H2 is less than the first height difference threshold, so the first sequence point: point b at this time is taken as the target coordinate point corresponding to the current moment.

[0153] In this embodiment, the first height difference and the first height difference threshold are used to determine whether the current first sequence point is the target coordinate point. When the first height difference is greater than or equal to the first height difference threshold, it means that the first sequence point is an error point. At this time, the first sequence point is deleted, and the previous steps of determining the first height difference are repeated until the first height difference is less than the first height difference threshold. The influence of the error point can be eliminated and the target coordinate point can be found accurately.

[0154] In one embodiment, during practical application, the height of each target coordinate point may also exhibit outlier phenomena. Outlier phenomena refer to the occurrence of random abnormal values ​​in the data due to certain reasons. The height of the above-mentioned random abnormal values ​​is different from that of error points introduced by noise and other factors. Therefore, after determining the target coordinate points corresponding to each time, outlier correction can be performed on the height values ​​of each target coordinate point. In this embodiment, only correction is considered for isolated outlier phenomena.

[0155] The condition for determining whether the height of the target coordinate point at that moment is an outlier is given by the following formula:

[0156] ([HISTSIZE-2]-[HISTSIZE-1])×[HISTSIZE-2]-[HISTSIZE-3])>0 Formula (1)

[0157] And abs[HISTSIZE-2]-[HISTSIZE-1])>thresh_1 Formula (II)

[0158] And abs([HISTSIZE-2]-[HISTSIZE-3])>thresh_1 (Formula 3)

[0159] Where [HISTSIZE-1] represents the height of the target coordinate point at the previous time step, [HISTSIZE-2] represents the height of the target coordinate point at the current time step, [HISTSIZE-3] represents the height of the target coordinate point at the next time step, abs represents the calculation of the absolute value, and thresh_1 represents the preset value.

[0160] The process of correcting the height of outlier target coordinates is described in the following formula (IV):

[0161]

[0162] In one embodiment, such as Figure 5 As shown, step 108, based on the historical target coordinate points in the target coordinate point array, determines the tracking status of the target coordinate point at the current moment, including:

[0163] Step 502: Cluster the historical target coordinate points according to their height to obtain a target array, and determine the cluster centers of the target array.

[0164] In this embodiment of the application, according to the steps described in the above embodiments, the target coordinate points corresponding to each time moment can be obtained. The target coordinate points corresponding to historical time moments are called historical target coordinate points. As time changes, the number of historical target coordinate points increases in real time. In order to obtain a stable starting point for trajectory prediction, a threshold for the number of coordinate points can be preset. When the number of historical target coordinate points is less than the threshold, no operation is performed, and the historical target coordinate points are updated until the number of historical target coordinate points is greater than or equal to the threshold. Then, trajectory prediction is started for the altitude of the target coordinate points.

[0165] Among them, historical target coordinate points can be clustered according to their height to obtain multiple clusters. Each cluster includes several historical target coordinate points. Based on the number of historical target coordinate points in each cluster, the cluster with the most historical target coordinate points is taken as the target array, and the cluster center of the target array is determined.

[0166] In one embodiment, the height of the historical target coordinate points can also be directly clustered. The resulting clusters include the z-coordinates of the historical target coordinate points. Therefore, the cluster center of the determined target array is a height value.

[0167] The steps for directly clustering the heights of historical target coordinate points are as follows: Take the height of any historical target coordinate point as the initial cluster center. Subtract the heights of other historical target coordinate points from the initial cluster center. If the difference is less than or equal to a first preset threshold (set by staff), the height of that historical target coordinate point is assigned to the cluster corresponding to the initial cluster center. If the difference is greater than the first preset threshold, the height of that historical target coordinate point does not belong to the cluster corresponding to the initial cluster center. When the number of historical target coordinate point heights in that cluster exceeds a second preset threshold (set by staff), the clustering of that cluster is complete. Furthermore, for each additional historical target coordinate point height added to the cluster, the cluster center is adjusted based on that height. The adjustment steps are described in formula (V) below:

[0168] center=0.5×(center+historyHeight[i]) formula (5)

[0169] Where center represents the cluster center and historyHeight[i] represents the height of the i-th historical coordinate point.

[0170] Step 504: Starting from the cluster center, predict the trajectory of the target object based on the target coordinates at the current time to obtain the altitude trajectory of the target object.

[0171] In this embodiment of the application, the cluster center is used as the starting point for trajectory prediction. Based on the height of the target coordinate point at the current time, the trajectory of the target object is predicted to obtain the height trajectory of the target object.

[0172] In target tracking technology, the time interval between adjacent moments is extremely small. Therefore, the altitude of the target coordinate point at the current moment can be considered approximately equal to the altitude of the target coordinate point at the previous moment. Thus, when predicting a trajectory, the prediction begins from the starting point, which is taken as the predicted altitude for the current moment. Based on the predicted altitude and the altitude of the target coordinate point at the current moment, the optimal trajectory state value for the current moment is determined. This optimal trajectory state value is then used as the predicted altitude for the next moment. The optimal trajectory state value for the next moment is determined based on the predicted altitude and the altitude of the target coordinate point at the next moment, and so on. Following these steps, trajectory prediction is performed sequentially for the target object at each moment, where the altitude trajectory is composed of the predicted altitudes at each moment.

[0173] Step 506: Determine the tracking status of the target coordinate point at the current time based on the altitude track and the target coordinate point at the current time.

[0174] In this embodiment, the altitude track corresponding to the current moment can be compared with the target coordinate point corresponding to the current moment to determine whether the altitude of the target coordinate point corresponding to the current moment is the same as the altitude track at the current moment. If the difference between the altitude of the target coordinate point corresponding to the current moment and the altitude track at the current moment is large (greater than or equal to the second altitude difference threshold), it indicates that the altitude of the target coordinate point corresponding to the current moment has deviated. Then, the tracking status of the target coordinate point corresponding to the current moment can be determined based on the deviation of the target coordinate point altitude at each moment.

[0175] In this embodiment, the altitude of the target coordinate point is first predicted. When predicting the altitude, the historical target coordinate points are clustered based on their altitudes, and the cluster center of the target array is used as the starting point for the altitude prediction. Compared with using the altitude of a single target coordinate point as the starting point, using the cluster center as the starting point is more reliable and the resulting altitude track is more accurate.

[0176] Secondly, based on the altitude track and the target coordinates at the current moment, the tracking status of the target coordinates at the current moment is determined (whether the motion state of the target coordinates is normal). If the motion state of the target coordinates is abnormal, it indicates that the three-dimensional coordinates of other objects have been mixed into the associated three-dimensional coordinates. At this time, the two-dimensional predicted track can be adjusted accordingly based on the tracking status to avoid the influence of other objects on the track prediction, thereby improving the accuracy of the track prediction.

[0177] In one embodiment, such as Figure 6 As shown, step 506 involves determining the tracking status of the target coordinate point at the current moment based on the altitude track and the target coordinate point at the current time, including:

[0178] Step 602: Determine the altitude of the target coordinate point at the current moment and the second altitude difference between the altitude track at the current moment.

[0179] In this embodiment of the application, the height (z coordinate) of the target coordinate point corresponding to the current time is obtained, and the difference between the above z coordinate and the height of the altitude track at the current time is obtained. The absolute value of the difference is used as the second altitude difference.

[0180] Step 604: When the second height difference is greater than or equal to the second height difference threshold, increase the number of times the foot is lost by 1.

[0181] In this embodiment of the application, during the movement of the same target object, when the time interval is small enough, the height change of the target object between two adjacent moments is minimal. That is, the height change value of the target coordinate point should be less than a certain value, which is the maximum value of the height change of the target object between two adjacent moments. When the height difference between the two detected target coordinate points is greater than this value, it indicates that the height of the target coordinate point corresponding to the current moment has deviated from the altitude track, and the two target coordinate points do not belong to the same target object. At this time, the millimeter-wave radar loses tracking, and the number of times it loses tracking is the number of times the height of the target coordinate point deviates from the altitude track.

[0182] The staff can set a second altitude difference threshold and compare the second altitude difference with the second altitude difference threshold. If the second altitude difference is greater than or equal to the second altitude difference threshold, it means that the altitude of the target coordinate point at the current time deviates from the altitude track. At this time, the number of times the target has lost track is increased by 1, and the initial number of times the target has lost track is 0.

[0183] Step 606: Determine the tracking status of the target coordinate point at the current moment based on the number of tracking failures.

[0184] In this embodiment of the application, the tracking status of the target coordinate point at the current moment can be determined based on the number of tracking failures. The tracking status is used to characterize whether the target coordinate point is in a normal motion state or not.

[0185] When the number of tracking failures is small, the target coordinate point is considered to have lost tracking occasionally. At this time, it will not have a significant impact on the two-dimensional predicted trajectory of the target object, and the tracking status of the target coordinate point is that the target coordinate point is in a normal motion state. When the number of tracking failures reaches a certain value, the target coordinate point is considered to have lost tracking completely, and the tracking status of the target point is that the target point is in an abnormal motion state.

[0186] In this embodiment, the tracking status of the target coordinate point (whether the motion state of the target coordinate point is normal) is determined based on the second altitude difference between the height of the target coordinate point at the current moment and the height of the altitude track at the current moment. When the motion state of the target coordinate point is abnormal, it indicates that the three-dimensional coordinate points of other objects have been mixed in with the associated three-dimensional coordinate points. At this time, the two-dimensional predicted track can be adjusted accordingly based on the tracking status to avoid the influence of other objects on the track prediction, thereby improving the accuracy of the track prediction.

[0187] In one embodiment, step 606, determining the tracking status of the target coordinate point at the current moment based on the number of tracking failures, includes:

[0188] If the number of tracking errors is greater than or equal to the tracking error threshold, the tracking status of the target coordinate point at the current moment is determined to be the first state; or...

[0189] If the number of tracking errors is less than the tracking error threshold, the tracking status of the target coordinate point at the current moment is determined to be the second state.

[0190] The first state is used to characterize the target coordinate point as being in an abnormal motion state, and the second state is used to characterize the target coordinate point as being in a normal motion state.

[0191] In this embodiment, the number of tracking errors is compared with a tracking error threshold (set by staff). If the number of tracking errors is greater than or equal to the tracking error threshold, the tracking status of the target coordinate point at the current moment is determined to be a first state, which is used to characterize the target coordinate point as being in an abnormal motion state. If the number of tracking errors is less than the tracking error threshold, the tracking status of the target coordinate point at the current moment is determined to be a second state, which is used to characterize the target coordinate point as being in a normal motion state.

[0192] In this embodiment, by setting a threshold for the number of tracking failures, the tracking status of the target coordinate point is determined based on the number of tracking failures and the threshold. Only when the number of tracking failures reaches the threshold (greater than or equal to the number of tracking failures) is the tracking status of the target coordinate point considered to be in the first state, making the judgment of the tracking status more accurate and thus improving the accuracy of trajectory prediction.

[0193] In one embodiment, the method further includes:

[0194] If no second height difference greater than or equal to the second height difference threshold is detected within the preset time period, the number of times the target is lost will be reset to zero.

[0195] In this embodiment of the application, during the target tracking process, due to environmental factors and other factors such as noise in the millimeter-wave radar used, there may be some error points in the target coordinates. The height of the error points will also deviate from the altitude track, thereby increasing the number of tracking failures by 1. If no restrictions are imposed when counting the number of tracking failures, the error points will affect the judgment of the tracking status of the target coordinates.

[0196] To address the aforementioned issues, we define the condition for determining whether a target coordinate point is in the first state as follows: when the target coordinate point loses tracking several times within a short period of time, that is, when the number of times it loses tracking reaches a threshold greater than or equal to the number of times it loses tracking within a short period of time, the tracking status of the target coordinate point is considered to be in the first state. Taking a threshold of 4 times for the number of times it loses tracking as an example, if the number of times it loses tracking reaches 4 during a certain tracking process, and it takes a whole day, it is clear that the target coordinate point is not in an abnormal movement state at this time.

[0197] In this embodiment of the application, in actual application scenarios, error points exist in isolation. That is, within a certain time range, the number of error points is less than or equal to 1. Therefore, this time range can be used as a preset duration (which can be set by staff). If no second height difference is detected that is greater than or equal to the second height difference threshold within the preset duration, the number of tracking errors will be cleared to zero.

[0198] For example, taking a preset duration of 2 seconds as an example, if the second height difference is detected to be greater than or equal to the second height difference threshold at 0.5 seconds, the number of missed steps is increased by 1, and the number of missed steps is 1. If the second height difference is not detected to be greater than or equal to the second height difference threshold in the next 1.5 seconds, the number of missed steps is reset to zero, and the number of missed steps is 0.

[0199] In this embodiment of the application, by setting a preset time, if the second altitude difference is not detected to be greater than or equal to the second altitude difference threshold within the preset time, the number of tracking failures is cleared to zero. This can eliminate the impact of error points caused by factors such as noise on target tracking, thereby improving the accuracy of trajectory prediction and target tracking.

[0200] In one embodiment, such as Figure 7 As shown, step 110, based on the tracking status and the two-dimensional predicted trajectory, obtains the motion trajectory of the target object, including:

[0201] Step 702: If the tracking state is in the first state, interrupt the two-dimensional predicted trajectory.

[0202] Step 704: After a preset time period, a new two-dimensional predicted trajectory is created based on the three-dimensional point cloud within the preset time period.

[0203] Step 706: Based on the two-dimensional predicted trajectory and the new two-dimensional predicted trajectory, obtain the motion trajectory of the target object.

[0204] In this embodiment of the application, when the tracking state is the first state (the target coordinate point is in an abnormal motion state), it means that the target coordinate point at the current moment is the three-dimensional coordinate point of another target. That is, at the current moment, the three-dimensional coordinate points associated with the two-dimensional predicted trajectory include not only the three-dimensional coordinate point corresponding to the target, but also the three-dimensional coordinate points of other objects. At this time, the two-dimensional predicted trajectory obtained by predicting the trajectory based on the associated three-dimensional coordinate points at the current moment is not accurate. Therefore, the two-dimensional predicted trajectory can be interrupted first, and trajectory prediction can no longer be performed.

[0205] Other objects and the target object will not always intersect; they will only intersect within a certain time frame. Therefore, a preset duration can be set based on the actual tracking situation. After the preset duration, other objects and the target object will separate. At this point, a new track is created based on the 3D point cloud within the preset duration. The steps for creating the new track are as follows:

[0206] The three-dimensional point cloud at the last moment of a preset duration can be used as the starting point. Based on the starting point, the trajectory can be predicted to obtain the two-dimensional predicted trajectory at the current moment. Based on the three-dimensional coordinate points associated with the two-dimensional predicted trajectory at the current moment, the trajectory can be predicted to obtain the two-dimensional predicted trajectory at the next moment. Repeat the above steps to obtain a new two-dimensional predicted trajectory.

[0207] Finally, the optimal state value of the trajectory (i.e., the motion trajectory) at each moment can be determined based on the two-dimensional predicted trajectory and the three-dimensional coordinate points associated with the two-dimensional predicted trajectory. Furthermore, a new motion trajectory can be determined based on the new two-dimensional predicted trajectory and the three-dimensional coordinate points associated with the new two-dimensional predicted trajectory. The motion trajectory of the target object includes the aforementioned motion trajectory and the new motion trajectory.

[0208] In one embodiment, either nonlinear Kalman filtering or linear Kalman filtering can be selected for trajectory prediction.

[0209] Step 708: When the tracking state is in the second state, determine the motion trajectory of the target object based on the two-dimensional predicted trajectory and the three-dimensional coordinate points associated with the two-dimensional predicted trajectory.

[0210] In this embodiment of the application, if the tracking state is the second state, that is, the target coordinate point is in a normal motion state, the optimal state value of the track at each time (the motion track of the target object) is determined directly based on the two-dimensional predicted track and each three-dimensional coordinate point associated with the two-dimensional predicted track.

[0211] In this embodiment, when the tracking state is in the first state (the target coordinate point is in an abnormal motion state), the current two-dimensional predicted trajectory is interrupted to avoid continuing to generate trajectory prediction errors. After a preset time, that is, after other objects that intersect with the target object have moved away from the target object, the trajectory prediction is performed again. By adjusting the two-dimensional predicted trajectory in the above manner, the influence of other objects on the target object's trajectory prediction is avoided, thereby improving the accuracy of trajectory prediction.

[0212] When the tracking state is in the second state, this embodiment can determine the motion track based on the predicted two-dimensional predicted track and the actual detected three-dimensional coordinate points associated with the two-dimensional predicted track. This can reduce the error generated during the prediction process and the error generated during the millimeter-wave radar detection process to a certain extent, and obtain a more accurate motion track.

[0213] In one specific embodiment, the millimeter-wave radar is used to monitor the three-dimensional point cloud of the target object at each moment in real time, and transmit the point cloud data to the data processing module. The data processing module can determine the historical three-dimensional coordinate points associated with the target object's motion trajectory from the three-dimensional point cloud at the previous moment based on the motion trajectory of the target object at the previous moment. Then, the Kalman filter method can be used to predict the trajectory based on the historical three-dimensional coordinate points to obtain the two-dimensional predicted trajectory of the target object at the current moment, and determine the three-dimensional coordinate points associated with the two-dimensional predicted trajectory at the current moment from the three-dimensional point cloud at the current moment.

[0214] Obtain the height of each associated 3D coordinate point. Sort the multiple 3D coordinate points from highest to lowest height to obtain a point cloud sequence. The first 3D coordinate point in the point cloud sequence is designated as the first sequence point, and the second 3D coordinate point as the second sequence point. The height difference between the first and second sequence points is calculated to obtain the first height difference. Compare this first height difference with a first height difference threshold. If the first height difference is less than the first height difference threshold, the first sequence point is designated as the target coordinate point at the current moment. If the first height difference is greater than or equal to the first height difference threshold, the current first sequence point is removed from the point cloud sequence. Repeat the process of designating the first and second 3D coordinate points as the first and second sequence points until the first height difference between the first and second sequence points is less than the first height difference threshold. At this point, the first sequence point is designated as the target coordinate point at the current moment.

[0215] Based on the above steps, the target coordinates at each time point can be obtained. These historical target coordinates can then be clustered to create multiple clusters. The cluster containing the most historical target coordinates is selected as the target array, and the cluster center of this target array is used as the starting point for altitude trajectory prediction. Based on the starting point and the target coordinates at the current time, trajectory prediction is performed to obtain the altitude trajectory of the target object.

[0216] Secondly, the altitude of the target coordinate point at the current moment can be subtracted from the altitude of the trajectory at the current moment to obtain the second altitude difference. When the second altitude difference is greater than or equal to the second altitude difference threshold, the tracking failure count is increased by 1. Specifically, if the second altitude difference is detected to be greater than or equal to the second altitude difference threshold at 0.4s within a preset duration (e.g., 2s), the tracking failure count becomes 1. If the second altitude difference is not detected to be greater than or equal to the second altitude difference threshold within the next 1.6s, the tracking failure count is reset to zero. Then, the tracking failure count is restarted. If the second altitude difference is detected to be greater than or equal to the second altitude difference threshold 4 times within 2s, the tracking failure count is 4, and the tracking failure count threshold is 4, then the tracking status of the target coordinate point at the current moment is determined to be the first state (the target coordinate point is in an abnormal motion state).

[0217] When the tracking state is in the first state, the current two-dimensional predicted trajectory is interrupted. After a preset time, the three-dimensional point cloud at the last moment of the preset time can be used as the starting point. Based on the starting point, the trajectory is predicted to obtain the two-dimensional predicted trajectory at the current moment. Based on the three-dimensional coordinate points associated with the two-dimensional predicted trajectory at the current moment, the trajectory is predicted to obtain the two-dimensional predicted trajectory at the next moment. The above steps are repeated to obtain a new two-dimensional predicted trajectory. Based on the two-dimensional predicted trajectory and the new two-dimensional predicted trajectory, the motion trajectory of the target object is obtained.

[0218] When the tracking state is in the second state, the motion trajectory of the target object is determined based on the two-dimensional predicted trajectory and the three-dimensional coordinate points associated with the two-dimensional predicted trajectory.

[0219] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0220] Based on the same inventive concept, this application also provides a target tracking device for implementing the target tracking method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more target tracking device embodiments provided below can be found in the limitations of the target tracking method described above, and will not be repeated here.

[0221] In one embodiment, such as Figure 8 As shown, a target tracking device 800 is provided, including: a trajectory prediction module 802, a first determination module 804, a second determination module 806, a third determination module 808, and a fourth determination module 810, wherein:

[0222] The trajectory prediction module 802 is used to predict the trajectory based on the three-dimensional point cloud of the previous moment and the motion trajectory of the target object in the previous moment, so as to obtain the two-dimensional predicted trajectory of the target object in the current moment.

[0223] The first determining module 804 is used to determine each three-dimensional coordinate point associated with the two-dimensional predicted trajectory from the three-dimensional point cloud at the current moment.

[0224] The second determining module 806 is used to determine the target coordinate point from the associated three-dimensional coordinate points;

[0225] The third determining module 808 is used to determine the tracking status of the target coordinate point at the current moment based on the historical target coordinate points in the target coordinate point array. The historical target coordinate points are the target coordinate points in the three-dimensional point cloud collected at historical moments that are associated with the two-dimensional predicted trajectory of the target object at each historical moment. The tracking status is used to characterize whether the target coordinate point is in a normal motion state or an abnormal motion state.

[0226] The fourth determination module 810 is used to obtain the motion trajectory of the target object based on the tracking status and the two-dimensional predicted trajectory.

[0227] In this embodiment, the three-dimensional coordinate points monitored by the millimeter-wave radar are processed from both the horizontal plane direction and the vertical direction perpendicular to the horizontal plane. The two-dimensional predicted trajectory of the target object obtained by trajectory prediction of the three-dimensional point cloud can represent the motion trajectory of the target object. Next, trajectory prediction is performed on the target coordinate points among the three-dimensional coordinate points associated with the two-dimensional predicted trajectory to obtain the height trajectory of the target object. The height trajectory can determine whether the motion state of the target coordinate points is normal. When the motion state of the target coordinate points is abnormal, it indicates that the three-dimensional coordinate points of other objects have been mixed in with the associated three-dimensional coordinate points. At this time, the two-dimensional predicted trajectory can be adjusted accordingly based on the tracking status to avoid the influence of other objects on the trajectory prediction, thereby improving the accuracy of trajectory prediction and target tracking.

[0228] In one embodiment, the trajectory prediction module 802 is further configured to:

[0229] Obtain the historical 3D coordinates of the target object in the 3D point cloud at the previous moment, which are associated with the target object's trajectory at the previous moment.

[0230] Based on historical 3D coordinate points, a 2D predicted trajectory of the target object at the current moment is obtained.

[0231] In one embodiment, the second determining module 806 is further configured to:

[0232] Obtain the height of each associated 3D coordinate point;

[0233] Based on the height of each 3D coordinate point, the multiple 3D coordinate points are sorted from high to low to obtain a point cloud sequence;

[0234] The first and second three-dimensional coordinate points in the point cloud sequence are determined as the first sequence point and the second sequence point, respectively.

[0235] Based on the first height difference between the first sequence points and the second sequence points, the target coordinate point corresponding to the current moment is determined from the point cloud sequence.

[0236] In one embodiment, the second determining module 806 is further configured to:

[0237] Determine the first height difference between the first sequence point and the second sequence point;

[0238] When the first height difference is less than the first height difference threshold, the first sequence point is taken as the target coordinate point at the current time.

[0239] Alternatively, when the first height difference is greater than or equal to the first height difference threshold, the first sequence point is deleted from the point cloud sequence, and the step of determining the first and second three-dimensional coordinate points in the point cloud sequence as the first sequence point and the second sequence point is repeated until the first height difference between the first sequence point and the second sequence point is less than the first height difference threshold. The first sequence point is then used as the target coordinate point corresponding to the current time.

[0240] In one embodiment, the third determining module 808 is further configured to:

[0241] The historical target coordinate points are clustered based on their height to obtain a target array, and the cluster centers of the target array are determined.

[0242] Starting from the cluster center, the trajectory of the target object is predicted based on the target coordinates at the current time, and the altitude trajectory of the target object is obtained.

[0243] Based on the altitude track and the target coordinates at the current moment, determine the tracking status of the target coordinates at the current moment.

[0244] In one embodiment, the third determining module 808 is further configured to:

[0245] Determine the altitude of the target coordinate point at the current moment, and the second altitude difference between the altitude track at the current moment;

[0246] When the second height difference is greater than or equal to the second height difference threshold, the number of times the foot is lost is increased by 1;

[0247] Based on the number of times the target coordinates are lost, the tracking status of the target coordinates at the current moment is determined.

[0248] In one embodiment, the third determining module 808 is further configured to:

[0249] If the number of tracking errors is greater than or equal to the tracking error threshold, the tracking status of the target coordinate point at the current moment is determined to be the first state; or...

[0250] If the number of tracking errors is less than the tracking error threshold, the tracking status of the target coordinate point at the current moment is determined to be the second state.

[0251] The first state is used to characterize the target coordinate point as being in an abnormal motion state, and the second state is used to characterize the target coordinate point as being in a normal motion state.

[0252] In one embodiment, the apparatus further includes:

[0253] The zeroing module is used to reset the number of missed steps to zero if the second height difference is not detected to be greater than or equal to the second height difference threshold within a preset time period.

[0254] In one embodiment, the fourth determining module 810 is further configured to:

[0255] When the tracking status is in the first state, the two-dimensional predicted trajectory is interrupted;

[0256] After a preset time period, a new two-dimensional predicted trajectory is created based on the three-dimensional point cloud within the preset time period.

[0257] Based on the two-dimensional predicted trajectory and the new two-dimensional predicted trajectory, the motion trajectory of the target object is obtained;

[0258] Alternatively, when the tracking state is in the second state, the motion trajectory of the target object is determined based on the two-dimensional predicted trajectory and the three-dimensional coordinate points associated with the two-dimensional predicted trajectory.

[0259] Each module in the aforementioned target tracking device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0260] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a target tracking method. The display screen can be an LCD screen or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0261] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0262] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0263] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0264] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0265] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0266] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0267] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0268] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A target tracking method, characterized in that, The method includes: Based on the three-dimensional point cloud of the previous moment and the motion trajectory of the target object at the previous moment, trajectory prediction is performed to obtain the two-dimensional predicted trajectory of the target object at the current moment. From the three-dimensional point cloud at the current moment, determine each three-dimensional coordinate point associated with the two-dimensional predicted trajectory; Determine the target coordinate point from the associated three-dimensional coordinate points; Based on the historical target coordinate points in the target coordinate point array, the tracking status of the target coordinate point corresponding to the current moment is determined. The historical target coordinate points are the target coordinate points in the three-dimensional point cloud collected at historical moments that are associated with the two-dimensional predicted trajectory of the target object at each historical moment. The tracking status is used to characterize whether the target coordinate point is in a normal motion state or an abnormal motion state. Based on the tracking status and the two-dimensional predicted trajectory, the motion trajectory of the target object is obtained; Determining the tracking status of the target coordinate point at the current moment based on historical target coordinate points in the target coordinate point array includes: Based on the historical target coordinate points in the target coordinate point array, a trajectory is predicted, and based on the altitude trajectory obtained from the trajectory prediction, the tracking status of the target coordinate point at the current moment is determined to be either a normal motion state or an abnormal motion state.

2. The method according to claim 1, characterized in that, The process of predicting the trajectory based on the 3D point cloud from the previous moment and the target object's trajectory at the previous moment, to obtain the target object's 2D predicted trajectory at the current moment, includes: Obtain the historical 3D coordinates of the target object associated with its motion trajectory in the 3D point cloud at the previous moment; Based on the historical three-dimensional coordinate points, a trajectory prediction is performed to obtain the two-dimensional predicted trajectory of the target object at the current moment.

3. The method according to claim 1, characterized in that, Determining the target coordinate point from the associated three-dimensional coordinate points includes: Obtain the height of each of the associated three-dimensional coordinate points; Based on the height of each of the three-dimensional coordinate points, the multiple three-dimensional coordinate points are sorted from high to low to obtain a point cloud sequence; The three-dimensional coordinate points that are first and second in the point cloud sequence are determined as the first sequence point and the second sequence point, respectively. Based on the first height difference between the first sequence point and the second sequence point, the target coordinate point corresponding to the current moment is determined from the point cloud sequence.

4. The method according to claim 3, characterized in that, Determining the target coordinate point corresponding to the current moment from the point cloud sequence based on the first height difference between the first sequence point and the second sequence point includes: Determine the first height difference between the first sequence point and the second sequence point; When the first height difference is less than the first height difference threshold, the first sequence point is used as the target coordinate point corresponding to the current time. Alternatively, when the first height difference is greater than or equal to the first height difference threshold, the first sequence point is deleted from the point cloud sequence, and the step of determining the first and second three-dimensional coordinate points in the point cloud sequence as the first sequence point and the second sequence point is repeated until the first height difference between the first sequence point and the second sequence point is less than the first height difference threshold, and the first sequence point is used as the target coordinate point corresponding to the current time.

5. The method according to any one of claims 1 to 4, characterized in that, Determining the tracking status of the target coordinate point at the current moment based on historical target coordinate points in the target coordinate point array includes: The historical target coordinate points are clustered based on their heights to obtain a target array, and the cluster centers of the target array are determined. Starting from the cluster center, the target object is predicted based on the target coordinates at the current time to obtain the altitude trajectory of the target object. Based on the altitude track and the target coordinates at the current time, determine the tracking status of the target coordinates at the current time.

6. The method according to claim 5, characterized in that, The step of determining the tracking status of the target coordinate point at the current time based on the altitude track and the target coordinate point at the current time includes: Determine the altitude of the target coordinate point corresponding to the current moment, and the second altitude difference between the altitude track and the altitude at the current moment; When the second height difference is greater than or equal to the second height difference threshold, the number of times the foot is lost is increased by 1; Based on the number of times the tracking fails, the tracking status of the target coordinate point at the current moment is determined.

7. The method according to claim 6, characterized in that, Determining the tracking status of the target coordinate point at the current moment based on the number of tracking failures includes: If the number of tracking errors is greater than or equal to a threshold, the tracking status of the target coordinate point at the current time is determined to be the first state; or... If the number of tracking errors is less than the tracking error threshold, the tracking status of the target coordinate point at the current time is determined to be the second state. The first state is used to characterize the target coordinate point as being in an abnormal motion state, and the second state is used to characterize the target coordinate point as being in a normal motion state.

8. The method according to claim 6, characterized in that, The method further includes: if the second height difference is not detected to be greater than or equal to the second height difference threshold within a preset time period, then the number of times the foot is lost is reset to zero.

9. The method according to claim 7, characterized in that, The step of obtaining the motion trajectory of the target object based on the tracking status and the two-dimensional predicted trajectory includes: If the tracking state is the first state, the two-dimensional predicted trajectory is interrupted; After a preset time period, a new two-dimensional predicted trajectory is obtained by constructing a new trajectory based on the three-dimensional point cloud within the preset time period. Based on the two-dimensional predicted trajectory and the new two-dimensional predicted trajectory, the motion trajectory of the target object is obtained; Alternatively, when the tracking state is the second state, the motion trajectory of the target object is determined based on the two-dimensional predicted trajectory and the three-dimensional coordinate points associated with the two-dimensional predicted trajectory.

10. A target tracking device, characterized in that, The device includes: The trajectory prediction module is used to predict the trajectory based on the three-dimensional point cloud of the previous moment and the motion trajectory of the target object at the previous moment, so as to obtain the two-dimensional predicted trajectory of the target object at the current moment. The first determining module is used to determine each three-dimensional coordinate point associated with the two-dimensional predicted trajectory from the three-dimensional point cloud at the current moment. The second determining module is used to determine the target coordinate point from the associated three-dimensional coordinate points; The third determining module is used to determine the tracking status of the target coordinate point corresponding to the current time based on the historical target coordinate points in the target coordinate point array. The historical target coordinate points are target coordinate points in the three-dimensional point cloud collected at historical times that are associated with the two-dimensional predicted trajectory of the target object at each historical time. The tracking status is used to characterize whether the target coordinate point is in a normal motion state or an abnormal motion state. The fourth determining module is used to obtain the motion trajectory of the target object based on the tracking status and the two-dimensional predicted trajectory; The third determining module is specifically used for: Based on the historical target coordinate points in the target coordinate point array, a trajectory is predicted, and based on the altitude trajectory obtained from the trajectory prediction, the tracking status of the target coordinate point at the current moment is determined to be either a normal motion state or an abnormal motion state.

Images