Object tracking method and apparatus, device, and storage medium

Abstract

An object tracking method and apparatus, a device, and a storage medium. The method comprises: when an object to be tracked is comprised, acquiring real-time sensing data of said object at a current moment (S110); by means of Kalman filtering and on the basis of the real-time sensing data, and historical state data and historical tracking point data which are determined at a previous moment, determining current state data subjected to filtering processing at the current moment and current tracking point data (S120); determining predicted center point data at the current moment on the basis of the current state data, the current tracking point data, and the historical tracking point data (S130); and determining current tracking point prediction data on the basis of the predicted center point data and the real-time sensing data, so as to perform object tracking on the basis of the current tracking point prediction data, and using the current tracking point prediction data as current tracking point data for determining tracking point prediction data at a next moment (S140).

Description

Object tracking method, device, equipment and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on December 18, 2023, with application number 202311746622.4, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of data processing technology, for example, to object tracking methods, devices, equipment and storage media. Background Art

[0003] With the rapid development of autonomous driving technology, object tracking, as one of the key technologies for autonomous driving, is of great significance for achieving safe and accurate autonomous driving. However, existing object tracking solutions, such as center-point-based object tracking methods, often ignore dynamic factors in the driving environment, such as unstable or blocked detection of vehicles, pedestrians, and obstacles. This can lead to inaccurate tracking points and even misjudgment.

[0004] Summary of the Invention

[0005] The present application provides an object tracking method, apparatus, device and storage medium to improve object tracking accuracy.

[0006] An embodiment of the present application provides an object tracking method, the method comprising:

[0007] Acquiring real-time sensing data of the object to be tracked at a current moment, under the condition that the object to be tracked is included;

[0008] Determine the filtered current state data and current tracking point data at the current moment based on the real-time perception data and the historical state data and historical tracking point data determined at the previous moment by means of Kalman filtering;

[0009] Determining predicted center point data at the current moment based on the current state data, the current tracking point data, and the historical tracking point data;

[0010] Based on the predicted center point data and the real-time perception data, current tracking point prediction data is determined to perform object tracking based on the current tracking point prediction data, and the current tracking point prediction data is used as the current tracking point data to determine tracking point prediction data at a next moment.

[0011] An embodiment of the present application provides an object tracking device, which includes:

[0012] a perception data acquisition module configured to acquire real-time perception data of the object to be tracked at a current moment under a condition that the object to be tracked is included;

[0013] a Kalman filter module configured to determine, by Kalman filtering, the current state data and the current tracking point data at the current moment after filtering based on the real-time perception data and the historical state data and the historical tracking point data determined at the previous moment;

[0014] a center point prediction module configured to determine predicted center point data at the current moment based on the current state data, the current tracking point data, and the historical tracking point data;

[0015] The object tracking module is used in the model detection module and is configured to determine current tracking point prediction data based on the predicted center point data and the real-time perception data, so as to perform object tracking based on the current tracking point prediction data, and to use the current tracking point prediction data as the current tracking point data to determine the tracking point prediction data at a next moment.

[0016] An embodiment of the present application provides an electronic device, comprising:

[0017] at least one processor; and

[0018] a memory communicatively connected to at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by the at least one processor so that the at least one processor can execute the object tracking method of any embodiment of the present application.

[0020] An embodiment of the present application provides a computer-readable storage medium, which stores computer instructions. The computer instructions are used to enable a processor to implement the object tracking method of any embodiment of the present application when executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] FIG1 is a flowchart of an object tracking method provided according to an embodiment of the present application;

[0022] FIG2 is a schematic diagram of a tracking point to be selected according to an embodiment of the present application;

[0023] FIG3 is a flowchart of another object tracking method provided according to an embodiment of the present application;

[0024] FIG4 is a flowchart of an object tracking method provided according to an embodiment of the present application;

[0025] FIG5 is a structural block diagram of an object tracking device according to an embodiment of the present application;

[0026] FIG6 is a structural block diagram of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will describe the technical solutions in the embodiments of this application in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0028] It should be noted that the terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not listed or are inherent to these processes, methods, products or apparatuses.

[0029] Figure 1 is a flowchart of an object tracking method provided according to an embodiment of the present application. This embodiment is applicable to scenarios where object tracking is performed based on Kalman filtering, and can be executed by an object tracking device. The object tracking device can be implemented in the form of hardware and / or software and configured in a processor of an electronic device.

[0030] As shown in Figure 1, the object tracking method includes the following steps:

[0031] S110 , under the condition that the object to be tracked is included, obtaining real-time perception data of the object to be tracked at the current moment.

[0032] The object to be tracked is the object being tracked. The object to be tracked can be a person, animal, or other object. It is understood that the object to be tracked depends on the application scenario. For example, in an autonomous driving scenario, the object to be tracked can be obstacles around the vehicle; in a security detection scenario, the object to be tracked can be a person exhibiting unusual behavior or a suspicious object.

[0033] It is understandable that for intelligent driving or autonomous driving, the evaluation of the autonomous driving system depends largely on perception data, which can include images or point cloud data. By converting the environmental information of the vehicle's environment into digital signals, data analysis and data processing are carried out to provide decision-making and control for intelligent driving or autonomous driving.

[0034] In this embodiment, obtaining real-time perception data of the object to be tracked at the current moment includes: obtaining real-time perception data of the object to be tracked at the current moment based on a perception device. The perception device may include an image acquisition device and a radar device. For example, the image acquisition device may be a monocular camera or a binocular camera, and the corresponding perception data may include image data. The radar device may be at least one of a laser radar, a millimeter-wave radar, and an ultrasonic radar, and the corresponding real-time perception data may include point cloud data.

[0035] Optionally, the sensory data is preprocessed to update the sensory data. Point cloud data is filtered based on point cloud filtering methods to remove noise and outliers, thereby denoising the point cloud data. Point cloud filtering methods include Gaussian filtering, median filtering, and statistical filtering. This improves the quality and accuracy of the sensory data, providing a more reliable data foundation for subsequent processing.

[0036] In this embodiment, after obtaining the real-time perception data of the object to be tracked at the current moment under the condition that the object to be tracked is included, it also includes: obtaining the environmental perception data at the current moment under the condition that the object to be tracked is not included; performing target detection on the environmental perception data at the current moment based on a pre-trained target detection model to obtain current state data corresponding to the target object.

[0037] The target object is the object to be tracked, which may be a person, an animal, and / or an object. For different scenarios, the target object may be an object of interest corresponding to the application scenario.

[0038] The state data is determined based on the perception data of the object to be tracked. For example, the state data may include tracking point coordinates, orientation angle, and object size.

[0039] It is understandable that at the start moment, when the object to be tracked is outside the sensing range of the sensing device, there is no object to be tracked corresponding to the previous moment. Therefore, it is necessary to perform target detection on the environmental perception data to obtain state data corresponding to the target object and then track the target object at the next moment. The environmental perception data is filtered based on the state data corresponding to the target object to obtain the perception data corresponding to each target object.

[0040] The object detection model is configured to determine state information such as the type, number, and location of target objects in the environmental perception data. For example, the object detection model may be at least one of a spatial pyramid pooling network (SPP-Net), a region-convolutional neural network (R-CNN), a single shot multibox detector (SSD), a you only look once (YOLO) series model, or an OverFeat model.

[0041] It is understandable that before training the target detection model, it is necessary to first construct a training sample set corresponding to the target detection model so as to train the target detection model based on each training sample in the training sample set.

[0042] In this embodiment, constructing a training sample set includes: acquiring an environmental perception image based on a camera, determining, for each object in the environmental perception image, a regional image corresponding to the object, and adding a rectangular frame including all pixels of the current target object to multiple regions in the environmental perception image, using the center point coordinates of the rectangular frame, the object orientation angle, and the object size as a label of the current target object, and using the label and the environmental perception image as training samples to obtain a training sample set.

[0043] A target detection model is trained based on each training sample in a training sample set, including: for each training sample, inputting the environmental perception image in the current training sample into the target detection model to obtain the coordinates of the center point of the rectangular box corresponding to each object, the orientation angle of the object, and the size of the object; based on the label corresponding to the object in the training sample set, a loss value is determined to correct the model parameters in the target detection model based on the loss value; and the convergence of the loss function in the target detection model is used as a training goal to obtain the target detection model.

[0044] The environmental perception data is input into the pre-trained target detection model to obtain the coordinates of the center point, orientation angle and size of the rectangular box corresponding to each target object, and the above information is used as the status data corresponding to the target object; based on the status data, the environmental perception data is filtered to obtain real-time perception data corresponding to each target object.

[0045] Exemplarily, the current state data is m(k):{m_size(k), m_cent(k), m_yaw(k)}, where m_size(k) is the size data of the target object at the current moment, m_cent(k) is the center point data of the target object at the current moment, and m_yaw(k) is the orientation angle data of the target object at the current moment.

[0046] S120 , determining the current state data and the current tracking point data after filtering at the current moment based on the real-time perception data and the historical state data and the historical tracking point data determined at the previous moment by means of Kalman filtering.

[0047] Since the sensory data corresponding to the object to be tracked includes sensory data corresponding to all points of the object to be tracked, due to the large amount of data, one of the points can be selected as the tracking point, and the state data corresponding to the tracking point can be used as the tracking point data. It should be noted that the degree of movement of the object to be tracked can be determined based on the difference between the current state data corresponding to the real-time sensory data and the historical state data. Considering the object to be tracked as a whole, the current state data can be determined based on the degree of movement of the object to be tracked and the historical tracking point data; the current tracking point data is determined based on the current state data.

[0048] Tracking point data may include state data corresponding to the tracking point. For example, historical tracking point data may include the coordinates, orientation angle, and size of the tracking point at historical moments, while current state data may include the coordinates, orientation angle, and size of the tracking point at the current moment.

[0049] Historical state data can be state data corresponding to sensory data at a previous moment. State information can include at least one of the position, velocity, and acceleration of the tracked object. Predicted state information corresponds to sensory data at the current moment, while historical state information corresponds to sensory data at moments before the current moment.

[0050] Kalman filtering is a technique that uses linear system state equations to estimate the past, current, or future state of a tracked object. Kalman filtering assumes that the state data of tracking points is random and follows a Gaussian distribution. Each state data point has a corresponding mean and variance (representing uncertainty). Based on Kalman filtering, the optimal estimate of the current tracking point corresponding to the historical tracking point data is determined and used as the current tracking point data. The optimal estimate of the current state data corresponding to the historical state data is also determined. This optimal estimate is then corrected based on real-time perception data to obtain the current state data at the current moment after filtering.

[0051] In this embodiment, the current state data and current tracking point data after filtering at the current moment are determined based on the real-time perception data and the historical state data and historical tracking point data determined at the previous moment by means of Kalman filtering, including: determining the current state data based on the preset state transfer matrix of the Kalman filter, the real-time perception data and the historical state data; determining the current tracking point data based on the current state data and the historical tracking point data.

[0052] The preset state transfer matrix is ​​a correspondence between historical state data and current state data.

[0053] Optionally, the state transfer matrix is ​​obtained by pre-calibration. Calibration of the state transfer matrix may include: for each current state data, determining the current state data and the historical state data corresponding to the current data, further determining the corresponding relationship between the current state data and the historical state data, and based on the corresponding relationship, determining a state transfer matrix between the current state data and the historical state data.

[0054] Based on the preset state transfer matrix of Kalman filtering, real-time perception data and historical state data, determining the current state data includes: determining the optimal current state data corresponding to the historical state transfer data based on the preset state transfer matrix of Kalman filtering; and correcting the optimal current state data under the ideal state based on the real-time perception data to obtain the current state data after Kalman filtering processing.

[0055] It is understood that for all points of the object to be tracked, including both tracking points and points other than the tracking points, there is a correspondence between the real-time perception data corresponding to the tracking points and the perception data corresponding to the points other than the tracking points. Therefore, based on the state data of the object to be tracked and this correspondence, the tracking point perception data of the tracking point can be determined for the real-time perception data, thereby obtaining the current state data of the tracking point, i.e., the current tracking point data. This current tracking point data is then corrected based on the historical tracking point data to update the current tracking point data.

[0056] For each object to be tracked, based on the state transfer matrix in the Kalman filter algorithm and the historical state information corresponding to the object to be tracked at the previous moment, the state data at the current moment is predicted, and the predicted state data corresponding to each tracked object is determined and used as the current state data.

[0057] The state data corresponding to the current tracking point is determined based on the historical tracking point data, and the current tracking point data is obtained based on the current state data and the state data corresponding to the current tracking point.

[0058] Exemplarily, the current state data m(k) is subjected to denoising to obtain updated current state data x(k); the product of the state transition matrix (A) and the historical state data x(k-1) is determined based on formula (1), and the product is used as the predicted current state data x(k) at the current moment:

[0059] x(k)=Ax(k-1) (1);

[0060] Among them, x(k-1) represents the historical state data after filtering at the (k-1)th moment; x(k) includes the center point coordinates (x_cent), size (x_size), and heading angle (x_yaw) of the current tracking object; the center point coordinates (x_cent) of the current tracking object are used as the current tracking point data.

[0061] S130 : Determine the predicted center point data at the current moment based on the current state data, the current tracking point data, and the historical tracking point data.

[0062] The current state data also includes center point data at a historical moment. For example, the center point data at a historical moment may be the center point data of the object to be tracked at a previous moment.

[0063] It is understandable that since an object cannot remain absolutely still, for the same object to be tracked, the state information of the object may change from the current moment to the historical moment. Therefore, the degree of change of the object to be tracked can be determined based on the historical tracking point data and the current tracking point data.

[0064] In the case where the tracking point data includes center point data at historical moments, a degree of change of the object to be tracked can be determined based on the historical tracking point data and the current tracking point data to predict the degree of change of the center point data; based on the predicted degree of change of the center point data and the center point data at historical moments in the current state data, the predicted center point data at the current moment is determined.

[0065] In this embodiment, the predicted center point data at the current moment is determined based on the current state data, the current tracking point data and the historical tracking point data, including: based on a preset corresponding relationship, the predicted center point data at the current moment is determined according to the current state data, the current tracking point data and the historical tracking point data, wherein the preset corresponding relationship includes the corresponding relationship between the current tracking point data, the current state data, the historical tracking point data and the predicted center point data at the current moment.

[0066] It can be understood that when the time difference between the historical moment and the current moment remains unchanged, the degree of change of the historical tracking point data and the current tracking point data of the object to be tracked also remains unchanged; therefore, based on this degree of change, the corresponding relationship between the current tracking point data, the current state data, the historical tracking point data and the predicted center point data at the current moment is determined.

[0067] A first correspondence between historical tracking point data, current tracking point data and the degree of change, and a second correspondence between current state data, predicted center point data at the current moment and the degree of change are preset; based on the current tracking point data, historical tracking point data and the first correspondence, the degree of change of the object to be tracked is determined, and based on the degree of change, the current state data and the second correspondence, the predicted center point data is determined.

[0068] For example, based on formula (2), the current tracking point data (x_cent) at time k and the historical tracking point data (track_index) obtained at time (k-1) are used to determine the predicted center point data x_center(k) at the current time:

[0069] x_center(k)=x_cent(k)-g(track_index(k-1),x_size(k),x_yaw(k)) (2);

[0070] Among them, the current tracking point data (x_cent) is a 2-row and 1-column matrix consisting of the horizontal and vertical coordinates of the current tracking point center coordinates, g(track_index(k-1),x_size(k),x_yaw(k)) represents the intermediate process value from the current tracking point data to the predicted center point data, track_index(k-1) represents the tracking point data at the (k-1)th moment, that is, the historical tracking point data, x_size(k) is the size data in the current state data at the kth moment, and x_yaw(k) is the orientation angle in the current state data at the kth moment.

[0071] g(track_index(k-1),x_size(k),x_yaw(k)) is determined based on formula (3):

[0072] Among them, the parameters x_size_l and x_size_w respectively represent the length data and width data in the size data, and track_index_1 and track_index_2 are the horizontal and vertical coordinates of the tracking point track_index respectively.

[0073] S140: Determine current tracking point prediction data based on the predicted center point data and the real-time perception data, perform object tracking based on the current tracking point prediction data, and use the current tracking point prediction data as current tracking point data to determine tracking point prediction data at a next moment.

[0074] The current tracking point prediction data is obtained by correcting the predicted center point data based on the real-time perception data, and is the current center point data corresponding to the predicted center point data.

[0075] Considering that among the multiple points of the object to be tracked at the current moment, the point corresponding to the same position as the historical tracking point may be blocked, based on the predicted center point data and real-time perception data, other points of the object to be tracked are determined as the current tracking points, and based on the relative position relationship between the center point and other points, the current tracking point prediction data is obtained; the current tracking point prediction data is used as the current tracking point data, and then at the next moment, the current tracking point data is used as the historical tracking point data to continue tracking prediction.

[0076] In this embodiment, determining the current tracking point prediction data based on the predicted center point data and the real-time perception data includes: for at least one to-be-selected tracking point corresponding to the to-be-tracked object, determining the movement distance corresponding to the current to-be-selected tracking point based on the predicted center point data and the real-time perception data; and determining the current tracking point prediction data based on the movement distances and real-time perception data of all to-be-selected tracking points.

[0077] The tracking point to be selected may be any point corresponding to the object to be tracked, for example, a boundary point of the object to be tracked, or four corner points or a center point of a rectangular box corresponding to the object to be tracked.

[0078] The movement distance is the distance between the point to be selected and the point in the object to be tracked at the previous moment that is located at the same position as the point to be selected. For example, the movement distance can be the distance between the center point of the rectangular box corresponding to the object to be tracked at the current moment and the center point of the rectangular box corresponding to the object to be tracked at the previous moment.

[0079] For at least one to-be-selected tracking point corresponding to the to-be-tracked object, determining a movement distance corresponding to the current to-be-selected tracking point based on the predicted center point data and the real-time perception data, including: for each to-be-selected tracking point, determining predicted data for the to-be-selected tracking point based on the predicted center point data and a relative positional relationship between the center point and the to-be-selected tracking point; determining current tracking point data corresponding to the real-time perception data, and determining a distance between the current tracking data and the predicted data for the to-be-selected tracking point, using the distance as the movement distance corresponding to the to-be-selected tracking point.

[0080] For example, refer to Figure 2, where the solid-line box 201 represents the four corner points and the center point of the rectangular box corresponding to the real-time perception data at the current moment, and each point is taken as a tracking point to be selected; the dotted-line box 202 represents the tracking point corresponding to the historical moment, and the historical tracking point coordinates corresponding to the center point are set to (0, 0), the historical tracking point coordinates corresponding to the upper left corner point are set to (-1, 1), the historical tracking point coordinates corresponding to the upper right corner point are set to (1, 1), the historical tracking point coordinates corresponding to the lower left corner point are set to (-1, -1), and the historical tracking point coordinates corresponding to the lower right corner point are set to (1, -1); based on the predicted center point data x_cent at the current moment and the current state data m corresponding to the real-time perception data, the moving distances corresponding to the center point and the four corner points are determined respectively.

[0081] The corresponding calculation formula is as follows:

[0082] The naming convention for the above parameters includes: tr represents the relevant parameters of the upper right corner point, tl represents the relevant parameters of the upper left corner point, bl represents the relevant parameters of the lower right corner point, br represents the relevant parameters of the upper right corner point, and parameters other than tr, tl, bl, and br represent the relevant parameters corresponding to the center point. dis_tmp is the movement distance of the center point, dis_tmp_br is the movement distance of the lower right corner, dis_tmp_tr is the movement distance of the upper right corner, dis_tmp_bl is the movement distance of the lower left corner, and dis_tmp_tl is the movement distance of the upper left corner. x_tmp is the predicted center point data, x_tmp_tr is the predicted top right corner point data, and so on for other corner point data; m_tmp is the center point movement distance corresponding to the real-time perception data at time k.

[0083] Exemplarily, track_index_tr is set to the historical coordinates (1, 1) of the upper right corner, m_cent(k) represents the center point coordinates in the state data corresponding to the real-time perception data at moment k, m_size represents the size data in the state data corresponding to the real-time perception data at moment k, m_yaw represents the orientation angle data in the state data corresponding to the real-time perception data at moment k, dis_tmp_tr represents the moving distance between the predicted information x_tmp_tr and the measured information m_tmp_tr; and so on, the moving distances corresponding to other corner points can be determined.

[0084] Optionally, based on the movement distance and real-time perception data, current tracking point prediction data is determined. This includes: obtaining a target tracking point to be selected based on the movement distance statistical indicator; determining center point data corresponding to the real-time perception data based on the real-time perception data and the target tracking point to be selected; and using the center point data as the current tracking point prediction data.

[0085] Exemplarily, the average value of all moving distances is determined, and the difference of each moving distance relative to the average value is determined to obtain the minimum difference; based on the real-time perception data and the tracking point to be selected corresponding to the minimum difference, the center point data corresponding to the real-time perception data is determined; and the center point data is used as the current tracking point prediction data.

[0086] The technical solution of this embodiment performs noise reduction processing on the perception data, and determines the tracking point prediction data based on the movement distance of at least one tracking point to be selected, and then updates the state data corresponding to the tracking point prediction data and the real-time perception data to achieve target tracking. The accuracy of the state data of the object to be tracked can be used to improve the accuracy of object tracking.

[0087] The technical solution of the embodiment of the present application obtains real-time perception data of the object to be tracked at the current moment, under the condition that the object to be tracked is included; determines the current state data and current tracking point data after filtering at the current moment based on the real-time perception data and historical state data and historical tracking point data determined at the previous moment through Kalman filtering; determines the predicted center point data at the current moment based on the current state data, current tracking point data, and historical tracking point data; determines the current tracking point prediction data based on the predicted center point data and real-time perception data, performs object tracking based on the current tracking point prediction data, and uses the current tracking point prediction data as the current tracking point data to determine the tracking point prediction data at the next moment. This solves the problem of low object tracking accuracy and improves object tracking accuracy.

[0088] Figure 3 is a flowchart of another object tracking method provided in accordance with an embodiment of the present application. This embodiment is applicable to scenarios where object tracking is performed using a Kalman filter. This embodiment shares the same inventive concept as the object tracking method in the aforementioned embodiment. Building on the aforementioned embodiment, this embodiment describes the process of determining predicted data for the current tracking point based on predicted center point data and real-time perception data.

[0089] As shown in FIG3 , the object tracking method includes the following steps.

[0090] S210 : Acquire real-time perception data of the object to be tracked at the current moment, under the condition that the object to be tracked is included.

[0091] S220 , determining the current state data and the current tracking point data after filtering at the current moment based on the real-time perception data and the historical state data and the historical tracking point data determined at the previous moment by means of Kalman filtering.

[0092] S230: For at least one to-be-selected tracking point corresponding to the to-be-tracked object, determine a movement distance corresponding to the current to-be-selected tracking point based on the predicted center point data and the real-time perception data, so as to obtain a target tracking point corresponding to the minimum movement distance.

[0093] The target tracking point is the tracking point to be selected corresponding to the minimum moving distance.

[0094] For at least one tracking point to be selected corresponding to the object to be tracked, based on the predicted center point data and the real-time perception data, the movement distance corresponding to the current tracking point to be selected is determined, the minimum value of all movement distances is obtained, and the minimum movement distance is used as the minimum movement distance. The tracking point to be selected corresponding to the minimum movement distance is obtained and used as the target tracking point.

[0095] S240: Determine current tracking point prediction data based on the perception data of the target tracking point corresponding to the minimum moving distance in the real-time perception data.

[0096] Based on the relative position relationship between the target tracking point and the center point, and the historical tracking point data corresponding to the target tracking point, the center point data corresponding to the real-time perception data is determined to obtain the current tracking point prediction data.

[0097] For example, based on formula (5), the current tracking point prediction data x_update(k) is updated according to the historical tracking point data (track_index), the predicted center point data (x_center), and the current state data x(k) corresponding to the target tracking point:

[0098] x_update(k)=x_center(k)+g(track_index(k),X_size(k),x_yaw(k)) (5).

[0099] S250: Determine the current state prediction data at the current moment based on the current tracking point prediction data and the real-time perception data.

[0100] , based on the preset corresponding relationship, the current state at the current moment is predicted according to the current tracking point prediction data and real-time perception data after Kalman filtering to obtain the current state prediction data at the current moment.

[0101] Exemplarily, based on formula (6), the current state prediction data m_update(k) is determined according to the current measurement tracking point data:

[0102] m_update(k)=m_cent(k)+g(track_index(k),m_size(k),m_yaw(k)) (6);

[0103] Among them, g(track_index(k-1),x_size(k),x_yaw(k)) represents the intermediate process value from the current measurement tracking point data to the predicted measurement data.

[0104] S260: Based on the current state prediction data and the current tracking point prediction data at the current moment, the current tracking point prediction data is updated to perform object tracking based on the current tracking point prediction data, and the current tracking point prediction data is used as the current tracking point data to determine the tracking point prediction data at the next moment.

[0105] Based on the current state prediction data and the current tracking point prediction data at the current moment and the Kalman gain, the optimal tracking point at the current moment is determined to obtain current tracking point prediction data. The object is tracked based on the current tracking point prediction data, and the current tracking point prediction data is used as the current tracking point data to determine the tracking point prediction data at the next moment.

[0106] Exemplarily, based on formula (7), the current tracking point prediction data is updated according to the current state prediction data and the current tracking point prediction data to obtain the filtered tracking point data x_output(k) at the current moment:

[0107] Among them, K k is the Kalman gain, and H is the observation matrix.

[0108] Based on formula (8), the center point data x_final(k) corresponding to the filtered tracking point data x_output(k) at the current moment is determined:

[0109] Where x_center_output(k) is the center coordinate data of the tracking point corresponding to x_output(k). The prediction data of the tracking point at the next moment is updated based on the current x_output and track_index.

[0110] The technical solution of the embodiment of the present application is based on a Kalman filter method, which selects a tracking point to be selected corresponding to the minimum moving distance and uses it as the target tracking point. Then, based on the target tracking point, the current tracking point prediction data and the current state data are filtered and updated, thereby improving the accuracy of the current tracking point prediction data and the current state data, thereby improving the accuracy of object tracking.

[0111] FIG4 is a flow chart of an object tracking method provided according to an embodiment of the present application. As shown in FIG4 , the object tracking method includes the following steps.

[0112] S310 , under the condition that the object to be tracked is included, for each object to be tracked, based on a pre-trained target detection model, determining observation data corresponding to the real-time perception data at the current moment.

[0113] The object to be tracked is the obstacle of the target vehicle, and the observation data m includes obstacle size data, direction angle data and obstacle center point coordinate data.

[0114] During vehicle driving, obstacles are perceived based on the perception device to obtain real-time perception data corresponding to the current obstacles; this perception data is input into the target detection model to obtain observation data corresponding to the current obstacles.

[0115] S320: Determine the tracking point data at the current moment based on the historical tracking point data.

[0116] The historical tracking point data is the coordinate data of the center point of the object to be tracked at the previous moment, that is, the tracking index track_index. If the current moment is the first moment, track_index is (0,0).

[0117] S330: Determine the optimal estimated value corresponding to the current moment by means of Kalman filtering, and determine the predicted tracking point data at the current moment.

[0118] Using Kalman filtering, we determine the optimal estimate x(k) corresponding to the final state data x(k-1) at the previous moment. x(k) includes information such as the predicted tracking point center coordinate x_cent, size x_size, and heading angle x_yaw. The predicted tracking point center coordinate x_cent is used as the predicted tracking point data at the current moment.

[0119] S340 : Determine the predicted center point data at the current moment based on the preset corresponding relationship, the predicted tracking point data at the current moment, and the tracking point data at the historical moments.

[0120] Based on the correspondence relationship shown in formula (2), the predicted center point data x_center(k) at the current moment corresponding to the predicted tracking point data at the current moment and the tracking point data at the historical moment is determined.

[0121] S350: Determine and calculate historical tracking point data based on the predicted center point data at the current moment and the observed data at the current moment.

[0122] Based on formula (4), the moving distances of all the tracking points to be selected are determined to obtain the minimum moving distance. The tracking point to be selected corresponding to the minimum moving distance is used as the target tracking point. The center point coordinates are determined based on the target tracking point and used as the historical tracking point data.

[0123] S360: Determine the predicted tracking point data at the current moment based on the predicted center point data at the current moment and the historical tracking point data, and determine the observed tracking point data at the current moment based on the observed data at the current moment and the historical tracking point data.

[0124] S370: Update the current tracking point data based on the Kalman gain, the predicted tracking point data at the current moment, and the observed tracking point data at the current moment.

[0125] S380 : Perform object tracking based on the current tracking point prediction data, and use the current tracking point prediction data as current tracking point data to determine tracking point prediction data at a next moment.

[0126] The technical solution of this embodiment is based on the perception of obstacles during vehicle driving by sensing devices, and then tracking the obstacles by means of Kalman filtering, which improves the accuracy of object tracking and can provide an information basis for subsequent automatic obstacle avoidance and path planning of smart cars.

[0127] Figure 5 is a structural block diagram of an object tracking device provided according to an embodiment of the present application. This embodiment is applicable to scenarios where object tracking is performed based on Kalman filtering. The device can be implemented in the form of hardware and / or software and integrated into a processor of an electronic device with application development capabilities.

[0128] As shown in FIG5 , the object tracking device includes: a perception data acquisition module 501 configured to acquire real-time perception data of the object to be tracked at the current moment, provided that the object to be tracked is present; a Kalman filter module 502 configured to determine filtered current state data and current tracking point data at the current moment based on the real-time perception data and historical state data and historical tracking point data determined at the previous moment through Kalman filtering; a center point prediction module 503 configured to determine predicted center point data at the current moment based on the current state data, current tracking point data, and historical tracking point data; and an object tracking module 504, which is a model detection module configured to determine predicted current tracking point data based on the predicted center point data and the real-time perception data, perform object tracking based on the current tracking point predicted data, and use the current tracking point predicted data as the current tracking point data to determine predicted tracking point data at the next moment. This solves the problem of low object tracking accuracy and improves object tracking accuracy.

[0129] Optionally, the perception data acquisition module 501 includes a target detection unit, which is configured to:

[0130] Obtaining the current environmental perception data without including the object to be tracked;

[0131] Based on a pre-trained target detection model, target detection is performed on the environmental perception data at the current moment to obtain current state data corresponding to the target object.

[0132] Optionally, the Kalman filter module 502 is configured to:

[0133] Determining current state data based on a preset state transfer matrix of the Kalman filter, the real-time perception data, and historical state data;

[0134] The current tracking point data is determined based on the current state data and the historical tracking point data.

[0135] Optionally, the center point prediction module 503 is configured as follows:

[0136] Based on a preset correspondence, the predicted center point data at the current moment is determined according to the current state data, the current tracking point data and the historical tracking point data, wherein the preset correspondence includes the correspondence between the current tracking point data, the current state data, the historical tracking point data and the predicted center point data at the current moment.

[0137] Optionally, the object tracking module 504 is configured to:

[0138] For at least one to-be-selected tracking point corresponding to the to-be-tracked object, determining a movement distance corresponding to the current to-be-selected tracking point based on the predicted center point data and the real-time perception data;

[0139] The current tracking point prediction data is determined based on the movement distances and real-time perception data of all tracking points to be selected.

[0140] Optionally, the object tracking module 504 is configured to:

[0141] For at least one to-be-selected tracking point corresponding to the to-be-tracked object, determining a movement distance corresponding to the current to-be-selected tracking point based on the predicted center point data and the real-time perception data, so as to obtain a target tracking point corresponding to a minimum movement distance;

[0142] The current tracking point prediction data is determined based on the perception data of the to-be-selected tracking point corresponding to the minimum moving distance in the real-time perception data.

[0143] Optionally, the object tracking module 504 further includes a data updating unit, which is configured to:

[0144] Determining current state prediction data at a current moment based on the current tracking point prediction data and the real-time perception data;

[0145] The current tracking point prediction data is updated based on the current state prediction data at the current moment and the current tracking point prediction data.

[0146] The object tracking device provided in the embodiments of the present application can execute the object tracking method provided in any embodiment of the present application, and has a functional module corresponding to the execution method.

[0147] Figure 6 is a block diagram of an electronic device provided according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0148] As shown in FIG6 , the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 and a random access memory (RAM) 13, that is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the ROM 12 or the computer program loaded from the storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0149] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0150] The processor 11 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the object tracking method.

[0151] In some embodiments, the object tracking method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the object tracking method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the object tracking method in any other suitable manner (e.g., via firmware).

[0152] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0153] Computer programs for implementing the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable object tracking device, such that when executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer programs can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0154] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM) or a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0155] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0156] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0157] A computing system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. The client-server relationship arises through computer programs running on the respective computers and establishing a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within a cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and virtual private server (VPS) services.

[0158] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the multiple steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of this application can be achieved. This is not limited herein.

Claims

1. An object tracking method, comprising: Under the condition that the object to be tracked is included, obtaining real-time perception data of the object to be tracked at the current moment; Determine the current state data and the current tracking point data after filtering at the current moment according to the real-time perception data and the historical state data and the historical tracking point data determined at the previous moment by means of Kalman filtering; Determining the predicted center point data at the current moment based on the current state data, the current tracking point data and the historical tracking point data; Based on the predicted center point data and the real-time perception data, current tracking point prediction data is determined to perform object tracking based on the current tracking point prediction data, and the current tracking point prediction data is used as the current tracking point data to determine the tracking point prediction data at the next moment.

2. The method according to claim 1, after acquiring the real-time perception data of the object to be tracked at the current moment under the condition that the object to be tracked is included, further comprises: Acquire the current environment perception data without including the object to be tracked; Based on a pre-trained target detection model, target detection is performed on the environmental perception data at the current moment to obtain current state data corresponding to the target object.

3. The method according to claim 1, wherein: The method of determining the current state data and the current tracking point data after filtering at the current moment according to the real-time perception data and the historical state data and the historical tracking point data determined at the previous moment by means of Kalman filtering includes: Determine current state data based on a preset state transfer matrix of the Kalman filter, the real-time perception data, and the historical state data; The current tracking point data is determined based on the current state data and the historical tracking point data.

4. The method according to claim 1, wherein: The determining the predicted center point data at the current moment based on the current state data, the current tracking point data and the historical tracking point data includes: Based on a preset corresponding relationship, the predicted center point data at the current moment is determined according to the current state data, the current tracking point data and the historical tracking point data, wherein the preset corresponding relationship includes the corresponding relationship between the current tracking point data, the current state data, the historical tracking point data and the predicted center point data at the current moment.

5. The method according to claim 1, wherein: The determining the current tracking point prediction data based on the prediction center point data and the real-time perception data includes: For at least one to-be-selected tracking point corresponding to the object to be tracked, based on the predicted The center point data and the real-time sensing data are used to determine a moving distance corresponding to a current tracking point to be selected among the tracking points to be selected; The current tracking point prediction data is determined based on the moving distances of all tracking points to be selected and the real-time perception data.

6. The method according to claim 1 or 5, wherein: The determining the current tracking point prediction data based on the prediction center point data and the real-time perception data includes: For at least one to-be-selected tracking point corresponding to the to-be-tracked object, determining a moving distance corresponding to a current to-be-selected tracking point among the to-be-selected tracking points based on the predicted center point data and the real-time perception data, so as to obtain a target tracking point corresponding to a minimum moving distance; The current tracking point prediction data is determined based on the perception data corresponding to the target tracking point in the real-time perception data.

7. The method according to claim 1, 5 or 6, wherein: The determining the current tracking point prediction data based on the prediction center point data and the real-time perception data further includes: Determining current state prediction data at a current moment based on the current tracking point prediction data and the real-time perception data; Based on the current state prediction data at the current moment and the current tracking point prediction data, the current tracking point prediction data is updated.

8. An object tracking device, comprising: A perception data acquisition module, configured to acquire real-time perception data of the object to be tracked at a current moment under the condition that the object to be tracked is included; A Kalman filter module, configured to determine the current state data and the current tracking point data after filtering at the current moment according to the real-time perception data and the historical state data and the historical tracking point data determined at the previous moment by means of Kalman filtering; a center point prediction module, configured to determine the predicted center point data at the current moment based on the current state data, the current tracking point data and the historical tracking point data; The object tracking module is used for the model detection module, and is configured to determine the current tracking point prediction data based on the predicted center point data and the real-time perception data, so as to perform object tracking based on the current tracking point prediction data, and to use the current tracking point prediction data as the current tracking point data to determine the tracking point prediction data at the next moment.

9. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the object tracking method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a processor to implement the object tracking method according to any one of claims 1 to 7 when executed.

Images