A target tracking method, medium, and electronic device

Abstract

This application provides a target tracking method, medium, and electronic device. The method, applied to an electronic device, specifically includes: determining first category information and first category confidence of a first detection box in the detection data of frame T, where the first category confidence represents the probability that the target object in the first detection box belongs to the category represented by the first category information; if the first category confidence of the first detection box satisfies a first condition, a first matching strategy is adopted to match the first detection box with multiple trajectory boxes in the detection data of frame T; if the first category confidence of the first detection box does not satisfy the first condition, a second matching strategy is adopted to match the first detection box with multiple trajectory boxes in the detection data of frame T. Furthermore, this method can improve the matching rate between detection boxes and trajectory boxes while reducing the total computational load and improving matching efficiency.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and more particularly to a target tracking method, medium, and electronic device. Background Technology

[0002] Object tracking can be applied to fields such as intelligent video surveillance, autonomous driving, and unmanned supermarkets. For example, when object tracking technology is applied to autonomous driving, vehicles can predict the trajectories of pedestrians or other vehicles in the surrounding area, and then take measures such as braking in advance to ensure driving safety.

[0003] Specifically, the vehicle can first acquire detection data of the current driving environment over a period of time, such as point cloud data obtained through LiDAR. By performing target detection on each frame of point cloud data, the vehicle obtains the bounding boxes corresponding to each target object in each frame. Then, it determines which target object in the previous frame corresponds to each target object in the previous frame, thus determining the position of the same target object in different frames of point cloud data and obtaining the target object's trajectory over that period. Based on the target object's trajectory, the vehicle can predict the target object's possible next action. Then, based on the target object's possible next action, the vehicle can take measures such as braking to ensure driving safety.

[0004] The process involves first predicting the position of each detection box in the current frame based on the point cloud data from the previous frame, thus obtaining individual trajectory boxes. Then, the predicted trajectory boxes are matched with the detection boxes in the current frame to determine the detection box corresponding to each trajectory box. This allows the system to determine which target object in the current frame corresponds to in the previous frame.

[0005] However, when the target object is occluded or its size is blurred due to motion, the matching rate between the detection box and the trajectory box decreases. If the target object's detection box cannot match the corresponding trajectory box, the vehicle will directly mark the target object corresponding to the trajectory box as lost, making it impossible to predict the target object's trajectory further. This affects the accuracy of subsequent target tracking processing. For example, the vehicle may not be able to predict the trajectories of surrounding pedestrians or other vehicles, and therefore cannot take corresponding measures in advance for these targets, potentially leading to traffic accidents. Summary of the Invention

[0006] This application provides a target tracking method, medium, and electronic device. Different matching strategies are used for detection boxes with different confidence levels, which can improve the matching rate between detection boxes and trajectory boxes, reduce the total computation, and improve matching efficiency.

[0007] In a first aspect, a target tracking method is provided, comprising: determining first category information and first category confidence of a first detection box in the detection data of the Tth frame, wherein the first category confidence represents the probability that the target object in the first detection box belongs to the category represented by the first category information; if the first category confidence of the first detection box satisfies a first condition, a first matching strategy is adopted to match the first detection box with multiple trajectory boxes in the detection data of the Tth frame; if the first category confidence of the first detection box does not satisfy the first condition, a second matching strategy is adopted to match the first detection box with multiple trajectory boxes in the detection data of the Tth frame.

[0008] In the above scheme, different matching strategies are adopted for detection boxes with different confidence levels. Specifically, for targets that are occluded or whose size is blurred due to motion, the confidence level in the target detection stage is often low, corresponding to low-score detection boxes, and the second matching strategy can be used to match the trajectory box. On the other hand, for clearer and more complete targets, the position and size information of the detection box obtained in the target detection stage is more accurate, and the confidence level is also higher, corresponding to high-score detection boxes, and the first matching strategy is used to match the trajectory box.

[0009] For example, the first matching strategy can be more accurate, and thus a high-precision matching strategy can be used for low-scoring detection boxes to improve the matching rate, while a low-precision matching strategy can be used for low-scoring detection boxes to reduce the amount of computation.

[0010] In conjunction with the first aspect, in some implementations, the first condition includes the first category confidence level being greater than the confidence threshold.

[0011] In the above scheme, detection boxes that meet the first condition are considered high-scoring detection boxes and are matched using the second matching strategy. Detection boxes that do not meet the first condition are considered low-scoring detection boxes and are matched using the first matching strategy. This approach, using different matching strategies for detection boxes with different confidence levels, can improve the matching rate between detection boxes and trajectory boxes while reducing the total computational load and improving matching efficiency.

[0012] In conjunction with the first aspect, in some implementations, the detection data for frame T includes either the image data of frame T or the point cloud data of frame T.

[0013] In the above scheme, the electronic device can acquire image data from a camera or point cloud data from a lidar device.

[0014] In conjunction with the first aspect, in some implementations, the first matching strategy includes calculating the matching degree based on the size of the overlapping area between the detection box and the trajectory box, and the second matching strategy includes: calculating the matching degree based on the distance between the center points of the detection box and the trajectory box and the diagonal length of the minimum bounding body, or calculating the matching degree based on the distance between the center points of the detection box and the trajectory box, the size of the overlapping area, and the aspect ratio; the first matching strategy includes calculating the matching degree based on the distance between the center points of the detection box and the trajectory box and the diagonal length of the minimum bounding body, and the second matching strategy includes calculating the matching degree based on the distance between the center points of the detection box and the trajectory box, the size of the overlapping area, and the aspect ratio.

[0015] In the above scheme, for low-scoring detection boxes, a more precise matching strategy can be used to match them with the trajectory boxes, which can improve the matching rate between the detection boxes and the trajectory boxes. For high-scoring detection boxes, a lower-precision matching strategy with less computation can be used to match them with the trajectory boxes, achieving a higher matching rate while reducing computational load. Therefore, the target tracking method provided in this application can improve the matching rate between detection boxes and trajectory boxes while reducing the total computational load, thus improving matching efficiency.

[0016] In conjunction with the first aspect, in some implementations, the detection data corresponding to the Tth frame is the point cloud data of the Tth frame, and the first matching strategy includes calculating the matching degree based on the intersection-union ratio of the three-dimensional general overlapping region, and the second matching strategy includes calculating the matching degree based on the intersection-union ratio of the three-dimensional distance overlapping region, or the second matching strategy is to calculate the matching degree based on the intersection-union ratio of the three-dimensional completely overlapping region; the first matching strategy includes calculating the matching degree based on the intersection-union ratio of the three-dimensional distance overlapping region, and the second matching strategy includes calculating the matching degree based on the intersection-union ratio of the three-dimensional completely overlapping region.

[0017] In the above scheme, the acquired data is point cloud data. In the matching of detection boxes and trajectory boxes, 3D information is preserved. The 3D matching algorithm does not lose the spatial and angular information of the target object, making the target tracking algorithm more accurate.

[0018] In conjunction with the first aspect, in some implementations, if the first matching degree between the first trajectory box and the first detection box in the detection data of the Tth frame is greater than the matching degree threshold, it is determined that the first trajectory box and the first detection box are matched; or, the first detection box is matched with multiple trajectory boxes in the detection data of the Tth frame, and it is determined that the first detection box has the largest matching degree with the first detection box in the multiple trajectory boxes, then it is determined that the first trajectory box and the first detection box are matched.

[0019] In the above scheme, the detection box is sequentially matched with each trajectory box to calculate the matching degree. A trajectory box with a matching degree reaching a threshold is matched with the detection box, or a trajectory box with a matching degree reaching the threshold and having the highest matching degree is matched with the detection box. By setting a threshold, the trajectory box matching the detection box can be determined among multiple trajectory boxes. This allows us to determine which target object in the current frame corresponds to which target object in the previous frame, further determining the trajectory of the same target object in the temporal domain, and predicting the target object's subsequent actions.

[0020] In conjunction with the first aspect, in some implementations, based on the detection boxes from the first frame to the (T-1)th frame detection data, multiple trajectory boxes of the Tth frame detection data are predicted, and the multiple trajectory boxes include the first trajectory box.

[0021] In the above scheme, the trajectory bounding box of frame T is predicted based on the trajectory corresponding to the detection bounding box of the previous T-1 frames. Therefore, based on the trajectory of the target object during this period, the next action of the target object can be predicted. For example, when applied to the field of autonomous driving, if it is predicted that a pedestrian on the side may walk directly in front of the vehicle, the vehicle will take braking measures in advance.

[0022] In conjunction with the first aspect, in some implementations, the error between each detection box and the corresponding matched trajectory box in the detection data of frame T is determined to obtain the error of the detection data of frame T; based on the error of the detection data of frame T and the error of the detection data of the previous T-1 frames, an adaptive factor is determined, which is used to adjust the covariance matrix of the trajectory box of frame T+1.

[0023] In the above scheme, after each matching is completed, the error between the matched detection box and the trajectory box is calculated. The motion prediction model is adjusted by the error so that the motion prediction model can be more accurate when predicting the trajectory box in the next time.

[0024] In conjunction with the first aspect, in some implementations, the adaptive threshold is used to adjust the covariance matrix of the trajectory frame in frame T+1 based on the adaptive factor being greater than the adaptive threshold; and a preset factor is used to adjust the covariance matrix of the trajectory frame in frame T+1 based on the adaptive factor being less than or equal to the adaptive threshold.

[0025] Secondly, this application provides a target tracking device, including a determining unit and an employing unit. The determining unit is used to determine a first category information and a first category confidence level of a first detection box in the detection data of the Tth frame. The first category confidence level represents the probability that the target object in the first detection box belongs to the category represented by the first category information. The employing unit is further used to, if the first category confidence level corresponding to the first detection box satisfies a first condition, employ a first matching strategy to match the first detection box with multiple trajectory boxes of the detection data of the Tth frame. The employing unit is further used to, if the first category confidence level corresponding to the first detection box does not satisfy the first condition, employ a second matching strategy to match the first detection box with multiple trajectory boxes of the detection data of the Tth frame.

[0026] Thirdly, this application provides an electronic device, characterized in that it includes a processor and a memory, the memory being used to store instructions, and the processor being used to execute the instructions, wherein when the processor executes the instructions, it performs the method described in the first aspect.

[0027] Fourthly, this application provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores instructions that, when executed on an electronic device, perform the method described in the first aspect.

[0028] Fifthly, this application provides a computer program product, characterized in that the computer program product includes computer instructions, which, when executed by an electronic device, cause the electronic device to perform the method described in the first aspect.

[0029] In summary, the target tracking method, medium, and electronic device provided in this application employ different matching strategies for detection boxes with varying confidence levels. This not only improves the matching rate between detection boxes and trajectory boxes but also reduces the overall computational load, thereby increasing matching efficiency. Furthermore, the matching process preserves 3D information and utilizes a 3D matching algorithm, ensuring no loss of spatial and angular information about the target object, thus enhancing the accuracy of the target tracking algorithm. Moreover, after each matching step, the error between the matched detection box and trajectory box is calculated, and the motion prediction model is adjusted based on this error, resulting in greater accuracy in predicting trajectory boxes in subsequent iterations. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0031] Figure 1 This is a schematic diagram of the structure of a target detection and tracking system provided in an embodiment of this application;

[0032] Figure 2This is a schematic diagram of the structure of a conventional target tracking system provided in an embodiment of this application;

[0033] Figure 3 This is a schematic diagram illustrating the matching of a detection box and a trajectory box according to an embodiment of this application;

[0034] Figure 4 This is a schematic diagram of the structure of a target tracking system provided in an embodiment of this application;

[0035] Figure 5 This is a flowchart illustrating a target tracking method provided in an embodiment of this application;

[0036] Figure 6 This is a flowchart illustrating another target tracking method provided in an embodiment of this application;

[0037] Figure 7 This is a schematic diagram of the structure of a target tracking device provided in an embodiment of this application;

[0038] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0039] The illustrative embodiments of this application include, but are not limited to, target tracking methods, media, and electronic devices.

[0040] The following is combined Figure 1 This paper provides a detailed explanation of the application scenarios of target tracking methods in the field of autonomous driving.

[0041] First, the vehicle can collect data through data acquisition devices, such as onboard LiDAR or cameras, to obtain detection data of surrounding objects. Specifically, LiDAR provides point cloud data, while cameras provide image data.

[0042] For example, vehicle-mounted lidar emits lasers to scan and perceive the road driving environment at a certain frequency, such as 10 Hz or 20 Hz, and collects driving scene information to obtain point cloud data of surrounding objects during a period of time.

[0043] Then, the vehicle uses point cloud data to detect dynamic targets (such as pedestrians and vehicles) and static targets (such as traffic signs and traffic cones), and provides the detection box and corresponding category information for each target (including dynamic and static targets).

[0044] Furthermore, based on the detection results (detection boxes and category information), the vehicle associates and matches targets between multiple point cloud frames over a period of time, and marks the same target in multiple point cloud frames with the same tracking identity document (ID), thereby obtaining the trajectory of the target with the same tracking ID over a period of time.

[0045] Finally, automatic post-labeling processing will be performed, such as trajectory smoothing and uniform detection box size, to obtain the final labeled results, including the processed detection boxes, category information, and tracking ID. Then, the vehicle can determine the trajectory of a target with the same tracking ID over a certain period of time based on the tracking ID, and thus predict the target's next actions. For example, if it is predicted that a pedestrian on the side may walk directly in front of the vehicle, the vehicle will take braking measures in advance.

[0046] It is understandable that the above target tracking method can also be applied to other fields such as road monitoring and unmanned supermarkets.

[0047] The target tracking system that implements the above-mentioned target tracking method will be described in detail below. For example... Figure 2 As shown, in some embodiments, the target tracking system may include a detection result preprocessing module 210, a motion prediction module 220, a data matching module 230, and a trajectory management module 240.

[0048] The detection result preprocessing module 210 is used to acquire the detection data obtained by the data acquisition device, wherein, Figure 2 Taking a lidar as an example of a data acquisition device, the detection data of the Tth frame is identified according to the target detection model, resulting in multiple detection boxes for multiple targets. In some embodiments, the detection result preprocessing module 210 can also filter the identified candidate boxes and then retain the candidate boxes that meet the requirements as detection boxes. For example, if multiple candidate boxes identify the same target, a non-maximum suppression (NMS) algorithm can be used to determine one candidate box as the detection box for that target.

[0049] The motion prediction module 220 is used to predict the trajectory bounding box of each target object in the detection data of the Tth frame based on the detection bounding box of the previous frame or several previous frames (the detection data of the previous T-1 frames, where T is a positive integer). For example, a Kalman filter can be used as the motion model to predict the possible location of the target object in the detection data of the Tth frame.

[0050] The data matching module 230 is used to match detection boxes and trajectory boxes to determine the correspondence between each detection box and trajectory box in the T-th frame of detection data, and then to determine the correspondence between each target object in the T-th frame of detection data and each target object in the previous T-1 frames of detection data. When matching detection boxes and trajectory boxes, the data matching module 230 typically uses a distance algorithm, such as Euclidean distance based on the center point or intersection over union (IoU) of the overlapping area of ​​the detection box and trajectory box.

[0051] Then, the target object corresponding to the detection box that matches the trajectory box is marked with the same tracking ID as the trajectory box and sent to the motion prediction module 220 for the next motion prediction. Then, the trajectory boxes that do not match the detection box are sent to the trajectory management module 240 for managing the trajectory lifecycle.

[0052] For example, such as Figure 3 As shown, the detection result preprocessing module 210 obtains detection boxes B1 based on target object 1 and B2 based on target object 2 using the target detection algorithm based on the detection data of frame T. The detection result preprocessing module 210 also detects detection boxes B3 and B4 based on the detection data of frame T-1, where detection box B3 corresponds to target object ID1 and detection box B4 corresponds to target object ID2. The motion prediction module 220 can predict the position of target object ID1 as trajectory box P1 in the detection data of frame T based on detection box B3, and predict the position of target object ID2 as trajectory box P2 in the detection data of frame T based on detection box B4.

[0053] Then, the trajectory boxes P1, P2, and detection boxes B1 and B2 are matched. By calculating the IoU between these boxes, the two boxes with the largest overlap can be identified as the matched trajectory box and detection box, i.e., trajectory box P1 corresponds to the matched detection box B1, and trajectory box P2 corresponds to the matched detection box B2. Furthermore, the ID1 corresponding to target object 1 and the ID2 corresponding to target object 2 can be determined.

[0054] The trajectory management module 240 can determine that the target object corresponding to the current trajectory box has disappeared and does not appear in the detection data of frame T based on the trajectory box that does not match the detection box, and thus mark the trajectory as lost. Then, for the detection box that does not match the trajectory box, it determines that the target object corresponding to the detection box is a newly appeared target object, and then establishes a new trajectory for the target object.

[0055] In other words, for detection boxes that do not match a trajectory box, the above method directly marks the corresponding target object as lost, resulting in a low matching rate between the detection box and the trajectory box. This matching rate is even lower when the target object is occluded or blurred due to motion in the T-frame detection data. If the target object's detection box cannot match the corresponding trajectory box, the vehicle will directly mark the target object corresponding to the trajectory box as lost, preventing further trajectory prediction for that target object and affecting the accuracy of subsequent target tracking processing. For example, the vehicle may be unable to predict the trajectories of surrounding pedestrians or other vehicles, thus failing to take corresponding measures in advance, potentially leading to traffic accidents.

[0056] To address the issue of low matching rates between detection boxes and trajectory boxes, this application provides a target tracking method applied to an electronic device. This electronic device obtains multiple detection boxes, along with category information and confidence scores for each detection box, by performing target detection on the detection data of frame T. The detection data can be two-dimensional image data or three-dimensional data, such as point cloud data. Each detection box corresponds to a target object, and the category information indicates the category to which the target object belongs, such as pedestrian, vehicle, etc. The confidence score represents the probability that the target object belongs to that category.

[0057] Furthermore, after the electronic device predicts the corresponding trajectory boxes for these detection boxes in frame T, when matching the detection boxes and trajectory boxes, the electronic device classifies the detection boxes according to the confidence level of each detection box in the detection data of frame T, and adopts different matching strategies for different types of detection boxes. For example, for detection boxes with high confidence, that is, the probability that the target object belongs to the corresponding category is relatively high, it means that the target object can be accurately determined to belong to the corresponding category through the detection data of the target object. In other words, the target object is usually relatively clear and complete. At this time, a lower precision matching strategy can achieve a high matching rate. On the other hand, for detection boxes with low confidence, that is, the probability that the target object belongs to the corresponding category is relatively low, it means that it is difficult to accurately determine that the target object belongs to the corresponding category through the detection data of the target object. In other words, the target object may be occluded or its size may be blurred, as mentioned above. At this time, a higher precision matching strategy can be adopted to improve the matching rate between the detection boxes and trajectory boxes.

[0058] For example, an electronic device can categorize detection boxes into high-score and low-score boxes based on their confidence level. For high-score detection boxes, a low-precision matching strategy is used to match them with various trajectory boxes, such as matching based on the size of the overlapping region. For low-score detection boxes, a high-precision matching strategy is used to match them with various trajectory boxes, such as matching based on the distance between the center points of the low-score detection box and the trajectory box, the diagonal length of the minimum bounding volume, or matching based on the overlapping region, the distance between the center points, and the aspect ratio of the low-score detection box and the trajectory box. The high-precision matching strategy considers more geometric factors than the low-precision strategy, thus achieving higher matching accuracy.

[0059] It should be understood that electronic devices can classify detection frames into two or more levels according to their confidence level. This application does not specify the number of specific classifications of detection frames.

[0060] Thus, for targets that are occluded or whose size is blurred due to motion, the confidence level in the target detection stage is often low, corresponding to low-scoring detection boxes. More precise matching strategies and trajectory boxes can be used for matching, improving the matching rate between the detection boxes and trajectory boxes. Conversely, for clearer and more complete targets, the position and size information of the detection boxes obtained in the target detection stage is more accurate, resulting in higher confidence. This corresponds to high-scoring detection boxes, where a lower-precision matching strategy with less computation can be used to achieve a higher matching rate while reducing computational load. Therefore, the target tracking method provided in this application can improve the matching rate between detection boxes and trajectory boxes while reducing the total computational load, thereby improving matching efficiency.

[0061] Optionally, the electronic device can first use a low-precision matching strategy to match high-scoring detection boxes with trajectory boxes. For trajectory boxes that do not match any detection boxes, a high-precision matching strategy is then used to match them with low-scoring detection boxes. This way, low-scoring detection boxes only need to be matched with the remaining trajectory boxes that did not match high-scoring detection boxes, reducing computation and improving matching efficiency.

[0062] Optionally, after matching the detection box and the trajectory box, this application further calculates the error between the detection box and the matched trajectory box, and then uses this error for the next trajectory box prediction. Specifically, the electronic device sends the error back to the motion prediction model, which generates an adaptive factor based on the ratio of the error of the detection data in frame T to the mean of the error of the detection data in the previous T-1 frames. This adaptive factor is used to adjust the parameters of the motion prediction model, thereby improving the prediction accuracy of the trajectory box.

[0063] Optionally, both the trajectory bounding box and the detection bounding box can be 3D boxes, specifically represented by the center point position (x, y, z), dimensions (l, w, h), and yaw angle θ. The matching between the trajectory bounding box and the detection bounding box also employs a 3D calculation method. For example, the low-precision matching strategy used between high-scoring detection boxes and trajectory bounding boxes can be based on the 3D generalized intersection over union (GIoU) ​​ratio, while the high-precision matching strategy used between low-scoring detection boxes and trajectory bounding boxes can be based on the 3D distance intersection over union (DIoU) ratio or the complete intersection over union (CIOU) ratio.

[0064] The aforementioned electronic device can be any electronic device that can acquire detection data from a data acquisition device, such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), or specialized cameras (such as SLR cameras and point-and-shoot cameras). This application does not impose any restrictions on the specific type of the electronic device.

[0065] In summary, the target tracking method provided in this application employs different matching strategies for detection boxes with varying confidence levels. This not only improves the matching rate between detection boxes and trajectory boxes but also reduces the overall computational load, thereby increasing matching efficiency. Furthermore, the matching process preserves 3D information, utilizing a 3D matching algorithm to avoid losing spatial and angular information of the target object, thus enhancing the accuracy of the target tracking algorithm. Moreover, after each matching step, the error between the matched detection box and trajectory box is calculated, and the motion prediction model is adjusted based on this error, resulting in greater accuracy in predicting trajectory boxes in subsequent iterations.

[0066] The structure of the target tracking system provided in this application is described below. For example... Figure 4 As shown, Figure 4 A target tracking system is shown, including a detection result preprocessing module 410, a motion prediction module 420, a low-precision matching module 430, a high-precision matching module 440, and a trajectory management module 450.

[0067] The detection result preprocessing module 410 is used to acquire the detection data obtained by the data acquisition device, wherein, Figure 4 Taking a lidar as an example, the data acquisition device is used. Multiple detection boxes are acquired from the detection data, and the detection boxes in the T-th frame are divided into high-score detection boxes and low-score detection boxes. Specifically, the detection result preprocessing module 410 can classify the detection boxes according to a score threshold, such as a confidence level of 0.5. Detection boxes with a confidence level less than or equal to 0.5 are low-score detection boxes, and detection boxes with a confidence level greater than 0.5 are high-score detection boxes. This can also be used to execute the relevant content of the aforementioned detection result preprocessing module 210, which will not be elaborated here.

[0068] The motion prediction module 420 is used to predict the trajectory box corresponding to the T-th frame based on the detection boxes of the previous T-1 frames of detection data. For example, a Kalman filter can be used as a motion model to predict the possible location of the target object in the T-th frame of detection data.

[0069] The low-precision matching module 430 is used to match high-scoring detection boxes and trajectory boxes using a low-precision matching strategy, such as 3D GIoU matching. Then, the high-scoring detection boxes that match trajectory boxes are labeled with IDs and sent back to the motion prediction module 420 for the next trajectory box prediction. Trajectory boxes that do not match high-scoring detection boxes are sent back to the high-precision matching module 440. The low-precision matching module 430 can also calculate the error between the matched trajectory boxes and detection boxes, and then send the error to the motion prediction module 420.

[0070] The high-precision matching module 440 is used to match low-score detection boxes and trajectory boxes using a high-precision matching strategy, such as matching using 3D DIoU. Similarly, low-score detection boxes that have been matched with trajectory boxes are labeled with IDs and sent back to the motion prediction module 420 for the next trajectory box prediction. Trajectory boxes that have not yet been matched are also sent to the trajectory management module 450.

[0071] The trajectory management module 450 can mark a trajectory as lost if no matching detection box is found. Then, for each detection box that does not match a trajectory box, a new trajectory is created for the target object within that detection box.

[0072] It should be understood that the detection result preprocessing module 410 can further classify the detection boxes into more levels based on confidence, such as three or more categories. Corresponding to the increased number of levels, the target tracking system should also have more matching modules, and the accuracy of the matching strategies employed by different matching modules can vary. Specifically, a low-precision matching strategy is used for detection boxes with high confidence, while a high-precision matching strategy is used for detection boxes with low confidence.

[0073] For example, the matching strategy may also include CIoU, which has higher precision than GIoU. When the low-precision matching module 430 uses GIoU, the high-precision matching module 440 may also use CIoU. It should be understood that this application does not specifically limit the type of matching algorithm.

[0074] In summary, the target tracking system provided in this application employs different matching strategies for detection boxes with varying confidence levels. This not only improves the matching rate between detection boxes and trajectory boxes but also reduces the overall computational load, thereby increasing matching efficiency. Furthermore, the matching process preserves 3D information and utilizes a 3D matching algorithm, ensuring no loss of spatial and angular information about the target object, thus enhancing the accuracy of the target tracking algorithm. Moreover, after each matching step, the error between the matched detection box and trajectory box is calculated. This error is used to adjust the motion prediction model, resulting in greater accuracy in predicting trajectory boxes in subsequent iterations.

[0075] The following is combined Figure 5 The target tracking method provided in this application is described in detail. This target tracking method can be applied to electronic devices, which include or are connected to data acquisition devices, such as cameras, lidar, etc.

[0076] Figure 5 Specifically, this will be explained using the example of dividing the detection frame into high-precision detection frames and low-precision detection frames. Figure 5 As shown, the target tracking method may include the following steps:

[0077] S510: Obtain multiple detection boxes from the detection data of frame T.

[0078] The electronic device acquires the detection data of the Tth frame from the data acquisition device. The electronic device can perform target detection on the detection data of the Tth frame to obtain the detection box corresponding to each target object in the detection data of the Tth frame.

[0079] The detection data can be image data acquired by devices such as cameras, with each frame including color information (RGB) of multiple pixels. Detection data can also be three-dimensional data, such as point cloud data obtained from LiDAR scanning, with each frame including the spatial location information of each point, or possibly reflection intensity information and / or RGB information. The detection bounding box can have a regular or irregular shape. For example, the bounding box can be circular, elliptical, polygonal, or any other geometric shape. Target detection can also be processed by other electronic devices, which directly acquire the detection bounding boxes obtained by those devices.

[0080] The electronic device acquires multiple detection boxes from the T-th frame of detection data. These boxes include: the location of the detection box, the category of the target object corresponding to the detection box, and the probability (i.e., confidence level) that the target object belongs to that category. For example, when the detection data is point cloud data, the location of the detection box includes the center point position (x, y, z), size (l, w, h), and yaw angle θ. Target detection specifically involves a classification model obtaining the probability of the target object in the detection box belonging to each category based on the characteristics of the detection box. Then, based on the probability of each category, the category corresponding to the target object in the detection box is determined. For example, the category with the highest category probability is determined as the category of the target object in the detection box, and the confidence level of the detection box can be determined using this highest category probability.

[0081] It should be understood that the electronic device can also first acquire multiple candidate boxes based on the target detection data of the Tth frame, and then filter the candidate boxes to obtain the candidate boxes that meet the requirements as the detection boxes. For details, please refer to the above. Figure 4 And its related descriptions.

[0082] S520: Determine multiple trajectory boxes for the detection data of the Tth frame based on the detection data of the previous T-1 frames.

[0083] The electronic device also acquires detection data from the frame preceding the T-th frame or several preceding frames to obtain the bounding boxes corresponding to each target object in the previous T frames. Based on the bounding boxes from the previous T-1 frames, the electronic device predicts the trajectory of the target object corresponding to each bounding box, thus obtaining multiple bounding boxes. Each bounding box can represent the possible position of each target object in the T-th frame of detection data. For example, a Kalman filter can be used as a motion model to predict the possible position of the target object in the T-th frame of detection data. Specifically, this can be achieved through the aforementioned... Figure 4 The motion prediction module 420 in the middle is used for processing.

[0084] In some embodiments, the electronic device may first acquire the trajectory bounding box and then acquire the detection bounding box, that is, execute step S520 first and then execute step S510. Alternatively, steps S510 and S520 may be performed simultaneously. In other words, this application does not specifically limit the execution order of steps S510 and S520.

[0085] S530: Determine in sequence whether each detection box meets the first condition.

[0086] The first condition can be that the confidence level of the detection box is greater than a confidence threshold. The electronic device determines whether the detection box meets the first condition based on the comparison between the confidence level and the confidence threshold. If the confidence level of the detection box is greater than the confidence threshold, the detection box meets the first condition and is a high-score detection box, thus proceeding to step S540. If the confidence level of the detection box is less than or equal to the confidence threshold, the detection box does not meet the first condition and is a low-score detection box, thus proceeding to step S550.

[0087] S540: Uses the first matching strategy to match the trajectory box.

[0088] The electronic device sequentially matches detection boxes that meet the first condition with trajectory boxes using a first matching strategy. The first matching strategy can be a low-precision matching strategy. For detection boxes that meet the first condition, since the target object is relatively clear and complete, the position and size information of the detection box obtained in the target detection stage is relatively accurate and the confidence level is also high. Therefore, a high matching rate can be obtained by using only a low-precision matching strategy with less computation to match the trajectory boxes, while also reducing the amount of computation.

[0089] The first matching strategy is a low-precision matching strategy. For example, when the detection data is image data, the first matching strategy may specifically be GIoU. When the detection data is point cloud data, the first matching strategy may specifically be 3D GIoU.

[0090] The calculation method for 3D GIoU can be referenced in the following formula (1), where A represents the detection box, B represents the trajectory box, and C represents the minimum bounding volume of the detection box and the trajectory box. A and V B Let V represent the volumes of A and B respectively. C This represents the volume of the smallest bounding volume.

[0091]

[0092] S550: Uses the second matching strategy to match the trajectory box.

[0093] For detection boxes that do not meet the first condition, the electronic device uses a second matching strategy to match them with the trajectory box. This second matching strategy is a high-precision matching strategy. For detection boxes that do not meet the first condition, the target object is usually obscured by motion or its size is blurred due to motion. Therefore, a more precise matching strategy is needed to match them with the trajectory box, which can improve the matching rate between the detection box and the trajectory box.

[0094] The second matching strategy is a high-precision matching strategy. For example, when the detection data is image data, the second matching strategy can specifically be DIoU. When the detection data is point cloud data, the second matching strategy can specifically be 3D DIoU.

[0095] The calculation method for 3D DIoU can be found in formula (2) below. V c represents the distance between the center point of the detection box and the center point of the trajectory box. V This represents the length of the diagonal of the smallest bounding volume.

[0096]

[0097] In some embodiments, the electronic device may first execute step S540 and then execute step S550, that is, first perform matching calculations between the detection boxes that meet the first condition and all trajectory boxes, and then match the trajectory boxes that do not match with the detection boxes with the detection boxes that do not meet the second condition.

[0098] In other embodiments, the electronic device may first execute step S550 and then execute step S540, that is, first perform matching calculations between the detection boxes that do not meet the first condition and all trajectory boxes, and then match the trajectory boxes that do not match with the detection boxes that meet the second condition.

[0099] In other embodiments, the electronic device may also perform the above steps S540 and S550 in parallel, that is, directly divide the detection box into detection boxes that meet the first condition and detection boxes that do not meet the first condition, and then match the two types of detection boxes with all trajectory boxes respectively.

[0100] It should be understood that the detection boxes can be further divided into more levels based on confidence, meaning that the target tracking method can also include determining whether the detection box meets a second condition, etc. Consequently, different matching strategies with varying precision are employed for detection boxes with different confidence levels. These matching strategies, ranked from low to high precision, can include GIoU, DIoU, CIoU, etc. For example, when the first matching strategy is GIoU, the second matching strategy can also be CIoU.

[0101] S560: Determine if a trajectory box and a detection box match. If yes, proceed to step S570; otherwise, proceed to step S580.

[0102] When the matching value (e.g., GIoU or DIoU value) between the detection box and the trajectory box is greater than the matching threshold, it indicates that the detection box and the trajectory box match, and step S570 is executed. If no trajectory box and the detection box have a matching value greater than the matching threshold, it indicates that the detection box does not match a trajectory box, and step S580 is executed. If multiple trajectory boxes have matching values ​​greater than the threshold with the detection box, the trajectory box with the highest matching value can be determined as the trajectory box that matches the detection box.

[0103] S570: Mark the target object ID and determine the error between the matched detection box and the trajectory box.

[0104] For a matching detection box and a trajectory box, the electronic device marks the target object in the detection box as the same target object corresponding to the trajectory box. The electronic device also calculates the error between the two, which can be used for the next trajectory box prediction.

[0105] The error between the matched detection box and the trajectory box can be calculated using the following formula (3). Where V T L represents the residual vector of the detection data in frame T. T The detection box represents the detection data of frame T. H represents the bounding box of the detection data in the Tth frame obtained from the detection data of the previous T-1 frames. T This represents the state transition matrix.

[0106]

[0107] Furthermore, the electronic device can add the residual vector of each frame to the residual vector of the historical frames and calculate the mean of the residual vector of the historical frames.

[0108]

[0109] In some embodiments, the electronic device may also determine an adaptive factor based on the aforementioned error, and then update the motion model based on the adaptive factor. The adaptive factor s can be calculated using the following formula (5).

[0110]

[0111] When the adaptive factor is greater than the adaptive threshold, it indicates that the filter may be in an unstable state or a divergent state. In this case, the adaptive threshold is used to predict the trajectory box of the next frame of detection data. When the adaptive factor is less than or equal to the adaptive threshold, it indicates that the filter of the motion prediction model is in a steady state. In this case, the adaptive factor can be used to predict the trajectory box of the next frame of detection data, or a preset factor can be used to predict the trajectory box of the next frame of detection data, wherein the preset factor is less than or equal to the adaptive threshold.

[0112] The adaptive threshold can be set to 1. When the adaptive factor S≤1, it indicates that the filter of the motion prediction model is in a steady state. When S>1, it indicates that the filter may be in an unstable state or a divergent state. In practical applications, to avoid false alarms, a threshold S0>1 is chosen. When S>S0, S=S0, i.e., the adaptive algorithm is used; when S≤S0, S=1, i.e., the Kalman filter algorithm is used. Here, S0 can be set to 1.1.

[0113] The formula for adjusting the adaptive model based on the adaptive factor can be found in the following formula (6). Where CT ′ / (T-1) Φ represents the estimated value of the covariance matrix from frame T-1 to frame T. T-1 Let C represent the state transition matrix of frame T-1. T-1 Let Q represent the variance matrix of the (T-1)th frame. T-1 This represents the system noise matrix for the (T-1)th frame.

[0114]

[0115] S580: Perform trajectory management based on unmatched detection boxes and trajectory boxes.

[0116] For trajectory frames that do not match detection frames, the electronic device determines that the target object corresponding to the current trajectory frame has disappeared and does not appear in the detection data of frame T, thus marking the trajectory as lost. Then, for detection frames that do not match trajectory frames, it determines that the target object corresponding to the detection frame is a newly appeared target object, and thus establishes a new trajectory for that target object. Specifically, this can be achieved as follows: Figure 4 The trajectory management module 450 shown is executed.

[0117] In summary, the target tracking method provided in this application employs different matching strategies for detection boxes with varying confidence levels. This not only improves the matching rate between detection boxes and trajectory boxes but also reduces the overall computational load, thereby increasing matching efficiency. Furthermore, the matching process preserves 3D information, utilizing a 3D matching algorithm to avoid losing spatial and angular information of the target object, thus enhancing the accuracy of the target tracking algorithm. Moreover, after each matching step, the error between the matched detection box and trajectory box is calculated, and the motion prediction model is adjusted based on this error, resulting in greater accuracy in predicting trajectory boxes in subsequent iterations.

[0118] The following will also be combined with Figure 6 This application introduces another target tracking method. This target tracking method can be applied to electronic devices, which include or are connected to data acquisition devices. For example... Figure 6 As shown, the target tracking method may include the following steps:

[0119] S601: The target detection model detects multiple detection boxes from the detection data of the Tth frame.

[0120] The electronic device acquires the detection data of the Tth frame from the data acquisition device. The electronic device can perform target detection on the detection data of the Tth frame to obtain the detection box corresponding to each target object in the detection data of the Tth frame. For details, please refer to the aforementioned step S510, which will not be repeated here.

[0121] S602: The motion prediction model predicts multiple trajectory boxes from the detection data of the first T-1 frames.

[0122] The electronic device will also acquire the detection data of the frame preceding the T-frame detection data or the detection data of several previous frames to obtain the trajectory boxes corresponding to each target object in the previous T frames.

[0123] S603: Whether the confidence level of the detection box is greater than the confidence threshold.

[0124] The electronic device determines the category of a detection box based on a comparison between the confidence level of the detection box and a confidence threshold. If the confidence level of the detection box is greater than the confidence threshold, the detection box is a high-score detection box, and step S604 is executed. If the confidence level of the detection box is less than or equal to the confidence threshold, the detection box is a low-score detection box, and step S605 is executed.

[0125] S604: This detection box is determined to be a high-score detection box.

[0126] Since the detection box is a high-score detection box, the electronic device will use a low-precision matching strategy for matching, that is, execute step S606.

[0127] S605: The detection box is determined to be a low-score detection box.

[0128] Since the detection box is a low-score detection box, the electronic device will use a high-precision matching strategy for matching, that is, execute step S610.

[0129] S606: Uses GIoU for matching.

[0130] The electronic device matches the high-score detection boxes and trajectory boxes using GIoU and calculates the matching degree. For details, please refer to step S540 above; it will not be repeated here.

[0131] S607: Is the matching degree greater than the matching degree threshold?

[0132] If the matching value between the detection box and the trajectory box is greater than the matching threshold, it means that the detection box and the trajectory box match, and step S608 is executed. If no trajectory box and the detection box have a matching value greater than the matching threshold, it means that the detection box does not match the trajectory box, and step S609 is executed.

[0133] S608: Calculate the error between the matched detection box and the trajectory box.

[0134] For the matched detection box and trajectory box, the electronic device will also calculate the error between them. This error can be sent to the motion prediction model for the next trajectory box prediction. See step S570 above for details, which will not be repeated here.

[0135] S609: Output the tracking ID of the detection box.

[0136] For a matching detection box and a trajectory box, the electronic device marks the target object in the detection box as the same target object as the target object in the matching trajectory box, that is, it marks the target object in the detection box with a tracking ID.

[0137] S610: Matching is performed using DIoU.

[0138] The electronic device matches the low-score detection boxes and trajectory boxes using GIoU and calculates the matching degree. For details, please refer to step S550 above; it will not be repeated here.

[0139] S611: Is the matching degree greater than the matching degree threshold?

[0140] If the matching value between the detection box and the trajectory box is greater than the matching threshold, it means that the detection box and the trajectory box match, and step S612 is executed. If no trajectory box and the detection box have a matching value greater than the matching threshold, it means that the detection box does not match the trajectory box, and step S614 is executed.

[0141] S612: Calculate the error between the detection box and the trajectory box.

[0142] For the matched detection box and trajectory box, the electronic device will also calculate the error between them. This error can be sent to the motion prediction model for the next trajectory box prediction. See step S570 above for details, which will not be repeated here.

[0143] S613: Output the tracking ID of the detection box.

[0144] For a matching detection box and a trajectory box, the electronic device marks the target object in the detection box as the same target object as the target object in the matching trajectory box, that is, it marks the target object in the detection box with a tracking ID.

[0145] S614: Manage the corresponding trajectory.

[0146] For a trajectory bounding box that fails to match a detection box after two matching attempts, the corresponding trajectory is marked as lost. For a detection box that does not match a trajectory bounding box, a new trajectory is created for it.

[0147] In summary, the target tracking method provided in this application employs different matching strategies for detection boxes with varying confidence levels. This not only improves the matching rate between detection boxes and trajectory boxes but also reduces the overall computational load, thereby increasing matching efficiency. Furthermore, the matching process preserves 3D information, utilizing a 3D matching algorithm to avoid losing spatial and angular information of the target object, thus enhancing the accuracy of the target tracking algorithm. Moreover, after each matching step, the error between the matched detection box and trajectory box is calculated, and the motion prediction model is adjusted based on this error, resulting in greater accuracy in predicting trajectory boxes in subsequent iterations.

[0148] Specifically, the method provided in this application can improve multiple object tracking accuracy (MOTA), target tracking precision (MOT, MOTP), and reduce the target tracking ID switch rate (IDS).

[0149] For example, experiments were conducted using this application and traditional MOT on the 4D Bird's Eye View 4D (BEV4D) dataset. The BEV4D test set is a full-scale, all-category test set containing 30 scenes and over 40,000 objects, including urban and highway conditions, and categories such as cars, trucks, pedestrians, and bicycles, with roads including straight and turning routes. The experimental results are shown in Table 1 below.

[0150] Table 1

[0151] Way MOTA↑ MOTP↑ IDS↓ Traditional MOT 0.5251 0.7599 1769 This application 0.5860 0.7860 203

[0152] As mentioned above, tracking accuracy (MOTA) improved by 6.1%, tracking precision (MOTP) improved by 2.7%, and tracking ID jump rate (IDS) decreased by 88.52%.

[0153] The error between the matched detection box and the trajectory box is calculated using this application. The motion prediction model is then adjusted based on the error. The experimental results obtained are compared with those of the traditional MOT model, as shown in Table 2 below.

[0154] Table 2

[0155] Way MOTA↑ MOTP↑ IDS↓ Traditional MOT 0.401 0.751 16 This application 0.413 0.764 14

[0156] As mentioned above, tracking accuracy (MOTA) improved by 1.2%, tracking precision (MOTP) improved by 1.3%, and tracking ID jump rate (IDS) decreased by 12.5%.

[0157] In this application, the detection boxes are divided into high-score detection boxes and low-score detection boxes, and different matching strategies with different precision are used for matching. The experimental results obtained are compared with those of the traditional MOT as shown in Table 3 below.

[0158] Table 3

[0159] Way MOTA↑ MOTP↑ IDS↓ Traditional MOT 0.391 0.809 17 This application 0.606 0.831 4

[0160] As mentioned above, tracking accuracy (MOTA) is improved by 21.5%, tracking precision (MOTP) is improved by 2.2%, and tracking ID jump rate (IDS) is reduced by 76.5%.

[0161] The experimental results obtained by preserving the 3D information of the detection box and adopting a 3D matching strategy are shown in Table 4 below, which compares them with those of the traditional MOT.

[0162] Table 4

[0163] Way MOTA↑ MOTP↑ IDS↓ 2D IoU 0.288 0.798 887 3D GIoU 0.389 0.807 29 3D GIoU+3D DIoU 0.608 0.833 5

[0164] As mentioned above, compared to the 2D IoU matching strategy, the tracking accuracy MOTA is improved by 32%, the tracking precision MOTP is improved by 3.5%, and the tracking ID jump rate IDS is reduced by 99.4%. Compared to the single-stage 3D IoU matching strategy, the tracking accuracy MOTA is improved by 21.9%, the tracking precision MOTP is improved by 2.6%, and the tracking ID jump rate IDS is reduced by 82.8%.

[0165] The target tracking device provided in this application will be described in detail below, such as... Figure 7 As shown, the target tracking device 700 includes a determination unit 710, an adoption unit 720, and a prediction unit 730.

[0166] The determining unit 710 is used to determine the first category information and the first category confidence in the first detection box in the T-th frame detection data. The first category confidence represents the probability that the target object in the first detection box belongs to the category represented by the first category information.

[0167] Unit 720 is used to match the first detection box with multiple trajectory boxes of the detection data in frame T when the first category confidence level of the first detection box meets the first condition and a first matching strategy is adopted.

[0168] Unit 720 is also used to match the first detection box with multiple trajectory boxes of the detection data of the Tth frame if the first category confidence level corresponding to the first detection box does not meet the first condition.

[0169] In some embodiments, the first condition includes a first category confidence level greater than a confidence threshold.

[0170] In other embodiments, the detection data for the Tth frame includes the image data of the Tth frame or the point cloud data of the Tth frame.

[0171] In other embodiments, corresponding to the first matching strategy, the matching degree is calculated based on the size of the overlapping area between the detection box and the trajectory box. The second matching strategy includes: calculating the matching degree based on the distance between the center points of the detection box and the trajectory box and the diagonal length of the minimum bounding body, or calculating the matching degree based on the distance between the center points of the detection box and the trajectory box, the size of the overlapping area, and the aspect ratio.

[0172] In other embodiments, the detection data corresponding to the Tth frame is the point cloud data of the Tth frame, and the first matching strategy includes calculating the matching degree based on the intersection-union ratio of the three-dimensional general overlapping regions, and the second matching strategy includes calculating the matching degree based on the intersection-union ratio of the three-dimensional distance overlapping regions, or the second matching strategy is to calculate the matching degree based on the intersection-union ratio of the three-dimensional completely overlapping regions; the first matching strategy includes calculating the matching degree based on the intersection-union ratio of the three-dimensional distance overlapping regions, and the second matching strategy includes calculating the matching degree based on the intersection-union ratio of the three-dimensional completely overlapping regions.

[0173] In other embodiments, the determining unit 710 is further configured to determine that the first trajectory box and the first detection box match when the first matching degree of the first trajectory box and the first detection box in the detection data of the Tth frame is greater than a matching degree threshold. Alternatively, the determining unit 710 is further configured to match the first detection box with multiple trajectory boxes in the detection data of the Tth frame, and determine that the first detection box matches the first detection box when the matching degree of the first detection box with the first detection box in the multiple trajectory boxes is the largest.

[0174] In other embodiments, the prediction unit 730 is used to predict multiple trajectory boxes of the detection data of the Tth frame based on the detection boxes of the detection data from the first frame to the (T-1)th frame, the multiple trajectory boxes including the first trajectory box.

[0175] In other embodiments, the determining unit 710 is further configured to determine the error between each detection box and its corresponding matched trajectory box in the T-th frame detection data, thereby obtaining the error of the T-th frame detection data. The determining unit 710 is further configured to determine an adaptive factor based on the error of the T-th frame detection data and the error of the previous T-1 frames detection data, the adaptive factor being used to adjust the covariance matrix of the trajectory box in the T+1-th frame.

[0176] In other embodiments, the determining unit 710 is further configured to use the adaptive threshold to adjust the covariance matrix of the trajectory box in the (T+1)th frame based on the adaptive factor being greater than the adaptive threshold. The determining unit 710 is also configured to use a preset factor to adjust the covariance matrix of the trajectory box in the (T+1)th frame based on the adaptive factor being less than or equal to the adaptive threshold, where the preset factor is less than or equal to the adaptive threshold.

[0177] In summary, the target tracking device provided in this application employs different matching strategies for detection boxes with varying confidence levels. This not only improves the matching rate between detection boxes and trajectory boxes but also reduces the overall computational load, thereby increasing matching efficiency. Furthermore, the matching process preserves 3D information and utilizes a 3D matching algorithm, ensuring no loss of spatial and angular information about the target object, thus enhancing the accuracy of the target tracking algorithm. Moreover, after each matching step, the error between the matched detection box and trajectory box is calculated. This error is used to adjust the motion prediction model, resulting in greater accuracy in predicting trajectory boxes in subsequent iterations.

[0178] The methods of the embodiments of this application have been described in detail above. In order to facilitate better implementation of the above-described solutions of the embodiments of this application, relevant equipment for cooperating in implementing the above solutions is also provided below.

[0179] Figure 8 This is a schematic diagram of the structure of an electronic device 800 provided in this application. The electronic device 800 can be the target tracking device 800 mentioned above. For example... Figure 8As shown, the electronic device 800 includes a processor 810, a communication interface 820, and a memory 830. The processor 810, communication interface 820, and memory 830 can be interconnected via an internal bus 840, or they can communicate via wireless transmission or other means. This embodiment uses a connection via bus 840 as an example. Bus 840 can be a peripheral component interconnect express (PCIe) bus, an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc. Bus 840 can be divided into address bus, data bus, control bus, etc. In addition to the data bus, bus 840 can also include a power bus, control bus, and status signal bus. However, for clarity, all buses are labeled as bus 840 in the figure.

[0180] Processor 810 may consist of at least one general-purpose processor, such as a central processing unit (CPU), or a combination of a CPU and hardware chips. The aforementioned hardware chips may be application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or combinations thereof. The aforementioned PLDs may be complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), generic array logic (GALs), or any combination thereof. Processor 810 executes various types of digital storage instructions, such as software or firmware programs stored in memory 830, enabling electronic device 800 to provide a variety of services.

[0181] The memory 830 stores program code, which is executed under the control of the processor 810 to perform the processing steps of the target tracking method in the above embodiments. The program code may include one or more software modules, which can be... Figure 7 The software modules provided in the embodiments.

[0182] It should be noted that this embodiment can be implemented using a general-purpose physical server, such as an ARM server or an x86 server, or it can be implemented using a virtual machine based on a general-purpose physical server combined with NFV technology. A virtual machine refers to a complete computer system with complete hardware system functions simulated by software and running in a completely isolated environment. This application does not make any specific limitations.

[0183] The memory 830 may include volatile memory, such as random access memory (RAM); the memory 830 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); the memory 830 may also include combinations of the above types. The memory 830 may store program code for execution. Figure 5 Steps S510-S580 and their optional steps in the embodiment, or execution Figure 6 The steps S601-S614 and their optional steps in the embodiment will not be described in detail here.

[0184] The communication interface 820 can be a wired interface (e.g., an Ethernet interface), an internal interface (e.g., a high-speed serial computer expansion bus (peripheral component interconnect express, PCIe) bus interface), a wired interface (e.g., an Ethernet interface), or a wireless interface (e.g., a cellular network interface or a wireless LAN interface), for communicating with other devices or modules.

[0185] It needs to be explained that, Figure 8 This is merely one possible implementation of an embodiment of this application. In practical applications, the electronic device 800 may include more or fewer components, and this is not a limitation. For content not shown or described in the embodiments of this application, please refer to the foregoing. Figure 5 or Figure 6 The relevant descriptions in the embodiments will not be repeated here.

[0186] It should be understood that Figure 8 The electronic device shown can also be a computer cluster consisting of at least one server, which is not specifically limited in this application.

[0187] This application also provides a computer-readable storage medium storing instructions that, when executed on a processor. Figure 5 or Figure 6 The method flow shown is thus implemented.

[0188] This application also provides a computer program product that, when run on a processor, provides a solution for... Figure 5 or Figure 6 The method flow shown is thus implemented.

[0189] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0190] The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0191] The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that includes one or more sets of available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs (DVDs), or semiconductor media. Semiconductor media can be SSDs.

[0192] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A target tracking method applied to electronic devices, characterized in that, include: Determine the first category information and first category confidence of the first detection box in the detection data of frame T. The first category confidence represents the probability that the target object in the first detection box belongs to the category represented by the first category information. If the confidence level of the first category corresponding to the first detection box meets the first condition, the first matching strategy is adopted to match the first detection box with multiple trajectory boxes of the detection data of the Tth frame. The first condition includes that the confidence level of the first category is greater than the confidence threshold. If the first category confidence of the first detection box does not meet the first condition, or if the matching degree between the first detection box obtained by the first matching strategy and multiple trajectory boxes of the detection data of the Tth frame is less than or equal to the matching degree threshold, a second matching strategy is adopted to match the first detection box with multiple trajectory boxes of the detection data of the Tth frame. The algorithms for calculating the matching degree between the detection box and the trajectory box are different between the first matching strategy and the second matching strategy, and the accuracy of the first matching strategy is lower than that of the second matching strategy. If there is a trajectory frame in the multiple trajectory frames corresponding to the detection data of the Tth frame that has a matching degree greater than the matching degree threshold with the first detection frame, the target object of the first detection frame is marked as the target object of the corresponding matching trajectory frame, and the error between the first detection frame and the corresponding matching trajectory frame is determined. The error is used to adjust the prediction of the trajectory frame of the (T+1)th frame.

2. The method according to claim 1, characterized in that, The detection data for the Tth frame includes either the image data of the Tth frame or the point cloud data of the Tth frame.

3. The method according to claim 2, characterized in that, Corresponding to the first matching strategy, which includes calculating the matching degree based on the size of the overlap area between the detection box and the trajectory box, the second matching strategy includes: The matching degree can be calculated based on the distance between the center points of the detection box and the trajectory box and the diagonal length of the minimum bounding volume, or based on the distance between the center points of the detection box and the trajectory box, the size of the overlapping area, and the aspect ratio. The first matching strategy includes calculating the matching degree based on the distance between the center points of the detection box and the trajectory box and the diagonal length of the minimum bounding volume. The second matching strategy includes calculating the matching degree based on the distance between the center points of the detection box and the trajectory box, the size of the overlapping area, and the aspect ratio.

4. The method according to claim 2, characterized in that, The point cloud data corresponding to the detected data of the Tth frame is the point cloud data of the Tth frame, and, Corresponding to the first matching strategy, which includes calculating the matching degree based on the intersection-union ratio of the three-dimensional general overlapping regions, the second matching strategy includes: The matching degree is calculated based on the intersection-union ratio of the three-dimensional overlapping regions, or the second matching strategy is to calculate the matching degree based on the intersection-union ratio of the three-dimensional completely overlapping regions; The first matching strategy includes calculating the matching degree based on the intersection-union ratio of the three-dimensional overlapping regions, and the second matching strategy includes calculating the matching degree based on the intersection-union ratio of the three-dimensional completely overlapping regions.

5. The method according to any one of claims 3 or 4, characterized in that, The matching of the first detection box with multiple trajectory boxes of the T-frame detection data includes: If the first matching degree between the first trajectory box and the first detection box in the T-th frame detection data is greater than the matching degree threshold, then the first trajectory box and the first detection box are determined to be a match; or... The first detection box is matched with multiple trajectory boxes of the detection data in the Tth frame. If the first detection box has the largest first matching degree with the first trajectory box in the multiple trajectory boxes and the first matching degree is greater than the matching degree threshold, then the first trajectory box and the first detection box are determined to be matched.

6. The method according to claim 5, characterized in that, Also includes: Based on the detection bounding boxes from the first frame to the (T-1)th frame detection data, multiple trajectory bounding boxes of the Tth frame detection data are predicted, and the multiple trajectory bounding boxes include the first trajectory bounding box.

7. The method according to claim 1, characterized in that, The method further includes: Determine the error between each detection box and the corresponding matched trajectory box in the T-th frame detection data to obtain the error of the T-th frame detection data; An adaptive factor is determined based on the error of the detection data in frame T and the error of the detection data in the preceding T-1 frames. The adaptive factor is used to adjust the covariance matrix of the trajectory box in frame T+1.

8. The method according to claim 7, characterized in that, The adaptive factor is used to predict multiple trajectory boxes in the detection data of frame T+1, including: Based on the fact that the adaptive factor is greater than the adaptive threshold, the adaptive threshold is used to adjust the covariance matrix of the trajectory box in the (T+1)th frame; Based on the fact that the adaptive factor is less than or equal to the adaptive threshold, a preset factor is used to adjust the covariance matrix of the trajectory box in the (T+1)th frame, where the preset factor is less than or equal to the adaptive threshold.

9. A readable medium, characterized in that, The readable medium contains instructions that, when executed by the processor of the electronic device, cause the electronic device to perform the method of any one of claims 1 to 8.

10. An electronic device, characterized in that, include: Memory is used to store instructions executed by one or more processors of an electronic device; And a processor, one of the processors of the electronic device, for executing the instructions to cause the electronic device to implement the method of any one of claims 1 to 8.

Images