A target tracking method, system, storage medium, and electronic device

Abstract

The application provides a target tracking method, system, storage medium and electronic equipment, and relates to the field of image processing.The method comprises the following steps: acquiring video stream data; detecting whether there is an occluded target in the video stream data; if yes, judging whether the occluded target can be effectively detected; if the occluded target is effectively detected, judging whether the upper boundary of the target detection frame of the occluded target is displaced; if the upper boundary is displaced, keeping tracking of the occluded target; if the upper boundary is not displaced and the lower boundary is displaced, performing a lock frame operation starting from the current frame to track the occluded target; and the lock frame operation is used to fix the size and position of the tracking frame.The application can avoid switching the tracking ID of the target in the tracking process, can inhibit multi-shot and improve missed shots, and effectively improves the accuracy of target tracking.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to a target tracking method, system, storage medium, and electronic device. Background Technology

[0002] Roadside parking intelligent management systems typically capture vehicle entry and exit behavior information and vehicle identification information to match vehicles entering and exiting parking spaces, forming a complete parking receipt and enabling automatic roadside parking fee collection. Due to the fee-based nature of the system, both over- and under-captured images of target vehicles can affect the efficiency and accuracy of vehicle tracking management. Particularly when vehicles are obstructed by other vehicles or objects, tracking targets are easily lost, reducing the robustness of real-time tracking and hindering vehicle tracking management. Summary of the Invention

[0003] The purpose of this application is to provide a target tracking method, system, storage medium, and electronic device that can effectively track occluded targets.

[0004] To address the aforementioned technical problems, this application provides a target tracking method, the specific technical solution of which is as follows:

[0005] Acquire video stream data;

[0006] Detect whether there is an obscured target in the video stream data;

[0007] If so, determine whether the occluded target can be effectively detected;

[0008] If the occluded target is effectively detected, determine whether the upper boundary of the target detection box of the occluded target has shifted;

[0009] If the upper boundary is displaced, continue tracking of the occluded target;

[0010] If the upper boundary does not shift but the lower boundary moves, a frame locking operation is performed starting from the current frame to track the occluded target; the frame locking operation is used to fix the size and position of the tracking frame.

[0011] Optionally, detecting whether an occluded target exists in the video stream data includes:

[0012] Determine whether there exists a video frame in the video stream data where the intersection-union ratio between the target detection box and the suspected occlusion detection box is greater than zero, and the ordinate of the lower boundary of the target detection box is less than the ordinate of the lower boundary of the suspected occlusion detection box.

[0013] If so, it confirms that there is an obscured target in the video stream data.

[0014] Optionally, determining whether the upper boundary of the target detection box of the occluded target has shifted includes:

[0015] Determine the starting frame number in which the occluded target is in a stationary state;

[0016] Determine the coordinates of the center point of the target detection box in the video frame corresponding to the starting frame number;

[0017] If the difference between the coordinates of the center point and the coordinates of the second center point of the target detection box in the next video frame is less than a set threshold, it is determined that the upper boundary of the target detection box of the occluded target has not shifted.

[0018] If the difference between the coordinates of the center point and the coordinates of the second center point is not less than the set threshold, it is determined that the upper boundary of the target detection box of the occluded target has shifted.

[0019] Optionally, after determining whether the occluded target can be effectively detected, the method further includes:

[0020] If the occluded target cannot be detected, determine whether the occluded target is included in the low-score detection box with a confidence level lower than a preset value;

[0021] If the occluded target is included, continue tracking the occluded target;

[0022] If the occluded target is not included in the low-resolution detection frame, the frame locking operation is performed starting from the current frame.

[0023] Optionally, before determining whether the occluded target is contained in a low-score detection box with a confidence level below a preset value, the method further includes:

[0024] Target detection is performed on the video stream data. Detection boxes with a confidence level greater than the preset value are designated as high-scoring detection boxes, and detection boxes with a confidence level lower than the preset value are designated as low-scoring detection boxes.

[0025] Optionally, after acquiring the video stream data, the following may also be included:

[0026] Record the position information of the center point of the detection box corresponding to the target;

[0027] The detection box trajectory is obtained by fitting the center point position information;

[0028] The trajectory of the target is obtained by using a tracking prediction algorithm to predict its trajectory;

[0029] Determine the angle between the detection box trajectory and the predicted box trajectory;

[0030] A cost function is constructed based on the included angle; the cost function serves as the matching criterion between the detection box and the tracking trajectory.

[0031] Optionally, after fitting the detection box trajectory based on the center point position information, the method further includes:

[0032] The high-resolution detection box is matched with the tracking trajectory of each target.

[0033] For a tracking trajectory that does not match the high-resolution detection box, after reducing the cost threshold of the cost function, the step of matching the tracking trajectory with the high-resolution detection box is continued.

[0034] If, after lowering the cost function threshold during the matching process and continuing to perform the step of matching the high-scoring detection box with the tracking trajectory, there are still unmatched tracking trajectories, then the low-scoring detection box is used to match the unmatched tracking trajectories.

[0035] If there is an unknown tracking trajectory that cannot be matched by both the high-resolution detection box and the low-resolution detection box, a frame locking operation is performed on a preset number of video frames, and a match is performed when the target corresponding to the unknown tracking trajectory appears a second time.

[0036] If no matching target is found after the preset number of video frames, the unknown tracking trajectory is deleted.

[0037] If there is an unknown high-resolution detection box that does not match the tracking trajectory, a new tracking trajectory is created, and the target corresponding to the unknown high-resolution detection box is added as a new target.

[0038] This application also provides a target tracking system, including:

[0039] The video data acquisition module is used to acquire video stream data;

[0040] An occlusion detection module is used to detect whether there is an occluded target in the video stream data;

[0041] An occlusion detection module is used to determine whether the occluded target can be effectively detected when the occlusion judgment module determines that the occlusion judgment result is yes.

[0042] The displacement detection module is used to determine whether the upper boundary of the target detection box of the occluded target has been displaced when the occlusion detection module determines that the result is yes.

[0043] The tracking module is used to maintain tracking of the occluded target if the upper boundary is displaced; if the upper boundary is not displaced but the lower boundary moves, a frame locking operation is performed starting from the current frame to track the occluded target; the frame locking operation is used to fix the size and position of the tracking frame.

[0044] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0045] This application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method described above when it invokes the computer program in the memory.

[0046] This application provides a target tracking method, comprising: acquiring video stream data; detecting whether there is an occluded target in the video stream data; if so, determining whether the occluded target can be effectively detected; if the occluded target is effectively detected, determining whether the upper boundary of the target detection box of the occluded target has shifted; if the upper boundary has shifted, maintaining tracking of the occluded target; if the upper boundary has not shifted but the lower boundary has moved, performing a frame locking operation starting from the current frame; the frame locking operation is used to fix the size and position of the tracking box to track the occluded target.

[0047] When an occluded target exists in the video stream data, if the occluded target can be effectively detected, displacement detection is performed on the upper boundary of its target detection box. For targets whose upper boundary has not shifted but whose lower boundary has moved, its tracking box is fixed to maintain target tracking. This ensures that the target tracking does not change after the occluder is removed, maintaining the continuity of the entire target tracking trajectory. It avoids switching the target's tracking ID during the tracking process, suppresses multiple shots, improves missed shots, and effectively enhances the accuracy of target tracking.

[0048] This application also provides a target tracking system, a storage medium, and an electronic device, which have the aforementioned beneficial effects, and will not be elaborated further here. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0050] Figure 1 A flowchart illustrating a target tracking method provided in an embodiment of this application;

[0051] Figure 2 A flowchart illustrating the tracking algorithm provided in the embodiments of this application;

[0052] Figure 3 This is a schematic diagram of a target tracking system provided in an embodiment of this application. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0054] Please refer to Figure 1 , Figure 1 A flowchart of a target tracking method provided in this application embodiment, the method comprising:

[0055] S101: Acquire video stream data;

[0056] S102: Detect whether there is an obscured target in the video stream data; if so, proceed to S103;

[0057] S103: Determine whether the occluded target can be effectively detected; if yes, proceed to S104.

[0058] S104: Determine whether the upper boundary of the target detection box of the occluded target has shifted;

[0059] S105: If the upper boundary is displaced, continue tracking of the occluded target;

[0060] S106: If the upper boundary does not shift but the lower boundary moves, a frame locking operation is performed starting from the current frame to track the occluded target; the frame locking operation is used to fix the size and position of the tracking frame.

[0061] There is no limitation on what kind of video stream data is obtained here; it can be historical monitoring video stream data or real-time monitoring video stream data.

[0062] For step S102 in the above process, if there is no occlusion in the video stream data, the target can be normally identified and tracked. Taking a vehicle as an example, if the vehicle is not occluded by pedestrians, other vehicles or objects, the vehicle's features can be normally identified, including but not limited to the identification of information such as license plates, so as to achieve vehicle tracking and identification.

[0063] There are no specific limitations on how to determine the existence of an occluded target. One feasible method is to compare the intersection-over-union (IoU) ratio of the detection boxes of each target in the video stream data, i.e., the degree of overlap between the detection boxes corresponding to each target. Obviously, if the overlap area is large, it can be considered that occlusion exists. Specifically, if there is a video in the video stream data where the IoU ratio between the target detection box and the suspected occlusion detection box is greater than zero, and the y-axis coordinate of the target detection box is greater than the y-axis coordinate of the suspected occlusion detection box, then it can be confirmed that an occluded target exists in the video stream data.

[0064] Specifically, you can refer to the following formula:

[0065] IOU(Area1, Area2)>0&&BotY1 <BotY2;

[0066] Where IOU (intersection of union) is the intersection-to-union ratio, Area1 is the target detection bounding box, and Area2 is the suspected occlusion detection bounding box. The ordinate of the lower boundary of the target vehicle detection bounding box is BotY1, and the ordinate of the lower boundary of the suspected occlusion detection bounding box is BotY2.

[0067] If an occluded target exists, it is then necessary to determine whether the occluded target can be effectively detected. Effective detection refers to whether the occluded target can be detected by commonly used detection models or algorithms, such as whether the bounding box of the occluded target can be detected by the widely used YOLO series of object detection algorithms. When determining whether the occluded target can be effectively detected, a higher confidence level can be used to ensure detection accuracy.

[0068] If an obstruction exists and the target vehicle can be detected, it can be further determined whether the upper boundary of the detection frame of the obstructed target has shifted. The following process can be used in this case:

[0069] Step 1: Determine the starting frame number when the occluded target is in a stationary state;

[0070] The second step is to determine the coordinates of the center point of the target detection box in the video frame corresponding to the starting frame number.

[0071] Third step: If the difference between the coordinates of the center point and the coordinates of the second center point of the target detection box in the next video frame is less than a set threshold, it is determined that the upper boundary of the target detection box of the occluded target has not been displaced.

[0072] Fourth step: If the difference between the coordinates of the center point and the coordinates of the second center point is not less than the set threshold, determine that the upper boundary of the target detection box of the occluded target has shifted.

[0073] The determination method can be shown in the following formula: The starting frame number for the stabilization determination, that is, the starting frame number when the target enters a stationary state: K, where the coordinates of the target bounding box center point in frame K are (X... k Y k ).

[0074] (K,K+1,K+2,...,K+M);

[0075] (X k+1 –X k ) <F&&(Y k+1 –Y k ) <F。

[0076] In the above formula, M and F are both judgment parameters used in determining whether the upper boundary of the detection box of the occluded target has shifted. The specific settings of these judgment parameters are not limited here; they can be obtained by those skilled in the art through testing based on the actual scenario. The selection of F can be determined according to actual needs. In one feasible approach, the preferred parameters are M = 5 and F = 3. However, the judgment parameters are independent of factors such as image resolution and the length and width of the target box. Meeting the above conditions indicates that the upper boundary of the occluded target has not shifted.

[0077] In this embodiment of the application, when an occluded target exists in the video stream data, if the occluded target can be effectively detected, displacement detection is performed on the upper boundary of its target detection box. If both the upper and lower boundaries move, normal tracking is maintained, while a small number of N video frames can be retained, for example, N=3.

[0078] For targets whose upper boundary has not shifted but whose lower boundary has moved, lock their tracking box to maintain tracking of the target, so that the tracking of the target does not change after the occlusion is removed, and maintain the continuity of the entire tracking trajectory of the target; at the same time, retain a preset number of video frames, for example, N=50. If no detection box matching the tracking box is detected after N frames, delete the tracking trajectory of the locked tracking box.

[0079] In this embodiment of the application, when there is an occluded target in the video stream data, if the occluded target can be effectively detected, displacement detection is performed on the upper boundary of its target detection box. For targets whose upper boundary has not shifted but whose lower boundary has moved, its tracking box is fixed to maintain the tracking of the target. This ensures that the tracking of the target does not change after the occluder is removed, maintains the continuity of the entire tracking trajectory of the target, avoids switching the target's tracking ID during the tracking process, can suppress multiple shots and improve missed shots, and effectively improves the accuracy of target tracking.

[0080] Based on the previous embodiment, if the occluded target cannot be detected, it can be further determined whether the occluded target is included in a low-scoring detection frame with a confidence level lower than a preset value. If the occluded target is included in the low-scoring detection frame, tracking continues, and a first number of video frames are retained, for example, N=10. If the occluded target is not included in the low-scoring detection frame either, a frame-locking operation is performed starting from the current frame. The frame-locking operation is used to fix the size and position of the tracking frame, and a preset number of video frames can be retained. If the preset number is greater than the first number, the determination time for tracking loss is appropriately extended, for example, N=50. If no detection frame matching the tracking frame is detected after N frames, the tracking trajectory of the locked tracking frame is deleted.

[0081] This embodiment fully utilizes target information not detected by the detection model in weakly occluded scenes, maintaining the continuity of the entire tracking trajectory and avoiding target ID switching, thus effectively improving the accuracy of vehicle behavior judgment. Furthermore, as seen in the previous embodiment, this application can execute different processing strategies based on the characteristics of weak and strong occlusion scenes, effectively controlling the continuity of the target tracking process. It has excellent effects on suppressing multiple shots and improving missed shots.

[0082] Previously, tracking algorithms could be used to track and detect targets in video stream data, while simultaneously determining detection boxes corresponding to different confidence levels. Higher confidence levels indicate higher detection accuracy. The method for distinguishing between high-scoring and low-scoring detection boxes is not limited here; it can be set by those skilled in the art based on actual detection needs. For example, detection boxes with a confidence level greater than 0.6 can be saved as high-scoring detection boxes, and those greater than 0.2 but less than 0.6 can be saved as low-scoring detection boxes. It should be noted that since detection boxes with a confidence level less than 0.2 have a higher probability of being false positives, the threshold for selecting low-scoring detection boxes is set above 0.2. Therefore, for the same video frame, if the confidence level is low, more targets will be detected. That is, if occluded targets cannot be detected in high-scoring detection boxes, the confidence level can be further reduced to detect occluded targets.

[0083] Based on the above embodiments, in one feasible implementation, a frame locking operation can be performed starting from the current frame. The frame locking operation is used to fix the size and position of the tracking frame, maintaining continuous tracking of the occluded target. It should be noted that after performing the frame locking operation, the removal status of the occluder can be determined in real time, so that when the occluded target reappears after the occluder is removed, matching can be performed again. For targets subjected to "frame locking," only the position does not change; tracking continues. This helps to prevent the tracking ID from changing after the occluder is removed, avoiding target ID switching and effectively improving the accuracy of vehicle behavior judgment.

[0084] The tracking algorithm involved in the above embodiments of this application is described below. See also Figure 2 , Figure 2 This is a flowchart illustrating the tracking algorithm provided in an embodiment of this application. The process includes:

[0085] S201: Match the high-resolution detection bounding box with the tracking trajectory of each target;

[0086] S202: For a tracking trajectory that does not match the high-resolution detection box, after reducing the cost threshold of the cost function, continue to execute the step of matching the tracking trajectory with the high-resolution detection box;

[0087] S203: If, after reducing the cost function threshold in the matching process and continuing to execute the step of matching the high-scoring detection box with the tracking trajectory, there are still unmatched tracking trajectories, the low-scoring detection box is used to match the unmatched tracking trajectories.

[0088] S204: If there is an unknown tracking trajectory that cannot be matched by both the high-resolution detection box and the low-resolution detection box, perform a frame locking operation on a preset number of video frames, and perform matching when the target corresponding to the unknown tracking trajectory appears a second time;

[0089] S205: If no matching target is found after a preset number of video frames, delete the unknown tracking trajectory;

[0090] S206: If there is an unknown high-resolution detection box that does not match the tracking trajectory, create a new tracking trajectory and use the target corresponding to the unknown high-resolution detection box as the new target.

[0091] This embodiment does not limit how to predict the tracking trajectory of a target in video stream data; Kalman filtering algorithms can be used. Similarly, it does not limit how to determine high-resolution and low-resolution detection boxes, which can be determined by those skilled in the art based on actual detection needs. For example, detection boxes with a confidence score greater than 0.6 can be saved as high-resolution detection boxes, and those greater than 0.2 but less than 0.6 can be saved as low-resolution detection boxes. Each target in the video stream data corresponds to its own tracking trajectory. High-resolution detection boxes are preferentially used for tracking trajectory matching, while low-resolution detection boxes are mainly used for matching when strong occlusion occurs. When matching high-resolution or low-resolution detection boxes with the tracking trajectory, it is actually a matching between the detection box and the predicted box; that is, the center points of each predicted box constitute the tracking trajectory.

[0092] It should also be noted that a lower threshold can be set for the cost function. That is, after the cost function is reduced to the lower threshold, it will no longer be reduced, and the low-scoring detection box will be used directly to track the trajectory.

[0093] As can be seen, after matching the tracking trajectory using high-resolution and low-resolution detection boxes respectively, for unknown high-resolution detection boxes with high confidence but no matching trajectory, a new tracking trajectory is created for them, treating them as new targets. This situation usually occurs when a new target appears in the video stream data. For unknown tracking trajectories without matching detection boxes, a frame-locking operation is performed on a preset number of video frames. The purpose is to perform tracking trajectory matching again when they reappear. This situation usually occurs when the target leaves the frame corresponding to the video stream data.

[0094] The value of N can be set according to the specific scenario. If the bounding box operation is performed from the current frame, it can be used to fix the size and position of the tracking box. In this case, the value of N can be set to a larger value, such as N=50, to appropriately extend the tracking loss time and prevent the target ID from being replaced too quickly during the tracking process. It should also be noted that because the threshold of the target detection algorithm is lowered, many more detection boxes will be generated, including false positives. Therefore, low-scoring detection boxes can only be used for maintaining historical tracking trajectories and not for creating new targets. That is, if no match is successfully found in a low-scoring detection box, all low-scoring detection boxes can be deleted.

[0095] The cost function mentioned above is not limited here, and it may include the following steps:

[0096] Step 1: Record the center point position information of the detection box corresponding to the target;

[0097] The second step is to fit the center point position information to obtain the detection box trajectory.

[0098] The third step is to use a tracking prediction algorithm to predict the trajectory of the target and obtain the predicted bounding box trajectory.

[0099] Step 4: Determine the angle between the detection box trajectory and the prediction box trajectory;

[0100] Step 5: Construct a cost function based on the included angle; the cost function serves as the matching standard between the detection box and the tracking trajectory.

[0101] This application provides a feasible cost function in its embodiments:

[0102]

[0103] in:

[0104]

[0105]

[0106] In the above formula, ρ 2 (b d bp ) represents the Euclidean distance between the center points of the detection box and the predicted box, c represents the area of ​​the bounding rectangle of the union of the detection box and the predicted box, v represents the aspect ratio similarity, h represents the height, w represents the width, and α represents the weight parameters constructed using IOU and v, w d w represents the width of the detection box. p The width of the predicted bounding box is represented by θ, and the height has a similar meaning. θ represents the angle between the detection box trajectory and the predicted bounding box trajectory. The trajectory used is obtained by fitting three consecutive frames.

[0107] The angle between the detection box trajectory and the predicted box trajectory can be obtained by continuously recording the position of the center point of the vehicle detection box after obtaining the vehicle detection box, and then fitting a polynomial to the vehicle's motion trajectory. Substituting the coordinates of the vehicle detection box's center point into the simplified equation, and solving the system of equations, the coefficients of the polynomial are obtained, thus yielding the least squares fitting polynomial, which can be used as the vehicle's motion trajectory. The specific mathematical calculation process can be determined as follows:

[0108] Given a function y = f(x) at points x1, x2, ..., xn. n The function values ​​at points y1, y2, ..., y n Find a polynomial to fit the target's trajectory. Substitute the coordinates of the target detection box center point into the polynomial, such that:

[0109]

[0110] set up

[0111] but

[0112]

[0113] After rearranging the equations, we solve the system of equations to obtain a0 and a1, thus obtaining the least-squares fitting polynomial of f(x):

[0114] f(x) = a0 + a1x

[0115] The target's tracking trajectory can then be obtained, allowing determination of the angle between the detected bounding box trajectory and the predicted bounding box trajectory. A smaller angle indicates a more accurate prediction; incorporating this angle into the cost function effectively prevents tracking box drift when the target is occluded. This application's embodiments recover missed targets by lowering the detection threshold, ensuring the continuity of the tracking trajectory. Furthermore, the target tracking optimization algorithm, by incorporating the target motion angle into the cost function, improves matching accuracy and effectively enhances the target capture rate. Simultaneously, it differentiates between thresholds for tracking loss in different scenarios and provides fixed protection for tracking boxes in scenarios with strong occlusion.

[0116] If the target mentioned above is a vehicle, i.e., applied to a smart parking scenario, this application can solve the problem of multiple shots caused by the upward movement of the target detection box. The multiple shots caused by the upward movement of the target detection box mainly include the following two situations: a target outside the parking space is obscured by a vehicle behind it, causing the lower boundary of the target vehicle detection box to move upward, resulting in multiple shots of the target entering the parking space; or the target inside the parking space is moved away by opening the trunk, causing the lower boundary of the target vehicle detection box to move upward. Simultaneously, this application can also solve the problem of multiple shots caused by tracking box drift due to strong occlusion. The problem of multiple shots caused by tracking box drift due to strong occlusion mainly includes the following two situations: a parked vehicle at an intersection is obscured by a passing vehicle, causing the tracking box to drift and resulting in multiple shots; or a large bus passing by outside the parking space strongly obscures the target inside the parking space, causing the tracking box to drift and resulting in multiple shots. This application performs a frame-locking operation starting from the current frame when occlusion is detected. This frame-locking operation fixes the size and position of the tracking frame to track the occluded target. This not only ensures that the position of the tracking frame does not change drastically but also that the state of the target frame remains unchanged, avoiding multiple shots. Simultaneously, this application also avoids missed shots caused by strong occlusion from large vehicles near parking spaces. Strong occlusion from large vehicles near parking spaces interrupts the tracking trajectory of a target vehicle leaving the parking space, making it impossible to determine the target's actual behavior and resulting in missed shots. This application, by maintaining the continuity of the entire tracking trajectory and avoiding target ID switching during tracking, effectively improves the accuracy of vehicle behavior judgment, effectively solving the occlusion problem in smart parking scenarios. It not only reduces multiple shots but also improves missed shots, demonstrating strong robustness.

[0117] The target tracking system provided in the embodiments of this application is described below. The target tracking system described below can be referred to in correspondence with the target tracking method described above.

[0118] See Figure 3 , Figure 3 This application provides a schematic diagram of a target tracking system structure according to an embodiment of the present application. The present application also provides a target tracking system, including:

[0119] The video data acquisition module is used to acquire video stream data;

[0120] An occlusion detection module is used to detect whether there is an occluded target in the video stream data;

[0121] An occlusion detection module is used to determine whether the occluded target can be effectively detected when the occlusion judgment module determines that the occlusion judgment result is yes.

[0122] The displacement detection module is used to determine whether the upper boundary of the target detection box of the occluded target has been displaced when the occlusion detection module determines that the result is yes.

[0123] The tracking module is used to maintain tracking of the occluded target if the upper boundary is displaced; if the upper boundary is not displaced but the lower boundary moves, a frame locking operation is performed starting from the current frame to track the occluded target; the frame locking operation is used to fix the size and position of the tracking frame.

[0124] Based on the above embodiments, as a preferred embodiment, the occlusion determination module includes:

[0125] An occlusion determination unit is used to determine whether there is a video frame in the video stream data where the intersection-union ratio between the target detection box and the suspected occlusion detection box is greater than zero, and the vertical axis coordinate of the lower boundary of the target detection box is less than the vertical axis coordinate of the lower boundary of the suspected occlusion detection box; if so, it is confirmed that there is an occluded target in the video stream data.

[0126] Based on the above embodiments, as a preferred embodiment, the displacement detection module includes:

[0127] A frame number determination unit is used to determine the starting frame number when the occluded target is in a stationary state;

[0128] A coordinate determination unit is used to determine the coordinates of the center point of the target detection box in the video frame corresponding to the starting frame number;

[0129] The displacement detection unit is used to determine that the upper boundary of the target detection box of the occluded target has not been displaced if the difference between the coordinates of the center point and the coordinates of the second center point of the target detection box in the next video frame is less than a set threshold; and to determine that the upper boundary of the target detection box of the occluded target has been displaced if the difference between the coordinates of the center point and the coordinates of the second center point is not less than the set threshold.

[0130] Based on the above embodiments, as a preferred embodiment, it further includes:

[0131] The secondary detection module is used to determine whether the occluded target is included in the low-score detection box with a confidence level lower than a preset value if the occluded target cannot be detected.

[0132] The first locking frame module is used to continue tracking the occluded target if it contains the occluded target;

[0133] The second frame-locking module is used to perform the frame-locking operation starting from the current frame if the low-resolution detection frame does not contain the occluded target.

[0134] Based on the above embodiments, as a preferred embodiment, it further includes:

[0135] The detection box setting module is used to perform target detection on the video stream data, and to designate detection boxes with a confidence level greater than the preset value as high-scoring detection boxes, and to designate detection boxes with a confidence level lower than the preset value as low-scoring detection boxes.

[0136] Based on the above embodiments, as a preferred embodiment, it further includes:

[0137] The cost function setting module is used to record the center point position information of the detection box corresponding to the target; to fit the center point position information to obtain the detection box trajectory; to use a tracking prediction algorithm to predict the trajectory of the target to obtain the predicted box trajectory; to determine the angle between the detection box trajectory and the predicted box trajectory; to construct a cost function based on the angle; and to use the cost function as a matching standard between the detection box and the tracking trajectory.

[0138] Based on the above embodiments, as a preferred embodiment, it further includes:

[0139] The target tracking module is used to match the high-resolution detection boxes with the tracking trajectories of each target. For tracking trajectories that do not match the high-resolution detection boxes, the cost threshold of the cost function is reduced, and the step of matching the high-resolution detection boxes with the tracking trajectories continues. If, after reducing the cost function threshold during the matching process and continuing the step of matching the high-resolution detection boxes with the tracking trajectories, there are still unmatched tracking trajectories, the low-resolution detection boxes are used to match the unmatched tracking trajectories. If there are unknown tracking trajectories that cannot be matched by either the high-resolution or low-resolution detection boxes, a frame-locking operation is performed on a preset number of video frames, and matching is performed when the target corresponding to the unknown tracking trajectory appears a second time. If there are unknown high-resolution detection boxes that do not match the tracking trajectory, a new tracking trajectory is created, and the target corresponding to the unknown high-resolution detection box is added as a new target.

[0140] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed, can perform the steps provided in the above embodiments. The storage medium may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0141] This application also provides an electronic device that may include a memory and a processor. The memory stores a computer program, and when the processor calls the computer program in the memory, it can implement the steps provided in the above embodiments. Of course, the electronic device may also include various network interfaces, power supplies, and other components.

[0142] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. As the system provided in the embodiments corresponds to the method provided in the embodiments, the description is relatively simple; relevant parts can be found in the method section.

[0143] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

[0144] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A target tracking method, characterized in that, include: Acquire video stream data; Detect whether there is an obscured target in the video stream data; If so, determine whether the occluded target can be effectively detected; If the occluded target is effectively detected, determine whether the upper boundary of the target detection box of the occluded target has shifted; If the upper boundary is displaced, continue tracking of the occluded target; If the upper boundary does not shift but the lower boundary moves, a frame locking operation is performed starting from the current frame to track the occluded target; the frame locking operation is used to fix the size and position of the tracking frame.

2. The target tracking method according to claim 1, characterized in that, Detecting whether there is an occluded target in the video stream data includes: Determine whether there exists a video frame in the video stream data where the intersection-union ratio between the target detection box and the suspected occlusion detection box is greater than zero, and the ordinate of the lower boundary of the target detection box is less than the ordinate of the lower boundary of the suspected occlusion detection box. If so, it confirms that there is an obscured target in the video stream data.

3. The target tracking method according to claim 1, characterized in that, Determining whether the upper boundary of the target detection box of an occluded target has shifted includes: Determine the starting frame number in which the occluded target is in a stationary state; Determine the coordinates of the center point of the target detection box in the video frame corresponding to the starting frame number; If the difference between the coordinates of the center point and the coordinates of the second center point of the target detection box in the next video frame is less than a set threshold, it is determined that the upper boundary of the target detection box of the occluded target has not shifted. If the difference between the coordinates of the center point and the coordinates of the second center point is not less than the set threshold, it is determined that the upper boundary of the target detection box of the occluded target has shifted.

4. The target tracking method according to claim 1, characterized in that, After determining whether the occluded target can be effectively detected, the method further includes: If the occluded target cannot be detected, determine whether the occluded target is included in the low-score detection box with a confidence level lower than a preset value; If the occluded target is included, continue tracking the occluded target; If the occluded target is not included in the low-resolution detection frame, the frame locking operation is performed starting from the current frame.

5. The target tracking method according to claim 4, characterized in that, Before determining whether the occluded target is contained in a low-score detection box with a confidence level below a preset value, the method further includes: Target detection is performed on the video stream data. Detection boxes with a confidence level greater than the preset value are designated as high-scoring detection boxes, and detection boxes with a confidence level lower than the preset value are designated as low-scoring detection boxes.

6. The target tracking method according to claim 5, characterized in that, After acquiring the video stream data, the following is also included: Record the position information of the center point of the detection box corresponding to the target; The detection box trajectory is obtained by fitting the center point position information; The trajectory of the target is obtained by using a tracking prediction algorithm to predict its trajectory; Determine the angle between the detection box trajectory and the predicted box trajectory; A cost function is constructed based on the included angle; the cost function serves as the matching criterion between the detection box and the tracking trajectory.

7. The target tracking method according to claim 6, characterized in that, After fitting the detection box trajectory based on the center point position information, the method further includes: The high-resolution detection box is matched with the tracking trajectory of each target. For a tracking trajectory that does not match the high-resolution detection box, after reducing the cost threshold of the cost function, the step of matching the tracking trajectory with the high-resolution detection box is continued. If, after lowering the cost function threshold during the matching process and continuing to perform the step of matching the high-scoring detection box with the tracking trajectory, there are still unmatched tracking trajectories, then the low-scoring detection box is used to match the unmatched tracking trajectories. If there is an unknown tracking trajectory that cannot be matched by both the high-resolution detection box and the low-resolution detection box, a frame locking operation is performed on a preset number of video frames, and a match is performed when the target corresponding to the unknown tracking trajectory appears a second time. If no matching target is found after the preset number of video frames, the unknown tracking trajectory is deleted. If there is an unknown high-resolution detection box that does not match the tracking trajectory, a new tracking trajectory is created, and the target corresponding to the unknown high-resolution detection box is added as a new target.

8. A target tracking system, characterized in that, include: The video data acquisition module is used to acquire video stream data; An occlusion detection module is used to detect whether there is an occluded target in the video stream data; An occlusion detection module is used to determine whether the occluded target can be effectively detected when the occlusion judgment module determines that the occlusion judgment result is yes. The displacement detection module is used to determine whether the upper boundary of the target detection box of the occluded target has been displaced when the occlusion detection module determines that the result is yes. A tracking module is used to maintain tracking of the occluded target if the upper boundary is displaced; If the upper boundary does not shift but the lower boundary moves, a frame locking operation is performed starting from the current frame to track the occluded target; the frame locking operation is used to fix the size and position of the tracking frame.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the target tracking method as described in any one of claims 1-7.

10. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and the processor, when calling the computer program in the memory, implements the steps of the target tracking method as described in any one of claims 1-7.

Images