Target tracking detection method and device based on dynamic score and time window adjustment

Abstract

The application discloses a target tracking detection method and device based on dynamic score and time window adjustment, and relates to the field of data processing. In the method, target detection is performed on a current frame to obtain a plurality of detection objects; tracking is performed on each detection object, and a tracking ID is assigned; target detection objects with the same tracking ID in the current frame and a previous frame of the current frame are determined, the motion speed of the target detection objects is calculated, and a motion speed score is generated; a confidence score is calculated for each tracking ID; the motion speed score and the confidence score are fused to obtain a comprehensive dynamic score, and a target time window is obtained according to the comprehensive dynamic score; whether the cumulative number of frames is greater than or equal to the target time window is judged; if yes, target detection results corresponding to the tracking IDs in the target time window are obtained, and are subjected to convergence processing to generate object detection results. The technical scheme provided by the application improves the adaptability of target tracking.

Description

Technical Field

[0001] This application relates to the field of data processing, specifically to a target tracking and detection method and apparatus based on dynamic scoring and time window adjustment. Background Technology

[0002] With its widespread application in computer vision fields such as video surveillance, autonomous driving, and intelligent transportation, video target detection and tracking are core technologies for achieving scene understanding and intelligent decision-making. The fundamental task is to locate and identify targets of interest (such as pedestrians and vehicles) in real-time and accurately across consecutive video frames. However, detection decisions based on single-frame images are highly susceptible to interference from various factors in complex real-world environments, such as sensor noise and drastic changes in lighting conditions. These factors directly lead to the instability of detection results.

[0003] To address the instability of single-frame detection, existing technologies commonly employ a decision stabilization method based on multi-frame information fusion. This method typically introduces a fixed-length time window, and makes a comprehensive decision by aggregating and analyzing the detection results of multiple consecutive frames within this time window. For example, the system only confirms the existence of a target and outputs a result when the target is consistently detected in most frames within the window. This temporal filtering mechanism effectively smooths the detection results, filters out instantaneous and sporadic false alarms, and compensates for missed alarms caused by brief occlusions, thereby significantly improving the stability and continuity of the detection results and reducing meaningless alarm jitter.

[0004] However, the aforementioned stabilization method using a fixed time window has significant problems in terms of adaptability. For a target moving rapidly in the scene, a short fixed window may not be sufficient to collect enough information to confirm its trajectory and presence, easily leading to tracking failures or missed detections. Conversely, for a target that remains stationary for a long time or moves slowly, a long fixed window introduces unnecessary processing delays, reducing the system's response speed to target appearance or disappearance events. This fixed strategy cannot guarantee both detection stability and response speed, resulting in low adaptability for target tracking. Summary of the Invention

[0005] This application provides a target tracking and detection method and apparatus based on dynamic scoring and time window adjustment, which improves the adaptability of target tracking.

[0006] A first aspect of this application provides a target tracking and detection method based on dynamic scoring and time window adjustment. The method includes: performing target detection on a current frame to obtain a detection result containing multiple detected objects; tracking each detected object in the detection result and assigning a tracking ID to each detected object; identifying target detected objects with the same tracking ID in the current frame and the previous frame, calculating the motion speed of the target detected objects, and generating a motion speed score based on the motion speed; creating a corresponding confidence history sequence for each tracking ID, and calculating a confidence score based on the statistical characteristics of the confidence history sequence; fusing the motion speed score and the confidence score to obtain a comprehensive dynamic score, and adjusting the time window size based on the comprehensive dynamic score to obtain a target time window; determining whether the cumulative number of processed frames is greater than or equal to the target time window; if the cumulative number of processed frames is greater than or equal to the target time window, obtaining the target detection results corresponding to each tracking ID within the target time window, and performing convergence processing on the target detection results based on the tracking ID to generate object detection results. This application dynamically adjusts the time window size by calculating the target's motion speed score and confidence score. This allows for a longer time window for fast-moving targets to ensure stability, while shortening the time window for stationary or slow-moving targets to improve response speed. Simultaneously, the aggregation and processing of detection results based on tracking IDs further enhances detection stability and accuracy. This technical solution significantly improves the system's response speed and adaptability while maintaining detection stability, meeting the practical needs of complex scenarios.

[0007] By employing the above technical solution, target detection is performed on the current frame, resulting in detection results containing multiple objects. Each detected object is tracked, and a tracking ID is assigned to each object, achieving continuous tracking and identification of targets in the video sequence. By identifying target objects with the same tracking ID in the current and previous frames, the motion velocity of the target objects is calculated, and a motion velocity score is generated, achieving a quantitative assessment of the target's motion state. By creating a corresponding confidence history sequence for each tracking ID and calculating a confidence score based on the statistical characteristics of the confidence history sequence, dynamic evaluation of tracking confidence is achieved. By fusing the motion velocity score and the confidence score, a comprehensive dynamic score is obtained, and the time window size is adjusted according to the comprehensive dynamic score, achieving adaptive adjustment of the detection time window based on the comprehensive dynamic characteristics of the target, improving the timeliness and accuracy of tracking detection. By determining whether the cumulative number of processed frames has reached the target time window, and acquiring the detection results within the target time window for aggregation processing when it has, periodic output and updates of the tracked target are achieved, improving the real-time performance and continuity of the tracking detection results.

[0008] Optionally, creating a corresponding confidence history sequence for each tracking ID specifically includes: obtaining the detection confidence of the target detection object corresponding to the tracking ID in multiple consecutive frames, where the detection confidence represents the probability that the target detection object is a real target; maintaining a confidence queue of a preset fixed length for each tracking ID; when the detection confidence of the target detection object in the current frame is obtained, inserting the detection confidence into the tail of the confidence queue and removing the detection confidence at the head of the confidence queue to keep the length of the confidence queue unchanged.

[0009] By employing the above technical solution, the detection confidence of the target object corresponding to the tracking ID is obtained in multiple consecutive frames. A fixed-length confidence queue is maintained for each tracking ID, enabling continuous recording and updating of target tracking confidence. Dynamic updates to the confidence queue are achieved by inserting the detection confidence of the current frame at the tail of the queue and removing the detection confidence from the head of the queue, while maintaining a fixed queue length and preventing unlimited data growth. Maintaining the confidence queue enables dynamic evaluation of target tracking quality, providing a data foundation for subsequent confidence score calculations and improving the timeliness and continuity of confidence evaluation. Simultaneously, the fixed-length confidence queue limits the time scale of confidence evaluation, preventing the infinite accumulation of historical data and reducing computational complexity and storage overhead.

[0010] Optionally, the step of calculating the confidence score based on the statistical characteristics of the confidence history sequence specifically includes: calculating the mean of the confidence history sequence as the confidence amplitude for evaluating the confidence level; calculating the coefficient of variation of the confidence history sequence, where the coefficient of variation is the ratio of the standard deviation to the mean of the confidence history sequence, and generating a confidence stability for evaluating the stability of the confidence based on the coefficient of variation; performing a weighted summation of the confidence amplitude and the confidence stability based on preset weights to obtain the individual confidence score for each tracking ID; and calculating the average of the individual confidence scores for all tracking IDs in the current frame as the confidence score.

[0011] By employing the above technical solutions, the mean of the historical confidence sequence is calculated as the confidence amplitude, enabling an assessment of the overall level of target tracking confidence and reflecting the quality of tracking. The coefficient of variation of the historical confidence sequence is calculated as the confidence stability, assessing the stability of target tracking confidence and reflecting fluctuations in confidence during tracking. A weighted sum of the confidence amplitude and confidence stability yields an individual confidence score, providing a comprehensive assessment of confidence level and stability, fully reflecting the tracking quality of a single target. The average of the individual confidence scores for all tracking IDs in the current frame is calculated to obtain the overall confidence score, providing a holistic assessment of the tracking quality of all targets at the current moment and reflecting the overall quality of the current tracking state. The calculation of the confidence score provides a quantitative indicator for subsequent comprehensive dynamic evaluation, improving the adaptability and robustness of the tracking detection process.

[0012] Optionally, adjusting the time window size based on the comprehensive dynamic score to obtain the target time window specifically includes: setting an upper and lower limit for the time window size and calculating an adjustment step size based on the comprehensive dynamic score; obtaining the initial time window size corresponding to the previous frame of the current frame; comparing the comprehensive dynamic score with a first preset score threshold and a second preset score threshold, wherein the first preset score threshold is greater than the second preset score threshold; if it is determined that the comprehensive dynamic score is greater than the first preset score threshold, then reducing the initial time window size by the adjustment step size to obtain the target time window; if it is determined that the comprehensive dynamic score is less than the second preset score threshold, then increasing the initial time window size by the adjustment step size to obtain the target time window; if it is determined that the comprehensive dynamic score is greater than or equal to the second preset score threshold and the comprehensive dynamic score is less than or equal to the first preset score threshold, then maintaining the initial time window size to obtain the target time window.

[0013] By adopting the above technical solution, upper and lower limits for the time window size are set, and multiple dynamic score thresholds are introduced to divide the target's comprehensive dynamic score into different intervals. Based on the interval in which the comprehensive dynamic score falls, the step size is dynamically calculated and adjusted to achieve adaptive adjustment of the time window size. When the comprehensive dynamic score is high, it indicates that the target's movement is stable and the detection confidence is stable; the time window is reduced to improve response speed. When the comprehensive dynamic score is low, it indicates that the target's movement is violent or the detection confidence fluctuates greatly; the time window is increased to improve detection stability. When the comprehensive dynamic score is at a moderate level, the current time window remains unchanged. By introducing an adaptive time window adjustment mechanism driven by the comprehensive dynamic score, this invention can flexibly adjust the spatiotemporal scale according to the target's actual motion state and detection quality, improving response speed while ensuring detection stability. This overcomes the shortcomings of the fixed time window method in the prior art and improves the adaptability and real-time performance of tracking and detection.

[0014] Optionally, the target detection result includes the detection confidence score, category label, and bounding box coordinates of the target object. The aggregation processing of the target detection results based on the tracking ID to generate the object detection result specifically includes: grouping target detection results with the same tracking ID within the current time window into a target detection result group; for each target detection result group, counting the frequency of occurrence of each category label, and using the category label with the highest frequency as the output category for the corresponding tracking ID; performing a weighted average of the detection confidence scores of the target detection results within each target detection result group, and using this as the output confidence score for the corresponding tracking ID; performing a weighted average of the bounding box coordinates of the target detection results within each target detection result group, and using this as the output bounding box for the corresponding tracking ID; and generating the object detection result based on the output category, the output confidence score, and the output bounding box.

[0015] By employing the above technical solution, target detection results with the same tracking ID are grouped into a single target detection result group, enabling the categorization and aggregation of multiple detection results for the same target within a time window. By statistically analyzing the frequency of each category label within each target detection result group and selecting the category label with the highest frequency as the output category, a voting fusion of target categories is achieved, improving the reliability of category judgment. By weighted averaging the detection confidence scores within each target detection result group, an output confidence score is obtained, achieving a comprehensive evaluation of target confidence and improving the accuracy of confidence estimation. By weighted averaging the bounding box coordinates within each target detection result group, an output bounding box is obtained, achieving smooth optimization of target position and improving the stability of position estimation. Through convergence processing, multiple detection results within the time window are merged into a comprehensive detection output, enabling the updating and optimization of the target tracking status and improving the accuracy, stability, and continuity of tracking detection results.

[0016] Optionally, after tracking each of the detected objects in the detection results and assigning a tracking ID to each detected object, the method further includes: detecting whether the bounding box of the first detected object overlaps with the bounding box of the second detected object, wherein the first detected object and the second detected object are any two of the multiple detected objects and the detection confidence of the second detected object is higher than the detection confidence of the first detected object; if the overlapping area exceeds a preset overlap threshold, then updating the state of the first tracking ID corresponding to the first detected object to an occlusion state; if a second tracking ID exists, then updating the state of the second tracking ID to a prediction loss state, wherein the second tracking ID is a tracking ID for which no corresponding third detected object is found in the current frame; and processing the first detected object corresponding to the first tracking ID and the third detected object corresponding to the second tracking ID.

[0017] By employing the above technical solutions, the detection and quantification of target occlusion are achieved by detecting whether the bounding boxes of different detected objects overlap and determining whether the overlapping area exceeds a preset threshold. By updating the state of the tracking ID corresponding to the occluded detected object to the occlusion state, state labeling and tracking management of occluded targets are realized. By updating the state of the tracking ID for which no corresponding detected object is found in the current frame to the predicted loss state, early warning and state labeling for possible target loss are achieved. Specialized processing of occluded targets and predicted loss targets enables target state management and tracking strategy adjustment in complex tracking environments, improving the adaptability and robustness of the tracking detection algorithm. The introduction of occlusion and predicted loss states provides necessary state information for subsequent target re-identification and trajectory prediction, supporting the continuity and stability of the tracking process.

[0018] Optionally, the processing of the first detection object corresponding to the first tracking ID and the third detection object corresponding to the second tracking ID specifically includes: obtaining the historical confidence of the first detection object, and determining the detection confidence of the first detection object in the current frame based on the historical confidence; obtaining the historical position and historical velocity of the third detection object; predicting the virtual position of the third detection object in the current frame based on the historical position, the historical velocity, and a preset motion model; if a new detection object is associated with the second tracking ID in subsequent frames within a preset time period, then the re-identification is determined to be successful, and the state of the second tracking ID is restored to the tracked state; calculating the motion velocity corresponding to the second tracking ID based on the virtual position and the position of the new detection object in the subsequent frames.

[0019] By employing the above technical solutions, the historical confidence level of occluded targets is obtained, and the detection confidence level of occluded targets in the current frame is determined based on the historical confidence level. This achieves reasonable estimation and assignment of confidence levels under target occlusion, maintaining the continuity of confidence assessment. By obtaining the historical position and historical velocity of the predicted lost target, and predicting the virtual position of the target in the current frame based on historical information and a preset motion model, position prediction and compensation under target loss are achieved, providing position priors for subsequent target re-identification. By monitoring whether new detected objects are associated with the tracking ID of the predicted lost target in subsequent frames, the success of re-identification is determined, achieving continuous tracking and state recovery after target loss. By calculating the motion velocity of the predicted lost target based on the virtual position and the re-identified position, reasonable estimation and updating of the motion state during the brief loss of the target are achieved, maintaining the continuity of motion velocity assessment. Through specialized processing of occluded targets and predicted lost targets, the adaptability and recovery capability of the tracking detection algorithm in complex tracking environments are improved, enhancing the continuity and smoothness of the tracking trajectory.

[0020] Secondly, embodiments of this application provide a target tracking and detection device based on dynamic scoring and time window adjustment. The target tracking and detection device based on dynamic scoring and time window adjustment includes: one or more processors and a memory; the memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the target tracking and detection device based on dynamic scoring and time window adjustment to perform the method described in the first aspect and any possible implementation of the first aspect.

[0021] Thirdly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a target tracking and detection device based on dynamic scoring and time window adjustment, cause the target tracking and detection device based on dynamic scoring and time window adjustment to perform the method described in the first aspect and any possible implementation thereof.

[0022] Fourthly, embodiments of this application provide a computer program product containing instructions that, when the computer program product is run on a target tracking and detection device based on dynamic scoring and time window adjustment, causes the target tracking and detection device based on dynamic scoring and time window adjustment to perform the method described in the first aspect and any possible implementation thereof.

[0023] In summary, one or more technical solutions provided in this application have at least the following technical effects or advantages:

[0024] 1. This technical solution addresses the problem in traditional video target detection and tracking where fixed time windows cannot adapt to dynamic changes in target movement speed and detection confidence by employing dynamic scoring and time window adjustment. Through a scoring mechanism based on movement speed and confidence, the time window size is adaptively adjusted, thereby achieving effective aggregation and optimization of target detection results and improving detection stability, response speed, and adaptability.

[0025] 2. This technical solution constructs a historical confidence sequence, calculates a confidence score by combining the mean and coefficient of variation, and integrates it with the motion speed score to dynamically adjust the time window size. Through this mechanism, a longer time window can be allocated to fast-moving targets to enhance stability, while a shorter time window can be allocated to stationary or slowly moving targets to improve response speed. Furthermore, through the aggregation processing of target detection results, using operations such as category label frequency statistics, confidence-weighted averaging, and bounding box weighted averaging, the accuracy and consistency of the system's target detection results are further improved.

[0026] 3. This technical solution achieves stable target detection and tracking in complex scenarios, significantly reducing false alarms, missed alarms, and alarm jitter, and improving the system's ability to handle occluded or lost targets. Simultaneously, its dynamic adjustment mechanism achieves a balance between detection stability and response speed, adapting to the dynamic changes in multiple targets and scenarios, providing strong technical support for applications such as video surveillance and autonomous driving. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating the target tracking and detection method based on dynamic scoring and time window adjustment disclosed in the embodiments of this application;

[0028] Figure 2This is another flowchart illustrating the target tracking and detection method based on dynamic scoring and time window adjustment disclosed in the embodiments of this application;

[0029] Figure 3 This is a schematic diagram of the target tracking and detection method device based on dynamic scoring and time window adjustment provided in the embodiments of this application.

[0030] Explanation of reference numerals in the attached drawings: 301, Central Processing Unit; 302, Read-Only Memory; 303, Random Access Memory; 304, Bus; 305, Input / Output Interface; 306, Input Section; 307, Output Section; 308, Storage Section; 309, Communication Section; 310, Driver; 311, Removable Media. Detailed Implementation

[0031] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0032] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0033] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple system devices refer to two or more system devices, and multiple screen terminals refer to two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0034] This application provides a target tracking and detection method based on dynamic scoring and time window adjustment, referring to... Figure 1 , Figure 1This is a flowchart illustrating the target tracking and detection method based on dynamic scoring and time window adjustment provided in this application embodiment. The method is applied to a device, which is a server. The server can be a single server, a server cluster consisting of multiple servers, or a cloud computing service center. The server can execute a target tracking and detection program based on dynamic scoring and time window adjustment. The method includes steps S101 to S107, as follows:

[0035] Step S101: Perform target detection on the current frame to obtain detection results containing multiple detection objects.

[0036] In step S101, the current frame represents a single still image extracted from the video sequence. Object detection refers to the process of locating and identifying objects of interest in the current frame. A detected object refers to a specific instance identified through the object detection process, such as a car or a pedestrian. The detection result refers to a set containing all information about all detected objects in the current frame.

[0037] Specifically, the server receives a real-time or pre-stored video stream and decodes a specific frame of image data, which is the current frame. A pre-trained deep learning object detection model, such as the YOLO series, SSD, or Faster R-CNN, is deployed on the server. The server inputs the image data of the current frame into this object detection model for forward inference computation. After processing, the model outputs a series of detection information. For each identified potential object in the image, the model outputs bounding box coordinates indicating the object's position and size in the image, a class label indicating the object's class, and a detection confidence score quantifying the probability that the detection is a real object. The server integrates the bounding box coordinates, class labels, and detection confidence scores of all detected objects in the current frame to form a structured dataset, which is the detection result, for use in subsequent steps.

[0038] Step S102: Track each detected object in the detection results and assign a tracking ID to each detected object.

[0039] In step S102, tracking refers to the process of associating the same physical entity across consecutive video frames. A tracking ID is a unique identifier assigned to the same physical entity within the video sequence.

[0040] Specifically, after processing the detection results of the current frame, the server starts a multi-target tracker. This tracker maintains a list of tracking IDs for all currently active targets. For each detected object in the detection results generated in step S101, the tracker attempts to match the detected object with a tracking ID that already exists in the previous frame. The matching criteria typically include position prediction and appearance similarity. For position prediction, the tracker can use state estimation algorithms such as Kalman filters to predict the possible position of each known tracking ID in the current frame, and then calculate the overlap between the predicted position and the position of the detected object in the current frame. For appearance similarity, the tracker extracts the depth features of the image patch of the detected object and compares them with the appearance features of the existing tracking IDs. If a detected object has the highest matching score with an existing tracking ID and exceeds a preset threshold, the server assigns that existing tracking ID to the detected object. If a detected object cannot be successfully matched with any existing tracking ID, the server determines that the detected object is a newly appearing target and assigns it a new, unique tracking ID.

[0041] In one possible implementation, after tracking each detected object in the detection results and assigning a tracking ID to each detected object, the method further includes steps S201-S208, as follows:

[0042] Step S201: Detect whether the bounding box of the first detection object overlaps with the bounding box of the second detection object. The first detection object and the second detection object are any two detection objects among a plurality of detection objects, and the detection confidence of the second detection object is higher than the detection confidence of the first detection object.

[0043] In step S201, the detected object refers to the target instance identified by the target detection algorithm in a single frame image; the first detected object and the second detected object refer to two different detected objects arbitrarily selected from all detected objects in the same frame image, wherein the detection confidence of the second detected object is greater than the detection confidence of the first detected object; the bounding box refers to the rectangular coordinates used to mark the position and range of the detected object on the image; the detection confidence refers to the algorithm's confidence score that the detection result is a correct target; overlap means that the bounding boxes of the two detected objects have an intersection in the image space.

[0044] Specifically, when processing the detection results of the current frame, the server performs a pairwise comparison process. The server iterates through all pairs of detected objects. For each pair, the server first compares the detection confidence scores of the two objects. Based on the comparison results, the object with the lower confidence score is designated as the first detected object, and the object with the higher confidence score is designated as the second detected object. Subsequently, the server obtains the bounding box coordinates of the first and second detected objects, respectively. By comparing the coordinate values ​​of these two bounding boxes, the server calculates the length of the intersection interval between the two rectangles in the horizontal and vertical directions. If the intersection interval length in both directions is greater than zero, the two bounding boxes are determined to overlap, and this pair of overlapping detected objects is recorded for use in subsequent steps. This process continues until all possible detected objects have been checked.

[0045] Step S202: If the overlapping area exceeds the preset overlap threshold, the status of the first tracking ID corresponding to the first detection object is updated to the occlusion status.

[0046] In step S202, the overlap area refers to the area of ​​the intersection region between the bounding box of the first detected object and the bounding box of the second detected object on the image; the preset overlap threshold is a pre-set value used to determine whether the degree of overlap is significant; the first tracking ID refers to the unique identifier assigned to the first detected object; the state refers to the label used to describe the current tracking stage of a tracking ID; the occlusion state is a specific state that indicates that the object associated with the tracking ID is being occluded by other objects.

[0047] Specifically, for each pair of detected objects determined to have overlapping bounding boxes in step S201, the server performs a further quantitative analysis. The server first calculates the specific value of the overlapping area. Then, for standardized comparison, the server typically calculates an overlap ratio, for example, dividing the overlapping area by the total area of ​​the first detected object's bounding box, resulting in a ratio between 0 and 1. The server compares this calculated ratio with a preset overlap threshold. If the ratio is greater than the preset overlap threshold, it indicates that the first detected object is severely occluded by the second detected object. At this point, the server accesses the central data structure storing tracking information, finds the corresponding first tracking ID through the first detected object, and modifies the internal state attribute value of that tracking ID to an occlusion state.

[0048] Step S203: If a second tracking ID exists, update the state of the second tracking ID to the predicted loss state. The second tracking ID is the tracking ID of the third detection object for which no corresponding object is found in the current frame.

[0049] In step S203, the second tracking ID refers to a tracking ID that existed in a previous frame but failed to match any detected object in the detection results of the current frame; the current frame refers to a single image being processed in the video sequence; the third detected object refers to a hypothetical object that should be associated with the second tracking ID but was not detected in the current frame; the predicted loss state is a state that indicates that the object associated with the tracking ID has temporarily disappeared from the field of view, and the system will begin to maintain its trajectory through prediction.

[0050] Specifically, after matching and associating all detected objects with existing tracking IDs in the current frame, the server checks a list of all tracking IDs that were active in the previous frame. The server compares this list with the list of successfully matched tracking IDs in the current frame, filtering out all tracking IDs that failed to find a match in the current frame. These filtered tracking IDs are the second tracking IDs. For each second tracking ID, the server accesses its tracking information record and updates its status attribute from the previous state, such as "tracked," to "predicted lost." This status change signifies that the tracking of the target has switched from relying on actual observations to relying on model predictions.

[0051] Step S204: Obtain the historical confidence of the first detected object, and determine the detection confidence of the first detected object in the current frame based on the historical confidence.

[0052] In step S204, historical confidence refers to the sequence of detection confidence of the object associated with the first tracking ID over a series of consecutive frames in the past.

[0053] Specifically, when the state of the first tracking ID corresponding to the first detected object is updated to an occluded state, the server initiates a confidence smoothing process to address the sudden drop in confidence caused by occlusion. The server accesses the tracking history database associated with the first tracking ID and extracts the historical confidence values ​​of the object over the past N consecutive frames, forming a time series. Then, the server uses a time series smoothing algorithm, such as exponential moving average or simple moving average, to calculate a weighted average of these historical confidence values. This calculated smoothed confidence value will be used as the final detection confidence of the first detected object in the current frame, replacing the original detection confidence that may be lower due to occlusion. This enhances the robustness of tracking and prevents the target from being incorrectly judged as a tracking failure due to brief occlusion.

[0054] Step S205: Obtain the historical position and historical speed of the third detection object.

[0055] In step S205, the entity referred to by the third detection object is the temporarily lost target associated with the second tracking ID; the historical position refers to the sequence of bounding box center point coordinates of the target in a series of frames before it was lost; the historical velocity refers to the sequence of motion velocity vectors of the target before it was lost, calculated based on the historical position.

[0056] Specifically, for each second tracking ID whose state is updated to the predicted lost state, the server needs to prepare data for predicting its future position. The server queries the historical tracking data bound to that second tracking ID. From the historical data, the server extracts the bounding box position information of the target in the M most recently successfully detected frames; this information constitutes the historical position. Simultaneously, the server calculates or directly extracts the stored historical velocity based on these historical positions. The velocity is calculated by dividing the difference in position coordinates between two frames by the time interval between the two frames. Through this step, the server obtains a complete description of the lost target's motion state before disappearing, including position and velocity information.

[0057] Step S206: Based on historical position, historical velocity and preset motion model, predict the virtual position of the third detection object in the current frame.

[0058] In step S206, the preset motion model is a mathematical formula or algorithm, such as a uniform motion model or a Kalman filter, used to predict the current state of the target based on its past state; the virtual position refers to the position in the current frame image where the lost target is most likely to appear, calculated by the preset motion model.

[0059] Specifically, the server uses the historical position and historical velocity obtained in step S205 to calculate the virtual position of the second tracking ID, which is in a predicted lost state, in the current frame. If a uniform motion model is used, the server takes the last observed position of the target and adds it to the product of its last calculated velocity and the lost time to obtain the predicted position in the current frame. If a more complex Kalman filter is used as the preset motion model, the server performs a filter prediction step, propagating the internal state vector containing position and velocity forward by one time step, thereby obtaining a more accurate virtual position that includes uncertainty estimation. This calculated virtual position is recorded as an important basis for re-identifying the target in subsequent frames.

[0060] Step S207: If a new detected object is associated with the second tracking ID in a subsequent frame within a preset time period, the re-identification is determined to be successful, and the state of the second tracking ID is restored to the tracked state.

[0061] In step S207, the preset time period refers to a set time window, such as 30 frames, during which the system will continuously try to find the lost target; subsequent frames refer to video frames after the current frame; new detected object refers to a detected object that appears in a subsequent frame and is not associated with any existing tracking ID; re-identification success means that the system successfully identifies a new detected object as the previously lost target; tracked state is a normal state indicating that the target has been successfully detected and tracked.

[0062] Specifically, for each second tracking ID in the predicted lost state, the server will attempt to match new detected objects in each frame within the next preset time period. When processing each subsequent frame, the server compares the positions of all new detected objects in that frame with the virtual positions predicted by the second tracking ID in the previous frame. If the distance between the position of a new detected object and its virtual position is less than a preset distance threshold, and if the system also uses appearance features, the similarity between the appearance features of the new detected object and the appearance features in the second tracking ID's historical records is higher than a preset similarity threshold, then the server determines that the new detected object has been successfully associated with the second tracking ID, i.e., re-identification is successful. Once re-identification is successful, the server immediately restores the state of the second tracking ID from the predicted lost state to the tracked state and updates the attributes of the new detected object to the tracking information of the second tracking ID.

[0063] Step S208: Calculate the motion velocity corresponding to the second tracking ID based on the virtual location and the position of the newly detected object in subsequent frames.

[0064] In step S208, the velocity refers to the change in the target's position per unit time, which is usually represented as a vector containing horizontal and vertical components.

[0065] Specifically, after successful re-identification in step S207, the server needs to update the motion model of the second tracking ID to ensure the accuracy of future predictions. To calculate a more accurate motion velocity, the server utilizes two reliable location points before and after the target's loss. The server retrieves the actual observed position and timestamp of the target in the last frame before loss from the history of the second tracking ID, and then retrieves the position and timestamp of the newly detected object in subsequent frames when re-identification was successful. The server calculates the displacement between these two location points, i.e., the coordinate difference in the x and y directions. Then, the server calculates the time difference between the two timestamps. Finally, the server divides each component of the displacement by the time difference to obtain a new average motion velocity that spans the target loss phase. This newly calculated motion velocity will be updated in the tracking information of the second tracking ID for tracking and prediction in subsequent frames.

[0066] To facilitate understanding, the following concrete example will be used to explain steps S201-S208:

[0067] Suppose in a surveillance scenario, a pedestrian A with a detection confidence of 0.7 and a tracking ID of 101 is occluded by a vehicle B with a detection confidence of 0.9. Since the overlap area between the bounding box of pedestrian A and the bounding box of vehicle B exceeds a preset 70% threshold, the system updates the state of pedestrian A's tracking ID 101 to occluded. Simultaneously, due to occlusion, pedestrian A's original detection confidence in the current frame drops to 0.4. The system then retrieves its historical confidence scores from previous frames (0.72, 0.75, etc.), calculates a smoothed confidence score of 0.68, and uses this value to update the confidence score of the current frame, thus avoiding tracking interruption due to excessively low confidence. On the other side of the scene, a pedestrian C with a tracking ID of 202 walks behind a billboard, resulting in pedestrian C not being detected in the current frame. The system then recognizes that tracking ID 202 did not match a detected object in the current frame and updates its state to predicted lost state. The system immediately acquires the historical position and velocity of pedestrian C before disappearing, and predicts the virtual position where pedestrian C should appear in the next frame using a pre-defined motion model based on a Kalman filter. Two frames later, a new detected object without a tracking ID appears on the other side of the billboard, its position and features closely matching the virtual position predicted by the system for tracking ID 202. The system determines that re-identification is successful, associates tracking ID 202 with this new detected object, and restores its state to tracked. Finally, based on pedestrian C's last position before disappearing and its new position upon reappearance, as well as the time elapsed in between, the system recalculates an accurate average velocity and updates it to the tracking information of tracking ID 202 to ensure the accuracy of subsequent tracking.

[0068] Step S103: Determine the target detection objects with the same tracking ID in the current frame and the previous frame, calculate the motion speed of the target detection objects, and generate a motion speed score based on the motion speed.

[0069] In step S103, the previous frame refers to the processed frame immediately preceding the current frame in the time series. Motion speed refers to the rate at which the position of the detected target changes on the image plane. The motion speed score is a numerical value used to characterize the dynamic range of the target, quantified based on the calculated motion speed.

[0070] Specifically, the server searches for targets successfully tracked in both the current and previous frames, i.e., targets with the same tracking ID. For each such target, the server extracts the coordinates of its bounding box center point in both the current and previous frames. By calculating the Euclidean distance between these two center point coordinates on the 2D image plane, the pixel displacement of the target between the two frames can be obtained. Dividing this pixel displacement by the fixed time interval between the two frames yields the target's motion velocity. Considering the significant differences in motion velocities among different targets, the server maps the calculated raw motion velocity values ​​to a preset scoring range, such as 0 to 1, using a normalization function, such as the sigmoid function or a piecewise linear function. The faster the motion, the higher the corresponding motion velocity score, and vice versa. Finally, the server calculates the average motion velocity scores of all tracked targets in the current frame, generating a motion velocity score representing the overall motion intensity of the current scene.

[0071] Step S104: Create a corresponding confidence history sequence for each tracking ID, and calculate the confidence score based on the statistical characteristics of the confidence history sequence.

[0072] In step S104, the confidence history sequence refers to a sequence maintained for each tracking ID, recording the detection confidence of that ID over multiple consecutive frames. Statistical properties refer to mathematical quantities describing the distribution characteristics of the sequence data, such as mean and standard deviation. The confidence score is a numerical value calculated based on these statistical properties to characterize the overall stability of the detection results.

[0073] Specifically, the server associates a fixed-length first-in-first-out queue with each active tracking ID; this queue serves as the confidence history sequence. When a target with a specific tracking ID is detected in the current frame, the server obtains the confidence score for that detection and stores the score value at the tail of the queue corresponding to that tracking ID. If the queue length exceeds a preset limit, the oldest confidence score value at the head of the queue is removed. Next, the server performs statistical analysis on the confidence history sequence for each tracking ID, calculating the mean and standard deviation of the sequence. Based on these two statistics, the server can further calculate the coefficient of variation, which is the ratio of the standard deviation to the mean. The magnitude of the mean reflects the overall level of confidence, while the magnitude of the coefficient of variation reflects the fluctuation or instability of the confidence. The server combines the mean and coefficient of variation of the confidence scores for all tracking IDs, for example, through a weighted summation, to generate a confidence score that reflects the overall magnitude and stability of the confidence scores for all target detections in the current frame. The greater the confidence fluctuation, the higher the score.

[0074] In one possible implementation, a corresponding confidence history sequence is created for each tracking ID, specifically including steps S1041-S1043, as follows:

[0075] Step S1041: Obtain the detection confidence of the target object corresponding to the tracking ID in multiple consecutive frames. The detection confidence represents the probability that the target object is a real target.

[0076] In step S1041, consecutive frames refer to a group of image frames that are sequentially linked in the video time series. Detection confidence represents the probability that the object detection model determines a detected object to be a real target.

[0077] Specifically, when the server processes the video stream frame by frame, for each target detection object assigned a tracking ID, it extracts the detection confidence score corresponding to that target object from the detection results after the target detection is completed. This acquisition process is continuous. As video frames are continuously input and processed, the server records the detection confidence score for the same tracking ID in each frame. The server temporarily stores these detection confidence score values ​​acquired in chronological order, thereby forming a time-series data associated with that tracking ID, providing raw data input for the subsequent creation and maintenance of the confidence score history sequence.

[0078] Step S1042: For each tracking ID, maintain a confidence queue of a preset fixed length.

[0079] In step S1042, the preset fixed length refers to a fixed value specified during system initialization or configuration, used to limit the number of elements that can be accommodated in the queue. A confidence queue is a data structure used to store historical detection confidence records for a specific tracking ID, following a first-in, first-out (FIFO) principle.

[0080] Specifically, when the server assigns a new tracking ID to a newly detected target in step S102, it synchronously instantiates a data queue in memory for that tracking ID; this queue is the confidence queue. The capacity of the confidence queue is determined by a pre-set parameter, namely a preset fixed length. For example, if the preset fixed length is set to 10, the confidence queue can store a maximum of 10 detection confidence values. The server maintains such a confidence queue independently for all currently tracked and active tracking IDs in the current video, for dynamically recording their respective confidence changes.

[0081] Step S1043: When the detection confidence of the target object in the current frame is obtained, the detection confidence is inserted into the tail of the confidence queue, and the detection confidence at the head of the confidence queue is removed to keep the length of the confidence queue unchanged.

[0082] In step S1043, the tail of the queue represents the end of the confidence queue used for inserting new data elements. The head of the queue represents the end of the confidence queue where the earliest inserted data element is stored, which is also the end where data was removed.

[0083] Specifically, for the current frame, when the server processes a target detection object with an existing tracking ID, the server first obtains the detection confidence score corresponding to the target detection object from the detection results in step S101. Then, the server locates the confidence score queue corresponding to the tracking ID. The server performs an enqueue operation, adding the newly obtained detection confidence score value to the tail of the confidence score queue. Next, the server determines whether the current number of elements in the confidence score queue has reached a preset fixed length. If the queue is not full, the operation ends. If the queue is full, the server performs a dequeue operation, deleting the earliest stored detection confidence score value at the head of the confidence score queue. Through this enqueue-and-potential-dequeue operation, the server ensures that the confidence score queue always stores only the detection confidence scores of the most recent frames, and that the queue length does not exceed the preset fixed length, thus achieving a sliding update of historical confidence score data.

[0084] To facilitate understanding, the following concrete example will be used to explain steps S1041-S1043:

[0085] Suppose a server is analyzing a video and has assigned a tracking ID "person-01" to a pedestrian appearing in the video. The system sets a preset fixed length of 5 for maintaining confidence history. When processing the first frame, the pedestrian is detected with a detection confidence of 0.98. The server retrieves this confidence and stores it in the confidence queue corresponding to "person-01," at which point the queue content is [0.98]. When processing the second, third, and fourth frames, the pedestrian is continuously and clearly detected with detection confidences of 0.99, 0.97, and 0.98, respectively. The server inserts these values ​​into the tail of the queue in sequence, and the queue content becomes [0.98, 0.99], [0.98, 0.99, 0.97], and [0.98, 0.99, 0.97, 0.98], respectively. When processing the fifth frame, the pedestrian was briefly obscured by a pillar, causing the detection confidence to drop to 0.75. The server inserted 0.75 into the queue, which was now full, containing [0.98, 0.99, 0.97, 0.98, 0.75]. When processing the sixth frame, the pedestrian moved out of the obstruction, and the detection confidence recovered to 0.96. The server obtained 0.96 and inserted it into the tail of the confidence queue. Since the queue length had reached 5, the server removed the oldest data at the head of the queue, 0.98. The confidence queue was then updated to [0.99, 0.97, 0.98, 0.75, 0.96], maintaining the latest five confidence records.

[0086] In one possible implementation, a confidence score is calculated based on the statistical characteristics of the historical confidence sequence, specifically including steps S1044-S1047, as follows:

[0087] Step S1044: Calculate the mean of the historical confidence series as the confidence magnitude for evaluating the confidence level.

[0088] In step S1044, the mean value refers to the result obtained by summing all detection confidence values ​​in the confidence history sequence and dividing by the number of values. The confidence amplitude is an indicator used to quantify the average confidence level of a target being detected as a real target over a recent period.

[0089] Specifically, for each tracking ID in the current frame, the server first locates the corresponding confidence history sequence, i.e., the confidence queue maintained in step S1043. The server iterates through all the detection confidence values ​​in the confidence queue, summing them all to obtain a total. Then, the server divides this total by the preset fixed length of the confidence queue, and the result is the confidence amplitude of the tracking ID in the current stage. The server associates and stores the calculated confidence amplitude with the corresponding tracking ID for use in subsequent steps.

[0090] Step S1045: Calculate the coefficient of variation of the confidence history series. The coefficient of variation is the ratio of the standard deviation to the mean of the confidence history series. Based on the coefficient of variation, generate the confidence stability to assess the confidence stability.

[0091] In step S1045, the standard deviation is a statistic used to measure the dispersion of each detection confidence value relative to the mean in the confidence history sequence. The coefficient of variation is the ratio of the standard deviation to the mean, used to eliminate the influence of dimensions and reflect the relative volatility of the data. Confidence stability is an indicator used to quantify the stability of the change in the detection confidence of a target over a recent period.

[0092] Specifically, after completing step S1044, the server continues to use the historical confidence sequence for each tracking ID. First, the server calculates the standard deviation of the sequence. The calculation process includes: obtaining the difference between each confidence value and the mean in the sequence, squaring each difference, summing all squared differences, dividing the sum by the sequence length, and finally taking the square root of the result. Next, the server divides the calculated standard deviation by the mean obtained in step S1044 to obtain the coefficient of variation. Since a smaller coefficient of variation indicates more stable data, to make the confidence stability index larger to represent greater stability, the server can generate the confidence stability based on the coefficient of variation through an inverse mapping relationship, for example, using 1 minus the coefficient of variation as the confidence stability. In this way, a sequence with small fluctuations and a low coefficient of variation will obtain a high confidence stability close to 1.

[0093] Step S1046: Based on preset weights, perform a weighted summation of the confidence amplitude and confidence stability to obtain the individual confidence score for each tracking ID.

[0094] In step S1046, the preset weights are values ​​pre-set in the system configuration, corresponding to the importance of confidence amplitude and confidence stability in the final score, respectively. Weighted summation is a calculation method that multiplies each value by its respective weight and then sums them. The individual confidence score is a quantitative assessment of the overall credibility of the target represented by a single tracking ID, combining confidence magnitude and stability.

[0095] Specifically, the server reads two preset weight values: one for the confidence amplitude and the other for the confidence stability. The sum of the two weights is typically 1. For each tracking ID in the current frame, the server retrieves the confidence amplitude calculated in step S1044 and the confidence stability generated in step S1045. The server multiplies the confidence amplitude by its preset weight and the confidence stability by its preset weight, then adds these two products together. The result of this weighted sum is the individual confidence score for that tracking ID. The server repeats this process for all active tracking IDs in the current frame, calculating an independent individual confidence score for each tracking ID.

[0096] Step S1047: Calculate the average of the individual confidence scores of all tracking IDs in the current frame as the confidence score.

[0097] In step S1047, the confidence score is the final comprehensive indicator for evaluating the overall target detection and tracking quality of the current frame.

[0098] Specifically, after calculating individual confidence scores for all tracking IDs in the current frame, the server collects all these individual confidence scores. The server sums all these individual confidence scores to obtain a total score. Then, the server divides this total score by the total number of active tracking IDs in the current frame, and the resulting arithmetic mean is the final confidence score for the current frame. This confidence score reflects the overall confidence level of all tracked targets in the current frame and can be used for subsequent system decisions or status assessments.

[0099] Step S105: Integrate the motion speed score and confidence score to obtain a comprehensive dynamic score, and adjust the time window size according to the comprehensive dynamic score to obtain the target time window.

[0100] In step S105, the motion speed score represents the magnitude of the motion speed of the target object, used to reflect changes in the target's motion state; the confidence score represents the statistical characteristics of the historical confidence sequence corresponding to the tracking ID, used to evaluate the reliability of the target detection results; the comprehensive dynamic score is an evaluation index obtained by comprehensively considering the target's motion state and detection reliability, used to adaptively adjust the tracking detection time window; the target time window represents the size of the time window used for tracking detection decisions, determined after adjustment based on the comprehensive dynamic score.

[0101] Specifically, the server first obtains the motion speed score of the target object calculated in step S103 and the confidence score corresponding to the tracking ID calculated in step S104. Then, the server performs a weighted summation of the motion speed score and the confidence score according to a preset fusion strategy to obtain a comprehensive dynamic score. In the fusion strategy, the weight of the motion speed score is set to a negative value, and the weight of the confidence score is set to a positive value. The purpose of this setting is: when the target's motion speed is large, the motion speed score is high, and in order to improve the stability of tracking detection, a lower comprehensive dynamic score is required, thereby increasing the time window; when the confidence of the target detection result is high and stable, its confidence score is high, and in order to speed up the detection response, a higher comprehensive dynamic score is required, thereby decreasing the time window. The specific values ​​of the weights can be debugged and optimized according to the actual application scenario and system requirements. Next, the server calculates the target time window size for this tracking detection based on the obtained comprehensive dynamic score and using preset time window adjustment rules. The adjustment rules typically include setting several thresholds for the comprehensive dynamic score and the corresponding time window change. When the overall dynamic score is low, it indicates that the target is moving violently or has low confidence, and the server will increase the size of the time window accordingly. Conversely, when the overall dynamic score is above a certain threshold, it indicates that the target is moving smoothly and has consistently high confidence, and the server will decrease the size of the time window accordingly. At the same time, to avoid excessively large adjustments to the time window, the server also sets upper and lower limits for the time window size. Ultimately, the server uses the adjusted time window size as the target time window for this tracking, providing a basis for the subsequent accumulation and aggregation of detection results.

[0102] In one possible implementation, the time window size is adjusted based on the comprehensive dynamic score to obtain the target time window, specifically including steps S1051-S1056, as follows:

[0103] Step S1051: Set the upper and lower limits of the time window size, and calculate and adjust the step size based on the comprehensive dynamic score.

[0104] In step S1051, the upper and lower limits of the time window size refer to the maximum and minimum allowable values ​​set for the time window size. These limits constrain the adjustment range of the time window, preventing it from becoming too large and resulting in outdated historical information, or too small and resulting in insufficient information. The comprehensive dynamic score is a comprehensive indicator that quantitatively evaluates the movement or change of a target in the current scene. The adjustment step size is a value used to increase or decrease the time window size in each adjustment, and its magnitude is related to the comprehensive dynamic score.

[0105] Specifically, the server first performs parameter setting and calculation. The server loads the upper and lower limits of the time window size from a preset configuration file. These two values ​​together define the legal dynamic adjustment range of the time window size, preventing the time window from becoming too large or too small due to continuous adjustment, thus ensuring stable system operation. Subsequently, the server calculates the adjustment step size based on the comprehensive dynamic score obtained in the previous steps. This calculation is performed using a preset mapping function. For example, the server can use a linear function proportional to the absolute value of the comprehensive dynamic score to calculate the adjustment step size; that is, the adjustment step size equals the absolute value of the comprehensive dynamic score multiplied by a preset scaling factor, thereby achieving the effect that the further the score deviates from the central region, the larger the adjustment magnitude.

[0106] Step S1052: Obtain the initial time window size corresponding to the previous frame of the current frame.

[0107] In step S1052, the previous frame of the current frame refers to the frame image immediately preceding the current processing frame in the video sequence. The initial time window size refers to the size of the time window used by the system before processing the current frame, that is, the target time window size calculated from the previous frame, and this value will be used as the basis for this adjustment.

[0108] Specifically, the server accesses its internal state store or memory to read the target time window size saved for the previous frame. This value is the result obtained after the entire adjustment process was completed last time. For example, if the target time window size determined after processing the previous frame was 20 frames, then the initial time window size obtained by the server when processing the current frame will be 20 frames. For the first frame of the video stream, since there is no previous frame, the server uses a preset default time window size, such as 20 frames, as the initial time window size for processing the first frame.

[0109] Step S1053: Compare the comprehensive dynamic score with the first preset score threshold and the second preset score threshold, wherein the first preset score threshold is greater than the second preset score threshold.

[0110] In step S1053, the first preset score threshold and the second preset score threshold are two pre-set critical values ​​used to determine whether the comprehensive dynamic score is high or low. These two thresholds divide the numerical range of the comprehensive dynamic score into three intervals: high, medium, and low, providing a basis for subsequent decision-making.

[0111] Specifically, the server obtains the specific values ​​of the first and second preset score thresholds from the system configuration; for example, the first preset score threshold is 0.8 and the second preset score threshold is 0.3. Then, the server performs a numerical comparison operation, comparing the overall dynamic score of the current frame with these two thresholds. The comparison result determines the range to which the score belongs: a high dynamic range greater than the first preset score threshold, a low dynamic range less than the second preset score threshold, or a medium dynamic range greater than or equal to the second preset score threshold and less than or equal to the first preset score threshold.

[0112] Step S1054: If it is determined that the comprehensive dynamic score is greater than the first preset score threshold, the initial time window size is reduced and the adjustment step is adjusted to obtain the target time window.

[0113] In step S1054, the server first determines whether the overall dynamic score of the current frame is greater than a first preset score threshold. The overall dynamic score is calculated in step S105 by fusing motion velocity score and confidence score, reflecting the overall dynamic characteristics of the target in the current frame. A higher value indicates more stable target motion and more stable detection confidence. The first preset score threshold is a critical value pre-set in step S1053 to determine whether the overall dynamic score is high. When the overall dynamic score exceeds the first preset score threshold, it indicates that the target's motion state is stable, the detection result confidence is continuously high, and the dynamic characteristics are stable. In this case, the time window can be appropriately reduced to improve the detection response speed.

[0114] If the overall dynamic score is determined to be greater than the first preset score threshold, the server will start from the initial time window size obtained in step S1052 and reduce the time window size according to the adjustment step size calculated in step S1051 to obtain the target time window size for this tracking and detection. The initial time window size is the time window size corresponding to the previous frame of the current frame, serving as the starting point for this time window adjustment. The adjustment step size is the change in time window size calculated based on the overall dynamic score; the larger the value, the greater the reduction in time window size. By subtracting the adjustment step size from the initial time window size, the adjusted target time window size is obtained.

[0115] Step S1055: If it is determined that the comprehensive dynamic score is less than the second preset score threshold, the initial time window size is increased by the adjustment step to obtain the target time window.

[0116] In step S1055, the server first determines whether the overall dynamic score of the current frame is less than a second preset score threshold. The overall dynamic score is calculated in step S105 by fusing motion velocity score and confidence score, reflecting the overall dynamic characteristics of the target in the current frame. The lower the value, the more violent the target motion and the lower the detection reliability. The second preset score threshold is a critical value pre-set in step S1053 to determine whether the overall dynamic score is low, and it is usually less than the first preset score threshold. When the overall dynamic score is lower than the second preset score threshold, it indicates that the target's motion state is unstable, the detection result has low confidence, and it belongs to a situation of drastic changes in dynamic characteristics. At this time, it is necessary to appropriately increase the time window to improve the stability and robustness of tracking and detection.

[0117] If the overall dynamic score is determined to be less than the second preset score threshold, the server will start from the initial time window size obtained in step S1052 and increase the time window size according to the adjustment step size calculated in step S1051 to obtain the target time window size for this tracking and detection. The initial time window size is the time window size corresponding to the previous frame of the current frame, serving as the starting point for this time window adjustment. The adjustment step size is the change in time window size calculated based on the overall dynamic score; the larger the value, the greater the increase in the time window size. By adding the initial time window size to the adjustment step size, the adjusted target time window size is obtained.

[0118] Step S1056: If it is determined that the overall dynamic score is greater than or equal to the second preset score threshold and the overall dynamic score is less than or equal to the first preset score threshold, then maintain the initial time window size and obtain the target time window.

[0119] Specifically, when the server determines in step S1053 that the overall dynamic score falls between or equals the second preset score threshold and the first preset score threshold, it indicates that the current scene's dynamics are at a moderate and relatively stable level. In this case, the system considers the current initial time window size appropriate and requires no adjustment. Therefore, the server does not perform any addition or subtraction operations and directly uses the initial time window size as the target time window for the current frame. This operation ensures the stability of the time window size when the scene's dynamics do not change significantly.

[0120] To facilitate understanding, the following concrete example will be used to explain steps S1051-S1056:

[0121] Suppose the server is performing a road traffic monitoring task, requiring the tracking and detection of vehicle targets in the frame. In step S1051, the upper limit of the time window size is preset to 15 frames, and the lower limit to 3 frames, and the adjustment step size is calculated to be 2 frames based on the comprehensive dynamic score. In step S1052, the time window size corresponding to the previous frame is obtained as 6 frames, which is used as the initial time window size. In step S1053, the first threshold is preset to 0.75, and the second threshold is 0.25. After the calculations in the previous steps, the comprehensive dynamic score of vehicle target A in the current frame is 0.8, and the comprehensive dynamic score of vehicle target B is 0.2. For target A, since the comprehensive dynamic score of 0.8 is greater than the first threshold of 0.75, the condition of step S1054 is met. Therefore, the server subtracts the adjustment step size of 2 frames from the initial time window size of 6 frames for target A, resulting in a target time window size of 4 frames. For target B, since the comprehensive dynamic score of 0.2 is less than the second threshold of 0.25, the condition of step S1055 is met. Therefore, the server adds an adjustment step size of 2 frames to the initial time window size of 6 frames for target B, resulting in a target time window size of 8 frames. The server performs convergent analysis on the detection results of target A and target B in the current frame based on the target time windows of 4 frames and 8 frames respectively, and outputs the corresponding tracking trajectory and recognition information.

[0122] Step S106: Determine whether the cumulative number of processed frames is greater than or equal to the target time window.

[0123] In step S106, the cumulative number of processed frames refers to the number of video frames that have been processed continuously since the last object detection result was generated.

[0124] Specifically, the server maintains a frame counter internally. This frame counter is reset to zero after the system begins processing video or after each completion of the aggregation processing in step S107 and the generation of object detection results. Each time the server finishes processing a frame of video, i.e., after completing the calculations in steps S101 to S105, the value of this frame counter is incremented by one. In step S106, the server compares the current value of the frame counter with the size of the target time window most recently calculated in step S105. This comparison determines whether to trigger the gating condition for subsequent aggregation processing steps.

[0125] Step S107: If the cumulative number of processed frames is greater than or equal to the target time window, then obtain the target detection results corresponding to each tracking ID within the target time window, and perform aggregation processing on the target detection results based on the tracking ID to generate object detection results.

[0126] In step S107, the convergence process refers to integrating and refining multiple detection results belonging to the same tracking ID within a time window to generate a more stable and accurate single result. The object detection result is the final detection information output after the convergence process, representing the system's best estimate of the target within that time period.

[0127] Specifically, when the judgment result of step S106 is yes, the server initiates the aggregation processing procedure. The server retrieves and collects the detection results of all frames stored within the target time window that has just passed. Then, the server groups these detection results according to tracking ID. For each tracking ID group, the server performs the following operations: First, it counts the category labels of all detection results in the group and selects the category label with the most occurrences as the final output category for that tracking ID. Second, it performs a weighted average of the detection confidence scores of all detection results in the group, with the weights based on the confidence score itself or other factors, to obtain an aggregated output confidence score. Third, it performs a weighted average of the bounding box coordinates of all detection results in the group to obtain a smoothed output bounding box. Finally, the server combines the output category, output confidence score, and output bounding box for each tracking ID to form the final object detection result and outputs it. After outputting, the server clears the cache used to store the detection results within the time window and resets the cumulative processed frame count counter to zero, preparing to start the accumulation of the next dynamic time window.

[0128] In one possible implementation, the target detection result includes the detection confidence score, category label, and bounding box coordinates of the target object. The target detection result is then aggregated based on the tracking ID to generate the object detection result, specifically including steps S1071-S1075, as follows:

[0129] Step S1071: Group the target detection results with the same tracking ID within the current time window into a target detection result group.

[0130] In step S1071, the current time window refers to a series of consecutive video frames used for convergence processing; the tracking ID refers to a unique identifier assigned to the same physical target in the video sequence, used to associate the target between different frames; the target detection result refers to the set of attributes of a target identified by the target detection algorithm on a single frame image, including detection confidence, category label and bounding box coordinates; the target detection result group refers to the set formed by aggregating all target detection results with the same tracking ID within the current time window.

[0131] Specifically, the server first retrieves the target detection results for all frames within the current time window from memory. Then, the server creates a data structure, such as a mapping table with tracking IDs as keys and lists of target detection results as values, to store the grouped results. The server iterates through each target detection result within the time window, reading the tracking ID contained within that result. The server uses this tracking ID to look up the result in the mapping table. If the tracking ID already exists as a key, the current target detection result is appended to the list corresponding to that key. If the tracking ID does not exist, the server creates a new entry in the mapping table with the tracking ID as the key and a new list containing only the current target detection result as the value. Once all target detection results within the time window have been processed, each value in the mapping table—that is, each list of target detection results—constitutes a group of target detection results.

[0132] Step S1072: For the target detection results within each target detection result group, count the frequency of occurrence of each category label, and take the category label with the highest frequency of occurrence as the output category of the corresponding tracking ID.

[0133] In step S1072, the category label refers to the category of the identified target, such as a person or a vehicle; the frequency of occurrence refers to the number of times a specific category label appears within the target detection result group; and the output category refers to the unique category label finally determined by the corresponding tracking ID after statistical analysis.

[0134] Specifically, the server processes each target detection result group generated in step S1071 independently. For a target detection result group, the server initializes a frequency counter to record the occurrence frequency of each category label within the group. The server iterates through all target detection results in the target detection result group and extracts the category label from each result. For each extracted category label, the server increments the count value of that category label in the frequency counter. After the iteration is complete, the server checks the frequency counter and finds the category label with the highest count value. This category label with the highest occurrence frequency is determined as the output category of the tracking ID corresponding to that target detection result group. If multiple category labels have the same highest frequency during the statistical process, the server will use preset rules to select the final output category, such as selecting the category label with the higher average confidence among all detection results, or selecting the category label from the frame that appeared later in time.

[0135] Step S1073: Calculate the weighted average of the detection confidence scores of the target detection results within each target detection result group, and use it as the output confidence score for the corresponding tracking ID.

[0136] In step S1073, detection confidence refers to the quantitative measure of the target detection algorithm's confidence that a certain detection result is a correct target; weighted average refers to an average calculation method in which each value is multiplied by a specific weight when summing, which is used to reflect the importance of the value; output confidence refers to the single value representing the overall confidence of the tracking ID obtained by weighted averaging of all detection confidences within the target detection result group.

[0137] Specifically, the server continues processing each group of target detection results. For a group of target detection results, the server needs to calculate a comprehensive confidence score. The server uses a weighted average algorithm, with the detection confidence score of each target detection result as its weight. This allows detection results with higher confidence scores to have a greater impact on the final average. The server initializes two variables: the weighted confidence score sum and the weight sum, both with initial values ​​of zero. The server iterates through each target detection result in the group, extracting the detection confidence score. For each detection confidence score, the server squares it and adds it to the weighted confidence score sum, while simultaneously adding the detection confidence score itself to the weight sum. After iterating through all results in the group, the server divides the weighted confidence score sum by the weight sum; the quotient is the output confidence score for the corresponding tracking ID. If the weight sum is zero, the output confidence score is set to zero to avoid division by zero errors.

[0138] Step S1074: Calculate a weighted average of the bounding box coordinates of the target detection results within each target detection result group, and use this average as the output bounding box for the corresponding tracking ID.

[0139] In step S1074, bounding box coordinates refer to a set of values ​​used to locate the target on the image, usually represented as the x and y coordinates of the upper left corner of the rectangle, as well as the width and height of the rectangle; the output bounding box refers to a single bounding box that represents the average position and size of the tracking ID within the current time window after weighted averaging of the bounding box coordinates of all bounding boxes in the target detection result group.

[0140] Specifically, the server calculates a stable output bounding box for each group of object detection results. Similar to calculating the output confidence score, the server uses a weighted average method, also using the detection confidence score of each object detection result as the weights. The server initializes five variables: four to store the weighted sums of the bounding box's x-coordinate, y-coordinate, width, and height, respectively, and one variable to store the total weights; all variables are initialized to zero. The server iterates through each object detection result in the group, extracting the detection confidence score and the four coordinate values ​​of the bounding box. For each result, the server multiplies the bounding box's x-coordinate by its detection confidence score and adds it to the weighted sum of the x-coordinates. The same operation is performed on the y-coordinate, width, and height. Simultaneously, the detection confidence score is added to the total weights. After the iteration is complete, the server divides the weighted sum of the x-coordinates by the total weights to obtain the x-coordinate of the output bounding box. The y-coordinate, width, and height are calculated using the same method. These four calculated averages together constitute the output bounding box for the corresponding tracking ID.

[0141] Step S1075: Generate object detection results based on the output category, output confidence score, and output bounding box.

[0142] In step S1075, the object detection result refers to the final and unique detection result generated after the convergence processing of a target with a specific tracking ID within the current time window. This result integrates information from multiple frames.

[0143] Specifically, after completing all calculations for a tracking ID, the server integrates these results. The server combines the output category obtained in step S1072, the output confidence score obtained in step S1073, and the output bounding box obtained in step S1074, along with the corresponding tracking ID, into a structured data record. This newly generated data record is the final object detection result for that tracking ID within the current time window. The server repeats this process for all unique tracking IDs within the current time window, generating a unique object detection result for each tracking ID. Finally, the server outputs a set consisting of all these generated object detection results as input to subsequent processing modules.

[0144] The target tracking and detection device based on dynamic scoring and time window adjustment in the embodiments of this invention is described below from the perspective of hardware processing. Please refer to [link to relevant documentation]. Figure 3 This is a schematic diagram of the target tracking and detection device based on dynamic scoring and time window adjustment in the embodiments of this application.

[0145] It should be noted that, Figure 3The structure of the target tracking and detection device based on dynamic scoring and time window adjustment shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0146] like Figure 3 As shown, the target tracking and detection device based on dynamic scoring and time window adjustment includes a Central Processing Unit (CPU) 301, which can perform various appropriate actions and processes according to a program stored in Read-Only Memory (ROM) 302 or a program loaded from storage section 308 into Random Access Memory (RAM) 303, such as performing the methods described in the above embodiments. The RAM 303 also stores various programs and data required for device operation. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An Input / Output (I / O) interface 305 is also connected to the bus 304.

[0147] The following components are connected to I / O interface 305: input section 306 including audio input devices, push-button switches, etc.; output section 307 including a liquid crystal display (LCD) and audio output devices, indicator lights, etc.; storage section 308 including a hard disk, etc.; and communication section 309 including a network interface card such as a LAN (Local Area Network) card, modem, etc. Communication section 309 performs communication processing via a network such as the Internet. Drive 310 is also connected to I / O interface 305 as needed. Removable media 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 310 as needed so that computer programs read from them can be installed into storage section 308 as needed.

[0148] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by central processing unit (CPU) 301, it performs the various functions defined in the present invention.

[0149] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0150] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those shown in the drawings.

[0151] Specifically, the target tracking and detection device based on dynamic scoring and time window adjustment in this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, it implements the target tracking and detection method based on dynamic scoring and time window adjustment provided in the above embodiment.

[0152] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the target tracking and detection device based on dynamic scoring and time window adjustment described in the above embodiments; or it may exist independently and not assembled into the target tracking and detection device based on dynamic scoring and time window adjustment. The storage medium carries one or more computer programs, which, when executed by a processor of the target tracking and detection device based on dynamic scoring and time window adjustment, cause the target tracking and detection device based on dynamic scoring and time window adjustment to implement the target tracking and detection method based on IoT data encryption transmission provided in the above embodiments.

Claims

1. A target tracking and detection method based on dynamic scoring and time window adjustment, characterized in that, The method includes: Perform target detection on the current frame to obtain detection results containing multiple detected objects; Each of the detected objects in the detection results is tracked, and a tracking ID is assigned to each detected object; Identify target detection objects with the same tracking ID in the current frame and the previous frame, calculate the motion speed of the target detection objects, and generate a motion speed score based on the motion speed. A corresponding confidence history sequence is created for each tracking ID, and a confidence score is calculated based on the statistical characteristics of the confidence history sequence. By combining the motion speed score and the confidence score, a comprehensive dynamic score is obtained, and the time window size is adjusted according to the comprehensive dynamic score to obtain the target time window; Determine whether the cumulative number of processed frames is greater than or equal to the target time window; If the cumulative number of processed frames is greater than or equal to the target time window, then the target detection results corresponding to each tracking ID within the target time window are obtained, and the target detection results are aggregated based on the tracking ID to generate object detection results; After tracking each of the detected objects in the detection results and assigning a tracking ID to each detected object, the method further includes: The detection method checks whether the bounding box of the first detection object overlaps with the bounding box of the second detection object. The first detection object and the second detection object are any two of the multiple detection objects, and the detection confidence of the second detection object is higher than the detection confidence of the first detection object. The detection confidence represents the probability that the detection object is a real target. If the overlapping area exceeds a preset overlap threshold, the status of the first tracking ID corresponding to the first detected object will be updated to occlusion status. If a second tracking ID exists, the status of the second tracking ID is updated to the predicted loss status, where the second tracking ID is the tracking ID for which no corresponding third detection object is found in the current frame; Process the first detection object corresponding to the first tracking ID and the third detection object corresponding to the second tracking ID; The processing of the first detection object corresponding to the first tracking ID and the third detection object corresponding to the second tracking ID specifically includes: Obtain the historical confidence score of the first detected object, and determine the detection confidence score of the first detected object in the current frame based on the historical confidence score; Obtain the historical position and historical speed of the third detection object; Based on the historical position, the historical velocity, and the preset motion model, predict the virtual position of the third detected object in the current frame; If a new detected object is associated with the second tracking ID in a subsequent frame within a preset time period, the re-identification is determined to be successful, and the status of the second tracking ID is restored to the tracked state. Based on the virtual location and the position of the newly detected object in the subsequent frame, the motion speed corresponding to the second tracking ID is calculated.

2. The method according to claim 1, characterized in that, The step of creating a corresponding confidence history sequence for each tracking ID specifically includes: Obtain the detection confidence of the target object corresponding to the tracking ID in multiple consecutive frames; For each tracking ID, maintain a confidence queue of a preset fixed length; When the detection confidence of the target object in the current frame is obtained, the detection confidence is inserted into the tail of the confidence queue, and the detection confidence at the head of the confidence queue is removed to keep the length of the confidence queue unchanged.

3. The method according to claim 1, characterized in that, The calculation of the confidence score based on the statistical characteristics of the historical confidence sequence specifically includes: Calculate the mean of the historical confidence series as the confidence magnitude for evaluating the confidence level; Calculate the coefficient of variation of the confidence history series, where the coefficient of variation is the ratio of the standard deviation to the mean of the confidence history series, and generate a confidence stability based on the coefficient of variation to assess the stability of the confidence. Based on preset weights, the confidence amplitude and the confidence stability are weighted and summed to obtain the individual confidence score for each tracking ID; Calculate the average of the individual confidence scores for all tracking IDs in the current frame, and use this average as the confidence score.

4. The method according to claim 1, characterized in that, The step of adjusting the time window size based on the comprehensive dynamic score to obtain the target time window specifically includes: Set the upper and lower limits of the time window size, and calculate the adjustment step size based on the comprehensive dynamic score; Obtain the initial time window size corresponding to the previous frame of the current frame; The comprehensive dynamic score is compared with a first preset score threshold and a second preset score threshold, wherein the first preset score threshold is greater than the second preset score threshold. If it is determined that the comprehensive dynamic score is greater than the first preset score threshold, then the initial time window size is reduced by the adjustment step size to obtain the target time window; If it is determined that the comprehensive dynamic score is less than the second preset score threshold, then the initial time window size is increased by the adjustment step size to obtain the target time window; If it is determined that the overall dynamic score is greater than or equal to the second preset score threshold, and the overall dynamic score is less than or equal to the first preset score threshold, then the initial time window size is maintained to obtain the target time window.

5. The method according to claim 1, characterized in that, The target detection result includes the detection confidence score, category label, and bounding box coordinates of the target object. The aggregation processing of the target detection result based on the tracking ID to generate the object detection result specifically includes: Target detection results with the same tracking ID within the target time window are grouped into one target detection result group; For each target detection result group, the frequency of occurrence of each category label is counted, and the category label with the highest frequency of occurrence is taken as the output category of the corresponding tracking ID; The detection confidence scores of the target detection results within each target detection result group are weighted and averaged to obtain the output confidence score for the corresponding tracking ID. The bounding box coordinates of the target detection results within each target detection result group are weighted and averaged to obtain the output bounding box for the corresponding tracking ID. The object detection result is generated based on the output category, the output confidence level, and the output bounding box.

6. A target tracking and detection device based on dynamic scoring and time window adjustment, characterized in that, The target tracking and detection device based on dynamic scoring and time window adjustment includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to cause the target tracking and detection device based on dynamic scoring and time window adjustment to perform the method as described in any one of claims 1-5.

7. A computer-readable storage medium comprising instructions, characterized in that, When the instruction is executed on the target tracking and detection device based on dynamic scoring and time window adjustment, the target tracking and detection device based on dynamic scoring and time window adjustment performs the method as described in any one of claims 1-5.

Images