A temporally stable method and system for video target detection based on continuous frame association

Abstract

This application discloses a temporal stabilization method and system for video target detection based on continuous frame association, relating to the technical fields of computer vision, video target detection, target tracking post-processing, and stable display of detection results. The method includes performing frame-by-frame target detection on continuous video frames to obtain detection results, which include detection boxes, category labels, and confidence scores; constructing a stable reference coordinate system constrained by the detection results; performing multi-attribute temporal stabilization processing on the detection results based on the stable reference coordinate system; performing target trajectory continuity management based on the multi-attribute temporal stabilization processing results; and superimposing the stabilized detection boxes, category labels, and confidence scores onto the original video image for output. This application, based on the output of existing frame-by-frame target detection models, performs continuous frame temporal stabilization processing on the detection results, thereby improving the stability and continuity of target detection results in continuous video.

Description

Technical Field

[0001] This application relates to the technical fields of computer vision, video target detection, target tracking post-processing, and stable display of detection results, specifically to a temporal stabilization method and system for video target detection based on continuous frame association. Background Technology

[0002] Deep learning detection algorithms, such as YOLO, Faster R-CNN, SSD, and RT-DETR, have been widely applied in scenarios including video surveillance, drone inspection, vehicle recording, action cameras, industrial vision inspection, and smart terminals. In video applications, object detection models typically output target bounding boxes, target categories, and confidence information frame by frame. However, due to factors such as lighting variations, slight occlusion, image blurring, background interference, model prediction errors, and slight camera movement within a single frame, frame-by-frame detection results are prone to instability over continuous time series.

[0003] Specifically, this manifests as follows: the target bounding box center jitters between adjacent frames; the bounding box width and height fluctuate wildly; the category label frequently switches between similar categories; the confidence score jumps rapidly within a short period; and when a target experiences a short-term missed detection, low confidence, or temporary mismatch, the detection box may suddenly disappear, and subsequent successful matching may result in discontinuous display. While these issues may not directly affect the detection accuracy of a single-frame detection model, they significantly impact the readability, continuity, and user experience of detection results in continuous video.

[0004] Existing methods have proposed improving detection box jitter by approximating the output feature data within the target detection network. However, these methods can only reduce minor jitter in the position of the detection box in dynamic videos to a certain extent, resulting in limited performance improvement. Summary of the Invention

[0005] To address the problems existing in the prior art, this application proposes a temporal stabilization method and system for video target detection based on continuous frame association. Based on the output of the existing frame-by-frame target detection model, this application performs temporal stabilization processing on the detection results for continuous frames, thereby improving the stability and continuity of target detection results in continuous video.

[0006] This application is achieved through the following technical solution:

[0007] A temporally stable method for video target detection based on consecutive frame association includes:

[0008] Frame-by-frame target detection is performed on consecutive video frames to obtain detection results, which include detection bounding boxes, category labels, and confidence scores.

[0009] Construct a stable reference coordinate system based on the constraints of the detection results;

[0010] Multi-attribute temporal stabilization processing is performed on the detection results based on the stable reference coordinate system;

[0011] Based on the results of multi-attribute temporal stability processing, target trajectory continuity management is performed;

[0012] The stabilized detection bounding boxes, class labels, and confidence scores are then overlaid onto the original video image for output.

[0013] In some implementations, constructing a stable reference coordinate system based on the constraints of the detection results includes:

[0014] Construct a foreground region based on the detection bounding box of the current frame, and remove the foreground region from the image region of the current frame to obtain the background region;

[0015] Estimate the global motion transformation between adjacent frames in the background region;

[0016] A camera motion trajectory sequence is constructed based on the global motion transformation, and the camera motion trajectory sequence is smoothed to obtain a stable reference coordinate system for the stabilization processing of the detection results.

[0017] In some implementations, the background area is obtained in the following way:

[0018] All detection boxes in the current frame are merged into the foreground region. The foreground region is then subjected to morphological dilation. After removing the dilated foreground region from the current frame image region, the remaining part is used as the background region.

[0019] And / or, estimating the global motion transformation between adjacent frames in the background region includes:

[0020] Feature points are extracted within the background area, and the extracted feature points are matched with feature points in the background area of ​​the previous frame image.

[0021] By eliminating abnormal matching point pairs through a robust estimation method, the global motion transformation of the current frame relative to the previous frame is estimated. The global motion transformation is used to describe the single-step motion relationship between two adjacent frames.

[0022] And / or, the stable reference coordinate system is obtained in the following way:

[0023] The global motion transformations from the starting frame to the current frame are multiplied sequentially to obtain the cumulative motion state of the current frame. The cumulative motion states corresponding to different time frames are used to form the original camera motion trajectory sequence.

[0024] Temporal smoothing is performed on the original camera motion trajectory sequence to obtain a smoothed camera motion trajectory sequence;

[0025] Based on the difference between the cumulative motion state corresponding to the current frame and the smooth cumulative motion state corresponding to the current frame, a compensation transformation for the current frame is constructed; the compensation transformation for the current frame is used to map the detection result of the current frame to a stable reference coordinate system.

[0026] In some implementations, performing multi-attribute temporal stabilization processing on the detection results based on the stable reference coordinate system includes:

[0027] The detection box of the current frame is mapped to the observation box in the stable reference coordinate system and associated with the existing trajectory set for matching: for matching pairs with a matching cost lower than a preset matching threshold, they are determined to be valid matches; for detection boxes that have not formed valid matches, new candidate trajectories are created and set to the initial state; for historical trajectories that have not formed valid matches, the number of consecutive unmatched frames is accumulated.

[0028] For the current frame with a valid match, time-series stabilization processing is performed on the three display attributes: detection box, category label, and confidence score.

[0029] In some implementations, mapping the detection box of the current frame to an observation box in the stable reference coordinate system and associating it with an existing trajectory set includes:

[0030] For each historical trajectory in the trajectory set, the prediction box of the current frame is obtained by extrapolation based on the trajectory state of the previous frame and the historical motion trend.

[0031] For each set of historical trajectories and the detection boxes of the current frame, the association cost is calculated; where the association cost is obtained by weighted fusion of the overlap, distance and category constraints between the predicted box and the observation box of the current frame in the stable reference coordinate system.

[0032] Construct a cost matrix based on all calculated associated costs;

[0033] The Hungarian algorithm is used to perform global optimal matching on the cost matrix to obtain the matching relationship between the trajectory and the detection box with the minimum total cost.

[0034] In some implementations, the time-series stabilization processing of the three display attributes—the detection box, the category label, and the confidence score—includes:

[0035] For the center position of the detection box, exponential smoothing is used for stable updates: the candidate display box of the current frame is obtained by fusing the stable display box of the previous frame and the observation box of the current frame in the stable reference coordinate system.

[0036] For the width and height of the detection box, calculate the single-frame change rate. When the single-frame change rate exceeds the corresponding preset threshold, limit the single-frame update amplitude to within the corresponding maximum update amplitude; otherwise, update according to the exponential smoothing result.

[0037] For each category label, a cumulative score for each cross-frame associated target is maintained and continuously updated using a time-smoothing method; the category with the highest cumulative score is selected as the candidate new category for the current frame.

[0038] For confidence level, internal trajectory association and state judgment, the original detection confidence level is still used, and the external display output is a smoothed stable confidence level. Among them, the confidence level smoothing coefficient adopts a piecewise adaptive strategy: when the confidence level jump amplitude is greater than the confidence level jump threshold, the confidence level smoothing coefficient is taken as the smoothing coefficient under strong smoothing state; otherwise, the confidence level smoothing coefficient is taken as the smoothing coefficient under constant smoothing state.

[0039] In some implementations, a category switching threshold and a preset number of consecutive frames are set for the category label. The display category is switched to the candidate new category only when the cumulative score of the candidate new category exceeds the cumulative score of the current display category and the difference between the two is greater than the category switching threshold, and the preset number of consecutive frames is met continuously. Otherwise, the original display category remains unchanged.

[0040] In some implementations, the method of performing target trajectory continuity management based on multi-attribute temporal stability processing results includes:

[0041] For each target trajectory, four basic states are established: initialization state, stable tracking state, short-term hold state, and termination state. The transitions between states are managed based on the number of consecutive successful matching frames and the number of consecutive unmatched frames.

[0042] When the number of consecutive successful matches of a candidate trajectory in the initialization state reaches a preset threshold, it enters the stable tracking state; when a candidate trajectory in the initialization state experiences consecutive unmatches before the number of consecutive successful matches reaches the preset threshold, and the number of consecutive unmatched frames exceeds the termination threshold, or the center of the prediction box exceeds the image area, it enters the termination state.

[0043] The target trajectory that is continuously matched is in a stable tracking state. When the trajectory does not get a valid match in the current frame, it enters a short-term hold state.

[0044] When the trajectory fails to obtain a valid match in the current frame, and the number of consecutive unmatched frames does not exceed the short-term holding threshold, the trajectory is in the display holding phase of the short-term holding state, and the target's current position is predicted based on the historical motion trend to continue to maintain the target display;

[0045] When the number of consecutive unmatched frames is greater than the short-term holding threshold but does not exceed the termination threshold, the trajectory is in the non-display holding stage of the short-term holding state and is no longer displayed normally, but the trajectory information is still retained and participates in subsequent association judgment.

[0046] When the number of consecutive unmatched frames exceeds the termination threshold or the center of the prediction box exceeds the image area, the trajectory enters the termination state.

[0047] In some implementations, the method of performing target trajectory continuity management based on multi-attribute temporal stability processing results further includes: setting two types of auxiliary events: uncertain events and recovery display processing;

[0048] Among them, the uncertain event is as follows: For a matched target, an uncertainty index is calculated. When the uncertainty index exceeds the uncertainty threshold, an uncertain event is triggered. When the uncertainty index falls back below the recovery threshold, normal updates resume. The uncertainty threshold is greater than the recovery threshold.

[0049] Restore display processing: When a target in the non-display retention phase that is in a short-term hold state meets the association conditions again, the restore display processing is triggered, and the target is reintroduced into the normal display process.

[0050] On the other hand, this application also proposes a temporal stabilization system for video target detection based on continuous frame correlation, including:

[0051] The target detection unit is used to perform frame-by-frame target detection on consecutive video frames to obtain detection results, which include detection boxes, category labels, and confidence scores.

[0052] A reference coordinate system construction unit is used to construct a stable reference coordinate system based on the constraints of the detection results;

[0053] A stabilization processing unit is used to perform multi-attribute temporal stabilization processing on the detection result based on the stable reference coordinate system;

[0054] The state manager is used to perform target trajectory state continuity management based on the results of the multi-attribute temporal stability processing.

[0055] And an output unit, used to overlay the stabilized detection boxes, class labels and confidence scores onto the original video image for output.

[0056] This application proposes a temporal stabilization method for video target detection based on continuous frame association. This method optimizes the detection results through post-processing, building upon existing frame-by-frame target detection output. Specifically, it first constructs a foreground mask using the detection bounding boxes and estimates global motion parameters in the background region, reducing interference from moving targets on global motion estimation and improving the targeting of image compensation. Then, it establishes independent continuous frame stabilization mechanisms for the detection bounding box position and size, category label, and confidence score, effectively suppressing detection bounding box jitter, abrupt size changes, label flickering, and confidence score jumps. Simultaneously, a target state management mechanism handles cases of missed detections, occlusion, and short-term out-of-frame occurrences, reducing display interruptions in continuous video. Compared to the original frame-by-frame detection display method, this method effectively improves display stability and continuity.

[0057] Correspondingly, the temporal stabilization system for video target detection based on continuous frame association proposed in this application also possesses the same technical effects as described above. Attached Figure Description

[0058] The accompanying drawings, which are included to provide a further understanding of the embodiments of this application and form part of this application, do not constitute a limitation on the embodiments of this application. In the drawings:

[0059] Figure 1 This is a schematic diagram of the method flow proposed in the embodiments of this application; high-level sampling is processed in the same way.

[0060] Figure 2 This is a schematic diagram illustrating the process of constructing a stable reference coordinate system according to an embodiment of this application;

[0061] Figure 3 This is a schematic diagram of the multi-attribute temporal stability processing flow of the detection results in an embodiment of this application;

[0062] Figure 4 This is a schematic diagram of the target trajectory continuity management process according to an embodiment of this application;

[0063] Figure 5 This is a system principle block diagram proposed in the embodiments of this application;

[0064] Figure 6 This is a schematic diagram of the electronic device proposed in the embodiments of this application;

[0065] Figure 7 This is a schematic diagram of a computer-readable storage medium proposed in an embodiment of this application;

[0066] Figure reference numerals and corresponding component names:

[0067] 200-Video target detection temporal stabilization system, 201-Target detection unit, 202-Reference coordinate system construction unit, 203-Stabilization processing unit, 204-State manager, 205-Output unit, 300-Electronic device, 310-Memory, 320-Processor, 311-Computer program A, 400-Computer-readable storage medium, 411-Computer program B. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the embodiments and accompanying drawings. The illustrative embodiments and descriptions of this application are only for explaining this application and are not intended to limit this application.

[0069] Existing video object detection systems typically overlay the detection results frame by frame onto the video screen, leading to significant temporal instability in the detection results across continuous video. Specifically, the existing frame-by-frame object detection display method has the following main drawbacks:

[0070] (1) The timing of the detected attributes is unstable: When the target is relatively stable in the picture or the actual size changes little, due to the single-frame detection error, the center position of the detection box may still jump between adjacent frames, and the width and height may also change abnormally, resulting in visible position jitter and size change of the detection box in continuous video. At the same time, in the case of similar categories, partial occlusion, low resolution or complex background, the detection category may switch frequently frame by frame, and the confidence level is also prone to large fluctuations in a short period of time, causing the category label to flicker and the confidence level display to be unstable.

[0071] (2) Discontinuous display of target detection results: When the target is briefly missed, occluded, has low confidence, or briefly leaves the frame, the frame-by-frame detection method usually manifests as the detection box suddenly disappearing or suddenly appearing between consecutive frames, resulting in discontinuous display of detection results and affecting the user's continuous observation of the target status.

[0072] (3) Stability of continuous frame detection due to slight camera motion interference: In scenarios such as handheld shooting, vehicle-mounted cameras, and drones, slight camera shake will introduce global coordinate offset between adjacent frames. If the global camera disturbance is not distinguished from the actual movement of the target, the continuous frame stabilization of the detection box is easily affected by the global displacement of the image, resulting in a decrease in stabilization effect.

[0073] To address the aforementioned problems, this application proposes a temporal stabilization method for video target detection based on continuous frame association, such as... Figure 1 As shown, the temporally stable method for video object detection includes the following steps:

[0074] Step 1: Perform frame-by-frame target detection on consecutive video frames to obtain detection results, including detection boxes, category labels, and confidence scores;

[0075] Step 2: Construct a stable reference coordinate system based on the constraints of the detection results;

[0076] Step 3: Perform multi-attribute temporal stabilization processing on the detection results based on a stable reference coordinate system;

[0077] Step 4: Based on the results of multi-attribute temporal stability processing, perform target trajectory continuity management;

[0078] Step 5: Overlay the stabilized detection boxes, category labels, and confidence scores onto the original video image for output.

[0079] Furthermore, in step 1 of the embodiments of this application, the frame-by-frame target detection technology uses an existing frame-by-frame target detection model to achieve detection, and obtains the detection box, category label and confidence information, which will not be elaborated here.

[0080] Furthermore, in step 2 of this application embodiment, the process of constructing a stable reference coordinate system based on detection result constraints includes:

[0081] Construct a foreground region based on the detection box of the current frame, and remove the foreground region from the image region of the current frame to obtain an effective background region;

[0082] Estimate the global motion transformation between adjacent frames in the background region;

[0083] A camera motion trajectory sequence is constructed based on the global motion transformation, and the camera motion trajectory sequence is smoothed to obtain a stable reference coordinate system for stabilizing the detection results.

[0084] Specifically, the instability of video object detection results stems partly from the frame-by-frame prediction noise of the detection model itself, and partly from slight camera movement. In scenarios such as handheld shooting, rotating cameras, vehicle-mounted cameras, or drone shooting, camera or sensor movement may introduce global coordinate offsets between adjacent frames, causing additional positional changes in the detection boxes for the same object. If subsequent smoothing processing does not distinguish between these two sources, the coordinate offsets caused by the camera will be mixed with the prediction noise of the detection model, affecting the stabilization effect; without processing, slight camera shake will be directly reflected in the displayed detection boxes, causing visible overall drift or jitter in the detection boxes.

[0085] To address this issue, this application introduces a background region global motion estimation mechanism based on detection result constraints to construct a stable reference coordinate system synchronized frame by frame. This allows subsequent detection box association and stable updates to be performed in a coordinate space with reduced global disturbances, thereby reducing the interference of camera global motion on trajectory association and stable detection box updates. Figure 2 As shown, the process of constructing a stable reference coordinate system mainly includes:

[0086] Foreground exclusion and background region extraction: The foreground region is constructed using the detection results of the current frame, and the background region is obtained. Specifically, all detection boxes in the current frame are merged into the foreground region, and morphological dilation is performed to cover the target edge and its adjacent area. After removing the dilated foreground region from the current frame image region, the remaining part is taken as the background region, represented as: ;in, The current frame image region; The foreground region formed by merging all detection boxes in the current frame; For the current frame image, the first One detection box; This represents the number of detection boxes in the current frame image. This is an operator for morphological dilation of the foreground region; symbol " This indicates that the dilated foreground region is removed from the current frame's image region; The background area is considered. If the target occupies a large portion of the image or moves quickly, directly performing feature matching on the entire image will cause the target's motion to be mixed into the global motion estimation, leading to estimation errors. This application's embodiments reduce the interference of the foreground target's own motion on the global motion estimation by excluding the foreground, thus ensuring that the estimated motion transformation primarily reflects the global motion of the camera or the image.

[0087] Background feature extraction and matching: Extract feature points within the background region and match them with feature points in the background region of the previous frame image.

[0088] Global motion estimation: A robust estimation method is used to eliminate outlier matching pairs, estimating the global motion transformation of the current frame relative to the previous frame. This global motion transformation describes the single-step motion relationship between adjacent frames. To analyze camera motion trends over a longer timeframe and distinguish high-frequency jitter components, the single-step transformations of each frame are accumulated sequentially to construct the cumulative camera motion state from the video's starting frame to the current frame. Specifically, the first... The original camera cumulative motion state of the frame is the previous frame. The sequential multiplication of the frame global motion transform is represented as:

[0089] ;

[0090] Among them, the initial frame order It is the identity matrix; This represents the cumulative motion state of the previous frame. This represents the global motion transformation of the current frame relative to the previous frame. This represents the cumulative motion state of the current frame. The cumulative motion state corresponds to different time frames. This forms the original camera motion trajectory sequence.

[0091] Camera motion trajectory sequence construction and smoothing: Temporal smoothing is performed on the original camera motion trajectory sequence to obtain a smoothed camera motion trajectory sequence; where the smoothed cumulative motion state corresponding to the current frame is... Represented as: ;in, Indicates from the first Frame to the The original cumulative motion state sequence of the frame; Indicates the length of the time-domain smoothing window; This represents a time-domain smoothing operator, which can be implemented using a moving average method. When When this happens, the existing cumulative motion state from the start frame to the current frame can be used for smoothing.

[0092] Constructing a compensation transformation: The difference between the original camera motion trajectory sequence and the smoothed camera motion trajectory sequence reflects the high-frequency disturbance bias in the trajectory motion. Therefore, in this embodiment, a compensation transformation for the current frame can be constructed based on the difference between the cumulative motion state corresponding to the current frame and the smoothed cumulative motion state corresponding to the current frame, expressed as: ;in, This is a compensation transformation for the current frame; for The inverse transform. This compensation transform maps the detection results of the current frame to a stable reference coordinate system that reduces the impact of global disturbances. All subsequent stabilization processes are performed in this stable reference coordinate system, and then... The results are then mapped back to the original image coordinate system for display. for The inverse transform of .

[0093] Furthermore, in step 3 of this embodiment, multi-attribute temporal stabilization processing is performed on the detection results, including:

[0094] Map the detection box of the current frame to a stable reference coordinate system, and update the position of the detection box in time by combining the historical trajectory prediction results;

[0095] Constraints on the rate of change of the detection frame width and height are established to suppress abrupt changes in the size of the detection frame;

[0096] Establish a cumulative score mechanism for category labels and suppress category label flickering through a category switching threshold;

[0097] The confidence level is smoothed using time-series smoothing with jump suppression to reduce rapid fluctuations in the confidence level.

[0098] Specifically, after obtaining a stable reference coordinate system, this embodiment establishes relatively independent time-series stable channels for the three display attributes of the detection results: detection box, category label, and confidence level. The reason for processing these three attributes separately, rather than using a uniform smoothing method, is that their fluctuation characteristics and stability requirements are fundamentally different: the position and size of the detection box are continuous quantities, suitable for exponential smoothing and rate-of-change constraints; the category label is a discrete quantity and cannot be directly averaged, requiring a cumulative voting mechanism; the confidence level can be used for both internal judgment and external display, and the response requirements for the two differ, thus requiring separate processing. The specific processing procedure is as follows: Figure 3 As shown, it includes:

[0099] Stable reference coordinate system mapping: When mapping the detection box, the four corner points of the detection box are mapped through compensation transformation, and the smallest bounding rectangle of the four corner points after mapping is taken as the mapped detection box. The video background image remains the original output. The compensation transformation is only used to map the detection box coordinates to the stable reference coordinate system, thereby avoiding the image quality degradation caused by full-frame resampling.

[0100] Continuous frame association matching: Before entering each attribute-stabilized channel, it is necessary to associate the current frame detection result with the existing trajectory set. Specifically, let the current total number of... Historical trajectory and The current frame detection box, for the first Trajectory and the first For each detection box, construct the association cost function. , is represented as:

[0101] ;

[0102] in, This represents the prediction box for the m-th trajectory in the current frame (obtained by extrapolation based on the trajectory state of the previous frame and historical motion trends). This represents the observation box that the nth detection box is mapped to in the stable reference coordinate system after compensation transformation; and These are the center of the prediction box and the center of the observation box, respectively. For intersection, union, and comparison; To predict the diagonal length of the bounding box, the center distance is used for scale normalization. To prevent tiny constants with a denominator of zero; This is a category constraint term; it takes a smaller value when the trajectory category matches the detection category, and a larger value when they do not match. , and These are the weight coefficients for the overlap term, distance term, and category term, respectively. The L2 norm is used to calculate the distance between two points. The association cost function described above can be used to calculate the association cost between each pair of historical trajectories and the current frame detection box.

[0103] Construct a cost matrix based on all calculated associated costs. : ;in, for Real matrix; the first value in this cost matrix Line 1 Column elements represent the first The historical trajectory and the first The matching cost between the current detection boxes.

[0104] The Hungarian algorithm is used to perform global optimal matching of the cost matrix, obtaining the matching relationship between the trajectory and the detection box with the minimum total cost. If the cost of a matching pair is lower than a preset matching threshold, it is determined to be a valid match and proceeds to the subsequent temporal stabilization process. For detection boxes that have not formed a valid match, new candidate trajectories are established and set to the initial state. For historical trajectories that have not formed a valid match, the number of consecutive unmatched frames is accumulated, and the process proceeds to the subsequent target state continuity management process.

[0105] Perform attribute-based time-series stabilization processing:

[0106] I. Timing Stability of Detection Boxes: Detection box stability primarily aims to suppress minor fluctuations in the center position between adjacent frames, as well as abnormal changes in width and height over short periods. Specifically, the center position is updated using exponential smoothing, while the width and height are constrained by rate-of-change constraints. For instance, the center position of the detection box is stably updated using exponential smoothing, as shown below:

[0107] ;

[0108] in, Candidate display boxes for the current frame; This is the stable display frame for the previous frame; This is the observation frame of the current frame in a stable reference coordinate system; This is a smoothing coefficient used to adjust the fusion ratio between the historical display frame and the current frame observation frame.

[0109] The causes of abrupt changes in width and height differ from those of center position jitter. These changes are typically caused by local occlusion, truncation, or occasional errors in size estimation of the target. Furthermore, the reasonable rate of change in width and height for the same target across consecutive frames is much lower than the rate of change in position. Therefore, this application establishes separate rate-of-change constraints for the detection frame width and height. Taking width as an example, the rate of change for single-frame width is defined as the difference between the candidate width of the current frame and the stable display width of the previous frame, divided by the stable display width of the previous frame (a small constant is added to the denominator to avoid division by zero). This represents the relative change in width between the current frame and the previous frame. When this rate of change exceeds a preset threshold, the update magnitude of the current frame width is limited to within the preset maximum update magnitude for a single frame width; otherwise, the width is updated according to the exponential smoothing result described above. Height is processed in the same way, with corresponding height change rate thresholds and maximum height update amplitudes set. That is, the height change rate of a single frame is calculated as the difference between the candidate height of the current frame and the stable display height of the previous frame, divided by the stable display height of the previous frame (a small constant is added to the denominator to avoid division by zero), which is the relative change in height between the current frame and the previous frame. When the height change rate exceeds the preset height change rate threshold, the update amplitude of the current frame height is limited to within the preset maximum single-frame height update amplitude; otherwise, the height is updated according to the above exponential smoothing result.

[0110] Through the above processing, the embodiments of this application maintain the target following response capability while reducing the jitter and size change of the detection box caused by single-frame anomaly detection.

[0111] II. Stable Temporal Sequence of Category Labels: The instability of category labels mainly manifests as: in situations involving similar categories, partial occlusion, partial blurring, or low resolution, the category detected frame by frame exhibits intermittent changes between multiple categories. Directly displaying the frame-by-frame categories would cause the labels to flicker visually. To address this, this application maintains a cumulative score for each category for each cross-frame associated target and continuously updates it using a time-smoothing method, specifically as follows:

[0112] ;

[0113] in, For the current frame, the trajectory is categorized. The cumulative score; For the previous frame, the category The cumulative score; The category in the current frame detection result Normalized confidence score; The time smoothing coefficient for the score in each frame. The larger the value, the higher the weight of the historical cumulative score. The smaller the value, the greater the impact of the current frame detection result on the cumulative score.

[0114] Candidate display categories are obtained based on cumulative scores: the category with the highest cumulative score is selected as the candidate new category for the current frame. Since the cumulative score is a weighted sum of historical information from multiple frames, the impact of a single frame misjudgment on the score is limited, thus suppressing frame-by-frame fluctuations in category labels over time.

[0115] To further prevent frequent label switching when scores are close, a category switching threshold and a preset number of consecutive frames are set. The display category will only be switched to the candidate new category if the cumulative score of the candidate new category exceeds the cumulative score of the currently displayed category, the difference between the two is greater than the category switching threshold, and the preset number of consecutive frames is met consecutively; otherwise, the original display category will remain unchanged.

[0116] Through the above processing, misjudgment in a single frame will not immediately change the target display category. Only a continuously stable new category will trigger category switching, thereby reducing category label flickering.

[0117] III. Stable Confidence Level: When confidence levels are displayed frame by frame directly, numerical jumps can easily occur due to single-frame detection noise, leading users to misjudge the detection stability. However, excessive smoothing of the confidence level reduces the system's responsiveness to real changes in the target confidence level. To address this, this application's embodiment separates internal confidence level assessment from externally displayed confidence level: internal trajectory association, state judgment, and other logic continue to use the original detection confidence level to maintain sensitive response to real changes; the externally displayed stable confidence level, after smoothing, is represented as:

[0118] ;

[0119] in, The stability confidence level for the current frame to be displayed externally; The original detection confidence level for the current frame; The stability confidence level of the previous frame displayed externally; This represents the confidence smoothing coefficient for the current frame. The confidence smoothing coefficient employs a piecewise adaptive strategy: when... When the confidence level jump is greater than the confidence level jump threshold, set the confidence level smoothing coefficient... Take the smaller value To enhance smoothing suppression; otherwise, the confidence smoothing coefficient Take the larger value To maintain the ability to adapt to changes normally. Among them, The smoothing coefficient under strong smoothing conditions; Let be the smoothing coefficient under constant smoothing state, satisfying Both take values ​​in (0,1); To trigger a smooth confidence jump threshold.

[0120] Furthermore, in step 4 of this application embodiment, performing target trajectory state continuity management includes:

[0121] For each target trajectory, four basic states are established: initialization state, stable tracking state, short-term holding state, and termination state. The transition between each state is managed based on the number of consecutive successful matching frames and the number of consecutive unmatched frames.

[0122] Two types of auxiliary events are set up: uncertain events and display recovery processing. These are used to suppress abnormal detection frames and the re-inclusion of targets after short-term mismatch, respectively, in order to maintain the display continuity of detection results in abnormal scenarios.

[0123] Specifically, the discontinuity of video target detection results manifests not only in the jitter of the detection box position and size, but also in the display interruption caused by single-frame missed detections, low confidence, target scale changes, target entering or leaving the frame, or unstable trajectory matching in continuous video. Traditional frame-by-frame detection and display methods typically employ a "display when detected, disappear when not detected" strategy. When a target does not achieve stable detection results within a few consecutive frames, problems such as the detection box suddenly disappearing, reappearing, or trajectory numbering becoming discontinuous can easily occur. To address this, this application establishes a state continuity management mechanism for each target trajectory. The trajectory state includes four basic states: initialization state, stable tracking state, short-term holding state, and termination state. Simultaneously, two types of auxiliary events are set: uncertain events and display recovery processing, to handle abnormal detection frames and the re-inclusion problem after short-term mismatch, respectively.

[0124] The specific logic for managing the continuity of target trajectory status is as follows: Figure 4 As shown, it includes:

[0125] Initialization state: Newly emerging candidate trajectories that have not yet undergone continuous frame association matching are in the initialization state; when the number of consecutively matched frames of a candidate trajectory reaches a preset threshold, it enters the stable tracking state; when the number of consecutively matched frames of a candidate trajectory fails to match before reaching the preset threshold, and the number of consecutively unmatched frames exceeds the preset initialization termination threshold, or the center of the prediction box exceeds the image area, the candidate trajectory is considered not to have been effectively confirmed and enters the termination state.

[0126] Stable tracking state: The target trajectory that is continuously matched is in the stable tracking state, which is the main state in which the target is displayed normally; when the trajectory does not get a valid match in the current frame, it enters the short-term hold state.

[0127] The short-term hold state occurs when a trajectory fails to find a valid match in the current frame. This state includes a display hold phase and a non-display hold phase. When the number of consecutive unmatched frames does not exceed the short-term hold threshold, the trajectory is in the display hold phase. During this phase, the target's current position is predicted based on historical motion trends, and the target is maintained in the display, thus avoiding display interruption due to a single frame's missed detection. When the number of consecutive unmatched frames exceeds the short-term hold threshold but does not exceed the termination threshold, the trajectory is in the non-display hold phase and is no longer displayed normally, but the trajectory information is still retained and participates in subsequent association judgments. When the number of consecutive unmatched frames exceeds the termination threshold, or the center of the predicted bounding box exceeds the image area, the trajectory enters the termination state.

[0128] Termination State: When the number of consecutive unmatched frames of the target exceeds the termination threshold, or when the center of the predicted box goes out of the image area (i.e., out of frame), the trajectory enters the termination state and stops normal display.

[0129] Maintain a continuous matching success count for each target trajectory. Consecutive unmatch count When the trajectory is successfully matched in the current frame, Add 1, and Reset to 0; when the trajectory does not match a detection result in the current frame, the consecutive non-match count will be reset. Increment by 1, meaning the number of unmatched matches is accumulated frame by frame. Reset to 0.

[0130] when The trajectory enters a short-term display holding phase, predicting the current display position based on historical movement trends and continuing to display it for a short period of time;

[0131] when The trajectory enters a short-term hold phase, a non-display retention phase, where it is no longer displayed normally but still retains trajectory information and participates in subsequent association judgments. If the association conditions are met again in this phase, a display recovery process is triggered, bringing the target back into the normal display process and transitioning to a stable tracking state. Simultaneously, the consecutive unmatched count is updated. Reset to 0;

[0132] when Alternatively, if the center of the prediction bounding box extends beyond the image area, the trajectory enters a terminated state and stops displaying normally. The maximum number of consecutive unmatched frames allowed for short-term state maintenance; To determine the threshold for the number of consecutive unmatched frames at which the target will terminate display, and . Used to control the length of time the display continues in the event of a short-term missed detection. Used to control the length of time for retaining trajectory information and for final termination.

[0133] The two types of auxiliary event handling are as follows:

[0134] Uncertainty Events: For a matched target, the trajectory uncertainty index is defined as:

[0135] ;

[0136] in, This is an indicator of the uncertainty of the current frame trajectory; To ensure consistency in overlap between the predicted bounding box and the observed bounding box; This is the normalized position residual; , and These are the weighting coefficients for the overlap term, the position term, and the confidence fluctuation term, respectively.

[0137] When the uncertainty index exceeds the uncertainty threshold, an uncertainty event is triggered, reducing the impact of the current observation on the display frame and making the display result more dependent on the historical trajectory, thereby suppressing the display jump caused by abnormal detection frames; when the uncertainty index falls back below the recovery threshold, normal updates are resumed, where the uncertainty threshold is greater than the recovery threshold.

[0138] Resume Display Processing: When a target in the retention phase meets the association conditions again, the resume display processing is triggered, reintegrating the target into the normal display flow and reducing abrupt changes when the detection box reappears through a smooth transition. This processing is an auxiliary event and does not exist as a separate basic state.

[0139] Based on existing frame-by-frame target detection results, this embodiment first maps the detection results to a relatively stable reference coordinate system through foreground exclusion and background motion estimation, reducing the impact of slight camera movement on target association and display stability. Secondly, it performs stabilization processing on three display attributes—detection box, category label, and confidence level—to reduce detection box jitter, sudden changes in box size, category label flickering, and confidence level jumps. Finally, through state management such as initialization, stable tracking, short-term hold, and termination, combined with auxiliary events such as uncertain events and display recovery processing, it reduces display interruptions caused by single-frame missed detections, short-term low confidence, target entry / exit from the frame, or unstable temporary matching. Therefore, this embodiment transforms the original frame-by-frame detection output into a stable and continuous display result based on continuous frame target association.

[0140] Based on the same technical concept described above, this application also proposes a temporally stable video target detection system based on continuous frame association, such as... Figure 5 As shown, the video target detection temporal stabilization system 200 includes:

[0141] The object detection unit 201 is used to perform frame-by-frame object detection on consecutive video frames to obtain detection results, including detection boxes, category labels, and confidence scores. Specific object detection methods can be implemented using existing object detection models, which will not be elaborated here.

[0142] The reference coordinate system construction unit 202 is used to construct a stable reference coordinate system based on the constraints of the detection results. The specific method for constructing the stable reference coordinate system is as described in step 2 above, and will not be elaborated further here.

[0143] The stabilization unit 203 is used to perform multi-attribute temporal stabilization processing on the detection results based on a stable reference coordinate system. The specific multi-attribute temporal stabilization processing method is as described in step 3 above, and will not be repeated here.

[0144] State manager 204 is used to perform target trajectory state continuity management based on the results of multi-attribute temporal stability processing. The specific target trajectory state continuity management process is as described in step 4 above, and will not be repeated here.

[0145] And output unit 205, which is used to overlay the stabilized detection box, class label and confidence score onto the original video image for output.

[0146] Based on the same technical concept described above, this application also proposes an electronic device, such as... Figure 6 As shown, the electronic device 300 includes: a memory 310, a processor 320, and a computer program A311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program A311, it performs the following steps:

[0147] Perform frame-by-frame target detection on consecutive video frames to obtain detection results, including detection boxes, class labels, and confidence scores;

[0148] Construct a stable reference coordinate system based on the constraints of the detection results;

[0149] Multi-attribute temporal stabilization processing is performed on the detection results based on a stable reference coordinate system;

[0150] Based on the results of multi-attribute temporal stability processing, target trajectory continuity management is performed;

[0151] The stabilized detection bounding boxes, class labels, and confidence scores are then overlaid onto the original video image for output.

[0152] Optionally, when the processor 320 executes the computer program A311, it can implement any of the embodiments in the corresponding examples of the above-described video target detection timing stabilization method.

[0153] It should be noted that the electronic device proposed in this application embodiment is a device used to implement the above-described video target detection timing stabilization method. Therefore, based on the above-described video target detection timing stabilization method proposed in this application embodiment, those skilled in the art can understand the specific implementation method and various variations of the electronic device in this application embodiment. Therefore, how the electronic device specifically implements the above-described video target detection timing stabilization method will not be described in detail here. Any electronic device used by those skilled in the art to implement the above-described video target detection timing stabilization method falls within the scope of protection of this application.

[0154] Based on the same technical concept described above, embodiments of this application also propose a computer-readable storage medium, such as... Figure 7 As shown, the computer-readable storage medium 400 stores a computer program B411, which, when executed by a processor, performs the following steps:

[0155] Perform frame-by-frame target detection on consecutive video frames to obtain detection results, including detection boxes, class labels, and confidence scores;

[0156] Construct a stable reference coordinate system based on the constraints of the detection results;

[0157] Multi-attribute temporal stabilization processing is performed on the detection results based on a stable reference coordinate system;

[0158] Based on the results of multi-attribute temporal stability processing, target trajectory continuity management is performed;

[0159] The stabilized detection bounding boxes, class labels, and confidence scores are then overlaid onto the original video image for output.

[0160] Optionally, when the computer program B411 is executed by the processor, it can implement any of the embodiments corresponding to the above-described video target detection temporal stabilization method.

[0161] Furthermore, to verify the effectiveness of the temporally stable video target detection method proposed in the above embodiments, this application uses the MOT17 public benchmark dataset, selects three representative video sequences, uses YOLOv8n as the basic detection model, and sets three comparison methods: baseline (direct display of YOLOv8n frame-by-frame detection), EMA smoothing, and the temporally stable video target detection method proposed in this application (hereinafter referred to as this method).

[0162] Among them, the MOT17 (Multiple Object Tracking 2017) public benchmark dataset provides a wealth of real-world video sequences and high-quality human-annotated ground truth (GT) values, covering a variety of shooting scenarios and camera motion conditions, providing a reliable evaluation environment for verifying the temporal stability of detection results.

[0163] The three representative video sequences selected cover three typical shooting scenarios: stationary camera (MOT17-04-SDP), slight motion (MOT17-09-SDP), and significant motion (MOT17-13-SDP). Specifically, MOT17-04-SDP has 1050 frames and 83 ground truth (GT) tracks, indicating multiple targets and a long frame count, used to verify multi-target stabilization; MOT17-09-SDP has 525 frames and 26 GT tracks, used to verify motion compensation; and MOT17-13-SDP has 750 frames and 90 GT tracks, used to verify extreme scenarios.

[0164] Of the three comparison methods: the baseline, i.e., the YOLOv8n frame-by-frame detection results, is displayed directly without any post-processing; EMA smoothing is an exponential moving average of the continuous frame detection boxes in the YOLOv8n frame-by-frame detection results; this method performs post-processing on the YOLOv8n frame-by-frame detection results, comprehensively introducing motion compensation, multi-attribute stabilization, and state continuity management.

[0165] This application's embodiments design five stability evaluation indicators (as shown in Table 1), all using the ground truth (GT) trajectory as a reference. The indicators are calculated by grouping the display box and the GT box after performing IoU matching. Lower values ​​for all five indicators generally indicate greater stability.

[0166] Table 1 Five stability assessment indicators

[0167]

[0168] It should be noted that the frame center jitter index is calculated by subtracting the actual displacement of adjacent frames from the ground plane (GT) to reduce the impact of the actual target motion on the index, and to focus more on reflecting the timing fluctuations of the display frame.

[0169] The experimental results of the three comparison methods are shown in Table 2.

[0170] Table 2 Experimental results of three comparison methods

[0171]

[0172] As shown in Table 2, compared to the direct display of YOLOv8n frame-by-frame detection results, the EMA smoothing method reduced the average performance of the five indicators by approximately 14.4%, 41.8%, 41.6%, 52.5%, and 2.4%, respectively. This indicates that simple exponential smoothing has a certain mitigating effect on detection frame size fluctuations and confidence jumps. The present invention reduced the performance of the five indicators by approximately 23.2%, 46.5%, 44.1%, 55.1%, and 28.0%, respectively. Compared to the EMA smoothing method, the present invention reduced the performance of frame center jitter, width variation, height variation, confidence jump, and average number of display segments by approximately 5.4%, 8.1%, 4.6%, 5.2%, and 26.3%, respectively, indicating that the present invention shows a further improvement trend in this set of test sequences.

[0173] In sequences with significant camera motion, such as MOT17-13, the frame center jitter index of this invention increased slightly by about 3.0% compared to the baseline, indicating that even with large camera motion, this index may still be affected by global motion estimation errors, detection box matching bias, and temporal smoothing lag. In contrast, the EMA smoothing method increased this index by 33.9%, while this method decreased it by about 23.1%, indicating that this method still has a certain suppressive effect on frame center jitter in this sequence.

[0174] Overall, this method demonstrates good stability in terms of detection box width and height, confidence level, and display continuity. Regarding box center jitter, there are some differences between different sequences, but in some scenarios, its performance is close to that of the EMA method. Experimental results show that this method can reduce temporal fluctuations in detection results and improve the display stability and continuity of continuous video target detection results in most test scenarios.

[0175] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0176] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0177] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0178] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0179] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0180] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only a specific embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A temporally stable method for video target detection based on continuous frame association, characterized in that, include: Frame-by-frame target detection is performed on consecutive video frames to obtain detection results, which include detection bounding boxes, category labels, and confidence scores. Construct a stable reference coordinate system based on the constraints of the detection results; Multi-attribute temporal stabilization processing is performed on the detection results based on the stable reference coordinate system; Based on the results of multi-attribute temporal stability processing, target trajectory continuity management is performed; The stabilized detection bounding boxes, class labels, and confidence scores are then overlaid onto the original video image for output.

2. The temporal stabilization method for video target detection based on continuous frame association according to claim 1, characterized in that, The construction of a stable reference coordinate system based on the constraints of the detection results includes: Construct a foreground region based on the detection bounding box of the current frame, and remove the foreground region from the image region of the current frame to obtain the background region; Estimate the global motion transformation between adjacent frames in the background region; A camera motion trajectory sequence is constructed based on the global motion transformation, and the camera motion trajectory sequence is smoothed to obtain a stable reference coordinate system for the stabilization processing of the detection results.

3. The temporal stabilization method for video target detection based on continuous frame association according to claim 2, characterized in that, The background area is obtained as follows: All detection boxes in the current frame are merged into the foreground region. The foreground region is then subjected to morphological dilation. After removing the dilated foreground region from the current frame image region, the remaining part is used as the background region. And / or, estimating the global motion transformation between adjacent frames in the background region includes: Feature points are extracted within the background area, and the extracted feature points are matched with feature points in the background area of ​​the previous frame image. By eliminating abnormal matching point pairs through a robust estimation method, the global motion transformation of the current frame relative to the previous frame is estimated. The global motion transformation is used to describe the single-step motion relationship between two adjacent frames. And / or, the stable reference coordinate system is obtained in the following way: The global motion transformations from the starting frame to the current frame are multiplied sequentially to obtain the cumulative motion state of the current frame. The cumulative motion states corresponding to different time frames are used to form the original camera motion trajectory sequence. Temporal smoothing is performed on the original camera motion trajectory sequence to obtain a smoothed camera motion trajectory sequence; Based on the difference between the cumulative motion state corresponding to the current frame and the smooth cumulative motion state corresponding to the current frame, a compensation transformation for the current frame is constructed; the compensation transformation for the current frame is used to map the detection result of the current frame to a stable reference coordinate system.

4. A temporal stabilization method for video target detection based on continuous frame association according to any one of claims 1-3, characterized in that, The multi-attribute temporal stabilization processing of the detection results based on the stable reference coordinate system includes: The detection box of the current frame is mapped to the observation box in the stable reference coordinate system and associated with the existing trajectory set for matching: for matching pairs with a matching cost lower than a preset matching threshold, they are determined to be valid matches; for detection boxes that have not formed valid matches, new candidate trajectories are created and set to the initial state; for historical trajectories that have not formed valid matches, the number of consecutive unmatched frames is accumulated. For the current frame with a valid match, time-series stabilization processing is performed on the three display attributes: detection box, category label, and confidence score.

5. The temporal stabilization method for video target detection based on continuous frame association according to claim 4, characterized in that, The process of mapping the detection box of the current frame to the observation box in the stable reference coordinate system and associating and matching it with the existing trajectory set includes: For each historical trajectory in the trajectory set, the prediction box of the current frame is obtained by extrapolation based on the trajectory state of the previous frame and the historical motion trend. For each set of historical trajectories and the detection boxes of the current frame, the association cost is calculated; where the association cost is obtained by weighted fusion of the overlap, distance and category constraints between the predicted box and the observation box of the current frame in the stable reference coordinate system. Construct a cost matrix based on all calculated associated costs; The Hungarian algorithm is used to perform global optimal matching on the cost matrix to obtain the matching relationship between the trajectory and the detection box with the minimum total cost.

6. The temporal stabilization method for video target detection based on continuous frame association according to claim 4, characterized in that, The aforementioned time-series stabilization processing of the three display attributes—detection box, category label, and confidence score—includes: For the center position of the detection box, exponential smoothing is used for stable updates: the candidate display box of the current frame is obtained by fusing the stable display box of the previous frame and the observation box of the current frame in the stable reference coordinate system. For the width and height of the detection box, calculate the single-frame change rate. When the single-frame change rate exceeds the corresponding preset threshold, limit the single-frame update amplitude to within the corresponding maximum update amplitude; otherwise, update according to the exponential smoothing result. For each category label, a cumulative score for each cross-frame associated target is maintained and continuously updated using a time-smoothing method; the category with the highest cumulative score is selected as the candidate new category for the current frame. For confidence level, internal trajectory association and state judgment, the original detection confidence level is still used, and the external display output is a smoothed stable confidence level. Among them, the confidence level smoothing coefficient adopts a piecewise adaptive strategy: when the confidence level jump amplitude is greater than the confidence level jump threshold, the confidence level smoothing coefficient is taken as the smoothing coefficient under strong smoothing state; otherwise, the confidence level smoothing coefficient is taken as the smoothing coefficient under constant smoothing state.

7. The temporal stabilization method for video target detection based on continuous frame association according to claim 6, characterized in that, For category labels, a category switching threshold and a preset value for the number of consecutive frames are also set. The display category will only be switched to the candidate new category if the cumulative score of the candidate new category exceeds the cumulative score of the current display category and the difference between the two is greater than the category switching threshold, and the preset value for the number of consecutive frames is met continuously. Otherwise, the original display category will remain unchanged.

8. The temporal stabilization method for video target detection based on continuous frame association according to claim 4, characterized in that, The aforementioned target trajectory continuity management based on multi-attribute time-series stability processing results includes: For each target trajectory, four basic states are established: initialization state, stable tracking state, short-term hold state, and termination state. The transitions between states are managed based on the number of consecutive successful matching frames and the number of consecutive unmatched frames. When the number of consecutive successful matches of a candidate trajectory in the initialization state reaches a preset threshold, it enters the stable tracking state; when a candidate trajectory in the initialization state experiences consecutive unmatches before the number of consecutive successful matches reaches the preset threshold, and the number of consecutive unmatched frames exceeds the termination threshold, or the center of the prediction box exceeds the image area, it enters the termination state. The target trajectory that is continuously matched is in a stable tracking state. When the trajectory does not get a valid match in the current frame, it enters a short-term hold state. When the trajectory fails to obtain a valid match in the current frame, and the number of consecutive unmatched frames does not exceed the short-term holding threshold, the trajectory is in the display holding phase of the short-term holding state, and the target's current position is predicted based on the historical motion trend to continue to maintain the target display; When the number of consecutive unmatched frames is greater than the short-term holding threshold but does not exceed the termination threshold, the trajectory is in the non-display holding stage of the short-term holding state and is no longer displayed normally, but the trajectory information is still retained and participates in subsequent association judgment. When the number of consecutive unmatched frames exceeds the termination threshold or the center of the prediction box exceeds the image area, the trajectory enters the termination state.

9. The temporal stabilization method for video target detection based on continuous frame association according to claim 8, characterized in that, The aforementioned target trajectory continuity management based on multi-attribute time-series stability processing results also includes: setting two types of auxiliary events: uncertain events and recovery display processing; Among them, the uncertain event is as follows: For a matched target, an uncertainty index is calculated. When the uncertainty index exceeds the uncertainty threshold, an uncertain event is triggered. When the uncertainty index falls back below the recovery threshold, normal updates resume. The uncertainty threshold is greater than the recovery threshold. Restore display processing: When a target in the non-display retention phase that is in a short-term hold state meets the association conditions again, the restore display processing is triggered, and the target is reintroduced into the normal display process.

10. A temporally stable video target detection system based on continuous frame association, characterized in that, include: The target detection unit is used to perform frame-by-frame target detection on consecutive video frames to obtain detection results, which include detection boxes, category labels, and confidence scores. A reference coordinate system construction unit is used to construct a stable reference coordinate system based on the constraints of the detection results; A stabilization processing unit is used to perform multi-attribute temporal stabilization processing on the detection result based on the stable reference coordinate system; The state manager is used to perform target trajectory state continuity management based on the results of the multi-attribute temporal stability processing. And an output unit, used to overlay the stabilized detection boxes, class labels and confidence scores onto the original video image for output.

Images