A camera tracking quality evaluation method and system based on video frame smoothness quantization

Abstract

The application discloses a camera tracking quality evaluation method and system based on video picture smoothness quantization, comprising the following steps: acquiring a video recorded by a camera under a tracking test scene, target trajectory data of a main tracking target and a tracking response state, wherein the target trajectory data comprises a target frame, a target center point, a target visible state and a tracking error mark; generating an initial evaluation interval according to the target visible state, the tracking error mark and the tracking response state, wherein the initial evaluation interval is a video interval in which the main tracking target is visible, no tracking error occurs and a tracking function of the camera for the main tracking target is in an activated state. The application can objectively evaluate the smoothness of a camera target tracking picture and locate tracking abnormalities.

Description

Technical Field

[0001] This invention relates to the field of camera technology, and in particular to a camera tracking quality evaluation method and system based on video image smoothness measurement. Background Technology

[0002] With the development of computer vision, target detection, and pan-tilt control technologies, cameras with automatic target tracking capabilities are widely used in scenarios such as home care, video conferencing, security monitoring, pet care, and intelligent photography. After detecting a moving target, the camera can control the mechanical pan-tilt unit to rotate according to the target's position in the frame, or adjust the shooting field of view through electronic cropping, electronic image movement, etc., to keep the target within a predetermined area in the frame.

[0003] During target tracking, the camera needs to continuously adjust its position and movement based on the target's location and motion. If the tracking control parameters are not set appropriately, or if there is a mismatch between the target detection results, motion prediction results, and gimbal response, issues such as high-frequency image jitter, sudden stutters, repeated changes in direction, reverse correction after overtaking the target, frequent start-stop cycles, and the camera failing to respond promptly when the target deviates from the center of the frame can easily occur. These issues not only affect the viewing experience but also hinder objective comparisons between different products, software versions, and tracking control strategies.

[0004] Existing technologies already include methods for determining camera shake based on video motion. For example, Chinese patent CN104301712B discloses a video analysis-based method for detecting camera shake. This method extracts and matches global feature points from consecutive video frames, clusters the displacement vectors of these feature points, and determines whether video shake has occurred based on the clustering results, thus distinguishing between local object movement and overall camera shake. This method is primarily used to determine whether a surveillance camera exhibits overall shake.

[0005] Chinese patent CN105681663B discloses a video jitter detection method based on the geometric smoothness of inter-frame motion. This method segments the video based on the similarity between frames, performs feature point detection and matching on each segment, calculates the average translation amount, the variance of the translation amount, and the average angle between the translation vectors of adjacent frames, and compares these geometric quantities with corresponding thresholds to determine whether jitter has occurred in the video segment. This scheme can evaluate the jitter situation in general videos based on the geometric smoothness of inter-frame motion.

[0006] Chinese patent CN111242984B discloses a target tracking method based on a panning camera. This method detects and determines the target to be tracked, calculates the distance of the target from the center of the frame in both the horizontal and vertical directions, and controls the panning camera to rotate based on these distances, causing the target to move towards the center of the frame. The solution also employs multi-frame filtering, Kalman prediction, and matching to reduce camera shake caused by false target detections. This solution mainly involves controlling the camera to track the target based on its position.

[0007] The aforementioned existing solutions have made improvements from the perspectives of camera shake detection, inter-frame motion geometric smoothness evaluation, and target tracking control. However, based on the publicly available information, general video shake detection methods typically use the overall motion amplitude, displacement variance, or directional changes of the image as the basis for judgment, failing to distinguish between effective motion generated by the camera to follow the target and invalid shake, overshoot, sway, and start / stop anomalies; target tracking control methods mainly focus on how to control camera movement based on the positional deviation between the target and the center of the image, without forming an independent quality evaluation method for tracking results.

[0008] Especially in camera tracking test scenarios, the movement of the main target itself in the frame can affect the inter-frame feature matching results. If all image features, including the target region, are used directly to estimate the motion, the local motion of the main target may be mistaken for the global motion generated by the camera. At the same time, in low-texture, low-light, dynamic background, or partially occluded scenes, the background motion estimation results may also have insufficient reliability.

[0009] Furthermore, relying solely on the smoothness of the image motion trajectory makes it difficult to determine whether the camera's reverse movement is a reasonable follow adjustment or an ineffective swing. It also makes it difficult to determine whether the target crossing the center of the frame is normal movement or an overshoot behavior where the camera corrects itself after tracking the target. In cases where the target has clearly deviated from the center of the frame while the camera remains stationary, typical video jitter detection methods struggle to identify this as insufficient tracking response.

[0010] Therefore, there is a need to provide a tracking quality evaluation method, system, and storage medium for camera target tracking scenarios. This method should reliably extract the corresponding motion of the camera image while eliminating interference from the motion of the main target itself. It should also identify high-frequency jitter, swing, overshoot, abnormal start / stop, and invalid stillness by combining the motion vector of the image and the target center trajectory. Finally, it should output objective evaluation results that reflect the smoothness of the tracking image and the specific reasons for the loss of points. Summary of the Invention

[0011] To address the problems existing in the prior art, this invention provides a method and system for evaluating camera tracking quality based on video image smoothness measurement.

[0012] To achieve the above objectives, this invention provides a camera tracking quality evaluation method based on video frame smoothness measurement, comprising the following steps: Step S1: Acquire the video recorded by the camera in the tracking test scenario, the target trajectory data of the main tracking target, and the tracking response status. The target trajectory data includes the target bounding box, the target center point, the target visibility status, and the tracking error marker. Generate an initial evaluation interval based on the target visibility status, the tracking error marker, and the tracking response status. The initial evaluation interval is the video interval in which the main tracking target is visible, no tracking error has occurred, and the camera's tracking function for the main tracking target is active. Step S2: Construct a target bounding box extended mask for adjacent video frames within the initial evaluation interval, extract and match background feature points in the background region outside the target bounding box extended mask, estimate the global translation transformation matrix, affine transformation matrix, or homography transformation matrix between adjacent video frames using a robust estimation method, and extract the image motion vector sequence representing the displacement direction of the background image content from the previous video frame to the next video frame in the image plane from the global translation transformation matrix, the affine transformation matrix, or the homography transformation matrix. Step S3: Based on at least one of the following factors: background feature point matching inlier rate, background region area ratio, background motion consistency, and dynamic background interference level, calculate the background motion estimation reliability of each adjacent frame pair, and aggregate the background motion estimation reliability of multiple consecutive adjacent frame pairs to obtain the segment reliability of candidate video segments; determine the candidate video segments with segment reliability not lower than a preset reliability threshold as valid evaluation intervals; when the segment reliability is lower than the preset reliability threshold, expand the statistical window or switch the motion estimation method for the corresponding candidate video segment, and recalculate the segment reliability; include the candidate video segments with recalculated segment reliability not lower than the preset reliability threshold into the valid evaluation interval, and determine the scoring weight of the corresponding video segment based on the recalculated segment reliability; exclude the candidate video segments with recalculated segment reliability still lower than the preset reliability threshold and output a low confidence mark; perform tracking quality evaluation only within the valid evaluation interval; Step S4: Set a short window, a medium window, and a long window within the effective evaluation interval. The short window is used to detect high-frequency jitter or sudden pauses. The medium window is used to detect direction reversal, swing back, or overshoot. The long window is used to analyze the overall tracking trend and perform statistical analysis on the motion vector sequence of the image and the trajectory sequence of the target center point based on each sliding window. Step S5: Calculate vector velocity, vector acceleration, and vector jerk based on the motion vector sequence of the image, and calculate the average acceleration amplitude, P90 acceleration amplitude, average jerk amplitude, and P90 jerk amplitude based on the vector acceleration and the vector jerk. Step S6: Separate the low-frequency following motion and high-frequency jitter residuals from the motion vector sequence of the image, calculate the normalized high-frequency jitter intensity based on the separated high-frequency jitter residuals, and calculate the high-frequency direction change rate; Step S7: Detect effective direction reversal according to the motion vector sequence of the image, and identify swing back events according to the number of effective direction reversals within the preset time window and the improvement of the target center offset. The swing back event is an event in which multiple effective direction reversals occur within the preset time window and the target center offset does not reach the preset improvement condition. Step S8: Identify overshoot events based on the joint conditions of the error sign flipping between the target center and the image center, the peak error in the same direction before the flipping and the peak error in the opposite direction after the flipping exceeding the center tolerance threshold in the preset observation sub-window before and after the sign flipping, and the image motion direction after the error sign flipping being reversed relative to the direction before the sign flipping, and calculate the overshoot amplitude and overshoot recovery time corresponding to the overshoot event. Step S9: Determine the motion state or stationary state based on the comparison result between the screen motion amplitude and the preset motion threshold, and count the start-stop switching rate and invalid stationary ratio within the effective evaluation interval. The invalid stationary state is the state where the main tracking target is visible, the target center offset exceeds the preset offset threshold, and the screen motion amplitude is not greater than the preset motion threshold. Step S10: Analyze the trajectory sequence of the target center point in the image, and calculate at least one of the following target center stability indicators: target center offset change rate, number of target center direction reversals, target center oscillation intensity, and number of edge ejections; Step S11: Based on the average acceleration amplitude, the P90 acceleration amplitude, the average jerk amplitude, the P90 jerk amplitude, the normalized high-frequency jitter intensity, the high-frequency direction change rate, the swing event, the overshoot event, the start-stop switching rate, the proportion of invalid stillness, the target center stability index, and the scoring weight of each effective video segment, calculate the smoothness comprehensive score according to the penalty function calibrated by product type, target type, test scenario, user subjective score, expert score, historical problem sample or version regression test results, and output the smoothness comprehensive score, level, deduction of each sub-index, confidence level, main reasons for loss of points and corresponding time interval, so as to distinguish between effective following motion and invalid jitter, overshoot, swing, start-stop abnormality or invalid stillness in the target tracking process.

[0013] Preferably, in step S2: the target bounding box expansion mask is obtained by expanding the target bounding box outward according to a preset expansion ratio to exclude feature points in the main tracking target, the target edge shadow area, or the local occlusion area; when the global translation transformation matrix or the affine transformation matrix is ​​used, the translation components in the global translation transformation matrix or the affine transformation matrix are extracted as the horizontal and vertical displacements between adjacent video frames; when the homography transformation matrix is ​​used, preset background sampling points or background grid points are mapped through the homography transformation matrix, and the average value, median value, or representative value of the center region of the displacement before and after mapping is calculated as the image motion vector between adjacent video frames.

[0014] Preferably, in step S4, the duration of the short window is 0.2 to 0.5 seconds, the duration of the medium window is 0.5 to 1.0 seconds, and the duration of the long window is 1.0 to 2.0 seconds, and the duration of each window is converted into the corresponding number of window frames according to the frame rate of the video; In step S5, let the motion vector between frame i and frame i+1 be g_i, and the time interval between adjacent video frames be Δt. Then, the vector velocity V_i, the vector acceleration A_i, and the vector jerk J_i satisfy the following: V_i = g_i / Δt; A_i = (V_i - V_(i-1)) / Δt; J_i = (A_i - A_(i-1)) / Δt. The amplitudes of the vector acceleration and the vector jerk are calculated respectively, and the amplitudes of the vector acceleration and the vector jerk are statistically analyzed to obtain the average acceleration amplitude, the P90 acceleration amplitude, the average jerk amplitude, and the P90 jerk amplitude. Wherein, the P90 acceleration amplitude is the 90th percentile of the vector acceleration amplitude, and the P90 jerk amplitude is the 90th percentile of the vector jerk amplitude.

[0015] Preferably, in step S6, a moving average is performed on the motion vector sequence of the image to obtain a low-frequency following motion component, wherein the window length of the moving average is 0.5 seconds to 1.0 seconds; the low-frequency following motion component is subtracted from the motion vector sequence of the image to obtain a high-frequency jitter residual; the normalized high-frequency jitter intensity is the ratio of the average amplitude of the high-frequency jitter residual to the length of the image diagonal; In step S7, let the motion vector of the image between frame i and frame i+1 be: g_i = (dx_i, dy_i); A valid direction reversal in the horizontal direction satisfies: sign(dx_i) ≠ sign(dx_(i-1)), and |dx_i| > d_threshold, and |dx_(i-1)| > d_threshold; A valid direction reversal in the vertical direction satisfies: sign(dy_i) ≠ sign(dy_(i-1)), and |dy_i| > d_threshold, and |dy_(i-1)| > d_threshold; where d_threshold is the direction reversal determination threshold, which is 0.1% to 0.3% of the diagonal length of the image. The criteria for determining the swing event include: two or more valid direction reversals occurring within a medium window of 0.5 to 1.0 seconds, and the target center offset at the end of the medium window not being improved compared to the target center offset at the beginning of the medium window.

[0016] Preferably, in step S8, the error between the target center and the image center undergoes a sign flip along at least one coordinate axis within a preset time window. Within the preset observation sub-windows before and after the sign flip, the peak value of the same-direction error before the flip and the peak value of the opposite-direction error after the flip exceed the center tolerance threshold set along the corresponding coordinate axis, and the image motion direction after the sign flip is reversed relative to the image motion direction before the sign flip. The overshoot amplitude is the ratio of the peak value of the reverse error after the sign flip to the image size in the corresponding coordinate axis direction. The overshoot recovery time is the time from the moment the error sign flips to the time when the target center re-enters the central safe zone. The central safe zone is enclosed by a horizontal center tolerance threshold and a vertical center tolerance threshold. When the absolute value of the horizontal error of the target center is not greater than the horizontal center tolerance threshold and the absolute value of the vertical error is not greater than the vertical center tolerance threshold, it is determined that the target center has entered the central safe zone. In step S9, the motion state or the stationary state is determined based on the comparison result between the motion amplitude of the screen, motion_i and the preset motion threshold, m_threshold. The preset motion threshold, m_threshold, is 0.05% to 0.2% of the diagonal length of the screen. When the motion amplitude of the screen, motion_i, is greater than the preset motion threshold, m_threshold, the corresponding video frame is determined to be in motion; otherwise, the corresponding video frame is determined to be in stationary state. The start-stop switching rate is the ratio of the number of times the motion state and the stationary state are switched within the effective evaluation interval to the effective evaluation duration; The invalid still frame is a video frame within the valid evaluation interval where the main tracking target is visible, the target center offset exceeds the preset offset threshold, and the motion amplitude motion_i is not greater than the preset motion threshold m_threshold. The invalid still frame ratio is the ratio of the number of invalid still frames to the number of target visible frames within the valid evaluation interval.

[0017] Preferably, in step S3, the background motion estimation reliability R_i satisfies: R_i=α×N_inlier / N_match+β×Area_bg / Area_img-γ×E_dynamic; Wherein, N_inlier is the number of background feature point matching inliers determined by the robust estimation method, N_match is the number of background feature point matching points, Area_bg is the effective area of ​​the background region, Area_img is the image area, E_dynamic is the dynamic background interference index, and α, β and γ are preset parameters or parameters obtained by calibration through test samples.

[0018] A candidate video segment is formed by K consecutive frames, and the candidate video segment includes K-1 adjacent frame pairs. The segment reliability R_seg of the candidate video segment satisfies: R_seg=[Σ_(i=1)^(K-1)R_i] / (K-1); The segment reliability R_seg is compared with the preset reliability threshold to determine whether the candidate video segment is included in the valid evaluation interval.

[0019] Preferably, in step S11: the corresponding penalty term P_k is calculated according to each sub-index x_k, wherein the penalty term P_k is one of a linear penalty function, a piecewise penalty function, or a saturation penalty function; When the penalty term P_k uses the saturation penalty function, the following conditions are met: P_k=P_k^max / {1+exp[-(x_k-μ_k) / s_k]}; Where P_k^max is the upper limit of the penalty for the k-th sub-index, and μ_k and s_k are parameters calibrated based on product type, target type, test scenario, user subjective rating, expert rating, historical problem samples or version regression test results; The smoothness score S satisfies: S=max{0,min[100,100-ΣP_k]}.

[0020] This invention also provides a camera tracking quality evaluation system based on video image smoothness measurement, comprising: The video input and trajectory acquisition module is used to acquire the video recorded by the camera in the tracking test scenario, the target trajectory data of the main tracking target, and the tracking response status. The target trajectory data includes the target bounding box, the target center point, the target visibility status, and the tracking error marker. The initial evaluation interval generation module is used to determine the video interval in which the main tracking target is visible, no tracking error has occurred, and the camera's tracking function for the main tracking target is active, based on the target visibility status, the tracking error marker, and the tracking response status. The background motion estimation module is used to construct a target bounding box extended mask for adjacent video frames within the initial evaluation interval, extract and match background feature points in the background region outside the target bounding box extended mask, estimate the global translation transformation matrix, affine transformation matrix or homography transformation matrix using a robust estimation method, and extract the image motion vector sequence representing the displacement direction of the background image content from the previous video frame to the next video frame in the image plane. The reliability and effective interval determination module is used to calculate the background motion estimation reliability of each adjacent frame pair based on at least one of the following: background feature point matching in-point rate, background region area ratio, background motion consistency, and dynamic background interference degree. It aggregates the background motion estimation reliability of multiple consecutive adjacent frame pairs to obtain the segment reliability of candidate video segments. Candidate video segments with segment reliability not lower than a preset reliability threshold are determined as effective evaluation intervals. When the segment reliability is lower than the preset reliability threshold, the module expands the statistical window or switches the motion estimation method for the corresponding candidate video segment and recalculates the segment reliability. Candidate video segments with recalculated segment reliability not lower than the preset reliability threshold are included in the effective evaluation interval, and the scoring weight of the corresponding video segment is determined based on the recalculated segment reliability. Candidate video segments with recalculated segment reliability still lower than the preset reliability threshold are excluded, and a low confidence marker is output. The multi-window index calculation module is used to set short windows, medium windows and long windows within the effective evaluation interval, and to calculate speed, acceleration, jerk, change in image motion direction and change in target center offset index based on the image motion vector sequence and the target trajectory data; The high-frequency jitter analysis module is used to separate the low-frequency following motion from the high-frequency jitter residual in the motion vector sequence of the image, and to calculate the normalized high-frequency jitter intensity and high-frequency direction change rate. The tracking event recognition module is used to identify direction reversal events, swing back events, and overshoot events based on the motion vector sequence of the screen and the target trajectory data, to calculate the start-stop switching rate and the proportion of invalid stationary states, and to calculate the target center stability index. The comprehensive scoring and cause localization module is used to calculate the smoothness comprehensive score based on the scoring weights of each sub-indicator, tracked event, and each valid video segment, according to the calibrated penalty function, and output the smoothness comprehensive score, level, deduction score of each sub-indicator, confidence level, main reasons for loss of points, and corresponding time interval.

[0021] Preferably, the event tracking and identification module includes: The swing recognition unit is used to count the number of effective directional reversals within the middle window and determine the improvement of the target center offset within the middle window. When the number of effective directional reversals reaches a preset number and the target center offset does not reach a preset improvement condition, a swing event is determined to have occurred. The overshoot identification unit is used to identify overshoot events based on the following joint conditions: the error between the target center and the image center causes a sign flip; the peak value of the same direction error before the flip and the peak value of the reverse error after the flip exceed the center tolerance threshold in the preset observation sub-window before and after the sign flip; and the image motion direction after the sign flip is reversed relative to the direction before the sign flip. The unit also calculates the overshoot amplitude and overshoot recovery time corresponding to the overshoot event. The start / stop and invalid stillness identification unit is used to determine the motion state or stillness state based on the comparison result of the screen motion amplitude and the preset motion threshold, calculate the start / stop switching rate based on the number of switching between the motion state and the stillness state and the effective evaluation time, and determine the corresponding video frame as an invalid still frame when the main tracking target is visible, the target center offset exceeds the preset offset threshold and the screen motion amplitude is not greater than the preset motion threshold, and calculate the invalid stillness ratio based on the ratio of the number of invalid still frames to the number of target visible frames in the effective evaluation interval.

[0022] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the camera tracking quality evaluation method based on video image smoothness quantification described in steps S1 to S11 above.

[0023] The technical solution of this invention has the following beneficial effects: (1) The present invention generates an initial evaluation interval based on the target visibility state, tracking error mark and tracking response state, and excludes video segments with target occlusion, target loss, tracking error and tracking function not in the active state for the main tracking target from the evaluation range. At the same time, it retains video segments with tracking function activated but not yet generating effective screen motion, so that subsequent calculations can cover response delay, control dead zone or invalid static state, and reduce the interference of irrelevant video segments on the evaluation results.

[0024] (2) The present invention expands the target box of the main tracking target to form a target box expansion mask, and extracts and matches background feature points in the background area outside the target box expansion mask. This can eliminate the influence of the main tracking target, target edge shadow and feature points in the local occlusion area, thereby obtaining the global image motion formed by camera rotation, electronic cropping or image movement more accurately, and reducing the probability of misjudging the target's own movement as camera shake.

[0025] (3) The present invention calculates the reliability of background motion estimation based on the background feature point matching inlier rate, effective area of ​​background region, background motion consistency and dynamic background interference degree, and performs processing such as expanding the statistical window, switching motion estimation mode, reducing scoring weight, excluding or outputting low confidence markers on low reliability video segments, which can improve the stability and reliability of evaluation results in low texture, low light, dynamic background and partial occlusion scenes.

[0026] (4) The present invention sets up short windows, medium windows and long windows to analyze high-frequency jitter, sudden pauses, direction reversals, swings, overshoots and overall tracking trends at different time scales. It describes the changes in screen motion through indicators such as speed, acceleration, jerk and high-frequency jitter residual. It can take into account both fast small jitter and medium and low frequency anomalies with long duration, and avoid missing anomalies caused by using only a single time window.

[0027] (5) The present invention combines the image motion vector and the target center offset to identify swing events. When multiple effective directional reversals occur within a preset time window and the target center offset does not meet the preset improvement conditions, the corresponding motion is identified as a swing, thereby distinguishing between reasonable directional adjustments made to bring the target back to the center of the image and invalid repeated swings that do not produce an effective centering effect.

[0028] (6) The present invention identifies overshoot events by combining the following conditions: the target center error sign flips, the same-direction error peak and the opposite-direction error peak before and after the sign flip exceed the center tolerance threshold, and the image motion direction reverses after the sign flip. This can reduce the misjudgment of the situation where the target normally passes through the center of the image as overshoot. Furthermore, it can calculate the overshoot amplitude and overshoot recovery time to quantify the degree of deviation and recovery speed after the camera chases the target.

[0029] (7) The present invention calculates the start-stop switching rate based on the switching between the motion state and the static state of the screen, and identifies invalid static when the target is visible, the target center offset exceeds the preset offset threshold and the screen motion amplitude is not greater than the preset motion threshold. It can detect the problem of frequent stopping and starting, response delay or failure to perform follow adjustment in time during the camera tracking process.

[0030] (8) The present invention analyzes both the smoothness of the motion of the image and the stability of the target center, which can avoid mistaking the situation of "the camera is not moving but the target has deviated from the center" as smooth tracking, so that the evaluation results not only reflect whether the image is shaking, but also whether the target is stably kept in a reasonable position in the image.

[0031] (9) This invention converts indicators such as acceleration, jerk, high-frequency jitter, direction reversal, sway, overshoot, start-stop switching, invalid stillness and target center stability into corresponding penalty items, calculates the smoothness comprehensive score, and outputs the evaluation level, deduction of each sub-indicator, confidence level, main reasons for loss of points and corresponding time interval. It can further locate specific tracking anomalies from a single score, and provide a basis for optimizing parameters such as tracking sensitivity, motion damping, target prediction, start response and stop response.

[0032] (10) This invention mainly utilizes video recorded by the camera and target trajectory data of the main tracking target to complete the evaluation. It does not require reading the motor control commands, gimbal angle, motion speed, or controller parameters inside the camera. Therefore, it can be applied to cameras with different hardware structures and different tracking control algorithms, and it is also convenient to carry out horizontal testing between different products and regression testing between different software versions of the same product. The above-mentioned beneficial effects are all achieved by technical means such as target box expansion mask, background motion reliability, multi-time scale analysis, joint determination of image motion and target center trajectory, and comprehensive score output. Attached Figure Description

[0033] Figure 1 This is a flowchart illustrating the overall process of screen smoothness evaluation according to an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the principle of target bounding box expansion mask and background global motion estimation according to an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the high-frequency jitter detection principle of an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the principle of direction reversal and oscillation detection according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the overshoot detection principle according to an embodiment of the present invention; Figure 6 This is a schematic diagram illustrating the calculation principle of a multi-frame sliding window according to an embodiment of the present invention; Figure 7 This is a stability analysis diagram of the target center trajectory according to an embodiment of the present invention; Figure 8 This is a schematic diagram illustrating the start / stop switching and invalid stationary detection principle of an embodiment of the present invention; Figure 9 This is a schematic diagram illustrating the structure of the smoothness sub-index penalty term according to an embodiment of the present invention; Figure 10This is a schematic diagram (example) of the smoothness comprehensive score mapping level according to an embodiment of the present invention; Figure 11 This is a flowchart illustrating the reliability and confidence level processing of background motion estimation according to an embodiment of the present invention. Detailed Implementation

[0034] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0035] Reference Figures 1 to 11 This invention uses video recorded by a camera in a tracking test scenario, target trajectory data of the main tracked target, and tracking response status as evaluation inputs. By using a target bounding box expansion mask, the influence of the main tracked target's own motion on the global motion estimation of the image is eliminated. Based on the reliability of the background motion estimation, an effective evaluation interval that can participate in the tracking quality evaluation is determined. Within the effective evaluation interval, short, medium, and long windows are used to analyze high-frequency jitter, direction reversal, swing, overshoot, start-stop anomalies, and overall tracking trends, respectively. Combined with the offset change of the target center point relative to the center of the image, the smoothness comprehensive score, evaluation level, deduction of each sub-index, confidence level, and main reasons for loss of points are calculated.

[0036] This invention does not require the acquisition of gimbal rotation angle, motor speed, motor drive command, proportional-integral-derivative control parameters, or other internal control data of the camera. It can complete the tracking quality evaluation based solely on the video recorded by the camera and the target trajectory data of the main tracking target. It can be used for regression testing between different software versions of the same camera product, for horizontal comparison testing between different camera products, and for verifying the anomaly localization and optimization effect of the camera tracking control algorithm.

[0037] like Figure 1 As shown, the camera tracking quality evaluation method based on video image smoothness measurement provided by the present invention includes steps S1 to S11.

[0038] In step S1, the video recorded by the camera under test, the target trajectory data of the main tracked target, and the tracking response status are acquired under a preset tracking test scenario. The tracking test scenario can be a scenario in which the target moves laterally at a constant speed, a scenario in which the target moves along a curved path, a scenario in which the target moves at a rapid speed change, a scenario in which the target suddenly starts or stops, a scenario in which the target reappears after a brief occlusion, or a combination of two or more of the above scenarios.

[0039] The primary tracking target can be a person, vehicle, pet, or other moving object that can be automatically identified and followed by the camera. The camera can be a camera with a mechanical pan-tilt head, or a camera that achieves target tracking through electronic cropping, electronic zoom, or digital image movement.

[0040] The target trajectory data includes the target bounding box, target center point, target visibility status, and tracking error marker in each video frame. The target bounding box in the i-th frame can be represented as: bbox_i=(x_i, y_i, w_i, h_i); Where x_i and y_i represent the x-coordinate and y-coordinate of the top-left corner of the target box in the i-th frame, respectively, and w_i and h_i represent the width and height of the target box, respectively.

[0041] The target center point in the i-th frame can be represented as: c_i = (c_x,i, c_y,i); in: c_x,i = x_i + w_i / 2; c_y,i = y_i + h_i / 2.

[0042] The target visibility state can include a visible state, a partially occluded state, a completely occluded state, and a lost state. The tracking error marker is used to indicate whether the object currently being followed by the camera is a pre-determined primary tracking target.

[0043] Target trajectory data can be obtained through manual frame-by-frame annotation, target detection models, target tracking models, external target trajectory annotation files, or a combination of the above methods. When target trajectory data is automatically acquired using target detection models or target tracking models, target boxes with detection confidence levels lower than the preset confidence levels can be manually reviewed or corrected to reduce the impact of target detection errors on tracking quality evaluation results.

[0044] The tracking response status indicates whether the camera's tracking function for the main target is active. The tracking response status can be determined based on a pre-set test procedure, manual annotation information, external test records, camera operating status, or changes in the target center offset. The tracking function being active includes situations where the camera has entered the tracking control process for the main target, but no effective image movement has yet occurred due to response delay, control dead zone, or the image motion amplitude not exceeding a preset motion threshold.

[0045] An initial evaluation interval is generated based on the target visibility status, tracking error markers, and tracking response status. This initial evaluation interval is the video interval where the main tracking target is visible, the camera has not made a tracking error, and the camera's tracking function for the main tracking target is active.

[0046] When generating the initial evaluation interval, video segments where the target is completely occluded or lost, video segments where the camera follows a non-primary target, transitional video segments where the camera has not yet entered the tracking function activation state after the test starts, video segments where the camera exits the tracking process after the test ends, and video segments that cannot be motion estimated due to missing video frames, serious encoding errors, or unrecognizable images.

[0047] The acquired video is preprocessed. This preprocessing may include standardizing the video frame rate, establishing a video timeline, reading the image width W and image height H, and calculating the diagonal length D of the image. The diagonal length D satisfies: D = sqrt(W² + H²).

[0048] For example, when the video resolution is 2560×1440 pixels, the diagonal length D of the screen is approximately 2938 pixels; when the video frame rate is 25fps, the time interval Δt between adjacent video frames is 0.04 seconds.

[0049] In step S2, such as Figure 2 As shown, a target bounding box expansion mask is constructed for adjacent video frames within the initial evaluation interval. Based on the target bounding box bbox_i in the i-th frame, it is expanded outwards according to a preset expansion ratio r to form the target bounding box expansion region. The preset expansion ratio r can be 5% to 20% of the target bounding box width and height.

[0050] The target bounding box extension mask is used to exclude feature points in the area where the main tracking target is located, the target edge shadow area, the blurred area formed by the target movement, and the local occlusion area adjacent to the main tracking target, thereby avoiding misinterpreting the movement of the main tracking target itself in the image as global image movement formed by camera rotation or image cropping.

[0051] Background feature points are extracted from the background region outside the target bounding box extended mask. The background feature points can be ORB feature points, FAST feature points, SIFT feature points, SURF feature points, or other image feature points that can be used for matching adjacent video frames.

[0052] In one specific implementation, the ORB algorithm is used to extract background feature points. The upper limit of the number of ORB feature points is set to 500, the scale factor is set to 1.2, and the number of image pyramid layers is set to 8.

[0053] Background feature points in frame i are matched with background feature points in frame i+1 to obtain background feature point matching pairs. A random sampling consensus algorithm is used to robustly estimate the background feature point matching pairs to filter out erroneous matching points and determine the in-place points of the background feature point matching.

[0054] The number of iterations for the random sampling consensus algorithm can be set from 500 to 3000, and the inlier distance threshold can be set from 1 to 5 pixels. In one specific implementation, the number of iterations for the random sampling consensus algorithm is set to 2000, and the inlier distance threshold is set to 3 pixels.

[0055] The global translation transformation matrix, affine transformation matrix, or homography transformation matrix between adjacent video frames is estimated by matching interior points based on background feature points. When the camera image mainly shows overall movement in the horizontal and vertical directions, the global translation transformation matrix can be used; when the image movement involves translation, rotation, or scale changes simultaneously, the affine transformation matrix can be used; when the camera's viewing angle changes significantly or the image has perspective changes, the homography transformation matrix can be used.

[0056] When using a global translation transformation matrix or an affine transformation matrix, the translation components in the matrix are extracted as the horizontal displacement dx_i and vertical displacement dy_i between adjacent video frames.

[0057] When using a homography transformation matrix, multiple background sampling points or background grid points are set outside the target bounding box extended mask. The background sampling points or background grid points are mapped through the homography transformation matrix. The displacement of each sampling point before and after mapping is calculated, and the average value, median value, or representative value of the central region displacement of each sampling point is used as the image motion vector between adjacent video frames.

[0058] The motion vector between frame i and frame i+1 is represented as follows: g_i = (dx_i, dy_i).

[0059] The motion amplitude of the image, motion_i, satisfies: motion_i=sqrt(dx_i²+dy_i²).

[0060] The normalized motion amplitude norm_motion_i satisfies: norm_motion_i = motion_i / D.

[0061] By using a target bounding box to expand the mask and estimating global image motion using only the background area, the impact of the main tracking target's own movement on the camera's image motion estimation can be reduced, avoiding misjudging local movement of the target in the image as global image motion caused by camera rotation, electronic image movement, or image cropping.

[0062] In step S3, such as Figure 11 As shown, the reliability of background motion estimation is calculated based on at least one of the following: background feature point matching inlier rate, background region area ratio, background motion consistency, and dynamic background interference degree.

[0063] In one specific implementation, the background motion estimation reliability R_i satisfies: R_i=α×N_inlier / N_match+β×Area_bg / Area_img-γ×E_dynamic; Where N_inlier is the number of background feature points matched by the random sampling consensus algorithm, N_match is the total number of background feature points matched, Area_bg is the effective area of ​​the background outside the target box extended mask, Area_img is the total area of ​​the image, E_dynamic is the dynamic background interference index, and α, β and γ are preset parameters or parameters obtained by calibration based on test samples.

[0064] A candidate video segment is formed by K consecutive frames, and the candidate video segment includes K-1 adjacent frame pairs. The segment reliability R_seg of the candidate video segment satisfies: R_seg=[Σ_(i=1)^(K-1)R_i] / (K-1).

[0065] Alternatively, the 10th percentile value of each R_i within the candidate video segment can be used as the segment reliability to reduce the impact of a small number of low-reliability frames on the average value.

[0066] When the total number of background feature point matching points N_match is 0, the background motion estimation reliability R_i is set to 0 and a low confidence mark is output. The background feature point matching in-point rate, background area ratio and dynamic background interference index are all normalized to the range of 0 to 1. α, β and γ are non-negative parameters, and the calculated background motion estimation reliability R_i is restricted to the range of 0 to 1.

[0067] Background motion consistency can be determined based on the degree of dispersion between the displacement vectors corresponding to the interior points matched by the background feature points. When the displacement directions and amplitudes corresponding to each interior point are relatively similar, the background motion consistency is high; when the displacement directions or amplitudes corresponding to each interior point differ significantly, the background motion consistency is low.

[0068] The dynamic background interference index can be determined based on at least one of the following: the proportion of the local moving area to the background area, the dispersion of the displacement direction of the background feature points, and the spatial clustering of the background feature points in the image. For example, when most matched inliers are concentrated in the local area where a swaying curtain, a dynamic display screen, a passing vehicle, or a non-prime tracker is located, the dynamic background interference level can be determined to be high.

[0069] Candidate video segments whose segment reliability R_seg is not lower than the preset reliability threshold R_th are identified as valid evaluation intervals. When the segment reliability R_seg is lower than the preset reliability threshold R_th, the statistical window for the corresponding candidate video segment is expanded or the motion estimation method is switched, and the segment reliability R_seg is recalculated.

[0070] Expanding the statistical window increases the number of adjacent frame pairs participating in the segment reliability statistics, and recalculates the segment reliability based on the matching intrapoint rate, background area ratio, and dynamic background interference level of each adjacent frame pair within the expanded window.

[0071] Switching motion estimation methods can include changing from background feature point matching-based motion estimation to enhanced correlation coefficient image registration, or to sparse optical flow motion estimation. It can also reduce the weight of feature points within local dynamic background regions, remove spatially anomalous background feature points, or re-divide the background grid and select grids with higher motion consistency for motion estimation.

[0072] Candidate video segments whose recalculated segment reliability R_seg is not lower than the preset reliability threshold R_th are included in the valid evaluation interval, and the scoring weight of the corresponding video segment is determined according to the recalculated segment reliability R_seg. The higher the segment reliability, the higher the scoring weight of the corresponding video segment; the closer the segment reliability is to the preset reliability threshold, the lower the scoring weight of the corresponding video segment.

[0073] In one method of determining scoring weights, for the j-th candidate video segment whose segment reliability R_seg,j is not lower than a preset reliability threshold R_th, the scoring weight w_j satisfies: w_j=w_min+(1-w_min)×(R_seg,j-R_th) / (1-R_th); Where w_min is the preset minimum scoring weight, and w_j is restricted to the range from w_min to 1.

[0074] Candidate video segments whose recalculated segment reliability R_seg is still lower than the preset reliability threshold R_th are excluded from the valid evaluation interval, and a low confidence mark is output. Subsequent velocity, acceleration, jerk, high-frequency jitter, sway, overshoot, start / stop anomalies, and target center stability indices are only calculated within the valid evaluation interval.

[0075] In step S4, such as Figure 6As shown, short, medium, and long windows are set within the effective evaluation interval. The short window is used to detect high-frequency jitter or sudden drops, and its duration can be set to 0.2 to 0.5 seconds; the medium window is used to detect directional reversals, swings, or overshoots, and its duration can be set to 0.5 to 1.0 seconds; the long window is used to analyze the overall tracking trend, and its duration can be set to 1.0 to 2.0 seconds.

[0076] The duration of each window is converted into the number of frames per second (fps) based on the video frame rate. The number of frames per second (fps) can be determined according to the following relationship: N_window=round(T_window×fps); Where T_window is the window duration and N_window is the corresponding window frame count.

[0077] For example, when the video frame rate is 25fps, the short window can be set to 0.2 seconds, corresponding to 5 frames; the medium window to 0.5 seconds, corresponding to approximately 13 frames; and the long window to 1.0 second, corresponding to 25 frames. Each sliding window can slide along the effective evaluation interval in steps of 1 frame, and the velocity, acceleration, jerk, change in image motion direction, and change in target center offset are calculated at each window position.

[0078] In step S5, let the motion vector between frame i and frame i+1 be g_i, and the time interval between adjacent video frames be Δt. Then the vector velocity V_i satisfies: V_i = g_i / Δt.

[0079] The vector acceleration A_i satisfies: A_i=(V_i-V_(i-1)) / Δt.

[0080] The vector accelerometer J_i satisfies: J_i=(A_i-A_(i-1)) / Δt.

[0081] The magnitudes of the vector velocity, vector acceleration, and vector jerk respectively satisfy: |V_i|=sqrt(V_x,i²+V_y,i²); |A_i|=sqrt(A_x,i²+A_y,i²); |J_i|=sqrt(J_x,i²+J_y,i²).

[0082] The vector acceleration amplitude and vector jerk amplitude within the effective evaluation interval are statistically analyzed to obtain the average acceleration amplitude, P90 acceleration amplitude, average jerk amplitude, and P90 jerk amplitude.

[0083] The P90 acceleration amplitude is the 90th percentile of the acceleration amplitude of each vector within the valid evaluation interval. The P90 jerk amplitude is the 90th percentile of the jerk amplitude of each vector within the valid evaluation interval.

[0084] Using the P90 index can reduce the dominance of extreme outliers or sporadic motion estimation errors on the overall statistical results, while retaining sensitivity to large acceleration and jerk changes. When making horizontal comparisons between videos of different resolutions, the acceleration amplitude and jerk amplitude can be divided by the diagonal length D of the screen to obtain the normalized acceleration index and normalized jerk index.

[0085] In step S6, such as Figure 3 As shown, a moving average is applied to the motion vector sequence of the image to obtain the low-frequency following motion component. The duration of the moving average window can be set from 0.5 seconds to 1.0 seconds.

[0086] The low-frequency motion component can be represented as: g_low(i)=MA(g_i,L); Where MA represents moving average processing, and L represents the number of frames corresponding to the moving average window length.

[0087] Subtracting the low-frequency following motion component from the original image motion vector sequence yields the high-frequency jitter residual: g_high(i)=g_i-g_low(i).

[0088] The amplitude of the high-frequency jitter residual, residual_i, satisfies: residual_i=sqrt(residual_x,i²+residual_y,i²).

[0089] The normalized high-frequency jitter intensity norm_jitter satisfies: norm_jitter=mean(residual_i) / D.

[0090] The low-frequency tracking motion component mainly reflects the continuous tracking motion generated by the camera in order to keep the main tracking target in a reasonable position in the frame, while the high-frequency jitter residual mainly reflects the frequent, small-amplitude and rapidly changing invalid jitter in the frame.

[0091] It is also possible to calculate the high-frequency directional change rate within a short window based on the directional changes of motion vectors in adjacent frames. The direction angle of the motion vectors in adjacent frames can be expressed as: θ_i=atan2(dy_i, dx_i).

[0092] The change in adjacent directions Δθ_i satisfies: Δθ_i=|atan2[sin(θ_i-θ_(i-1)), cos(θ_i-θ_(i-1))]|.

[0093] Only frame pairs whose motion amplitude exceeds a preset directional statistical threshold are counted; when Δθ_i is greater than the preset directional change threshold θ_dir, it is recorded as a high-frequency directional change. The high-frequency directional change rate is the ratio of the number of high-frequency directional changes within a short window to the effective duration of the short window.

[0094] In step S7, such as Figure 4 As shown, valid direction reversal is detected based on the sequence of image motion vectors. Let the image motion vectors between frame i and frame i+1 be: g_i = (dx_i, dy_i).

[0095] In this embodiment, the image motion vector g_i represents the displacement direction of the background image content from frame i to frame i+1 in the image plane; when converted to the equivalent camera motion direction, the opposite direction of the image motion vector is taken. Subsequent direction changes, direction reversals, swings, and overshoot determinations all adopt a unified image motion vector direction convention.

[0096] Effective direction reversal in the horizontal direction satisfies: sign(dx_i)≠sign(dx_(i-1)); |dx_i|>d_threshold; |dx_(i-1)|>d_threshold.

[0097] Effective direction reversal in the vertical direction satisfies: sign(dy_i)≠sign(dy_(i-1)); |dy_i|>d_threshold; |dy_(i-1)|>d_threshold.

[0098] Wherein, d_threshold is the direction reversal determination threshold, which can be set to 0.1% to 0.3% of the screen diagonal length D.

[0099] By setting a direction reversal determination threshold, minor fluctuations near zero can be eliminated, avoiding misjudging video encoding noise, background feature point matching errors, or slight mechanical vibrations as valid direction reversals.

[0100] The offset of the target center relative to the image center can be expressed as: e_i = (e_x,i, e_y,i); in: e_x,i = c_x,i - W / 2; e_y,i = c_y,i - H / 2.

[0101] The magnitude of the target center offset can be expressed as: |e_i|=sqrt(e_x,i²+e_y,i²).

[0102] Within a mid-window period of 0.5 to 1.0 seconds, the number of valid direction reversals is counted, and the target center offset at the beginning and end of the mid-window is compared. If two or more valid direction reversals occur within the mid-window, and the target center offset at the end of the mid-window is not improved compared to the target center offset at the beginning of the mid-window, a swingback event is determined to have occurred.

[0103] The lack of improvement in the target center offset can be defined as either the target center offset at the end of the middle window being no less than the target center offset at the beginning of the middle window, or the reduction in the target center offset at the end of the middle window relative to the target center offset at the beginning of the middle window not reaching the preset improvement amount.

[0104] A swing event indicates that the camera repeatedly adjusts between multiple directions of motion, but this adjustment does not bring the main tracked target closer to the center of the image, which is an invalid swing that does not make an effective contribution to centering the target.

[0105] In step S8, such as Figure 5 As shown, overshoot events are identified based on the error sign flipping of the target center relative to the image center and the direction of image movement.

[0106] Taking the horizontal direction as an example, the horizontal error of the target center relative to the image center is: e_x,i = c_x,i - W / 2.

[0107] When the target center moves from one side of the image center to the other side, the horizontal error e_x,i undergoes a sign flip.

[0108] An overshoot event simultaneously meets the following conditions: the error between the target center and the image center undergoes sign flipping along at least one coordinate axis within a preset time window; within the preset observation sub-windows before and after the sign flipping, the peak value of the same-direction error before the flipping and the peak value of the reverse-direction error after the flipping exceed the center tolerance threshold on the corresponding coordinate axis, respectively; after the error sign flipping, the image motion direction is reversed relative to the image motion direction before the sign flipping.

[0109] The center tolerance threshold in the horizontal direction can be set to 5% to 10% of the screen width W, and the center tolerance threshold in the vertical direction can be set according to the screen height H in a corresponding proportion.

[0110] For example, when the camera rotates to the right, causing the main target to gradually approach the center of the screen from the right side; when the main target crosses the center of the screen and enters the left side, and then the camera's movement direction changes from rotating to the right to rotating to the left, it can be determined that the camera has overshooted and then corrected itself after overshooting the target.

[0111] The overshoot amplitude OS_x in the horizontal direction can be expressed as: OS_x = max(|e_x|) / W.

[0112] The overshoot amplitude OS_y in the vertical direction can be expressed as: OS_y = max(|e_y|) / H.

[0113] The overshoot recovery time is the time from the moment the target center error sign flips to the moment the target center re-enters the center safety zone. The center safety zone is jointly defined by the horizontal center tolerance threshold and the vertical center tolerance threshold; when |e_x| is not greater than the horizontal center tolerance threshold and |e_y| is not greater than the vertical center tolerance threshold, the target center is determined to have entered the center safety zone.

[0114] By simultaneously employing three conditions—error sign flipping, error peak values ​​before and after sign flipping exceeding the center tolerance threshold, and reversed image motion direction—the probability of a target being misjudged as an overshoot event when it normally crosses the center of the image can be reduced.

[0115] In step S9, such as Figure 8 As shown, the motion state of the corresponding video frame is determined based on the motion amplitude `motion_i` and the preset motion threshold `m_threshold`. The preset motion threshold `m_threshold` can be set to 0.05% to 0.2% of the diagonal length `D` of the image.

[0116] When motion_i > m_threshold, the corresponding video frame is determined to be in motion; when motion_i ≤ m_threshold, the corresponding video frame is determined to be in stillness.

[0117] Count the number of transitions between motion and stationary states within the valid evaluation interval. The start-stop rate StartStopRate satisfies: StartStopRate=N_switch / T_valid; Where N_switch is the number of times the motion state and the stationary state are switched within the valid evaluation interval, and T_valid is the valid evaluation duration.

[0118] The start-stop switching count can include the number of times the camera starts transitioning from a stationary state to a moving state and the number of times it stops transitioning from a moving state to a stationary state. A high start-stop switching rate indicates that the camera frequently exhibits a stepped movement with stops and starts during tracking.

[0119] When the main tracking target is visible, the target center offset exceeds the preset offset threshold, and the motion amplitude of the image is not greater than the preset motion threshold, the corresponding video frame is determined to be an invalid still frame.

[0120] The InvalidStillRatio satisfies: InvalidStillRatio=N_invalid / N_visible; Where N_invalid represents the number of invalid static frames within the valid evaluation interval, and N_visible represents the number of frames visible to the main tracking target within the valid evaluation interval.

[0121] Invalid stillness is used to identify situations where the main target being tracked has deviated from the center of the frame, but the camera still has not produced an effective tracking motion. A high percentage of invalid stillness indicates that the camera's response to changes in the target's position is insufficient or has a large response delay.

[0122] In step S10, as Figure 7 As shown, the trajectory sequence of the center point of the main tracking target in the image is analyzed. The target center offset sequence is as follows: e_i = c_i - c_image; Where c_image = (W / 2, H / 2) is the center of the image.

[0123] The target center stability index may include at least one of the following: the rate of change of target center offset, the number of times the target center direction reverses, the intensity of target center oscillation, and the number of times the edge is thrown out.

[0124] The rate of change of the target center offset can be calculated based on the difference in target center offset between adjacent video frames and the time interval. The number of times the target center direction is reversed can be determined based on the directional change of the target center offset in the horizontal or vertical direction and a preset offset threshold.

[0125] The target central oscillation intensity, center_oscillation, can be expressed as: center_oscillation=mean(‖e_i-MA(e_i)‖2); Where MA(e_i) represents the moving average result of the target center offset sequence, and ||·|2 represents the two-dimensional Euclidean norm.

[0126] The normalized target center oscillation intensity, norm_center_oscillation, satisfies: norm_center_oscillation=center_oscillation / D.

[0127] The edge flick count is the number of times the center point of the main tracking target enters the preset edge area of ​​the screen from the preset center area or the normal tracking area. The screen edge area can be determined according to a preset ratio of screen width and screen height.

[0128] The target center stability index is used to reflect whether the main tracking target can be stably maintained in a reasonable position in the frame during the camera tracking process, avoiding the use of smooth frame movement as the sole criterion for judging good tracking performance.

[0129] In step S11, as Figure 9 and Figure 10 As shown, the smoothness comprehensive score is calculated based on the sub-indicators obtained in steps S5 to S10 and the scoring weights of each effective video segment.

[0130] The sub-indicators involved in the scoring may include average acceleration amplitude, P90 acceleration amplitude, average jerk amplitude, P90 jerk amplitude, normalized high-frequency jitter intensity, high-frequency direction change rate, number of direction reversals, number of oscillation events, number of overshoot events, overshoot amplitude, overshoot recovery time, start-stop switching rate, percentage of invalid stationary states, target center oscillation intensity, and number of edge-flipping events.

[0131] Calculate the corresponding penalty term P_k based on each sub-index x_k. The penalty term P_k can be a linear penalty function, a piecewise penalty function, or a saturation penalty function.

[0132] When a saturation penalty function is used, the penalty term P_k satisfies: P_k=P_k^max / {1+exp[-(x_k-μ_k) / s_k]}; Where P_k^max is the upper limit of the penalty for the k-th sub-index, and μ_k and s_k are parameters calibrated based on product type, target type, test scenario, user subjective rating, expert rating, historical problem samples or version regression test results.

[0133] In one parameter calibration method, each sub-index is calculated and normalized for historical test samples with user subjective ratings or expert ratings. The constraint is to minimize the error between the smoothness comprehensive score and the user subjective rating or expert rating, and to ensure that the score does not increase with the degree of anomaly. P_k^max, μ_k, and s_k are determined by grid search. After the parameters are determined, each parameter is associated with and saved with the corresponding product type, target type, test scenario, and sample version.

[0134] The above saturation penalty function can also be expressed as: P_k=P_k^max×sigmoid[(x_k-μ_k) / s_k】; in: sigmoid(z)=1 / [1+exp(-z)].

[0135] The overall smoothness score S satisfies: S=max{0,min[100,100-ΣP_k]}.

[0136] The above comprehensive score calculation relationship can also be expressed as: S=clip(100-ΣP_k,0,100); The clip function is used to limit the calculation result to the range of 0 to 100.

[0137] When different valid video segments have different scoring weights, the sub-indicators or penalty items within each video segment can be weighted and aggregated according to their respective scoring weights before calculating the overall smoothness score. For video segments with lower reliability but which can participate in the evaluation after compensation, their contribution to the sub-indicator statistics or penalty items should be reduced to mitigate the impact of motion estimation errors on the final score.

[0138] In one specific implementation, the baseline score is set to 100 points, the upper limit for the acceleration penalty item is set to 15 points, the upper limit for the jerk penalty item is set to 15 points, the upper limit for the high-frequency jitter penalty item is set to 20 points, the upper limit for the direction reversal and sway penalty item is set to 20 points, the upper limit for the overshoot penalty item is set to 20 points, the upper limit for the start-stop anomaly penalty item is set to 10 points, and the upper limit for the target center stability penalty item is set to 10 points. The above upper limits for penalty items are only used to illustrate one implementation method and do not constitute a limitation on the scope of protection of this invention.

[0139] The overall smoothness score can be mapped to multiple evaluation levels. For example, 90 to 100 points is level 5, indicating that the picture is smooth and natural without obvious stuttering; 80 to 89 points is level 4, indicating that there are occasional slight adjustments but they do not significantly affect the viewing experience; 65 to 79 points is level 3, indicating that there are perceptible stutters, overshoots, or bounces but they are still within an acceptable range; 50 to 64 points is level 2, indicating that there are frequent jitters, bounces, or abnormal start-stops; and below 50 points is level 1, indicating that the picture is severely unstable and significantly affects the viewing experience.

[0140] The output evaluation results include the overall smoothness score, evaluation level, values ​​of each sub-index, deductions for each sub-index, confidence level of motion estimation, main reasons for deductions, and the time intervals corresponding to the main reasons for deductions.

[0141] The main reasons for losing points may include at least one of the following: high-frequency jitter, frequent directional reversals, swayback, overshoot, excessively long overshoot recovery time, abnormal start-stop, invalid stillness, target center oscillation, edge ejection, or low confidence in motion estimation.

[0142] By outputting the specific reasons for the loss of points and the corresponding time interval, the anomaly type of the camera tracking algorithm can be identified, providing a basis for optimizing parameters such as tracking sensitivity, gimbal motion damping, target position prediction, speed control, start response, or stop response.

[0143] The product parameters, index values, and comprehensive scores in the following examples are exemplary constructed data used to illustrate the calculation relationships and output formats of this invention, and are not intended as actual product test results, experimental verification results, or proof of technical effects.

[0144] In one sample evaluation, an indoor PTZ camera that supports human tracking was used as the evaluation object. The test video resolution was 2560×1440 pixels, the video frame rate was 25fps, and the test target moved along an S-shaped path at a speed of about 0.5m / s.

[0145] The target bounding box, target center point, target visibility status, and tracking error markers are obtained by combining the target detection model with manual verification. After excluding video segments with target occlusion, target loss, tracking errors, and when the camera has not yet entered the tracking function activation state, an initial evaluation interval of approximately 38 seconds is obtained.

[0146] The ORB algorithm was used to extract background feature points outside the target bounding box extended mask. The target bounding box expansion ratio was set to 10%, the maximum number of ORB feature points was set to 500, the scale factor was set to 1.2, and the number of image pyramid layers was set to 8. The random sampling consensus algorithm was used to estimate the affine transformation matrix, with 2000 iterations and an inlier distance threshold of 3 pixels.

[0147] For video segments where the inlier ratio or background area ratio is lower than a preset value, the motion estimation is re-performed using the enhanced correlation coefficient image registration method. Video segments that still do not meet the reliability requirements after re-estimation are excluded and marked as low-confidence video segments.

[0148] The short window was set to 0.2 seconds, the medium window to 0.5 seconds, and the long window to 1.0 seconds. In this example, the normalized P90 acceleration amplitude was 0.018, the normalized P90 jerk amplitude was 0.042, the normalized high-frequency jitter intensity was 0.006, the direction reversal frequency was 14.2 times / minute, the oscillation event was 3 times, the overshoot event was 2 times, the average overshoot recovery time was 0.8 seconds, the start-stop switching rate was 9.5 times / minute, the proportion of invalid stillness was 7%, and the normalized target center oscillation intensity was 0.011.

[0149] In this example, the acceleration penalty is 2.7 points, the jerk penalty is 6.3 points, the high-frequency jitter penalty is 12 points, the direction reversal and sway penalty is 10 points, the overshoot penalty is 16 points, the start-stop anomaly penalty is 7 points, and the target center stability penalty is 0 points. Based on these penalties, the overall smoothness score is 46 points, corresponding to an evaluation level of 1. The main reasons for the low score are overshoot and high-frequency jitter.

[0150] The above values ​​are exemplary constructed data, used only to illustrate the calculation process and output format of the present invention, and are not intended as actual product test results, experimental verification results or proof of technical effects.

[0151] In the horizontal comparison test of products, product A and product B can record tracking videos under the same test environment, the same target movement path, the same target speed, and the same lighting conditions, and use the same target trajectory acquisition method, window parameters, reliability threshold, and penalty function parameters for evaluation.

[0152] In one example, the normalized P90 acceleration of product A is 0.008, and that of product B is 0.023; the normalized P90 jerk of product A is 0.015, and that of product B is 0.058; the normalized high-frequency jitter intensity of product A is 0.003, and that of product B is 0.009; the direction reversal frequency of product A is 6.2 times / minute, and that of product B is 18.5 times / minute; no swingback or overshoot events were detected in product A, while product B detected 5 swingback events and 3 overshoot events; the overall smoothness score of product A is 89.5 points, and that of product B is 52 points. Under the aforementioned exemplary parameter settings, the above results illustrate that the present invention can quantitatively compare the smoothness of the tracking images of different products.

[0153] The same product can also be evaluated under different test scenarios. The target moving laterally at a constant speed is mainly used to evaluate stable following ability; the target moving along an S-shaped path is mainly used to evaluate overshoot and sway during directional changes; the target running fast is mainly used to evaluate high-frequency jitter and start-stop anomalies during high-speed response; the target suddenly stopping or starting is mainly used to evaluate response delay and stepped movement; the target reappearing after a brief occlusion is mainly used to evaluate overshoot and recovery time during target reacquisition.

[0154] In one scene evaluation example, the smoothness score for the scene of the target moving horizontally at a constant speed was 92 points, corresponding to level 5; the smoothness score for the scene of the target moving along an S-shaped path was 78.5 points, corresponding to level 3, with the main reason for the loss being overshoot; the smoothness score for the scene of the target running quickly was 61 points, corresponding to level 2, with the main reasons for the loss being high-frequency jitter and abnormal start-stop; the smoothness score for the scene of the target suddenly stopping or starting was 58.5 points, corresponding to level 2, with the main reason for the loss being abnormal start-stop; and the smoothness score for the scene of the target reappearing after a brief occlusion was 65 points, corresponding to level 3, with the main reason for the loss being overshoot after the target reappeared.

[0155] In software version regression testing, the same camera can be tested with both the unoptimized and optimized software versions installed, and tracking videos recorded under the same test scenario, using the same evaluation parameters. When the overall smoothness score of the optimized software version improves, and the intensity of high-frequency jitter, the frequency of direction reversal, the number of swing events, the number of overshoot events, or the start-stop switching rate decreases, it can be determined that the software version optimization has improved the smoothness of the tracking footage.

[0156] For example, the smoothness scores of the unoptimized software version in scenarios where the target moves along an S-shaped path, the target runs quickly, and the target suddenly stops or starts are 78.5, 61, and 58.5, respectively; the smoothness scores of the optimized software version in the same scenarios are 85, 72, and 68, respectively. Under the aforementioned exemplary parameter settings, the above results illustrate that the present invention can characterize the change in the smoothness of the tracking screen before and after software version optimization.

[0157] The method of this invention, a trajectory smoothness evaluation method based on feature point matching and random sampling consensus algorithm, an evaluation method based on inter-frame motion amplitude statistics, and a multi-index jitter evaluation method based on optical flow or affine transformation can also be run on the same set of tracking test videos.

[0158] General video smoothness evaluation methods can output overall smoothness, amplitude statistics, or jitter level. However, in video segments where the camera reverses its correction after the target crosses the center of the frame, they can usually only identify sudden changes in motion trajectory and are difficult to determine overshoot events by combining the target center error sign flip and the reverse direction of the image motion. In video segments where the target center offset has not been improved and the camera swings repeatedly, it is difficult to identify swing events based on the improvement of the target center offset. In video segments where the target has deviated from the center but the image is still stationary, it is also difficult to identify invalid stillness.

[0159] Ablation experiments can be used to remove the target bounding box extension mask, multi-timescale windows, overshoot joint judgment conditions, swing back improvement judgment conditions, or background motion estimation reliability processing, and the false detection rate, false negative rate, correlation with user subjective scores, or consistency with expert scores can be compared.

[0160] After removing the target bounding box extension mask, the main tracking target's own motion can easily contaminate the global image motion estimation; after removing short, medium, and long windows, it is difficult to simultaneously cover high-frequency jitter and mid-to-low-frequency swaying; after removing the overshoot joint judgment condition, the process of the main tracking target normally crossing the center of the image can easily be misjudged as overshoot; after removing the target center offset improvement judgment, it is difficult to distinguish between reasonable direction adjustment and invalid repeated swaying; after removing the background motion estimation reliability processing, low-texture, low-light, or dynamic background video segments can easily lead to distorted evaluation results.

[0161] In low-texture or low-light scenes, when the number of background feature points is below a minimum threshold or the inlier rate of the random sampling consensus algorithm is below a preset threshold, the short or medium window can be enlarged, and the algorithm can switch to enhanced correlation coefficient image registration or sparse optical flow estimation, while reducing the scoring weight of the corresponding video segment. Video segments whose reliability is still insufficient after compensation will not participate in the comprehensive scoring; only a low confidence warning for motion estimation will be output in the evaluation results.

[0162] In dynamic background scenes, such as swaying curtains, flickering displays, passing vehicles, or multiple people moving around, the system can determine the reliability of a background region based on the spatial distribution and motion consistency of the matching inner points of background feature points. When the matching inner points of background feature points are concentrated in a local dynamic area, the weight of that area is reduced or the feature points in that area are removed to avoid misinterpreting dynamic background motion as global camera motion.

[0163] In scenes involving electronic zoom, image cropping, rolling shutter, or motion blur, translation, rotation, and scaling components can be decomposed from the affine transformation matrix or homography transformation matrix. When the scaling component exceeds a preset threshold, the corresponding video segment is recorded as an electronic zoom or image cropping interference video segment, and its confidence information is output separately. When rolling shutter or severe motion blur causes motion estimation failure, an estimation failure flag is output, and the corresponding video segment is excluded or its impact on the overall score is reduced.

[0164] This invention can be applied to the quality evaluation of human tracking using indoor PTZ cameras. In this application, the camera can be mounted on an indoor ceiling, wall, or furniture, and the distance between the main tracking target and the camera can be 2 to 5 meters. Due to the limited indoor space and the frequent changes in target orientation, the evaluation can focus on the frequency of orientation reversal, the number of swing events, the overshoot amplitude, and the overshoot recovery time.

[0165] This invention can also be applied to the quality evaluation of vehicle tracking in outdoor security cameras. In this application, the distance between the main tracking target and the camera can be 10 meters to 50 meters, and the vehicle speed can be 10 km / h to 40 km / h. Due to the large variation in vehicle speed, the evaluation can focus on the P90 acceleration amplitude, P90 jerk amplitude, and the rate of change of target center offset.

[0166] This invention can also be applied to the low-position tracking quality evaluation of pet cameras. Since pets' movement paths are irregular and their directions change frequently, the evaluation can focus on the intensity of target center oscillations, high-frequency jitter, number of directional reversals, and number of edge flicks.

[0167] This invention also provides a camera tracking quality evaluation system based on video image smoothness measurement. The system includes a video input and trajectory acquisition module, an initial evaluation interval generation module, a background motion estimation module, a reliability and effective interval determination module, a multi-window index calculation module, a high-frequency jitter analysis module, a tracking event recognition module, and a comprehensive scoring and cause localization module.

[0168] The video input and trajectory acquisition module is used to acquire the video recorded by the camera in the tracking test scenario, the target trajectory data of the main tracking target, and the tracking response status. The initial evaluation interval generation module is used to determine the video interval where the main tracking target is visible, no tracking error has occurred, and the camera's tracking function for the main tracking target is active, based on the target visibility status, tracking error markers, and tracking response status.

[0169] The background motion estimation module is used to construct a bounding box extended mask for adjacent video frames within the initial evaluation interval, extract and match background feature points in the background region outside the bounding box extended mask, estimate the global translation transformation matrix, affine transformation matrix or homography transformation matrix using a robust estimation method, and extract the image motion vector sequence representing the displacement direction of the background image content in the image plane.

[0170] The reliability and effective interval determination module is used to calculate the background motion estimation reliability of each adjacent frame pair. It aggregates the background motion estimation reliability of multiple consecutive adjacent frame pairs to obtain the segment reliability of candidate video segments. Candidate video segments with segment reliability not lower than a preset reliability threshold are determined as effective evaluation intervals. When the segment reliability is lower than the preset reliability threshold, the statistical window of the corresponding candidate video segment is expanded or the motion estimation method is switched, and the segment reliability is recalculated. Candidate video segments with recalculated segment reliability not lower than the preset reliability threshold are included in the effective evaluation interval and their corresponding scoring weights are determined. Candidate video segments with recalculated segment reliability still lower than the preset reliability threshold are excluded and low confidence labels are output.

[0171] The multi-window index calculation module is used to set short, medium and long windows within the effective evaluation range, and to calculate indices such as speed, acceleration, jerk, change in image motion direction and change in target center offset.

[0172] The high-frequency jitter analysis module is used to separate low-frequency following motion from high-frequency jitter residuals, and to calculate the normalized high-frequency jitter intensity and high-frequency direction change rate.

[0173] The tracking event recognition module may include a swing back recognition unit, an overshoot recognition unit, and a start / stop and invalid stillness recognition unit. The swing back recognition unit is used to identify swing back events based on the number of effective direction reversals within the middle window and the improvement of target center offset; the overshoot recognition unit is used to identify overshoot events based on the target center error sign flip, the same-direction error peak and reverse error peak before and after the sign flip, and the direction of screen motion, and calculates the overshoot amplitude and overshoot recovery time; the start / stop and invalid stillness recognition unit is used to determine the motion state or stillness state based on the screen motion amplitude, and calculates the start / stop switching rate and the proportion of invalid stillness.

[0174] The comprehensive scoring and cause localization module is used to calculate the comprehensive smoothness score based on the scoring weights of each sub-indicator, tracked event, and each valid video segment, and output the evaluation level, deduction score of each sub-indicator, confidence level, main reasons for loss of score, and corresponding time interval.

[0175] Each of the above modules can be implemented by one or more processors executing computer programs, or by software modules, hardware circuits, or a combination of software and hardware. Different modules can run on the same computing device or be distributed across different computing devices, and transmit video, target trajectory data, intermediate calculation results, or evaluation results via wired or wireless communication.

[0176] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the camera tracking quality evaluation method based on video image smoothness measurement described in steps S1 to S11 above.

[0177] The computer-readable storage medium may include a read-only memory, a random access memory, a solid-state drive, a magnetic disk, an optical disk, a flash memory, or other non-transitory storage media capable of storing computer programs.

[0178] It should be noted that the specific algorithm names, parameter ranges, thresholds, scoring values, evaluation levels, and exemplary constructed data listed in this specification are only for illustrating specific implementations of the present invention. Those skilled in the art can adjust the above parameters according to camera resolution, video frame rate, camera product type, main tracking target type, test scenario, and user perception requirements without departing from the technical concept of the present invention.

[0179] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Without departing from the technical concept of the present invention, those skilled in the art can make corresponding adjustments or substitutions. The scope of protection of the present invention is determined by the claims.

Claims

1. A camera tracking quality evaluation method based on video image smoothness measurement, characterized in that, Includes the following steps: Step S1: Acquire the video recorded by the camera in the tracking test scenario, the target trajectory data of the main tracking target, and the tracking response status. The target trajectory data includes the target bounding box, the target center point, the target visibility status, and the tracking error marker. Generate an initial evaluation interval based on the target visibility status, the tracking error marker, and the tracking response status. The initial evaluation interval is the video interval in which the main tracking target is visible, no tracking error has occurred, and the camera's tracking function for the main tracking target is active. Step S2: Construct a target bounding box extended mask for adjacent video frames within the initial evaluation interval, extract and match background feature points in the background region outside the target bounding box extended mask, estimate the global translation transformation matrix, affine transformation matrix, or homography transformation matrix between adjacent video frames using a robust estimation method, and extract the image motion vector sequence representing the displacement direction of the background image content from the previous video frame to the next video frame in the image plane from the global translation transformation matrix, the affine transformation matrix, or the homography transformation matrix. Step S3: Based on at least one of the following factors: background feature point matching inlier rate, background region area ratio, background motion consistency, and dynamic background interference level, calculate the background motion estimation reliability of each adjacent frame pair, and aggregate the background motion estimation reliability of multiple consecutive adjacent frame pairs to obtain the segment reliability of candidate video segments; determine the candidate video segments with segment reliability not lower than a preset reliability threshold as valid evaluation intervals; when the segment reliability is lower than the preset reliability threshold, expand the statistical window or switch the motion estimation method for the corresponding candidate video segment, and recalculate the segment reliability; include the candidate video segments with recalculated segment reliability not lower than the preset reliability threshold into the valid evaluation interval, and determine the scoring weight of the corresponding video segment based on the recalculated segment reliability; exclude the candidate video segments with recalculated segment reliability still lower than the preset reliability threshold and output a low confidence mark; perform tracking quality evaluation only within the valid evaluation interval; Step S4: Set a short window, a medium window, and a long window within the effective evaluation interval. The short window is used to detect high-frequency jitter or sudden pauses. The medium window is used to detect direction reversal, swing back, or overshoot. The long window is used to analyze the overall tracking trend and perform statistical analysis on the motion vector sequence of the image and the trajectory sequence of the target center point based on each sliding window. Step S5: Calculate vector velocity, vector acceleration, and vector jerk based on the motion vector sequence of the image, and calculate the average acceleration amplitude, P90 acceleration amplitude, average jerk amplitude, and P90 jerk amplitude based on the vector acceleration and the vector jerk. Step S6: Separate the low-frequency following motion and high-frequency jitter residuals from the motion vector sequence of the image, calculate the normalized high-frequency jitter intensity based on the separated high-frequency jitter residuals, and calculate the high-frequency direction change rate; Step S7: Detect effective direction reversal according to the motion vector sequence of the image, and identify swing back events according to the number of effective direction reversals within the preset time window and the improvement of the target center offset. The swing back event is an event in which multiple effective direction reversals occur within the preset time window and the target center offset does not reach the preset improvement condition. Step S8: Identify overshoot events based on the joint conditions of the error sign flipping between the target center and the image center, the peak error in the same direction before the flipping and the peak error in the opposite direction after the flipping exceeding the center tolerance threshold in the preset observation sub-window before and after the sign flipping, and the image motion direction after the error sign flipping being reversed relative to the direction before the sign flipping, and calculate the overshoot amplitude and overshoot recovery time corresponding to the overshoot event. Step S9: Determine the motion state or stationary state based on the comparison result between the screen motion amplitude and the preset motion threshold, and count the start-stop switching rate and invalid stationary ratio within the effective evaluation interval. The invalid stationary state is the state where the main tracking target is visible, the target center offset exceeds the preset offset threshold, and the screen motion amplitude is not greater than the preset motion threshold. Step S10: Analyze the trajectory sequence of the target center point in the image, and calculate at least one of the following target center stability indicators: target center offset change rate, number of target center direction reversals, target center oscillation intensity, and number of edge ejections; Step S11: Based on the average acceleration amplitude, the P90 acceleration amplitude, the average jerk amplitude, the P90 jerk amplitude, the normalized high-frequency jitter intensity, the high-frequency direction change rate, the swing event, the overshoot event, the start-stop switching rate, the proportion of invalid stillness, the target center stability index, and the scoring weight of each effective video segment, calculate the smoothness comprehensive score according to the penalty function calibrated by product type, target type, test scenario, user subjective score, expert score, historical problem sample or version regression test results, and output the smoothness comprehensive score, level, deduction of each sub-index, confidence level, main reasons for loss of points and corresponding time interval, so as to distinguish between effective following motion and invalid jitter, overshoot, swing, start-stop abnormality or invalid stillness in the target tracking process.

2. The camera tracking quality evaluation method based on video image smoothness quantification according to claim 1, characterized in that, In step S2: The target bounding box expansion mask is obtained by expanding the target bounding box outward according to a preset expansion ratio, so as to exclude feature points in the main tracking target, the target edge shadow area, or the local occlusion area. When the global translation transformation matrix or the affine transformation matrix is ​​used, the translation components in the global translation transformation matrix or the affine transformation matrix are extracted as the horizontal and vertical displacements between adjacent video frames. When the homography transformation matrix is ​​used, the preset background sampling points or background grid points are mapped through the homography transformation matrix, and the average value, median value, or representative value of the central region of the displacement before and after mapping is calculated as the image motion vector between adjacent video frames.

3. The camera tracking quality evaluation method based on video image smoothness measurement according to claim 1, characterized in that: In step S4, the duration of the short window is 0.2 to 0.5 seconds, the duration of the medium window is 0.5 to 1.0 seconds, and the duration of the long window is 1.0 to 2.0 seconds. The duration of each window is converted into the corresponding number of window frames according to the frame rate of the video. In step S5, let the motion vector between frame i and frame i+1 be g_i, and the time interval between adjacent video frames be Δt. Then the vector velocity V_i, the vector acceleration A_i, and the vector jerk J_i satisfy the following: V_i = g_i / Δt; A_i = (V_i - V_(i-1)) / Δt; J_i = (A_i - A_(i-1)) / Δt. The magnitudes of the vector acceleration and the vector jerk are calculated separately, and the magnitudes of the vector acceleration and the vector jerk are statistically analyzed to obtain the average acceleration magnitude, the P90 acceleration magnitude, the average jerk magnitude, and the P90 jerk magnitude. Wherein, the P90 acceleration amplitude is the 90th percentile of the vector acceleration amplitude, and the P90 jerk amplitude is the 90th percentile of the vector jerk amplitude.

4. The camera tracking quality evaluation method based on video image smoothness quantification according to claim 1, characterized in that: In step S6, a moving average is performed on the motion vector sequence of the image to obtain a low-frequency following motion component. The window length of the moving average is 0.5 seconds to 1.0 seconds. The low-frequency following motion component is subtracted from the motion vector sequence of the image to obtain a high-frequency jitter residual. The normalized high-frequency jitter intensity is the ratio of the average amplitude of the high-frequency jitter residual to the length of the image diagonal. In step S7, let the motion vector of the image between frame i and frame i+1 be: g_i = (dx_i, dy_i); A valid direction reversal in the horizontal direction satisfies: sign(dx_i) ≠ sign(dx_(i-1)), and |dx_i| > d_threshold, and |dx_(i-1)| > d_threshold; A valid direction reversal in the vertical direction satisfies: sign(dy_i) ≠ sign(dy_(i-1)), and |dy_i| > d_threshold, and |dy_(i-1)| > d_threshold; where d_threshold is the direction reversal determination threshold, which is 0.1% to 0.3% of the diagonal length of the image. The criteria for determining the swing event include: two or more valid direction reversals occurring within a medium window of 0.5 to 1.0 seconds, and the target center offset at the end of the medium window not being improved compared to the target center offset at the beginning of the medium window.

5. The camera tracking quality evaluation method based on video image smoothness quantification according to claim 1, characterized in that: In step S8, the error between the target center and the image center undergoes a sign flip along at least one coordinate axis within a preset time window. Within the preset observation sub-windows before and after the sign flip, the peak value of the same-direction error before the flip and the peak value of the opposite-direction error after the flip exceed the center tolerance threshold set along the corresponding coordinate axis, and the image motion direction after the sign flip is reversed relative to the image motion direction before the sign flip. The overshoot amplitude is the ratio of the peak value of the reverse error after the sign flip to the image size in the corresponding coordinate axis direction. The overshoot recovery time is the time from the moment the error sign flips to the time when the target center re-enters the central safe zone. The central safe zone is enclosed by a horizontal center tolerance threshold and a vertical center tolerance threshold. When the absolute value of the horizontal error of the target center is not greater than the horizontal center tolerance threshold and the absolute value of the vertical error is not greater than the vertical center tolerance threshold, it is determined that the target center has entered the central safe zone. In step S9, the motion state or the stationary state is determined based on the comparison result between the motion amplitude of the screen, motion_i and the preset motion threshold, m_threshold. The preset motion threshold, m_threshold, is 0.05% to 0.2% of the diagonal length of the screen. When the motion amplitude of the screen, motion_i, is greater than the preset motion threshold, m_threshold, the corresponding video frame is determined to be in motion; otherwise, the corresponding video frame is determined to be in stationary state. The start-stop switching rate is the ratio of the number of times the motion state and the stationary state are switched within the effective evaluation interval to the effective evaluation duration; The invalid still frame is a video frame within the valid evaluation interval where the main tracking target is visible, the target center offset exceeds the preset offset threshold, and the motion amplitude motion_i is not greater than the preset motion threshold m_threshold. The invalid still frame ratio is the ratio of the number of invalid still frames to the number of target visible frames within the valid evaluation interval.

6. The camera tracking quality evaluation method based on video image smoothness quantification according to claim 1, characterized in that, In step S3, the background motion estimation reliability R_i satisfies: R_i=α×N_inlier / N_match+β×Area_bg / Area_img-γ×E_dynamic; Wherein, N_inlier is the number of background feature point matching inliers determined by the robust estimation method, N_match is the number of background feature point matching points, Area_bg is the effective area of ​​the background region, Area_img is the image area, E_dynamic is the dynamic background interference index, and α, β and γ are preset parameters or parameters obtained by calibration through test samples. A candidate video segment is formed by K consecutive frames, and the candidate video segment includes K-1 adjacent frame pairs. The segment reliability R_seg of the candidate video segment satisfies: R_seg=[Σ_(i=1)^(K-1)R_i] / (K-1); The segment reliability R_seg is compared with the preset reliability threshold to determine whether the candidate video segment is included in the valid evaluation interval.

7. The camera tracking quality evaluation method based on video image smoothness quantification according to claim 1, characterized in that, In step S11: calculate the corresponding penalty term P_k according to each sub-index x_k, wherein the penalty term P_k is one of a linear penalty function, a piecewise penalty function, or a saturation penalty function; When the penalty term P_k uses the saturation penalty function, the following conditions are met: P_k=P_k^max / {1+exp[-(x_k-μ_k) / s_k]}; Where P_k^max is the upper limit of the penalty for the k-th sub-index, and μ_k and s_k are parameters calibrated based on product type, target type, test scenario, user subjective rating, expert rating, historical problem samples or version regression test results; The smoothness score S satisfies: S=max{0,min[100,100-ΣP_k]}.

8. A camera tracking quality evaluation system based on video image smoothness measurement, characterized in that, include: The video input and trajectory acquisition module is used to acquire the video recorded by the camera in the tracking test scenario, the target trajectory data of the main tracking target, and the tracking response status. The target trajectory data includes the target bounding box, the target center point, the target visibility status, and the tracking error marker. The initial evaluation interval generation module is used to determine the video interval in which the main tracking target is visible, no tracking error has occurred, and the camera's tracking function for the main tracking target is active, based on the target visibility status, the tracking error marker, and the tracking response status. The background motion estimation module is used to construct a target bounding box extended mask for adjacent video frames within the initial evaluation interval, extract and match background feature points in the background region outside the target bounding box extended mask, estimate the global translation transformation matrix, affine transformation matrix or homography transformation matrix using a robust estimation method, and extract the image motion vector sequence representing the displacement direction of the background image content from the previous video frame to the next video frame in the image plane. The reliability and effective interval determination module is used to calculate the background motion estimation reliability of each adjacent frame pair based on at least one of the following: background feature point matching in-point rate, background region area ratio, background motion consistency, and dynamic background interference degree. It aggregates the background motion estimation reliability of multiple consecutive adjacent frame pairs to obtain the segment reliability of candidate video segments. Candidate video segments with segment reliability not lower than a preset reliability threshold are determined as effective evaluation intervals. When the segment reliability is lower than the preset reliability threshold, the module expands the statistical window or switches the motion estimation method for the corresponding candidate video segment and recalculates the segment reliability. Candidate video segments with recalculated segment reliability not lower than the preset reliability threshold are included in the effective evaluation interval, and the scoring weight of the corresponding video segment is determined based on the recalculated segment reliability. Candidate video segments with recalculated segment reliability still lower than the preset reliability threshold are excluded, and a low confidence marker is output. The multi-window index calculation module is used to set short windows, medium windows and long windows within the effective evaluation interval, and to calculate speed, acceleration, jerk, change in image motion direction and change in target center offset index based on the image motion vector sequence and the target trajectory data; The high-frequency jitter analysis module is used to separate the low-frequency following motion from the high-frequency jitter residual in the motion vector sequence of the image, and to calculate the normalized high-frequency jitter intensity and high-frequency direction change rate. The tracking event recognition module is used to identify direction reversal events, swing back events, and overshoot events based on the motion vector sequence of the screen and the target trajectory data, to calculate the start-stop switching rate and the proportion of invalid stationary states, and to calculate the target center stability index. The comprehensive scoring and cause localization module is used to calculate the smoothness comprehensive score based on the scoring weights of each sub-indicator, tracked event, and each valid video segment, according to the calibrated penalty function, and output the smoothness comprehensive score, level, deduction score of each sub-indicator, confidence level, main reasons for loss of points, and corresponding time interval.

9. The camera tracking quality evaluation system based on video image smoothness measurement according to claim 8, characterized in that, The event tracking and identification module includes: The swing recognition unit is used to count the number of effective directional reversals within the middle window and determine the improvement of the target center offset within the middle window. When the number of effective directional reversals reaches a preset number and the target center offset does not reach a preset improvement condition, a swing event is determined to have occurred. The overshoot identification unit is used to identify overshoot events based on the following joint conditions: the error between the target center and the image center causes a sign flip; the peak value of the same direction error before the flip and the peak value of the reverse error after the flip exceed the center tolerance threshold in the preset observation sub-window before and after the sign flip; and the image motion direction after the sign flip is reversed relative to the direction before the sign flip. The unit also calculates the overshoot amplitude and overshoot recovery time corresponding to the overshoot event. The start / stop and invalid stillness identification unit is used to determine the motion state or stillness state based on the comparison result of the screen motion amplitude and the preset motion threshold, calculate the start / stop switching rate based on the number of switching between the motion state and the stillness state and the effective evaluation time, and determine the corresponding video frame as an invalid still frame when the main tracking target is visible, the target center offset exceeds the preset offset threshold and the screen motion amplitude is not greater than the preset motion threshold, and calculate the invalid stillness ratio based on the ratio of the number of invalid still frames to the number of target visible frames in the effective evaluation interval.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the camera tracking quality evaluation method based on video image smoothness measurement as described in any one of claims 1 to 7.

Images