A method for evaluating a video picture jitter level

Abstract

A method for evaluating the shaking level of a video picture, comprising the following steps: frame processing is performed on a video to be detected, and the resolution of each frame is down-sampled to obtain a frame sequence; each adjacent two frames in the frame sequence are input into a real-time optical flow extraction network based on deep learning to extract an optical flow map; for adjacent frames, feature points are uniformly selected from a previous frame, i.e., a preceding frame, the coordinates of the feature points are added to corresponding optical flow components output by the optical flow extraction network to obtain the coordinates of the feature points in a subsequent frame, affine transformation estimation is performed, and an affine transformation matrix of the entire frame between the i-th frame and the i+1-th frame is output; a cumulative transformation sequence of the video to be detected is calculated; an evaluation index is calculated for the cumulative transformation sequence of the video, and a weighted operation is performed, a shaking score of each second of the video to be detected is output, whether the video to be detected shakes and the degree of shaking are determined according to a shaking degree threshold value. The present application can improve the accuracy and robustness of video shaking detection and evaluation under different shaking conditions.

Description

Technical Field

[0001] The present invention belongs to the technical field of video picture jitter assessment, and in particular relates to a method for assessing the jitter level of a video picture. Background Art

[0002] Video jitter detection is fundamental to post-processing and adjustment of videos. Researchers have conducted extensive research based on video analysis in fields such as video processing, video stabilization, and computer vision. Although several methods have been proposed, existing detection algorithms are inaccurate and slow, making them inadequate for real-time detection in scenarios such as surveillance. Some commonly used methods are insensitive to videos captured under conditions of large lens displacement and strong jitter within a short period of time, others are insensitive to minor jitter, some are unsuitable for detecting rotational motion, and others are unsuitable for scenarios with low camera jitter frequencies.

[0003] The existing methods are as follows:

[0004] 1) Grayscale projection method: Based on the principle of grayscale distribution consistency of overlapping similar regions in the image, the local grayscale information of adjacent video frames is used to obtain the vector motion relationship. The algorithm mainly consists of grayscale projection correlation calculations in both row and column directions of different regions.

[0005] 2) Block matching: This is the most commonly used algorithm in video stabilization systems. This method divides the current frame into blocks, where every pixel has the same motion vector. For each block, the best match is searched within a specific range of the reference frame to estimate the global motion vector of the video sequence.

[0006] Grayscale projection is effective for scenes with only translational jitter. It cannot estimate rotational motion vectors, cannot distinguish between smooth and rapid video motion and video jitter, and cannot detect slight video jitter.

[0007] Block matching methods typically require block segmentation and estimate the global motion vector based on the motion vector within each block. This makes them ineffective in detecting video jitter in certain scenarios. For example, a frame may be divided into four grids, three of which are stationary, while one contains a moving object. Furthermore, block matching methods typically require Kalman filtering to process the calculated motion vectors, which is computationally expensive and lacks real-time performance. This makes them ineffective in detecting large camera movement and strong jitter within a short period of time.

[0008] The patent document "A Video Image Jitter Detection Method Based on TSN Network" jointly applied for by Gosuncn Technology Group Co., Ltd. and Xidian University (application date: September 6, 2019, CN110602487A) is too slow to detect high-resolution videos and cannot be applied to scenarios requiring real-time detection, such as surveillance.

[0009] The patent document applied for by Suning Cloud Computing Co., Ltd. is "A method and device for detecting video jitter" (application date: June 21, 2019, publication number CN110248048A); on the one hand, its accuracy depends on the selection of feature points. When there are many moving foreground objects in the picture, the feature points tend to gather on the moving foreground objects, and it is easy to judge a stable picture as a jittery picture, resulting in a misjudgment of picture jitter. Summary of the Invention

[0010] To overcome the shortcomings of the above-mentioned prior art, the present invention aims to provide a method for assessing video image jitter levels, which can improve the accuracy and robustness of video jitter detection and assessment under different jitter scenarios. In particular, for scenes with slight jitter, scenes with strong jitter such as large displacement and rotation, or scenes with slow jitter with a low jitter frequency, it can reduce the misjudgment of stable motion scenes, improve the detection processing speed, and ensure the jitter detection performance and speed in scenes requiring real-time detection, such as surveillance.

[0011] In order to achieve the above object, the technical solution adopted by the present invention is:

[0012] A method for evaluating the jitter level of a video image comprises the following steps:

[0013] Step 1: First, the video to be detected is divided into frames, and the resolution of each frame is downsampled to obtain a frame sequence (P1, P2, ..., P i , P i+1 ,……,P n ), where P i represents the i-th frame of the video to be detected, and n represents the total number of frames of the video to be detected;

[0014] Step 2: Input every two adjacent frames in the frame sequence obtained in step 1 into the real-time optical flow extraction network based on deep learning to extract the optical flow map;

[0015] Step 3: For adjacent frames P i and P i+1 , in the previous frame, P i Uniformly select feature points on the image, add the coordinates of the feature points to the corresponding optical flow components output by the optical flow extraction network to obtain the feature points in the next frame, i.e., P i+1 Corresponding coordinates; perform affine transformation estimation on the two sets of coordinates and output the i-th frame P i To the i+1th frame P i+1 The overall affine transformation matrix T between frames i ;

[0016] Step 4: Calculate the cumulative transformation sequence L of the video to be detected;

[0017] Step 5: Calculate six evaluation indicators for the cumulative transformation sequence L of the video in step 4 and perform weighted operations, output the jitter score for each second of the video to be detected, and determine whether the video to be detected is jittery and the degree of jitter based on the jitter degree threshold.

[0018] The step 2 is specifically as follows:

[0019] Get inter-frame optical flow: the adjacent frames P i and P i+1 Input a dense optical flow tracking network FastFlowNet based on deep learning (reference Kong L, Shen C, Yang J.Fastflownet:A lightweight network for fast optical flow estimation[C] / / 2021IEEE International Conference on Robotics and Automation(ICRA).IEEE,2021:10310-10316), output P i to P i+1 The optical flow map F i ;

[0020]

[0021] Where i represents the index of the video frame, i=1, 2, ..., n-1; F i Represents the i-th frame P i To the i+1th frame P i+1 The optical flow matrix of P i The horizontal coordinate index of the pixel point in P, j = 1, 2, ..., 720; k represents P i The vertical coordinate index of the pixel point, k = 1, 2, ..., 1280; Indicates P i The pixel point with coordinate index (j, k) is from P i to P i+1 The motion vector of dx represents P i The pixel point with coordinate index (j, k) is from P i to P i+1 The row offset component, dy represents P i The pixel point with coordinate index (j, k) is from P i to P i+1 The column offset component of .

[0022] The step 3 is specifically as follows:

[0023] Perform affine transformation estimation and output the overall affine transformation matrix between frames;

[0024] Step 3.1: In P i Uniformly select m feature points from the matrix and record the coordinates of these feature points to obtain the feature point coordinate matrix S. i ;

[0025]

[0026] Where i represents the index of the video frame, i=1, 2, ..., n-1; S i Indicates that in the i-th frame P i The matrix composed of all the feature points selected above; Represents the i-th frame P i The horizontal coordinate of the first feature point selected on Represents the i-th frame P i The vertical coordinate of the first feature point selected; Represents the i-th frame P i The horizontal coordinate of the second feature point selected on Represents the i-th frame P i The vertical coordinate of the second feature point selected above; m represents the total number of feature points; Represents the i-th frame P i The horizontal coordinate of the mth feature point selected on Represents the i-th frame P i The ordinate of the mth feature point selected above;

[0027] Step 3.2: The optical flow map F output in step 2 i Find the feature point coordinate matrix S in step 3.1 i The corresponding motion vector matrix M i ;

[0028]

[0029] Where i represents the index of the video frame, i=1, 2, ..., n-1; M i Represents the i-th frame P i The matrix composed of the motion vectors of the m feature points selected above; Represents the i-th frame P i The horizontal coordinate of the first feature point selected on Represents the i-th frame P i The vertical coordinate of the first feature point selected; Indicates P i The median coordinate index is The first feature point from P i to P i+1 The motion vector of Represents the i-th frame P iThe horizontal coordinate of the second feature point selected on Represents the i-th frame P i The vertical coordinate of the second feature point selected; Indicates P i The median coordinate index is The second characteristic point of P i to P i+1 The motion vector of m represents the total number of feature points. Represents the i-th frame P i The horizontal coordinate of the mth feature point selected on Represents the i-th frame P i The ordinate of the mth feature point selected above; Indicates P i The median coordinate index is The mth feature point from P i to P i+1 motion vector; each pixel on the video frame has an optical flow component, according to the coordinates of the feature point You can index its corresponding optical flow component

[0030] Step 3.3: Calculate the feature points at P i+1 The coordinates of P i The coordinate matrix of the feature points selected in P is added to the motion vector matrix of the feature points to obtain the coordinate matrix of the feature points selected in P i+1 The coordinate matrix S in i+1 , that is, S i+1 =S i +M i ;

[0031] Step 3.4: S i With S i+1 Input into the opencv affine transformation estimation function and output the i-th frame P i To the i+1th frame P i+1 The affine transformation matrix T i ,

[0032]

[0033] Where i represents the index of the video frame, i=1, 2, ..., n-1; T i Represents the i-th frame P i To the i+1th frame P i+1 The overall affine transformation matrix of i Represents the i-th frame P i To the i+1th frame P i+1 The rotation angle of the overall affine transformation; Δx i Represents the i-th frame Pi To the i+1th frame P i+1 The row offset component of the overall affine transformation; Δy i Represents the i-th frame P i To the i+1th frame P i+1 The column offset component of the overall affine transformation.

[0034] Further, all frames of the video to be detected are processed according to steps 2-3 to obtain an affine transformation parameter sequence H;

[0035]

[0036] where Δx 1 , Δy 1 , Δθ 1 Δx represents the row offset component, column offset component and rotation angle of the overall affine transformation from the first frame P1 to the second frame P2; 2 , Δy 2 , Δθ 2 Δx represents the row offset component, column offset component and rotation angle of the overall affine transformation from the second frame P2 to the third frame P3; n-1 , Δy n-1 , Δθ n-1 Indicates the n-1th frame P n- To the nth frame P n The row offset component, column offset component, and rotation angle of the overall affine transformation.

[0037] The step 4 is specifically as follows:

[0038] Accumulate the affine transformation parameter sequence H row by row to obtain the cumulative transformation sequence L of the video to be detected;

[0039]

[0040] where Δx 1 , Δy 1 , Δθ 1 Δx represents the row offset component, column offset component and rotation angle of the overall affine transformation from the first frame P1 to the second frame P2; 2 , Δy 2 , Δθ 2 Δx represents the row offset component, column offset component and rotation angle of the overall affine transformation from the second frame P2 to the third frame P3; i , Δy i , Δθ i Represents the i-th frame P i To the i+1th frame P i+1 The row offset component, column offset component and rotation angle of the overall affine transformation, i represents the index of the video frame, i = 1, 2, ..., n-1.

[0041] The step 5 is specifically:

[0042] The evaluation index is calculated: the frame rate of the video to be detected is Rfps, and R frames are taken as an evaluation unit, that is, the video is evaluated once per second, and six evaluation indexes are designed. The six evaluation indexes of the Tth second of the video to be detected are E 1,T , E 2,T , E 3,T , E 4,T , E 5,T , and E 6,T , and the calculation method is as follows:

[0043]

[0044] Wherein j represents the evaluation index, j = 1, 2, 3; T represents the Tth second of the video to be detected; E j,T represents the jth evaluation index of the Tth second of the video to be detected; R represents the frame rate of the video to be detected; L is the cumulative transformation sequence; i represents the row index value of the cumulative transformation sequence L,

[0045]

[0046] Wherein k represents the evaluation index, k = 4, 5, 6; T represents the Tth second of the video to be detected; E k,T represents the kth evaluation index of the Tth second of the video to be detected; R represents the frame rate of the video to be detected; L is the cumulative transformation sequence; i represents the row index value of the cumulative transformation sequence L,

[0047] The weighted score of the video to be detected per second is calculated, and the level of jitter is judged;

[0048] score T = E 1,T + E 2,T + 100 x E 3,T + E 4,T + E 5,T + 100 x E 6,T

[0049] Wherein T represents the Tth second of the video to be detected, score T represents the weighted score of the Tth second of the video to be detected; E 1,T , E 2,T , E 3,T , E 4,T , E 5,T , and E 6,T represent the six evaluation indexes of the Tth second of the video to be detected.

[0050] The level evaluation threshold is set, if scoreT 11.8, the video is not shaking; if 1.8≤score T <9, the video jitter level is 1; if 9≤score T <20, the video jitter level is 2; if 20≤score T <45, the video jitter level is 3; if 45≤score T , the video jitter level is 4.

[0051] Beneficial effects of the present invention:

[0052] The present invention uses the dense optical flow network FastFlowNet based on deep learning to extract motion information of adjacent frames. The FastFlowNet network extracts optical flow with high accuracy and speed, which improves the accuracy of jitter detection in scenes with slight jitter or strong jitter scenes with large displacement and rotation, and realizes the function of real-time detection of high-resolution video.

[0053] The present invention evenly selects feature points on the video frame, so that the distribution of feature points takes into account both the foreground and background of the picture, and can more accurately estimate the overall motion of the picture;

[0054] The present invention sets 6 evaluation indicators, taking into account the jitter detection accuracy of various jitter situations such as horizontal and vertical jitter, rotational jitter evaluation, high-frequency fast jitter, low-frequency slow jitter, etc.

[0055] The present invention sets reasonable weighting coefficients and level evaluation thresholds, reduces misjudgment of stable videos, and subdivides jitter levels, thereby better understanding the degree of video jitter. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0057] The present invention will be described in further detail below with reference to the accompanying drawings.

[0058] like Figure 1 The specific steps are as follows:

[0059] Step 1: Read the detection video, perform frame processing, and downsample each frame to 1280×720 resolution to obtain the frame sequence P i (i=1,2,3,…,n), where P i represents the i-th frame of the video to be detected, and n represents the total number of frames of the video to be detected;

[0060] Step 2: Get inter-frame optical flow: P i and P i+1Input a dense optical flow tracking network FastFlowNet based on deep learning and output P i to P i+1 The optical flow map F i , i=1,2,3,…,n-1;

[0061]

[0062] Where i represents the index of the video frame, i=1, 2, ..., n-1; F i Represents the i-th frame P i To the i+1th frame P i+1 The optical flow matrix of P i The horizontal coordinate index of the pixel point in P, j = 1, 2, ..., 720; k represents P i The vertical coordinate index of the pixel point, k = 1, 2, ..., 1280; Indicates P i The pixel point with coordinate index (j, k) is from P i to P i+1 The motion vector of dx represents P i The pixel point with coordinate index (j, k) is from P i to P i+1 The row offset component, dy represents P i The pixel point with coordinate index (j, k) is from P i to P i+1 The column offset component of .

[0063] Step 3: Perform affine transformation estimation and output the overall affine transformation matrix between frames;

[0064] Step 3.1: In P i Evenly select more than 200 feature points, record the coordinates of these feature points, and obtain the feature point coordinate matrix S i ;

[0065]

[0066] Where i represents the index of the video frame, i=1, 2, ..., n-1; S i Indicates that in the i-th frame P i The matrix composed of all the feature points selected above; Represents the i-th frame P i The horizontal coordinate of the first feature point selected on Represents the i-th frame P i The vertical coordinate of the first feature point selected; Represents the i-th frame P i The horizontal coordinate of the second feature point selected on Represents the i-th frame P i The vertical coordinate of the second feature point selected above; m represents the total number of feature points; Represents the i-th frame P i The horizontal coordinate of the mth feature point selected on Represents the i-th frame P i The vertical coordinate of the mth feature point selected on .

[0067] Step 3.2: The optical flow map F output in step 2 i Find the feature point coordinate matrix S in step 3.1 i The corresponding motion vector matrix M i .

[0068]

[0069] Where i represents the index of the video frame, i=1, 2, ..., n-1; M i Represents the i-th frame P i The matrix composed of the motion vectors of the m feature points selected above; Represents the i-th frame P i The horizontal coordinate of the first feature point selected on Represents the i-th frame P i The vertical coordinate of the first feature point selected; Indicates P i The median coordinate index is The first feature point from P i to P i+1 The motion vector of Represents the i-th frame P i The horizontal coordinate of the second feature point selected on Represents the i-th frame P i The vertical coordinate of the second feature point selected; Indicates P i The median coordinate index is The second characteristic point of P i to P i+1 The motion vector of m represents the total number of feature points. Represents the i-th frame P i The horizontal coordinate of the mth feature point selected on Represents the i-th frame P i The ordinate of the mth feature point selected above; Indicates P i The median coordinate index is The mth feature point from P i to P i+1 motion vector.

[0070] Step 3.3: Calculate the feature points at P i+1 The coordinates of P i The coordinate matrix of the feature points selected in P is added to the motion vector matrix of the feature points to obtain the coordinate matrix of the feature points selected in P i+1 The coordinate matrix S in i+1 , that is, S i+1 =S i +M i ;.

[0071] Step 3.4: S i With S i+1 Input into the opencv affine transformation estimation function and output the i-th frame P i To the i+1th frame P i+1 The affine transformation matrix T i ;

[0072]

[0073] Where i represents the index of the video frame, i=1, 2, ..., n-1; T i Represents the i-th frame P i To the i+1th frame P i+1 The overall affine transformation matrix of i Represents the i-th frame P i To the i+1th frame P i+1 The rotation angle of the overall affine transformation; Δx i Represents the i-th frame P i To the i+1th frame P i+1 The row offset component of the overall affine transformation; Δy i Represents the i-th frame P i To the i+1th frame P i+1 The column offset component of the overall affine transformation.

[0074] Step 4: Process all frames of the video to be detected according to steps 2-3 to obtain the affine transformation parameter sequence H.

[0075]

[0076] where Δx 1 , Δy 1 , Δθ 1 Δx represents the row offset component, column offset component and rotation angle of the overall affine transformation from the first frame P1 to the second frame P2; 2 , Δy 2 , Δθ 2 Δx represents the row offset component, column offset component and rotation angle of the overall affine transformation from the second frame P2 to the third frame P3; n-1 , Δy n-1 , Δθ n-1Indicates the n-1th frame P n- To the nth frame P n The row offset component, column offset component, and rotation angle of the overall affine transformation;

[0077] Step 5: Accumulate the affine transformation parameter sequence H row by row to obtain the cumulative transformation sequence L of the video to be detected.

[0078]

[0079] where Δx 1 , Δy 1 , Δθ 1 Δx represents the row offset component, column offset component and rotation angle of the overall affine transformation from the first frame P1 to the second frame P2; 2 , Δy 2 , Δθ 2 Δx represents the row offset component, column offset component and rotation angle of the overall affine transformation from the second frame P2 to the third frame P3; i , Δy i , Δθ i Represents the i-th frame P i To the i+1th frame P i+1 The row offset component, column offset component and rotation angle of the overall affine transformation, i represents the index of the video frame, i = 1, 2, ..., n-1;

[0080] Step 6: Calculate the evaluation index: Read the frame rate of the video to be tested as Rfps, and use R frames as an evaluation unit, that is, one second of video is evaluated once. Design six evaluation indicators, and the six evaluation indicators of the T-th second of the video to be tested are E 1,T 、E 2,T 、E 3,T 、E 4,T 、E 5,T , E 6,T , the calculation method is as follows:

[0081]

[0082] Where j represents the evaluation index, j = 1, 2, 3; T represents the Tth second of the video to be detected; E j,T represents the jth evaluation index at the Tth second of the video to be detected; R represents the frame rate of the video to be detected; L is the cumulative transformation sequence; i represents the row index value of the cumulative transformation sequence L,

[0083]

[0084] Where k represents the evaluation index, k = 4, 5, 6; T represents the Tth second of the video to be detected; E k,Trepresents the kth evaluation index at the Tth second of the video to be detected; R represents the frame rate of the video to be detected; L is the cumulative transformation sequence; i represents the row index value of the cumulative transformation sequence L,

[0085] Step 7: Calculate the weighted score of the video to be tested per second and determine the level of jitter;

[0086] score T =E 1,T +E 2,T +100×E 3,T +E 4,T +E 5,T +100×E 6,T

[0087] Where T represents the Tth second of the video to be detected, score T Indicates the weighted score of the Tth second of the video to be detected; E 1,T 、E 2,T 、E 3,T 、E 4,T 、E 5,T 、E 6,T Indicates the six evaluation indicators at the Tth second of the video to be detected.

[0088] Set the grade evaluation threshold. If the score T <1.8, the video is not shaking; if 1.8≤score T <9, the video jitter level is 1; if 9≤score T <20, the video jitter level is 2; if 20≤score T <45, the video jitter level is 3; if 45≤score T , the video jitter level is 4.

[0089] Step 8: Execute steps 6-7 second by second, and output the score and jitter level of each second of the video to be tested.

Claims

1. A method for evaluating the jitter level of a video image, characterized in that: The following steps are included: Step 1: First, the video to be detected is divided into frames, and the resolution of each frame is downsampled to obtain a frame sequence (P1, P2, ..., P i , P i+1 ,……,P n ), where P i represents the i-th frame of the video to be detected, and n represents the total number of frames of the video to be detected; Step 2: Input every two adjacent frames in the frame sequence obtained in step 1 into the real-time optical flow extraction network based on deep learning to extract the optical flow map; Step 3: For adjacent frames P i and P i+1 , in the previous frame, P i Uniformly select feature points on the image, add the coordinates of the feature points to the corresponding optical flow components output by the optical flow extraction network to obtain the feature points in the next frame, i.e., P i+1 Corresponding coordinates; perform affine transformation estimation on the two sets of coordinates and output the i-th frame P i To the i+1th frame P i+1 The overall affine transformation matrix T between frames i ; Step 4: Calculate the cumulative transformation sequence L of the video to be detected; Step 5: Calculate six evaluation indicators for the cumulative transformation sequence L of the video in step 4 and perform weighted operations, output the jitter score for each second of the video to be detected, and determine whether the video to be detected is jittery and the degree of jitter based on the jitter degree threshold.

2. The method for evaluating the jitter level of a video image according to claim 1, wherein: The step 2 is specifically as follows: Get inter-frame optical flow: the adjacent frames P i and P i+1 Input a dense optical flow tracking network FastFlowNet based on deep learning and output P i to P i+1 The optical flow map F i ; Where i represents the index of the video frame, i=1, 2, ..., n-1; F i Represents the i-th frame P i To the i+1th frame P i+1 The optical flow matrix of P i The horizontal coordinate index of the pixel point in P, j = 1, 2, ..., 720; k represents P i The vertical coordinate index of the pixel point, k = 1, 2, ..., 1280; Indicates P i The pixel point with coordinate index (j, k) is from P i to P i+1 The motion vector of dx represents P i The pixel point with coordinate index (j, k) is from P i to P i+1 The row offset component, dy represents P i The pixel point with coordinate index (j, k) is from P i to P i+1 The column offset component of .

3. The method for evaluating the jitter level of a video image according to claim 1, wherein: The step 3 is specifically as follows: Perform affine transformation estimation and output the overall affine transformation matrix between frames; Step 3.1: In P i Uniformly select m feature points from the matrix and record the coordinates of these feature points to obtain the feature point coordinate matrix S. i ; Where i represents the index of the video frame, i=1, 2, ..., n-1; S i Indicates that in the i-th frame P i The matrix composed of all the feature points selected above; Represents the i-th frame P i The horizontal coordinate of the first feature point selected on Represents the i-th frame P i The vertical coordinate of the first feature point selected; Represents the i-th frame P i The horizontal coordinate of the second feature point selected on Represents the i-th frame P i The vertical coordinate of the second feature point selected above; m represents the total number of feature points; Represents the i-th frame P i The horizontal coordinate of the mth feature point selected on Represents the i-th frame P i The ordinate of the mth feature point selected above; Step 3.2: The optical flow map F output in step 2 i Find the feature point coordinate matrix S in step 3.1 i The corresponding motion vector matrix M i ; Where i represents the index of the video frame, i=1, 2, ..., n-1; M i Represents the i-th frame P i The matrix composed of the motion vectors of the m feature points selected above; Represents the i-th frame P i The horizontal coordinate of the first feature point selected on Represents the i-th frame P i The vertical coordinate of the first feature point selected; Indicates P i The median coordinate index is The first feature point from P i to P i+1 The motion vector of Represents the i-th frame P i The horizontal coordinate of the second feature point selected on Represents the i-th frame P i The vertical coordinate of the second feature point selected; Indicates P i The median coordinate index is The second characteristic point of P i to P i+1 The motion vector of m represents the total number of feature points. Represents the i-th frame P i The horizontal coordinate of the mth feature point selected on Represents the i-th frame P i The ordinate of the mth feature point selected above; Indicates P i The median coordinate index is The mth feature point from P i to P i+1 motion vector; each pixel on the video frame has an optical flow component, according to the coordinates of the feature point You can index its corresponding optical flow component Step 3.3: Calculate the feature points at P i+1 The coordinates of P i The coordinate matrix of the feature points selected in P is added to the motion vector matrix of the feature points to obtain the coordinate matrix of the feature points selected in P i+1 The coordinate matrix S in i+1 , that is, S i+1 =S i +M i ; Step 3.4: S i With S i+1 Input into the opencv affine transformation estimation function and output the i-th frame P i To the i+1th frame P i+1 The affine transformation matrix T i , Where i represents the index of the video frame, i=1, 2, ..., n-1; T i Represents the i-th frame P i To the i+1th frame P i+1 The overall affine transformation matrix of i Represents the i-th frame P i To the i+1th frame P i+1 The rotation angle of the overall affine transformation; Δx i Represents the i-th frame P i To the i+1th frame P i+1 The row offset component of the overall affine transformation; Δy i Represents the i-th frame P i To the i+1th frame P i+1 The column offset component of the overall affine transformation.

4. The method for evaluating the jitter level of a video image according to claim 1, wherein: Process all frames of the video to be detected according to steps 2-3 to obtain the affine transformation parameter sequence H; where Δx 1 , Δy 1 , Δθ 1 Δx represents the row offset component, column offset component and rotation angle of the overall affine transformation from the first frame P1 to the second frame P2; 2 , Δy 2 , Δθ 2 Δx represents the row offset component, column offset component and rotation angle of the overall affine transformation from the second frame P2 to the third frame P3; n-1 , Δy n-1 , Δθ n-1 Indicates the n-1th frame P n-1 To the nth frame P n The row offset component, column offset component, and rotation angle of the overall affine transformation.

5. The method for evaluating the jitter level of a video image according to claim 1, wherein: The step 4 is specifically as follows: Accumulate the affine transformation parameter sequence H row by row to obtain the cumulative transformation sequence L of the video to be detected; where Δx 1 , Δy 1 , Δθ 1 Δx represents the row offset component, column offset component and rotation angle of the overall affine transformation from the first frame P1 to the second frame P2; 2 , Δy 2 , Δθ 2 Δx represents the row offset component, column offset component and rotation angle of the overall affine transformation from the second frame P2 to the third frame P3; i , Δy i , Δθ i Represents the i-th frame P i To the i+1th frame P i+1 The row offset component, column offset component and rotation angle of the overall affine transformation, i represents the index of the video frame, i = 1, 2, ..., n-1.

6. The method for evaluating the jitter level of a video image according to claim 1, wherein: The step 5 is specifically as follows: Calculate the evaluation index: read the frame rate of the video to be detected as Rfps, take R frames as an evaluation unit, that is, evaluate once per second of video, and design six evaluation indicators. The six evaluation indicators of the T-th second of the video to be detected are E 1,T 、E 2,T 、E 3,T 、E 4,T 、E 5,T , E 6,T , the calculation method is as follows: Where j represents the evaluation index, j = 1, 2, 3; T represents the Tth second of the video to be detected; E j,T represents the jth evaluation index at the Tth second of the video to be detected; R represents the frame rate of the video to be detected; L is the cumulative transformation sequence; i represents the row index value of the cumulative transformation sequence L, Where k represents the evaluation index, k = 4, 5, 6; T represents the Tth second of the video to be detected; E k,T represents the kth evaluation index at the Tth second of the video to be detected; R represents the frame rate of the video to be detected; L is the cumulative transformation sequence; i represents the row index value of the cumulative transformation sequence L, 7. The method for evaluating the jitter level of a video image according to claim 6, wherein: Calculate the weighted score of the video to be tested per second and determine the level of jitter; score T =And 1,T +E 2,T +100×E 3,T +E 4,T +E 5,T +100×E 6,T Where T represents the Tth second of the video to be detected, score T Indicates the weighted score of the Tth second of the video to be detected; E 1,T 、E 2,T 、E 3,T 、E 4,T 、E 5,T 、E 6,T Represents the six evaluation indicators at the Tth second of the video to be detected; Set the grade evaluation threshold. If the score T <1.8, the video is not shaking; if 1.8≤score T <9, the video jitter level is 1; if 9≤score T <20, the video jitter level is 2; if 20≤score T <45, the video jitter level is 3; if 45≤score T , the video jitter level is 4.

Images