An environment-adaptive law enforcement video correction method and system

Abstract

The present invention relates to the technical field of video correction, and discloses an environment-adaptive law enforcement video correction method and system. The method comprises: obtaining law enforcement video image frames, calculating the motion information of each pixel in the law enforcement video image frames based on optical flow analysis; extracting feature points of adjacent law enforcement video image frames based on the principle of consistency of spatial motion information, and connecting the feature points to form motion feature trajectories; using a differential weight analysis method to set different weights for different motion feature trajectories; performing a geometric transformation on the law enforcement video to be corrected based on the assigned trajectory weights, correcting the motion trajectories of the feature points in the video and stabilizing the video to obtain a corrected law enforcement video. The method of the present invention selects feature points with the smallest static gradient change amplitude and dynamic motion direction as feature points of adjacent image frames to obtain the feature trajectory of the law enforcement video, and corrects the feature trajectory of camera shake, thereby achieving correction of the law enforcement video.

Description

Technical Field

[0001] The present invention relates to the technical field of video correction, and in particular to an environment-adaptive law enforcement video correction method and system. Background Art

[0002] Handheld law enforcement video recorders are essential equipment for law enforcement, offering portability and irreplaceability. However, during use, these recorders can experience significant vibration, resulting in rotational vibration in recorded law enforcement videos, severely impacting their viewability. To address this issue, this patent proposes an environmentally adaptive law enforcement video correction method and system. Summary of the Invention

[0003] In view of this, the present invention provides an environment-adaptive law enforcement video correction method, the purpose of which is that (1) the traditional LK optical flow method needs to meet the assumption that the target object changes very little in consecutive frames and the pixel brightness remains unchanged, but there may be more severe jitter when shooting law enforcement videos. Therefore, this scheme establishes a pyramid-based LK optical flow method, and through several downsampling processes, the violent motion that does not meet the assumption of the LK optical flow method is converted into tiny motion, thereby obtaining accurate pixel motion information; (2) based on the static features of the image and the pixel motion features, the feature points with the smallest static gradient change amplitude and dynamic motion direction are selected as the feature points of adjacent image frames to obtain the feature trajectory of the law enforcement video, and the feature trajectory of the shooting camera jitter is separated from the feature trajectory. By correcting the feature trajectory, the correction of the law enforcement video is achieved.

[0004] To achieve the above objectives, the present invention provides an environment-adaptive law enforcement video correction method, comprising the following steps:

[0005] S1: Obtain law enforcement video image frames, and calculate motion information of each pixel in the law enforcement video image frames based on optical flow analysis, wherein the LK algorithm based on the improved pyramid is the main method of the optical flow analysis;

[0006] S2: Extract feature points of adjacent frames of law enforcement video based on the consistency principle of spatial motion information, and connect the feature points to form motion feature trajectories;

[0007] S3: Use the differential weight analysis method to set different weights for different motion feature trajectories, highlight the feature trajectories with larger jitter amplitudes, and achieve trajectory separation;

[0008] S4: Performing geometric transformation on the law enforcement video to be corrected according to the assigned trajectory weights, correcting the motion trajectory of the feature points in the video and stabilizing the video to obtain a corrected law enforcement video.

[0009] As a further improvement method of the present invention:

[0010] Optionally, obtaining the law enforcement video image frame in step S1 includes:

[0011] Obtain the law enforcement video to be corrected, and divide the law enforcement video into several law enforcement video image frames. After division, the law enforcement video image frame set is {f i |i∈[1,N]}, where f i represents the i-th video image frame of the law enforcement video, and N represents the total number of video image frames divided into the law enforcement video.

[0012] Optionally, calculating the motion information of each pixel in the law enforcement video image frame based on optical flow analysis in step S1 includes:

[0013] For any law enforcement video image frame f i , using the image grayscale processing method to transform the law enforcement video image frame f i Converted into a grayscale image, the law enforcement video image frame f i Any pixel point f i The grayscale processing result of (x,y) is:

[0014] g i (x,y)=max{R i (x,y),G i (x,y),B i (x,y)}

[0015] in:

[0016] g i (x,y) represents the pixel f i The grayscale value of (x, y) is used as the pixel value. (x, y) represents the pixel f i The position (x, y), that is, the law enforcement video image frame f i The pixel at row x and column y;

[0017] R i (x,y),G i (x,y),B i (x,y) represent pixel points f i (x,y) is the color value in the R, G, and B color channels;

[0018] In an embodiment of the present invention, the law enforcement video image frame is a still image in the law enforcement video;

[0019] The motion information of each pixel in the law enforcement video image frame is calculated based on optical flow analysis. i , any pixel f i The (x,y) motion information extraction process is:

[0020] S11: Construct pixel point f i The neighborhood pixel area of ​​(x,y), where the pixel point f i (x, y) is the center of the neighborhood pixel area, and the neighborhood pixel area is a square area of ​​3×3 pixels. The pixel point f i The neighborhood pixel area of ​​(x,y) is:

[0021] {f i (x j ,y j )|j∈[0,8]}

[0022] in:

[0023] f i (x j ,y j ) represents the pixel f i The jth neighboring pixel of (x,y), f i (x0,y0) represents the pixel f i (x,y), if the law enforcement video image frame f i There is no pixel f in i (x j ,y j ), then mark the pixel as blank and the pixel value is 0;

[0024] S12: Constructing a pixel motion information model based on the LK algorithm:

[0025] (A i ) T (B i ) 2 A i V i =(A i ) T (B i ) 2 C i

[0026]

[0027]

[0028] V i =[v i,X (x,y),v i,Y (x,y)] T

[0029] B i =diag(d(g i (x0,y0)),d(g i (x1,y1)),...,d(gi (x8,y8)))

[0030]

[0031] in:

[0032] Represents pixel f i (x j ,y j )’s pixel value in the horizontal direction of the video image frame, Represents pixel f i (x j ,y j )’s pixel value in the vertical direction of the video image frame, Represents pixel f i (x j ,y j ) in the direction of the video image frame timing sequence, wherein the video image frame timing sequence feature is the timing information of an adjacent and ordered video image frame sequence;

[0033] T stands for transpose;

[0034] diag(·) represents a diagonal matrix;

[0035] d(g i (x j ,y j )) represents pixel f i (x j ,y j ), the closer the pixel is to the center of the neighborhood pixel area, the greater the pixel weight;

[0036] v i,X (x,y) represents the pixel f i (x,y) velocity in the horizontal direction, v i,Y (x,y) represents the pixel f i (x,y) velocity in the vertical direction;

[0037] In the embodiment of the present invention, the LK (Lucas-Kanade) algorithm assumes that the brightness of the object target between adjacent image frames remains constant and that the motion of the object target is slight.

[0038] S13: Solve the pixel point motion information model to obtain the pixel point f i Initial motion information V of (x,y) i :

[0039] V i =[(A i ) T(B i ) 2 A i ] -1 (A i ) T (B i ) 2 C i

[0040] S14: The video image frame f i As the bottom image of the pyramid, image downsampling is performed based on the bottom image to construct a pyramid, wherein the number of layers of the pyramid is 4, and the image downsampling process is:

[0041] L a =down(L a-1 )*Gauss 5×5

[0042] in:

[0043] Gauss 5×5 A Gaussian kernel of size 5 × 5 pixels;

[0044] * indicates the convolution operation;

[0045] a∈[0,3] represents the number of layers of the pyramid, and L0 represents the bottom image;

[0046] aown(L a-1 ) represents the image L a-1 Perform downsampling processing;

[0047] S15: Calculate the pixel points f of different layers of the pyramid i (x, y) motion information, the pixel point f i The calculation process of (x, y) motion information is from S11 to S13;

[0048] S16: Get pixel point f based on pyramid and LK algorithm i Global motion information of (x,y):

[0049]

[0050] in:

[0051] V i (a) represents the pixel point f obtained based on the a-th layer pyramid image i (x, y) motion information, V i (0) represents pixel f i Initial motion information V of (x, y) i ;

[0052] Vi (x, y) represents the pixel f i Global motion information of (x, y), including pixel f i (x, y) velocity V in the horizontal direction i，X (x, y) and the velocity V in the vertical direction i，Y (x, y).

[0053] Optionally, the step S2 of extracting feature points of adjacent image frames of the law enforcement video based on the principle of consistency of spatial motion information includes:

[0054] Based on the principle of consistency of spatial motion information, feature points of adjacent image frames in law enforcement videos are extracted. For any image frame f in law enforcement videos, i , the feature point extraction process in the law enforcement video image frame is:

[0055] S21: Construct image frame f i Scale space:

[0056]

[0057] in:

[0058] g i (x, y) represents the image frame f i Any pixel point f i The pixel value at (x, y);

[0059] σ is the scale factor. The number of scale factors set is 2. Each scale factor is different from each other. A two-layer scale space is constructed. The scale factors of the two-layer scale space constructed are σ1=2 and σ2=3 respectively.

[0060] S22: Constructing Gaussian difference space G based on scale space i (x, y, σ):

[0061] G i (x, y, σ) = L i (x, y, σ2)-L i (x, y, σ1)

[0062] Each sampling point in the Gaussian difference space is detected. If the sampling point is the maximum or minimum value of the eight adjacent sampling points at this scale, the sampling point is marked as a feature point s. i (x′, y′), the feature point s i The pixel point corresponding to (x′, y′) is f i (x′, y′);

[0063] S23: Calculate the feature point s i (x′, y′) corresponds to the pixel point fi The gradient direction θ of (x′, y′) i (x′, y′) and the gradient value u(x′, y′):

[0064]

[0065]

[0066] in:

[0067] g i (x′, y′) is the pixel f i The pixel value at (x′, y′);

[0068] For any adjacent law enforcement video image frame f i , f i+1 , calculate the law enforcement video image frame f i Feature point s i (x′, y′) and f i+1 Feature point s i+ 1 ( x″, y″) motion information direction and feature gradient direction changes

[0069] d[s i (x′, y′), s i+1 (x″, y″)]:

[0070] d[s i (x′, y′), s i+1 (x″, y″)] = |θ i (x′, y′)-θ i+1 (x″,y″)|+|α i (x′, y′)-α i+1 (x″, y″)|

[0071]

[0072]

[0073] in:

[0074] α i (x′, y′) represents the feature point s i (x′, y′) corresponds to the pixel point f i (x′, y′) motion information direction, V i，X (x′, y′) represents the pixel f i The velocity of (x′, y′) in the horizontal direction, V i，Y (x′, y′) represents the pixel f i (x′, y′) velocity in the vertical direction;

[0075] α i+1 (x″, y″) represents the feature point s i+1 (x″, y″) corresponds to the pixel point f i+1 (x″, y″) motion information direction, V i+1，X (x″, y″) represents the pixel f i+1 (x″, y″) velocity in the horizontal direction, V i+1，Y (x″, y″) represents the pixel f i+1 (x″, y″) velocity in the vertical direction;

[0076] Select such that d[s i (x′,y′),s i+1 (x″,y″)] to achieve the minimum feature point s i+1 (x″,y″), as the law enforcement video image frame f i Feature point s i (x′, y′) adjacent feature points, and preset feature thresholds, if the feature gradient direction changes d[s i (x′,y′),s i+1 (x″, y″)] is higher than the feature threshold, which means that the feature point s i There are no adjacent feature points at (x′, y′);

[0077] Repeat the above steps to obtain the adjacent feature points of any feature point in the law enforcement video image frame in the adjacent image frame.

[0078] Optionally, in step S2, the extracted feature points are connected to form a motion feature trajectory, including:

[0079] Adjacent feature points of adjacent image frames are connected to form a motion feature trajectory. The process of forming the motion feature trajectory is as follows:

[0080] For any law enforcement video image frame f i Isolated feature points s i (x * ,y * ), determine the feature point s i (x * ,y * ) in the enforcement video image frame f i+1 Is there an adjacent feature point s? i (x ** ,y ** ), if it exists, connect the feature point s i (x * ,y * ) and s i (x ** ,y** ), forming an initial trajectory, and judging whether the terminal feature point of the initial trajectory has an adjacent feature point in the next law enforcement video image frame, if so, connecting the terminal feature point with the adjacent feature point of the next law enforcement video image frame,

[0081] Repeat this step, otherwise end with feature point s i (x * ,y * ) is the process of forming the initial motion feature trajectory, and the length of the trajectory is at least three feature points;

[0082] The isolated feature point means that the feature point is not an adjacent feature point of any feature point of the previous law enforcement video image frame;

[0083] The constructed motion feature trajectory set is {τ q |q∈[1,Q]}, where τ q represents the qth motion feature trajectory, and Q represents the total number of motion feature trajectories formed.

[0084] Optionally, in step S3, different weights are set for different motion feature trajectories using a differential weight analysis method, including:

[0085] Identify camera shake trajectories in the motion feature trajectory set. If the frame length of a motion feature trajectory is less than 0.1×N, it is considered a camera shake trajectory, where N represents the total frame length of the law enforcement video and the frame length of the motion feature trajectory is the total number of feature points in the trajectory.

[0086] Different weights are set for different motion feature trajectories using the differential weight analysis method. q The weight of the i-th law enforcement video image frame for:

[0087]

[0088] in:

[0089] γ q Represents the motion feature trajectory τ q The inclination angle;

[0090] r q Represents the motion feature trajectory τ q The length of the trajectory;

[0091] num i represents the number of trajectories passing through the i-th law enforcement video image frame;

[0092] β q,i ={0,1}, if the motion feature trajectory τ qAfter the i-th law enforcement video image frame, β q,i =1, otherwise β q,i =0;

[0093] Represents the motion feature trajectory τ q Passing through the feature points of the i-th law enforcement video image frame, G(·;σ′) represents a Gaussian function with a standard deviation of σ′.

[0094] Optionally, in step S4, geometric transformation is performed on the law enforcement video to be corrected according to the assigned trajectory weights, the motion trajectory of the feature points in the video is corrected, and the video is stabilized, including:

[0095] The law enforcement video to be corrected is geometrically transformed according to the assigned trajectory weights, and the geometric transformation matrix F is:

[0096] F={F i |i∈[1,N]}

[0097] in:

[0098] F i The geometric transformation formula of the i-th law enforcement video image frame is based on F i Performing correction processing on the i-th law enforcement video image frame;

[0099] Construct the correction objective function H(F) of law enforcement video:

[0100]

[0101]

[0102] in:

[0103] H(F) is the correction objective function. By solving the geometric transformation matrix F that minimizes H(F), the law enforcement video to be corrected is geometrically transformed based on F to correct the motion trajectory of the feature points in the video and stabilize the video to obtain the corrected law enforcement video.

[0104] The correction objective function H(F) is optimized and solved. The optimization solution process of the correction objective function H(F) is as follows:

[0105] S41: Initialize the geometric transformation matrix F 0 , the mth iteration result of the geometric transformation matrix is ​​F m , the initial value of m is 1, the error threshold is ∈, where ∈ is 0.3;

[0106] S42: Calculate the partial derivative matrix of the correction target function H(F) And update the geometric transformation matrix:

[0107]

[0108] in:

[0109] μ represents the damping factor, which is set to 0.15;

[0110] S43: If Then stop the algorithm iteration, F m is the optimal geometric transformation matrix F * Otherwise, set m=m+1 and return to step S42;

[0111] Using the optimal geometric transformation matrix F * Correcting law enforcement videos, The geometric transformation formula of the i-th law enforcement video image frame is represented by:

[0112] In order to solve the above problems, the present invention provides an environment-adaptive law enforcement video correction system, characterized in that the system includes:

[0113] A video frame motion information extraction module is used to obtain law enforcement video image frames and calculate the motion information of each pixel in the law enforcement video image frames based on optical flow analysis;

[0114] A feature trajectory extraction device is used to extract feature points of adjacent image frames of law enforcement videos based on the principle of consistency of spatial motion information, connect the feature points to form motion feature trajectories, and use a differential weight analysis method to set different weights for different motion feature trajectories, highlighting feature trajectories with larger jitter amplitudes to achieve trajectory separation;

[0115] The correction parameter solving device is used to perform geometric transformation on the law enforcement video to be corrected according to the assigned trajectory weight, correct the motion trajectory of the feature points in the video and stabilize the video.

[0116] In order to solve the above problem, the present invention further provides an electronic device, comprising:

[0117] a memory storing at least one instruction; and

[0118] The processor executes the instructions stored in the memory to implement the above-mentioned environment-adaptive law enforcement video correction method.

[0119] In order to solve the above problems, the present invention also provides a computer-readable storage medium, which stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the above-mentioned environment-adaptive law enforcement video correction method.

[0120] Compared with the existing technology, the present invention proposes an environment-adaptive law enforcement video correction method, which has the following advantages:

[0121] First, this scheme proposes a pixel motion feature extraction method by constructing pixel point f i The neighborhood pixel area of ​​(x,y), where the pixel point f i (x, y) is the center of the neighborhood pixel area, and the neighborhood pixel area is a square area of ​​3×3 pixels. The pixel point f i The neighborhood pixel area of ​​(x,y) is:

[0122] {f i (x j ,y j )|j∈[0,8]}

[0123] Where: f i (x j ,y j ) represents the pixel f i The jth neighboring pixel of (x,y), f i (x0,y0) represents the pixel f i (x,y), if the law enforcement video image frame f i There is no pixel f in i (x j ,y j ), then mark the pixel as blank and the pixel value is 0; construct a pixel point motion information model based on the LK algorithm:

[0124] (A i ) T (B i ) 2 A i V i =(A i ) T (B i ) 2 C i

[0125]

[0126]

[0127] V i =[v i,X (x,y),v i,Y (x,y)] T

[0128] B i =diag(d(g i (x0,y0)),d(g i(x1,y1)),...,d(g i (x8,y8)))

[0129]

[0130] in: Represents pixel f i (x j ,y j )’s pixel value in the horizontal direction of the video image frame, Represents pixel f i (x j ,y j )’s pixel value in the vertical direction of the video image frame, Represents pixel f i (x j ,y j ) in the direction of the video image frame timing sequence, wherein the video image frame timing sequence feature is the timing information of the adjacent and ordered video image frame sequence; T represents transpose; diag(·) represents a diagonal matrix; d(g i (x j ,y j )) represents pixel f i (x j ,y j ), the closer the pixel is to the center of the neighborhood pixel area, the greater the weight of the pixel; v i,X (x,y) represents the pixel f i (x,y) velocity in the horizontal direction, v i,Y (x,y) represents the pixel f i (x, y) speed in the vertical direction; solve the pixel motion information model to obtain the pixel f i Initial motion information V of (x,y) i :

[0131] V i =[(A i ) T (B i ) 2 A i ] -1 (A i ) T (B i ) 2 C i

[0132] The video image frame f i As the bottom image of the pyramid, image downsampling is performed based on the bottom image to construct a pyramid, wherein the number of layers of the pyramid is 4, and the image downsampling process is:

[0133] L a =down(L a-1 )*Gauss 5×5

[0134] Among them: Gauss 5×5 is a Gaussian kernel of 5×5 pixels; * indicates the convolution operation; a∈[0,3] indicates the number of pyramid layers, L0 indicates the bottom image; down(L a-1 ) represents the image L a-1 Perform downsampling processing; calculate the pixel points f of different layers of images in the pyramid i (x, y) motion information; get pixel point f based on pyramid and LK algorithm i Global motion information of (x,y):

[0135]

[0136] Where: V i (a) represents the pixel point f obtained based on the a-th layer pyramid image i (x,y) motion information, V i (0) represents pixel f i Initial motion information V of (x,y) i ; V i (x,y) represents the pixel f i Global motion information of (x,y), including pixel f i The horizontal velocity V of (x,y) i,X (x,y) and the velocity V in the vertical direction i,Y (x, y). The traditional LK optical flow method assumes that the target object changes little between consecutive frames and that pixel brightness remains constant. However, when shooting law enforcement videos, there may be significant jitter. Therefore, this solution establishes a pyramid-based LK optical flow method. Through several downsampling processes, it converts violent movements that do not meet the LK optical flow assumptions into subtle movements, thereby obtaining accurate pixel motion information.

[0137] At the same time, this scheme proposes a method for extracting features from adjacent frames. This scheme extracts feature points from adjacent image frames in law enforcement videos based on the principle of consistency of spatial motion information. For any image frame f in a law enforcement video, i The feature point extraction process in the law enforcement video image frame is as follows: construct the image frame f i The scale space of:

[0138]

[0139] Where: g i (x,y) represents the image frame fi Any pixel point f i The pixel value of (x, y); σ is the scale factor, the number of scale factors set is 2, each scale factor is different, and a two-layer scale space is constructed. The scale factors of the two-layer scale space constructed are σ1=2, σ2=3 respectively; Gaussian difference space G is constructed based on the scale space u (x,y,σ):

[0140] G u (x,y,σ)=L u (x,y,σ2)-L i (x,y,σ1)

[0141] Each sampling point in the Gaussian difference space is detected. If the sampling point is the maximum or minimum value of the eight adjacent sampling points at this scale, the sampling point is marked as a feature point s. u (x′, y′), the feature point s u The pixel point corresponding to (x′,y′) is f i (x′,y′); calculate the feature point s i The pixel point f corresponding to (x′,y′) i The gradient direction θ of (x′,y′) i (x′, y′) and gradient value u(x′, y′):

[0142]

[0143]

[0144] Where: g i (x′, y′) is the pixel f i (x′, y′) pixel value; for any adjacent law enforcement video image frame f i , f i+1 , calculate the law enforcement video image frame f i Feature point s i (x′, y′) and f i+1 Feature point s i+1 (x″, y″) motion information direction and feature gradient direction change d[s i (x′, y′), s i+1 (x″, y″)]:

[0145] d[s i (x′, y′), s i+1 (x″, y″)] = |θ i (x′, y′)-θ i+1 (x″,y″)|+|α i (x′, y′)-α i+1(x″, y″)|

[0146]

[0147]

[0148] Where: α i (x′, y′) represents the feature point s i (x′, y′) corresponds to the pixel point f i (x′, y′) motion information direction, V i，X (x′, y′) represents the pixel f i The velocity of (x′, y′) in the horizontal direction, V i，Y (x′, y′) represents the pixel f i (x′, y′) velocity in the vertical direction; α i+1 (x″, y″) represents the feature point s i+1 (x″, y″) corresponds to the pixel point f i+1 (x″, y″) motion information direction, V i+1，X (x″, y″) represents the pixel f i+1 (x″, y″) velocity in the horizontal direction, V i+1，Y (x″, y″) represents the pixel f i+1 (x″, y″) velocity in the vertical direction; select the velocity such that d[s i (x′, y′), s i+1 (x″, y″)] to achieve the minimum feature point s i+1 (x″, y″), as the law enforcement video image frame f i Feature point s i (x′, y′) adjacent feature points, and preset feature thresholds, if the feature gradient direction changes d[s i (x′, y′), s i+1 (x″, y″)] is higher than the feature threshold, which means that the feature point s i (x′, y′) does not have adjacent feature points; repeat the above steps to obtain the adjacent feature points of any feature point in the law enforcement video image frame in the adjacent image frame. Based on the static features of the image and the motion characteristics of the pixel points, this solution selects the feature points with the smallest static gradient change amplitude and dynamic motion direction as the feature points of the adjacent image frames, obtains the characteristic trajectory of the law enforcement video, and separates the characteristic trajectory of the camera shake from the characteristic trajectory. By correcting the characteristic trajectory, the law enforcement video can be corrected. BRIEF DESCRIPTION OF THE DRAWINGS

[0149] Figure 1 A flowchart of an environment-adaptive law enforcement video correction method provided by one embodiment of the present invention;

[0150] Figure 2 A functional module diagram of an environment-adaptive law enforcement video correction system provided by one embodiment of the present invention;

[0151] Figure 3 A schematic structural diagram of an electronic device for implementing an environment-adaptive law enforcement video correction method provided by one embodiment of the present invention.

[0152] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0153] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0154] The embodiments of the present application provide an environment-adaptive law enforcement video correction method. The method can be executed by at least one of the following electronic devices, including a server and a terminal, that can be configured to execute the method provided by the embodiments of the present application. In other words, the method can be executed by software or hardware installed on a terminal or server device, where the software can be a blockchain platform. The server can include, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0155] Example 1:

[0156] S1: Obtain law enforcement video image frames, and calculate the motion information of each pixel in the law enforcement video image frames based on optical flow analysis, wherein the LK algorithm based on the improved pyramid is the main method of the optical flow analysis.

[0157] The step S1 of obtaining the law enforcement video image frame includes:

[0158] Obtain the law enforcement video to be corrected, and divide the law enforcement video into several law enforcement video image frames. After division, the law enforcement video image frame set is {f i |i∈[1,N]}, where f i represents the i-th video image frame of the law enforcement video, and N represents the total number of video image frames divided into the law enforcement video.

[0159] The step S1 calculates the motion information of each pixel in the law enforcement video image frame based on optical flow analysis, including:

[0160] For any law enforcement video image frame f i , using the image grayscale processing method to transform the law enforcement video image frame f i Converted into a grayscale image, the law enforcement video image frame f i Any pixel point f iThe grayscale processing result of (x,y) is:

[0161] g i (x,y)=max{R i (x,y),G i (x,y),B i (x,y)}

[0162] in:

[0163] g i (x,y) represents the pixel f i The grayscale value of (x, y) is used as the pixel value. (x, y) represents the pixel f i The position (x, y), that is, the law enforcement video image frame f i The pixel at row x and column y;

[0164] R i (x,y),G i (x,y),B i (x,y) represent pixel points f i (x,y) is the color value in the R, G, and B color channels;

[0165] The motion information of each pixel in the law enforcement video image frame is calculated based on optical flow analysis. i , any pixel f i The (x,y) motion information extraction process is:

[0166] S11: Construct pixel point f i The neighborhood pixel area of ​​(x,y), where the pixel point f i (x, y) is the center of the neighborhood pixel area, and the neighborhood pixel area is a square area of ​​3×3 pixels. The pixel point f i The neighborhood pixel area of ​​(x,y) is:

[0167] {f i (x j ,y j )|j∈[0,8]}

[0168] in:

[0169] f i (x j ,y j ) represents the pixel f i The jth neighboring pixel of (x,y), f i (x0,y0) represents the pixel f i (x,y), if the law enforcement video image frame f i There is no pixel f in i (xj ,y j ), then mark the pixel as blank and the pixel value is 0;

[0170] S12: Constructing a pixel motion information model based on the LK algorithm:

[0171] (A i ) T (B i ) 2 A i V i =(A i ) T (B i ) 2 C i

[0172]

[0173]

[0174] V i =[v i,X (x,y),v i,Y (x,y)] T

[0175] B i =diag(d(g i (x0,y0)),d(g i (x1,y1)),...,d(g i (x8,y8)))

[0176]

[0177] in:

[0178] Represents pixel f i (x j ,y j )’s pixel value in the horizontal direction of the video image frame, Represents pixel f i (x j ,y j )’s pixel value in the vertical direction of the video image frame, Represents pixel f i (x j ,y j ) in the direction of the video image frame timing sequence, wherein the video image frame timing sequence feature is the timing information of an adjacent and ordered video image frame sequence;

[0179] T stands for transpose;

[0180] diag(·) represents a diagonal matrix;

[0181] d(g i (x j ,y j )) represents pixel f i (x j ,y j ), the closer the pixel is to the center of the neighborhood pixel area, the greater the pixel weight;

[0182] v i,X (x,y) represents the pixel f i (x,y) velocity in the horizontal direction, v i,Y (x,y) represents the pixel f i (x,y) velocity in the vertical direction;

[0183] S13: Solve the pixel point motion information model to obtain the pixel point f i Initial motion information V of (x,y) i :

[0184] V i =[(A i ) T (B i ) 2 A i ] -1 (A i ) T (B i ) 2 C i

[0185] S14: The video image frame f i As the bottom image of the pyramid, image downsampling is performed based on the bottom image to construct a pyramid, wherein the number of layers of the pyramid is 4, and the image downsampling process is:

[0186] L a =down(L a-1 )*Gauss 5×5

[0187] in:

[0188] Gauss 5×5 A Gaussian kernel of size 5 × 5 pixels;

[0189] * indicates the convolution operation;

[0190] a∈[0,3] represents the number of layers of the pyramid, and L0 represents the bottom image;

[0191] down(La-1 ) represents the image L a-1 Perform downsampling processing;

[0192] S15: Calculate the pixel points f of different layers of the pyramid i (x, y) motion information, the pixel point f i The calculation process of (x, y) motion information is from S11 to S13;

[0193] S16: Get pixel point f based on pyramid and LK algorithm i Global motion information of (x,y):

[0194]

[0195] in:

[0196] V i (a) represents the pixel point f obtained based on the a-th layer pyramid image i (x,y) motion information, V i (0) represents pixel f i Initial motion information V of (x,y) i ;

[0197] V i (x,y) represents the pixel f i Global motion information of (x,y), including pixel f i The horizontal velocity V of (x,y) i,X (x,y) and the velocity V in the vertical direction i,Y (x,y).

[0198] S2: Based on the principle of consistency of spatial motion information, feature points of adjacent image frames of law enforcement videos are extracted and connected to form motion feature trajectories.

[0199] In step S2, feature points of adjacent image frames of the law enforcement video are extracted based on the principle of consistency of spatial motion information, including:

[0200] Based on the principle of consistency of spatial motion information, feature points of adjacent image frames in law enforcement videos are extracted. For any image frame f in law enforcement videos, i , the feature point extraction process in the law enforcement video image frame is:

[0201] S21: Construct image frame f i Scale space:

[0202]

[0203] in:

[0204] gi (x,y) represents the image frame f i Any pixel point f i The pixel value at (x,y);

[0205] σ is the scale factor. The number of scale factors set is 2. Each scale factor is different from each other. A two-layer scale space is constructed. The scale factors of the two-layer scale space constructed are σ1 = 2 and σ2 = 3 respectively.

[0206] S22: Constructing Gaussian difference space G based on scale space i (x,y,σ):

[0207] G i (x,y,σ)=L i (x,y,σ2)-L i (x,y,σ1)

[0208] Each sampling point in the Gaussian difference space is detected. If the sampling point is the maximum or minimum value of the eight adjacent sampling points at this scale, the sampling point is marked as a feature point s. i (x′, y′), the feature point s i The pixel point corresponding to (x′,y′) is f i (x′,y′);

[0209] S23: Calculate the feature point s i The pixel point f corresponding to (x′,y′) i The gradient direction θ of (x′,y′) i (x′, y′) and gradient value u(x′, y′):

[0210]

[0211]

[0212] in:

[0213] g i (x′,y′) is the pixel f i The pixel value at (x′,y′);

[0214] For any adjacent law enforcement video image frame f i , f i+1 , calculate the law enforcement video image frame f i Feature point s i (x′,y′) and f i+1 Feature point s i+1 (x″, y″) motion information direction and feature gradient direction changes

[0215] d[si (x′,y′),s i+1 (x″,y″)]:

[0216] d[s i (x′,y′),s i+1 (x″,y″)]=|θ i (x′,y′)-θ i+1 (x″,y″)|+|α i (x′,y′)-α i+1 (x″,y″)|

[0217]

[0218]

[0219] in:

[0220] α i (x′,y′) represents the feature point s i The pixel point f corresponding to (x′,y′) i (x′, y′) motion information direction, V i,X (x′,y′) represents the pixel f i The velocity of (x′,y′) in the horizontal direction, V i,Y (x′,y′) represents the pixel f i (x′,y′) velocity in the vertical direction;

[0221] α i+1 (x″,y″) represents the feature point s i+1 The pixel point f corresponding to (x″,y″) i+1 (x″,y″) motion information direction, V i+1,X (x″,y″) represents the pixel f i+1 The velocity of (x″,y″) in the horizontal direction, V i+1,Y (x″″,y″) represents the pixel f i+1 (x″,y″) velocity in the vertical direction;

[0222] Select such that d[s i (x′,y′),s i+1 (x″,y″)] to achieve the minimum feature point s i+1 (x″,y″), as the law enforcement video image frame f i Feature point s i (x′, y′) adjacent feature points, and preset feature thresholds, if the feature gradient direction changes d[s i (x′,y′),s i+1 (x″, y″)] is higher than the feature threshold, which means that the feature point si There are no adjacent feature points at (x′, y′);

[0223] Repeat the above steps to obtain the adjacent feature points of any feature point in the law enforcement video image frame in the adjacent image frame.

[0224] In step S2, the extracted feature points are connected to form a motion feature trajectory, including:

[0225] Adjacent feature points of adjacent image frames are connected to form a motion feature trajectory. The process of forming the motion feature trajectory is as follows:

[0226] For any law enforcement video image frame f i Isolated feature points s i (x * ,y * ), determine the feature point s i (x * ,y * ) in the enforcement video image frame f i+1 Is there an adjacent feature point s? i (x ** ,y ** ), if it exists, connect the feature point s i (x * ,y * ) and s i (x ** ,y ** ), forming an initial trajectory, and judging whether the terminal feature point of the initial trajectory has an adjacent feature point in the next law enforcement video image frame, if so, connecting the terminal feature point with the adjacent feature point of the next law enforcement video image frame,

[0227] Repeat this step, otherwise end with feature point s i (x * ,y * ) is the process of forming the initial motion feature trajectory, and the length of the trajectory is at least three feature points;

[0228] The isolated feature point means that the feature point is not an adjacent feature point of any feature point of the previous law enforcement video image frame;

[0229] The constructed motion feature trajectory set is {τ q |q∈[1,Q]}, where τ q represents the qth motion feature trajectory, and Q represents the total number of motion feature trajectories formed.

[0230] S3: Use the differential weight analysis method to set different weights for different motion feature trajectories, highlight the feature trajectories with larger jitter amplitudes, and achieve trajectory separation.

[0231] In step S3, different weights are set for different motion feature trajectories using a differential weight analysis method, including:

[0232] Identify camera shake trajectories in the motion feature trajectory set. If the frame length of a motion feature trajectory is less than 0.1×N, it is considered a camera shake trajectory, where N represents the total frame length of the law enforcement video and the frame length of the motion feature trajectory is the total number of feature points in the trajectory.

[0233] Different weights are set for different motion feature trajectories using the differential weight analysis method. q The weight of the i-th law enforcement video image frame for:

[0234]

[0235] in:

[0236] γ q Represents the motion feature trajectory τ q The inclination angle;

[0237] r q Represents the motion feature trajectory τ q The length of the trajectory;

[0238] num i represents the number of trajectories passing through the i-th law enforcement video image frame;

[0239] β q,i ={0,1}, if the motion feature trajectory τ q After the i-th law enforcement video image frame, β q,i =1, otherwise

[0240] β q,i =0;

[0241] Represents the motion feature trajectory τ q Passing through the feature points of the i-th law enforcement video image frame, G(·;σ′) represents a Gaussian function with a standard deviation of σ′.

[0242] S4: Performing geometric transformation on the law enforcement video to be corrected according to the assigned trajectory weights, correcting the motion trajectory of the feature points in the video and stabilizing the video to obtain a corrected law enforcement video.

[0243] In step S4, geometric transformation is performed on the law enforcement video to be corrected according to the assigned trajectory weights, the motion trajectory of the feature points in the video is corrected, and the video is stabilized, including:

[0244] The law enforcement video to be corrected is geometrically transformed according to the assigned trajectory weights, and the geometric transformation matrix F is:

[0245] F={F i |i∈[1,N]}

[0246] in:

[0247] F i The geometric transformation formula of the i-th law enforcement video image frame is based on F i Performing correction processing on the i-th law enforcement video image frame;

[0248] Construct the correction objective function H(F) of law enforcement video:

[0249] argminH(F)

[0250]

[0251] in:

[0252] H(F) is the correction objective function. By solving the geometric transformation matrix F that minimizes H(F), the law enforcement video to be corrected is geometrically transformed based on F to correct the motion trajectory of the feature points in the video and stabilize the video to obtain the corrected law enforcement video.

[0253] The correction objective function H(F) is optimized and solved. The optimization solution process of the correction objective function H(F) is as follows:

[0254] S41: Initialize the geometric transformation matrix F 0 , the mth iteration result of the geometric transformation matrix is ​​F m , the initial value of m is 1, the error threshold is ∈, where ∈ is 0.3;

[0255] S42: Calculate the partial derivative matrix of the correction target function H(F) And update the geometric transformation matrix:

[0256]

[0257] in:

[0258] μ represents the damping factor, which is set to 0.15;

[0259] S43: If Then stop the algorithm iteration, F m is the optimal geometric transformation matrix F * Otherwise, set m=m+1 and return to step S42;

[0260] Using the optimal geometric transformation matrix F * Correcting law enforcement videos, The geometric transformation formula of the i-th law enforcement video image frame is represented by:

[0261] Example 2:

[0262] like Figure 2 , which is a functional module diagram of an environment-adaptive law enforcement video correction system provided by an embodiment of the present invention, which can implement the environment-adaptive law enforcement video correction method in Example 1.

[0263] The environment-adaptive law enforcement video correction system 100 described in the present invention can be installed in an electronic device. Depending on the functionality implemented, the system can include a video frame motion information extraction module 101, a feature trajectory extraction device 102, and a correction parameter calculation device 103. A module, also referred to as a unit, refers to a series of computer program segments that can be executed by an electronic device processor and perform a fixed function, and is stored in the electronic device's memory.

[0264] The video frame motion information extraction module 101 is used to obtain law enforcement video image frames and calculate the motion information of each pixel in the law enforcement video image frames based on optical flow analysis;

[0265] Feature trajectory extraction device 102 is used to extract feature points of adjacent image frames of the law enforcement video based on the principle of consistency of spatial motion information, connect the feature points to form motion feature trajectories, and use the differential weight analysis method to set different weights for different motion feature trajectories, highlighting feature trajectories with larger jitter amplitudes to achieve trajectory separation;

[0266] The correction parameter solving device 103 is used to perform geometric transformation on the law enforcement video to be corrected according to the assigned trajectory weights, correct the motion trajectory of the feature points in the video and stabilize the video.

[0267] In detail, each module in the environment-adaptive law enforcement video correction system 100 of the embodiment of the present invention adopts the same method as above when in use. Figure 1 The technical means are the same as the environmental adaptive law enforcement video correction method described in, and can produce the same technical effects, so they will not be repeated here.

[0268] Example 3:

[0269] like Figure 3 1 is a schematic diagram of the structure of an electronic device for implementing an environment-adaptive law enforcement video correction method provided by an embodiment of the present invention.

[0270] The electronic device 1 may include a processor 10 , a memory 11 and a bus, and may further include a computer program stored in the memory 11 and executable on the processor 10 , such as a program 12 .

[0271] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 11 can also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Furthermore, the memory 11 can also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device 1, such as the code of the program 12, etc., but can also be used to temporarily store data that has been output or is to be output.

[0272] In some embodiments, the processor 10 may be comprised of an integrated circuit, such as a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 10 is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and circuits. It executes or runs programs or modules stored in the memory 11 (such as the program 12 for performing law enforcement video correction) and calls data stored in the memory 11 to perform various functions of the electronic device 1 and process data.

[0273] The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable connection and communication between the memory 11 and at least one processor 10, etc.

[0274] Figure 3Only the electronic device with components is shown, and it can be understood by those skilled in the art that Figure 3 The structure shown does not constitute a limitation on the electronic device 1 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0275] For example, although not shown, the electronic device 1 may further include a power source (e.g., a battery) for powering various components. Preferably, the power source may be logically connected to the at least one processor 10 via a power management device, thereby enabling functions such as charge management, discharge management, and power consumption management via the power management device. The power source may further include any of one or more DC or AC power sources, a recharging device, a power failure detection circuit, a power transferor or inverter, a power status indicator, and the like. The electronic device 1 may further include various sensors, Bluetooth modules, Wi-Fi modules, and the like, which are not further described herein.

[0276] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 1 and other electronic devices.

[0277] Optionally, the electronic device 1 may further include a user interface, which may be a display or an input unit (such as a keyboard). Optionally, the user interface may also be a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.

[0278] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0279] The program 12 stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When executed in the processor 10, it can achieve the following:

[0280] Obtain law enforcement video image frames, and calculate motion information of each pixel in the law enforcement video image frames based on optical flow analysis;

[0281] Based on the principle of consistency of spatial motion information, feature points of adjacent image frames of law enforcement videos are extracted and connected to form motion feature trajectories.

[0282] Different weights are set for different motion feature trajectories using the differential weight analysis method;

[0283] The law enforcement video to be corrected is geometrically transformed according to the assigned trajectory weights, the motion trajectory of the feature points in the video is corrected and the video is stabilized to obtain the corrected law enforcement video.

[0284] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to Figures 1 to 3 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0285] It should be noted that the serial numbers of the above-mentioned embodiments of the present invention are for descriptive purposes only and do not represent the advantages or disadvantages of the embodiments. In addition, the terms "including", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method comprising the element.

[0286] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0287] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An environment-adaptive law enforcement video correction method, characterized in that: The method comprises: S1: Obtain law enforcement video image frames, and calculate motion information of each pixel in the law enforcement video image frames based on optical flow analysis, wherein the LK algorithm based on the improved pyramid is the main method of the optical flow analysis; S2: Extracting feature points of adjacent frames of the law enforcement video based on the principle of consistency of spatial motion information, and connecting the feature points to form a motion feature trajectory, wherein the feature point extraction process includes: Based on the principle of consistency of spatial motion information, feature points of adjacent image frames in law enforcement videos are extracted. For any image frame in law enforcement video, , the feature point extraction process in the law enforcement video image frame is: S21: Constructing image frame The scale space of: in: Represents an image frame Any pixel in Pixel value of is the scale factor, the number of scale factors set is 2, each scale factor is different, and a two-layer scale space is constructed. The scale factors of the constructed two-layer scale space are ; S22: Constructing Gaussian difference space based on scale space : Each sampling point in the Gaussian difference space is detected. If the sampling point is the maximum or minimum value of the eight adjacent sampling points at this scale, the sampling point is marked as a feature point. , the feature points The corresponding pixel is ; S23: Calculate the feature point Corresponding pixel The gradient direction and gradient values : in: Pixel Pixel value of For any adjacent law enforcement video image frames , , calculate the law enforcement video image frame Feature Points as well as Feature Points The direction of motion information and the change of feature gradient direction : in: Representing feature points Corresponding pixel The direction of motion information, Represents pixel points The speed in the horizontal direction, Represents pixel points Speed ​​in the vertical direction; Representing feature points Corresponding pixel The direction of motion information, Represents pixel points The speed in the horizontal direction, Represents pixel points Speed ​​in the vertical direction; Select so that Reach the minimum feature point , as law enforcement video image frames Middle feature point The adjacent feature points and preset feature thresholds, if the feature gradient direction changes The minimum value of is higher than the feature threshold, which means that the feature point There are no adjacent feature points; Repeat the above steps to obtain the adjacent feature points of any feature point in the law enforcement video image frame in the adjacent image frame; S3: Use the differential weight analysis method to set different weights for different motion feature trajectories; S4: Performing geometric transformation on the law enforcement video to be corrected according to the assigned trajectory weights, correcting the motion trajectory of the feature points in the video and stabilizing the video to obtain a corrected law enforcement video.

2. The method for environmentally adaptive law enforcement video correction according to claim 1, wherein: The step S1 of obtaining the law enforcement video image frame includes: Obtain the law enforcement video to be corrected and divide it into several law enforcement video image frames. The law enforcement video image frame set after division is ,in represents the i-th video image frame of the law enforcement video, and N represents the total number of video image frames divided into the law enforcement video.

3. The method for environmentally adaptive law enforcement video correction according to claim 2, wherein: The step S1 calculates the motion information of each pixel in the law enforcement video image frame based on optical flow analysis, including: For any law enforcement video image frame , using the image grayscale processing method to convert the law enforcement video image frame Converted into a grayscale image, the law enforcement video image frame Any pixel in The grayscale processing result is: in: Represents pixel points The grayscale value of the pixel is used as the pixel value. Represents pixel points The position of the law enforcement video image frame The pixel at row x and column y; Represents pixel points Color values ​​in the R, G, and B color channels; Based on optical flow analysis, the motion information of each pixel in the law enforcement video image frame is calculated. , any pixel The motion information extraction process is as follows: S11: Constructing pixels The neighborhood pixel area, where the pixel point is the center of the neighborhood pixel area, and the neighborhood pixel area is A square area of ​​pixel size, the pixel point The neighborhood pixel area is: in: Represents pixel points The jth neighboring pixel of Represents pixel points , if the law enforcement video image frame There is no pixel in , then mark the pixel as blank and the pixel value is 0; S12: Constructing a pixel motion information model based on the LK algorithm: in: Represents pixel points The partial derivative of the pixel value in the horizontal direction of the video image frame, Represents pixel points The partial derivative of the pixel value in the vertical direction of the video image frame, Represents pixel points The partial derivative of the pixel value in the direction of the video image frame timing sequence, wherein the video image frame timing feature is the timing information of an adjacent and ordered video image frame sequence; T stands for transpose; represents a diagonal matrix; Represents pixel points The closer the pixel is to the center of the neighborhood pixel area, the greater the weight of the pixel; Represents pixel points The speed in the horizontal direction, Represents pixel points Speed ​​in the vertical direction; S13: Solve the pixel point motion information model to obtain the pixel point Initial motion information : S14: Video image frame As the bottom image of the pyramid, image downsampling is performed based on the bottom image to construct a pyramid, wherein the number of layers of the pyramid is 4, and the image downsampling process is: in: for pixel-sized Gaussian kernel; Represents the convolution operation; Indicates the number of layers of the pyramid, Represents the underlying image; Represents the image Perform downsampling processing; S15: Calculate the pixel points of different layers of the pyramid The motion information of the pixel point The calculation process of motion information is from S11 to S13; S16: Get pixel points based on pyramid and LK algorithm Global motion information of: in: Indicates that based on Pixel points obtained from the layer pyramid image Sports information, Represents pixel points Initial motion information ; Represents pixel points Global motion information, including pixel points Speed ​​in the horizontal direction and vertical speed .

4. The method for environmentally adaptive law enforcement video correction according to claim 1, wherein: In step S2, the extracted feature points are connected to form a motion feature trajectory, including: Adjacent feature points of adjacent image frames are connected to form a motion feature trajectory. The process of forming the motion feature trajectory is as follows: For any law enforcement video image frame Isolated feature points , determine the feature points Image frames in law enforcement video Are there adjacent feature points? , if it exists, connect the feature points and , forming an initial trajectory, and judging whether the terminal feature point of the initial trajectory has an adjacent feature point in the next frame of law enforcement video image frame. If so, connect the terminal feature point with the adjacent feature point of the next frame of law enforcement video image frame, and repeat this step. Otherwise, end with the feature point A process for forming an initial motion feature trajectory, wherein the length of the trajectory is at least three feature points; The isolated feature point means that the feature point is not an adjacent feature point of any feature point of the previous law enforcement video image frame; The constructed motion feature trajectory set is ,in represents the qth motion feature trajectory, and Q represents the total number of motion feature trajectories formed.

5. The method for environmentally adaptive law enforcement video correction according to claim 4, wherein: In step S3, different weights are set for different motion feature trajectories using a differential weight analysis method, including: Identify the camera shake trajectory in the motion feature trajectory set. If the frame length of the motion feature trajectory is less than , it means that the motion feature trajectory is a camera shake trajectory, where N represents the total frame length of the law enforcement video, and the frame length of the motion feature trajectory is the total number of feature points in the trajectory; Different weights are set for different motion feature trajectories using a differential weight analysis method. The weight of the i-th law enforcement video image frame for: in: Represents motion feature trajectory The inclination angle; Represents motion feature trajectory The length of the trajectory; represents the number of trajectories passing through the i-th law enforcement video image frame; , if the motion feature trajectory After the i-th law enforcement video image frame, ,otherwise ; Represents motion feature trajectory After the feature points of the i-th law enforcement video image frame, The standard deviation is Gaussian function.

6. The method for environmentally adaptive law enforcement video correction according to claim 1, wherein: In step S4, geometric transformation is performed on the law enforcement video to be corrected according to the assigned trajectory weights, the motion trajectory of the feature points in the video is corrected, and the video is stabilized, including: The law enforcement video to be corrected is geometrically transformed according to the assigned trajectory weights, and the geometric transformation matrix F is: in: The geometric transformation formula of the i-th law enforcement video image frame is based on Performing correction processing on the i-th law enforcement video image frame; Constructing a correction objective function for law enforcement videos : in: To correct the objective function, we solve Achieve the minimum geometric transformation matrix F, perform geometric transformation on the law enforcement video to be corrected based on F, correct the motion trajectory of the feature points in the video and stabilize the video to obtain the corrected law enforcement video; Correction objective function Perform optimization and solve, the correction objective function The optimization solution process is: S41: Initialize the geometric transformation matrix , the mth iteration result of the geometric transformation matrix is , the initial value of m is 1, and the error threshold is ,in is 0.3; S42: Calculate the correction target function The partial derivative matrix of , and update the geometric transformation matrix: in: Indicates the damping factor, set it to 0.15; S43: If , then stop the algorithm iteration, is the optimal geometric transformation matrix , otherwise let , return to step S42; Using the optimal geometric transformation matrix obtained Correcting law enforcement videos, , The geometric transformation formula of the i-th law enforcement video image frame is represented by: .

7. An environment-adaptive law enforcement video correction system, characterized in that: The system comprises: A video frame motion information extraction module is used to obtain law enforcement video image frames and calculate the motion information of each pixel in the law enforcement video image frames based on optical flow analysis; A feature trajectory extraction device is used to extract feature points of adjacent image frames of law enforcement videos based on the principle of consistency of spatial motion information, connect the feature points to form motion feature trajectories, and use a differential weight analysis method to set different weights for different motion feature trajectories, highlighting feature trajectories with larger jitter amplitudes to achieve trajectory separation; A correction parameter solving device is used to perform geometric transformation on the law enforcement video to be corrected according to the assigned trajectory weight, correct the motion trajectory of the feature points in the video and stabilize the video, so as to realize an environment-adaptive law enforcement video correction method as described in claims 1-6.

Images