Video anti-shake system for eliminating interference of dynamic feature points

Abstract

The invention discloses a video anti-shake system for eliminating interference of dynamic feature points, and the system comprises a feature point extraction module which is used for extracting a plurality of feature points in an image from a video frame, and generating the motion trails of the feature points; the dynamic feature point filtering module comprises a gyroscope data acquisition unit and a trajectory analysis unit; the gyroscope data acquisition unit is used for acquiring rotation information of the camera in a shooting process; the track analysis unit is used for carrying out clustering analysis on the tracks of the feature points based on the motion tracks and the rotation information of the feature points so as to identify and filter out dynamic feature points; and the image smoothing processing module is used for calculating the conversion between the video frames according to the filtered static feature points and smoothing the video frames so as to reduce picture jitter. According to the method, the video anti-shake effect in a dynamic scene can be remarkably improved, so that a video picture is more stable, and the problems of image distortion and deformation caused by existence of a dynamic object are reduced.

Description

Technical Field

[0001] The invention belongs to the field of digital image processing, and in particular relates to a video anti-shake system for eliminating interference of dynamic feature points. Background Art

[0002] With the widespread use of video shooting equipment, video stabilization technology has become particularly important. The purpose of video stabilization is to reduce the shaking during video recording and improve the stability of the picture and the viewing experience. Traditional video stabilization methods mostly rely on sensor data or optical systems for physical stabilization, such as hardware devices such as gyroscopes and accelerometers to detect the motion state of the camera. However, when processing complex dynamic scenes, these hardware devices often cannot accurately capture the subtle shaking in the video content, resulting in poor stabilization effect.

[0003] At present, video stabilization technology based on optical flow estimates the motion trajectory of the camera by detecting feature points in video frames and analyzing the motion vectors between adjacent frames. Although this method can effectively reduce video jitter, there are many challenges in practical applications. Especially in dynamic scenes, dynamic objects in the video often introduce interference; the movement of dynamic objects is inconsistent with the movement of the camera. If the feature points caused by these dynamic objects are not processed, the stabilization algorithm will mistake these irrelevant movements for camera jitter, which will affect the stabilization effect.

[0004] In order to improve the effect of video stabilization, especially in dynamic scenes, traditional stabilization methods usually rely on the extraction and processing of feature points. However, the distribution of feature points is not always uniform, especially in scenes with complex video content, where feature points may be concentrated in certain areas and sparse in other areas. This uneven distribution can lead to unstable video stabilization effects. Therefore, how to effectively screen feature points and filter dynamic feature points has become one of the key issues in improving video stabilization performance. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide a video stabilization system that eliminates interference from dynamic feature points, which can significantly improve the video stabilization effect in dynamic scenes, make the video picture more stable, and reduce image distortion and deformation problems caused by the presence of dynamic objects.

[0006] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] A video anti-shake system for eliminating dynamic feature point interference of the present invention comprises: a feature point extraction module, a dynamic feature point filtering module and an image smoothing processing module;

[0008] The feature point extraction module is used to extract multiple feature points in the image from the video frame and generate motion trajectories of the feature points;

[0009] The dynamic feature point filtering module includes: a gyroscope data acquisition unit and a trajectory analysis unit;

[0010] A gyroscope data acquisition unit, used to obtain the rotation information of the camera during shooting;

[0011] The trajectory analysis unit performs cluster analysis on the trajectory of feature points based on the motion trajectory and rotation information of the feature points, thereby identifying and filtering out dynamic feature points;

[0012] The image smoothing processing module is used to calculate the transformation between video frames according to the filtered static feature points, and to perform smoothing processing on the video frames to reduce picture jitter.

[0013] Furthermore, the feature point extraction module specifically includes:

[0014] Preliminary feature point extraction: Preliminary feature points are extracted from the video frame based on the corner detection algorithm, and feature descriptors are generated based on the local neighborhood of the feature points;

[0015] Feature point extraction optimization: Screen the initially extracted feature points, select high-confidence feature points based on the extraction threshold, and ensure that the feature points are evenly distributed in space; when the number of feature points in a certain area is small, the density of feature points in the area is supplemented by lowering the extraction threshold to ensure the overall uniformity of feature point distribution;

[0016] Obtain feature trajectories: Track feature points in video frames using the optical flow method to generate motion trajectories of feature points; calculate motion vectors of feature points based on pixel changes between consecutive frames to obtain the trajectories of feature points for subsequent dynamic point filtering and static feature point classification.

[0017] Furthermore, the specific process of extracting the preliminary feature points includes:

[0018] (11) Read the video frame I at a certain time t of the video stream t ;

[0019] (12) Harris corner feature point detection algorithm is used to detect corner points from video frame I t Extract a set of feature points from Each feature point Contains its coordinate position And its feature descriptor

[0020] Furthermore, the specific process of feature point extraction optimization includes:

[0021] (21) Calculate the confidence of each feature point. Based on the characteristic point detection response value calculation, the response value is set The confidence It is expressed as:

[0022]

[0023] in, The higher the confidence, the more significant the feature point is in the local features;

[0024] (22) Set an extraction threshold τ to filter out feature points with confidence higher than the extraction threshold. The filtered feature point set is expressed as: Select a τ value to ensure that the feature points have sufficient confidence after screening;

[0025] (23) Use the grid method to divide the feature point distribution of the video frame into regions, dividing the video frame into several grid units of size m×m. For the feature point sparse area G ij , if the number of feature points in the region Below the preset lower threshold T min , then it is judged that the feature points in this area are sparse, and the feature point extraction threshold of this area needs to be adjusted;

[0026] (24) Obtaining video frame feature points P t , in Represents the feature point P of the video frame t The jth grid in the i-th row after being divided into m×m grid units.

[0027] Furthermore, the step (23) specifically includes:

[0028] (231) For the feature point sparse region G ij , reduce the extraction threshold τ, so that more feature points are extracted in this area and the density of feature points is supplemented; the new extraction threshold τ ij , according to the lack of feature points in the area, it is adjusted as follows:

[0029]

[0030] As the number of feature points in the region decreases, the new extraction threshold τ ij It will be reduced accordingly, so as to extract more feature points and ensure that the number of regional feature points meets the requirements;

[0031] (232) Determine whether the adjusted threshold meets the number of feature points Greater than or equal to the preset lower threshold Tmin If it is not satisfied, repeat step (231). If it is still not satisfied after five repetitions, stop and use the currently extracted feature point results.

[0032] Furthermore, the specific process of obtaining the feature trajectory includes:

[0033] (31) For the video frame at time t, I t The feature points in The video frame at time t+1 is I t+1 The corresponding feature points are in is the motion vector of the feature point between two frames, expressed as optical flow; when the image brightness is constant, the following equation is obtained:

[0034]

[0035] in, Represents pixels Gray value at time t; By Taylor expansion of the equation in time and space and ignoring high-order terms, the linear optical flow constraint equation is obtained, and the feature points are calculated

[0036] (32) Feature points As the starting point, repeat step (31) to calculate the optical flow of the feature point position in each video frame to obtain the motion trajectory of the feature point in multiple consecutive frames; the motion trajectory of the feature point is expressed as:

[0037]

[0038] in, Representing feature points In video frame I t+j The corresponding pixel position in Representing feature points The movement trajectory, I t+k The corresponding feature points in .

[0039] Furthermore, the gyroscope data acquisition unit specifically includes:

[0040] (41) Obtain gyroscope data and calculate the rotation matrix of the camera; read the angular velocity data w at the current time t of each video frame, w = (w x ,w y ,w z ), where w x ,w y ,w z Represent the angular velocity components on the x, y, and z axes respectively;

[0041] (42) Assuming that the attitude of the gyroscope at time t is C(t), then at time t+1, the attitude of the gyroscope C(t+1) is:

[0042]

[0043] Where E is the unit matrix, σ=|w|, From this, we can get the rotation matrix R of the camera from time t to time t+1 t for

[0044] Furthermore, the specific process of the trajectory analysis unit includes:

[0045] (51) Obtain all feature point trajectories P in the video frame at time t t ,The movement of dynamic feature points in the video frame includes the movement of the camera and the movement of the dynamic feature points themselves. The trajectory of the feature points corresponding to the dynamic feature points is inconsistent with the trajectory of the static feature points; according to the camera imaging principle Where K is the camera internal parameter matrix, P represents the three-dimensional coordinates corresponding to the pixel p on the video frame; the camera imaging position is also related to the three-dimensional spatial coordinates corresponding to the feature point. A kmeans clustering is performed according to the feature point trajectory to classify the dynamic feature points and the static feature points to obtain a set of k feature point trajectories;

[0046] (52) Obtain the classified feature points and filter out the dynamic feature points.

[0047] Furthermore, the step (52) specifically includes:

[0048] (521) Set the rotation matrix of the camera from time t to time t+1 to R and the displacement to T;

[0049] (522) For a certain feature point trajectory The corresponding feature points at time t and t+1 are and According to the camera imaging formula and the camera's position and posture, the following formula is obtained:

[0050]

[0051] in, For feature points The corresponding Z coordinate of the three-dimensional coordinate, For feature points The Z coordinate of the corresponding three-dimensional coordinate;

[0052] After simplification, we finally get the formula:

[0053]

[0054] According to the above formula, as long as enough feature point trajectories are provided, the rotation of the camera from time t to t+1 can be calculated through the feature point trajectories;

[0055] (523) For each feature point trajectory classified in step (51), the camera rotation matrix R1, R2, ..., R k , and then compare it with the real rotation matrix R of the camera, and use the Euclidean distance as the distance function D for similarity comparison i , the formula is D i =(RR i ) 2 , if D i Greater than the threshold D T , it means that the class corresponding to the rotation is the dynamic feature point class;

[0056] (524) Filter out all dynamic feature point classes and obtain the static feature point set at time t

[0057] Furthermore, the image smoothing processing module specifically includes:

[0058] (61) The video frame at time t in step (524) is obtained as I t , the corresponding static feature point set The video frame at time t+1 is I t+1 , the corresponding static feature point set Calculate the transformation relationship between video frames;

[0059] (62) The transformation matrices H1, H2, ..., H between the obtained adjacent video frames n-1 Perform smoothing; define the homography matrix The smoothing formula is:

[0060]

[0061] in, is Gaussian filtering, r represents the smoothing radius, is the smoothed homography matrix, is the homography matrix transformed from the 1st frame to the ξth frame;

[0062] (63) A remapping operation based on smooth transformation is performed on each frame to remap the video frame to the smoothed camera motion trajectory. The smoothed video frame for in,

[0063] After remapping all original video frames, smoothed video frames are obtained.

[0064] Furthermore, the step (61) specifically includes:

[0065] (611) The transformation between video frames is described by homography matrix, including translation, rotation, and scaling; for the static feature point set and static feature points Perform feature point matching and calculate the matching feature point pairs between the two sets based on the feature descriptors corresponding to the feature points and the nearest neighbor principle;

[0066] (612) Using the homography matrix H t To describe two video frames I t and I t+1 Relationship I t+1 =H t I t , set feature points and feature points For collections from and collection A pair of matching feature points satisfies the following formula:

[0067]

[0068] (613) According to the least squares method, the homography matrix H is calculated from the matched feature point pairs. t , the homography matrix describes the transformation between video frames, including translation, rotation and scaling.

[0069] Beneficial effects of the present invention:

[0070] The present invention extracts and optimizes feature points in video frames and dynamically adjusts feature point extraction thresholds to ensure uniform distribution of feature points in video frames. At the same time, feature point interference caused by dynamic objects is filtered out by analyzing feature point trajectories, thereby improving the accuracy and robustness of video anti-shake.

[0071] 1. The present invention adopts a strategy of different thresholds to make the distribution of feature points extracted from the video frame as even as possible, avoiding the situation where the feature points in a certain area are very sparse and the feature points in another area are very dense, thereby improving the quality of the feature point pairs obtained and laying a foundation for subsequent video stabilization.

[0072] 2. The present invention adopts dual modalities, visual modality and sensor (gyroscope) modality, to verify whether the acquired feature trajectory class belongs to a dynamic feature point, thereby filtering out the dynamic feature points to avoid their interference with subsequent video anti-shake. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 It is the application principle diagram of the system of the present invention;

[0074] Figure 2 Schematic diagram of the process for feature point extraction optimization;

[0075] Figure 3 This is a schematic diagram of the principle of the dynamic feature point filtering module;

[0076] Figure 4 Schematic diagram of the principle of the image smoothing processing module. DETAILED DESCRIPTION

[0077] In order to facilitate the understanding of those skilled in the art, the present invention is further described below in conjunction with embodiments and drawings. The contents mentioned in the implementation modes are not intended to limit the present invention.

[0078] Reference Figures 1 to 4 As shown, a video anti-shake system for eliminating dynamic feature point interference of the present invention comprises: a feature point extraction module, a dynamic feature point filtering module and an image smoothing processing module;

[0079] The feature point extraction module is used to extract multiple feature points in the image from the video frame and generate the motion trajectory of the feature points; specifically, it includes:

[0080] Preliminary feature point extraction: Preliminary feature points are extracted from the video frame based on the corner detection algorithm, and feature descriptors are generated based on the local neighborhood of the feature points;

[0081] Feature point extraction optimization: Screen the initially extracted feature points, select high-confidence feature points based on the extraction threshold, and ensure the uniform distribution of feature points in space; when the number of feature points in a certain area is small, the density of feature points in the area is supplemented by lowering the extraction threshold to ensure the uniformity of the overall feature point distribution;

[0082] Obtain feature trajectory: Track the feature points in the video frame through the optical flow method to generate the motion trajectory of the feature points; calculate the motion vector of the feature points based on the pixel changes between consecutive frames to obtain the trajectory of the feature points for subsequent dynamic point filtering and static feature point classification processing.

[0083] The specific process of extracting the preliminary feature points includes:

[0084] (11) Read the video frame I at a certain time t of the video stream t ;

[0085] (12) Harris corner feature point detection algorithm is used to detect corner points from video frame I t Extract a set of feature points from Each feature point Contains its coordinate position And its feature descriptor

[0086] The specific process of feature point extraction and optimization includes:

[0087] (21) Calculate the confidence of each feature point. Based on the characteristic point detection response value calculation, the response value is set The confidence It is expressed as:

[0088]

[0089] in, The higher the confidence, the more significant the feature point is in the local features;

[0090] (22) An extraction threshold τ is set to filter out feature points with confidence higher than the extraction threshold. The filtered feature point set is expressed as: Select a τ value to ensure that the feature points have sufficient confidence after screening;

[0091] (23) Use the grid method to divide the feature point distribution of the video frame into regions, dividing the video frame into several grid units of size m×m. For the feature point sparse area G ij , if the number of feature points in the region Below the preset lower threshold T min , then it is judged that the feature points in this area are sparse, and the feature point extraction threshold of this area needs to be adjusted;

[0092] (24) Obtaining video frame feature points P t , in Represents the feature point P of the video frame t The jth grid in the i-th row after being divided into m×m grid units.

[0093] Specifically, the step (23) comprises:

[0094] (231) For the feature point sparse region G ij , reduce the extraction threshold τ, so that more feature points are extracted in this area and the density of feature points is supplemented; the new extraction threshold τ ij , according to the lack of feature points in the area, it is adjusted as follows:

[0095]

[0096] As the number of feature points in the region decreases, the new extraction threshold τ ijIt will be reduced accordingly, so as to extract more feature points and ensure that the number of regional feature points meets the requirements;

[0097] (232) Determine whether the adjusted threshold meets the number of feature points Greater than or equal to the preset lower threshold T min If it is not satisfied, repeat step (231). If it is still not satisfied after five repetitions, stop and use the currently extracted feature point results.

[0098] The specific process of obtaining the feature trajectory includes:

[0099] (31) For the video frame at time t, I t The feature points in The video frame at time t+1 is I t+1 The corresponding feature points are in is the motion vector of the feature point between two frames, expressed as optical flow; when the image brightness is constant, the following equation is obtained:

[0100]

[0101] in, Represents pixels Gray value at time t; By Taylor expansion of the equation in time and space and ignoring high-order terms, the linear optical flow constraint equation is obtained, and the feature points are calculated

[0102] (32) Feature points As the starting point, repeat step (31) to calculate the optical flow of the feature point position in each video frame to obtain the motion trajectory of the feature point in multiple consecutive frames; the motion trajectory of the feature point is expressed as:

[0103]

[0104] in, Representing feature points In video frame I t+j The corresponding pixel position in Representing feature points The movement trajectory, I t+k The corresponding feature points in .

[0105] The dynamic feature point filtering module includes: a gyroscope data acquisition unit and a trajectory analysis unit;

[0106] A gyroscope data acquisition unit, used to obtain the rotation information of the camera during shooting;

[0107] The trajectory analysis unit performs cluster analysis on the trajectory of feature points based on the motion trajectory and rotation information of the feature points, thereby identifying and filtering out dynamic feature points;

[0108] Wherein, the gyroscope data acquisition unit specifically includes:

[0109] (41) Obtain gyroscope data and calculate the rotation matrix of the camera; read the angular velocity data w at the current time t of each video frame, w = (w x ,w y ,w z ), where w x ,w y ,w z Represent the angular velocity components on the x, y, and z axes respectively;

[0110] (42) Assuming that the attitude of the gyroscope at time t is C(t), then at time t+1, the attitude of the gyroscope C(t+1) is:

[0111]

[0112] Where E is the unit matrix, σ=|w|, From this, we can get the rotation matrix R of the camera from time t to time t+1 t for

[0113] The specific process of the trajectory analysis unit includes:

[0114] (51) Obtain all feature point trajectories P in the video frame at time t t ,The movement of dynamic feature points in the video frame includes the movement of the camera and the movement of the dynamic feature points themselves. The trajectory of the feature points corresponding to the dynamic feature points is inconsistent with the trajectory of the static feature points; according to the camera imaging principle Where K is the camera internal parameter matrix, P represents the three-dimensional coordinates corresponding to the pixel p on the video frame; the camera imaging position is also related to the three-dimensional spatial coordinates corresponding to the feature point. A kmeans clustering is performed according to the feature point trajectory to classify the dynamic feature points and the static feature points to obtain a set of k feature point trajectories;

[0115] (52) Obtain the classified feature points and filter out the dynamic feature points.

[0116] Specifically, the step (52) includes:

[0117] (521) Set the rotation matrix of the camera from time t to time t+1 to R and the displacement to T;

[0118] (522) For a certain feature point trajectory The corresponding feature points at time t and t+1 are and According to the camera imaging formula and the camera's position and posture, the following formula is obtained:

[0119]

[0120] in, For feature points The corresponding Z coordinate of the three-dimensional coordinate, For feature points The Z coordinate of the corresponding three-dimensional coordinate;

[0121] After simplification, we finally get the formula:

[0122]

[0123] According to the above formula, as long as enough feature point trajectories are provided, the rotation of the camera from time t to t+1 can be calculated through the feature point trajectories;

[0124] (523) For each feature point trajectory classified in step (51), the camera rotation matrix R1, R2, ..., R k , and then compare it with the real rotation matrix R of the camera, and use the Euclidean distance as the distance function D for similarity comparison i , the formula is D i =(RR i ) 2 , if D i Greater than the threshold D T , it means that the class corresponding to the rotation is the dynamic feature point class;

[0125] (524) Filter out all dynamic feature point classes and obtain the static feature point set at time t

[0126] The image smoothing processing module is used to calculate the transformation between video frames according to the filtered static feature points, and to smooth the video frames to reduce the picture jitter; specifically, it includes:

[0127] (61) The video frame at time t in step (524) is obtained as I t , the corresponding static feature point set The video frame at time t+1 is I t+1 , the corresponding static feature point set Calculate the transformation relationship between video frames;

[0128] (62) The transformation matrices H1, H2, ..., H between the obtained adjacent video frames n-1 Perform smoothing; define the homography matrix The smoothing formula is:

[0129]

[0130] in, is Gaussian filtering, r represents the smoothing radius, is the homography matrix after smoothing, is the homography matrix transformed from the 1st frame to the ξth frame;

[0131] (63) A remapping operation based on smooth transformation is performed on each frame to remap the video frame to the smoothed camera motion trajectory. The smoothed video frame for in,

[0132] After remapping all original video frames, smoothed video frames are obtained.

[0133] Specifically, the step (61) comprises:

[0134] (611) The transformation between video frames is described by homography matrix, including translation, rotation, and scaling; for the static feature point set and static feature points Perform feature point matching and calculate the matching feature point pairs between the two sets based on the feature descriptors corresponding to the feature points and the nearest neighbor principle;

[0135] (612) Using the homography matrix H t To describe two video frames I t and I t+1 Relationship I t+1 =H t I t , set feature points and feature points For collections from and collection A pair of matching feature points satisfies the following formula:

[0136]

[0137] (613) According to the least squares method, the homography matrix H is calculated from the matched feature point pairs. t , the homography matrix describes the transformation between video frames, including translation, rotation and scaling.

[0138] The present invention has many specific application paths. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements can be made without departing from the principle of the present invention. These improvements should also be regarded as the protection scope of the present invention.

Claims

1. A video anti-shake system for eliminating interference from dynamic feature points, characterized in that: include: Feature point extraction module, dynamic feature point filtering module and image smoothing processing module; The feature point extraction module is used to extract multiple feature points in the image from the video frame and generate motion trajectories of the feature points; The dynamic feature point filtering module includes: a gyroscope data acquisition unit and a trajectory analysis unit; A gyroscope data acquisition unit, used to obtain the rotation information of the camera during shooting; The trajectory analysis unit performs cluster analysis on the trajectory of feature points based on the motion trajectory and rotation information of the feature points, thereby identifying and filtering out dynamic feature points; The image smoothing processing module is used to calculate the transformation between video frames according to the filtered static feature points and perform smoothing processing on the video frames.

2. The video anti-shake system for eliminating dynamic feature point interference according to claim 1, characterized in that: The feature point extraction module specifically includes: Preliminary feature point extraction: Preliminary feature points are extracted from the video frame based on the corner detection algorithm, and feature descriptors are generated based on the local neighborhood of the feature points; Feature point extraction optimization: Screen the initially extracted feature points, select high-confidence feature points based on the extraction threshold, and ensure that the feature points are evenly distributed in space; when the number of feature points in a certain area is small, the density of feature points in the area is supplemented by lowering the extraction threshold to ensure the overall uniformity of feature point distribution; Obtain feature trajectory: Track the feature points in the video frame through the optical flow method to generate the motion trajectory of the feature points; calculate the motion vector of the feature points based on the pixel changes between consecutive frames to obtain the trajectory of the feature points.

3. The video anti-shake system for eliminating dynamic feature point interference according to claim 2, characterized in that: The specific process of extracting the preliminary feature points includes: (11) Read the video frame I at a certain time t of the video stream t ; (12) Harris corner feature point detection algorithm is used to detect corner points from video frame I t Extract a set of feature points from Each feature point Contains its coordinate position And its feature descriptor 4. The video stabilization system for eliminating dynamic feature point interference according to claim 3, characterized in that: The specific process of feature point extraction optimization includes: (21) Calculate the confidence of each feature point. Based on the characteristic point detection response value calculation, the response value is set Confidence It is expressed as: in, The higher the confidence, the more significant the feature point is in the local features; (22) An extraction threshold τ is set to filter out feature points with confidence higher than the extraction threshold. The filtered feature point set is expressed as: Select a τ value to ensure that the feature points have sufficient confidence after screening; (23) Use the grid method to divide the feature point distribution of the video frame into regions, dividing the video frame into several grid units of size m×m. For the feature point sparse area G ij , if the number of feature points in the region Below the preset lower threshold T min , then it is judged that the feature points in this area are sparse, and the feature point extraction threshold of this area needs to be adjusted; (24) Obtaining video frame feature points P t , in Represents the feature point P of the video frame t The jth grid in the i-th row after being divided into m×m grid units.

5. The video anti-shake system for eliminating dynamic feature point interference according to claim 4, characterized in that: The step (23) specifically comprises: (231) For the feature point sparse region G ij , reduce the extraction threshold τ, so that more feature points are extracted in this area and the density of feature points is supplemented; the new extraction threshold τ ij , according to the lack of feature points in the area, it is adjusted as follows: As the number of feature points in the region decreases, the new extraction threshold τ ij It will be reduced accordingly, so as to extract more feature points and ensure that the number of regional feature points meets the requirements; (232) Determine whether the adjusted threshold meets the number of feature points Greater than or equal to the preset lower threshold T min If it is not satisfied, repeat step (231). If it is still not satisfied after five repetitions, stop and use the currently extracted feature point results.

6. The video anti-shake system for eliminating dynamic feature point interference according to claim 4, characterized in that: The specific process of obtaining the feature trajectory includes: (31) For the video frame at time t, I t The feature points in The video frame at time t+1 is I t+1 The corresponding feature points are in is the motion vector of the feature point between two frames, expressed as optical flow; when the image brightness is constant, the following equation is obtained: in, Represents pixels Gray value at time t; By Taylor expansion of the equation in time and space and ignoring high-order terms, the linear optical flow constraint equation is obtained, and the feature points are calculated (32) Feature points As the starting point, repeat step (31) to calculate the optical flow of the feature point position in each video frame to obtain the motion trajectory of the feature point in multiple consecutive frames; the motion trajectory of the feature point is expressed as: in, Representing feature points In video frame I t+j The corresponding pixel position in Representing feature points The movement trajectory, Indicates I t+k The corresponding feature points in .

7. The video anti-shake system for eliminating dynamic feature point interference according to claim 1, characterized in that: The gyroscope data acquisition unit specifically includes: (41) Obtain gyroscope data and calculate the rotation matrix of the camera; read the angular velocity data w at the current time t of each video frame, w = (w x ,w y ,w z ), where w x ,w y ,w z Represent the angular velocity components on the x, y, and z axes respectively; (42) Assuming that the attitude of the gyroscope at time t is C(t), then at time t+1, the attitude of the gyroscope C(t+1) is: Where E is the unit matrix, σ=|w|, Therefore, from time t to time t+1, the rotation matrix R of the camera is obtained. t for 8. The video stabilization system for eliminating dynamic feature point interference according to claim 1, characterized in that: The specific process of the trajectory analysis unit includes: (51) Obtain all feature point trajectories P in the video frame at time t t ,The movement of dynamic feature points in the video frame includes the movement of the camera and the movement of the dynamic feature points themselves. The trajectory of the feature points corresponding to the dynamic feature points is inconsistent with the trajectory of the static feature points; according to the camera imaging principle Where K is the camera internal parameter matrix, P represents the three-dimensional coordinates corresponding to the pixel p on the video frame; the camera imaging position is also related to the three-dimensional spatial coordinates corresponding to the feature point. A kmeans clustering is performed according to the feature point trajectory to classify the dynamic feature points and the static feature points to obtain a set of k feature point trajectories; (52) Obtain the classified feature points and filter the dynamic feature points therein, specifically including: (521) Set the rotation matrix of the camera from time t to time t+1 to R and the displacement to T; (522) For a certain feature point trajectory The corresponding feature points at time t and t+1 are and According to the camera imaging formula and the camera's position and posture, the following formula is obtained: in, For feature points The corresponding Z coordinate of the three-dimensional coordinate, For feature points The Z coordinate of the corresponding three-dimensional coordinate; After simplification, we finally get the formula: (523) For each feature point trajectory classified in step (51), the camera rotation matrix R1, R2, ..., R k , and then compare it with the real rotation matrix R of the camera, and use the Euclidean distance as the distance function D for similarity comparison i , the formula is D i =(RR i ) 2 , if D i Greater than the threshold D T , it means that the class corresponding to the rotation is the dynamic feature point class; (524) Filter out all dynamic feature point classes and obtain the static feature point set at time t 9. The video anti-shake system for eliminating dynamic feature point interference according to claim 8, characterized in that: The image smoothing processing module specifically includes: (61) The video frame at time t in step (524) is obtained as I t , the corresponding static feature point set The video frame at time t+1 is I t+1 , the corresponding static feature point set Calculate the transformation relationship between video frames; (62) The transformation matrices H1, H2, ..., H between the obtained adjacent video frames n-1 Perform smoothing; define the homography matrix The smoothing formula is: in, is Gaussian filtering, r represents the smoothing radius, is the smoothed homography matrix, is the homography matrix transformed from the 1st frame to the ξth frame; (63) A remapping operation based on smooth transformation is performed on each frame to remap the video frame to the smoothed camera motion trajectory. The smoothed video frame for in, After remapping all original video frames, smoothed video frames are obtained.

10. The video anti-shake system for eliminating dynamic feature point interference according to claim 9, characterized in that: The step (61) specifically comprises: (611) The transformation between video frames is described by homography matrix, including translation, rotation, and scaling; for the static feature point set and static feature points Perform feature point matching and calculate the matching feature point pairs between the two sets based on the feature descriptors corresponding to the feature points and the nearest neighbor principle; (612) Using the homography matrix H t To describe two video frames I t and I t+1 Relationship I t+1 =H t I t , set feature points and feature points For collections from and collection A pair of matching feature points satisfies the following formula: (613) According to the least squares method, the homography matrix H is calculated from the matched feature point pairs. t , the homography matrix describes the transformation between video frames, including translation, rotation and scaling.

Images