Video three-dimensional reconstruction key frame extraction method and device based on mutual verification weighted matching

Abstract

The invention discloses a video three-dimensional reconstruction key frame extraction method and device based on mutual verification weighted matching, and the method comprises the steps: obtaining an input video stream, and carrying out the pre-classification of an image scene in the input video stream; calculating image feature points and descriptors by using an SIFT (Scale Invariant Feature Transform) detection algorithm, capturing dynamic changes of matching points of adjacent frames based on FLANN matching and pyramid LK optical flow calculation, and screening according to an intersection of FLANN matching and pyramid LK optical flow calculation results to obtain strong matching feature points; and based on the strong matching feature points, calculating vanishing points, endowing surrounding points within a preset range of the vanishing points with weights, constructing a robust index of the similarity of adjacent frames, obtaining the similarity of the adjacent frames, and realizing screening of the key frames of the video stream. According to the invention, the detection precision and stability are improved, and the robustness of the algorithm in diversified scenes is ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of video 3D reconstruction, and specifically relates to a method and device for extracting key frames for video 3D reconstruction based on mutual verification weighted matching, which can be used in visual matching positioning to provide crowdsourced images that support 3D reconstruction needs for the construction of visual space feature reference maps in an unrestricted and wide-area range. Background Art

[0002] Using navigation and positioning technology to obtain accurate location information has become a necessity in daily life. Traditional location acquisition methods rely on maps and GNSS, but face difficulties in environments with large signal interference. Therefore, visual navigation and positioning technology has become a current research hotspot. This technology can use image information for analysis and recognition, solve the problem of satellite navigation failure, and achieve more accurate positioning and real-time navigation.

[0003] Visual positioning technology requires building a 3D model in advance, and then achieving positioning by matching the current image with the model. Traditional 3D reconstruction relies on irregular images from multiple perspectives to restore the 3D coordinates of the scene points and the camera pose. There are a large number of image frames in the video, which can provide rich scene information, so video-based 3D reconstruction is of great significance. However, there are many redundant images in the video, so it is necessary to screen the video sequence and extract the key frames that meet the 3D reconstruction requirements, reduce the amount of reconstruction calculations, and improve efficiency and robustness.

[0004] At present, the methods for obtaining key frames can be mainly divided into time interval extraction, frame difference extraction and optical flow extraction. Among them, the time interval extraction method does not rely on the specific image content or features, but uses the time interval information to evenly select frames; the frame difference extraction method selects key frames by comparing the similarity of adjacent frames. The current methods for measuring similarity mainly include pixel difference method, color histogram method, structural similarity index method, etc.; the optical flow extraction method calculates the image light spot motion information and extracts key frames by analyzing the size and direction of the vector field of the video motion.

[0005] However, the selection of key frames must not only consider image quality and information richness, but also the dynamic changes of the scene. Regardless of which of the above methods is used to extract key frames, there are a series of challenges, which can be summarized into the following three aspects:

[0006] (1) Videos are composed of image sequences with different scene features. During video acquisition, scene changes are dynamic. When there are fast-moving objects in the scene, it will cause significant visual changes, resulting in a sharp increase in optical flow. This change may cause deviations in the optical flow estimation of the object, thus affecting the extraction of key frames.

[0007] (2) There are still some defects when calculating the similarity matching of adjacent frames. The time interval-based method may lead to missing or redundant images at different motion speeds; the histogram calculation in the frame difference method often fails to capture local details and lacks robustness to lighting changes; the optical flow extraction method may fail when dealing with complex backgrounds, fast motion or occlusion.

[0008] (3) When selecting video key frames, the contribution values ​​of image pixels in the calculation of the similarity of adjacent frames vary in different scenes. In scenes with large motion changes, the pixel differences between adjacent frames increase significantly, resulting in a significant increase in the contribution values ​​of certain areas. In relatively static or simple scenes, the picture changes are small, and the contribution values ​​of many pixels are relatively low. Summary of the invention

[0009] To solve the above technical problems, the present invention provides a method for extracting key frames for video 3D reconstruction based on mutual verification weighted matching, which can pre-classify video scenes and effectively divide scenes. It combines SIFT feature point matching and optical flow mutual verification mechanism to screen strong matching points of adjacent frames, considers the contribution of the image vanishing point and its surrounding points and weights them to construct a robust similarity index, obtains the similarity of adjacent frames and realizes key frame screening of video streams.

[0010] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0011] A method for extracting key frames for video 3D reconstruction based on mutual verification weighted matching, the method comprising:

[0012] Step 1: Obtain an input video stream and pre-classify the image scenes included in the input video stream;

[0013] Step 2: Use the SIFT detection algorithm to solve the image feature points and descriptors, capture the dynamic changes of the matching points of adjacent frames based on FLANN matching and pyramid LK optical flow calculation, and obtain strong matching feature points based on the intersection of FLANN matching and pyramid LK optical flow calculation results;

[0014] Step 3: Based on the strongly matched feature points, the vanishing point is solved and weights are assigned to the surrounding points within the preset range of the vanishing point, a robust index of adjacent frame similarity is constructed, the adjacent frame similarity is obtained, and key frame screening of the video stream is realized.

[0015] On the other hand, the present invention provides a video 3D reconstruction key frame extraction device based on mutual verification weighted matching, comprising:

[0016] A preprocessing unit, used for acquiring an input video stream and pre-classifying image scenes included in the input video stream;

[0017] The feature point matching unit is used to solve the image feature points and descriptors using the SIFT detection algorithm, capture the dynamic changes of the matching points of adjacent frames based on FLANN matching and pyramid LK optical flow calculation, and obtain strong matching feature points based on the intersection of the FLANN matching and pyramid LK optical flow calculation results;

[0018] The key frame extraction unit is used to solve the vanishing point based on the strong matching feature points and assign weights to the surrounding points within the preset range of the vanishing point, construct a robust indicator of the similarity of adjacent frames, obtain the similarity of adjacent frames, and realize the key frame screening of the video stream.

[0019] In a third aspect, the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned video three-dimensional reconstruction key frame extraction method based on mutual verification weighted matching.

[0020] In a fourth aspect, the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the aforementioned method for extracting key frames for three-dimensional reconstruction of video based on mutual verification weighted matching.

[0021] The beneficial effects of the present invention are:

[0022] The present invention uses the data of the terminal gyroscope as a reference, and by segmenting the scene, classifies and evaluates the image features in different environments, thereby improving the detection accuracy and ensuring the robustness of the algorithm in diverse scenes;

[0023] The present invention extracts and describes the SIFT feature points of the image, comprehensively considers FLANN matching and pyramid LK optical flow method, adopts the idea of ​​mutual verification to eliminate matching point pairs with poor quality, and improves the accuracy and stability of matching detection;

[0024] Considering the differences in pixel contribution values ​​in different scenes, a flexible strategy is adopted in similarity calculation to ensure more accurate identification of key frames. Among them, the contribution value of a pixel point to the similarity is usually related to the Euclidean distance from the point to the vanishing point. Therefore, after completing the scene pre-classification, the image vanishing point is calculated, and with this as the center, a flexible weighting strategy is applied to the surrounding pixels according to different scenes. Through weighted calculation, a more robust similarity index is constructed, which improves the robustness of the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 The present invention is a flow chart of a method for extracting key frames of video 3D reconstruction based on mutual verification weighted matching. DETAILED DESCRIPTION

[0026] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0027] In view of the problems in the extraction of key frames of video streams, such as the principle of similarity judgment of adjacent frames is difficult to apply to the current scene, the acquired data is redundant or missing, and the key frame images do not meet the requirements of three-dimensional reconstruction, the present invention proposes a key frame extraction method for video three-dimensional reconstruction based on mutual verification weighted matching. The video stream data shot by a handheld mobile terminal is used as input, and the acquired gyroscope data is used to pre-classify the image scene. The SIFT detection algorithm is used to solve the image feature points and descriptors. The FLANN matching and pyramid LK optical flow are used to capture the dynamic changes of adjacent frame matching points. Strong matching points are obtained by intersection screening and elimination, and the vanishing points are further solved and weights are assigned to the surrounding points as a robust judgment indicator of adjacent frame similarity. The similarity of adjacent frames is obtained to realize the screening of video stream key frames. Specifically, the following steps are included:

[0028] Step 1: Obtain an input video stream and pre-classify the image scenes included in the input video stream;

[0029] During the video shooting process, the camera passes through multiple scenes, and the video frames in different scenes change differently. The present invention focuses on the image changes of turns and straight roads. On straight roads, the edge pixels of the image change, and when turning, the whole picture changes. In order to effectively extract feature points and obtain the similarity of adjacent frames, these differences need to be classified and calculated. According to the video collected by the mobile terminal, the gyroscope data is used to calculate and perform effective image scene classification.

[0030] When the user rotates the device, the gyroscope can record the angular velocity data of the handheld device on each coordinate axis. When turning, the handheld device mainly rotates around the y-axis. Combining the y-direction angular velocity data and the terminal timestamp, the angle in this direction is integrated:

[0031] ,

[0032] in, is the current azimuth, is the sensor y-direction angular velocity data, are the current and starting timestamps respectively, and the initial azimuth is set to 0. A linear regression analysis is performed on the scatter plot, and the best fitting line is obtained by estimating the data points by least squares:

[0033] ,

[0034] in, is the curve fitting value, which has two unknown parameters. represents the slope of the fitting curve, Represents the intercept. Take the self-collected video as input, extract and input N groups of gyroscope data obtained by measuring the gyroscope sensor accuracy, etc. , Indicates the timestamp, Indicates the angular acceleration value of the gyroscope in the direction recorded at the corresponding timestamp. The least squares method is used to estimate the parameters, requiring the sensor observation value The weighted sum of squares of the deviations is minimized, and then the slope of the fitting curve is obtained. , according to the slope of the fitting curve The image scene is pre-classified. When the broken line has an inflection point, that is, the slope is larger, it means that the image is at a road bend, and when it is relatively stable, that is, the slope tends to 0, it corresponds to a straight line position. The adjacent frames of the video are obtained in sequence, and the video frame is judged according to the above calculation whether it is at a road bend, and the bend road sequence diagram and the straight road sequence diagram are divided as the input for subsequent calculations.

[0035] Step 2: Use the SIFT detection algorithm to solve the image feature points and descriptors, capture the dynamic changes of matching points in adjacent frames based on FLANN matching and pyramid LK optical flow calculation, and obtain strong matching points based on the intersection of FLANN matching and pyramid LK optical flow calculation results;

[0036] First, SIFT operators of adjacent frames are extracted, and the two technologies of image FLANN matching and pyramid LK optical flow calculation are combined to obtain robust matching point pairs of adjacent frames with high efficiency. The specific implementation is as follows:

[0037] Step 2.1: Detect feature points based on SIFT detection algorithm; including:

[0038] Step 2.1.1, construct the Gaussian difference scale space and determine the potential feature extreme points in the scale space;

[0039] The SIFT algorithm uses the Gaussian difference operator (DoG), and the convolution kernel uses the difference of the Gaussian kernel to construct the scale space, which is expressed as:

[0040] ,

[0041] in, is the Gaussian linear kernel, is the original image, represents the convolution operation, , is the intermediate parameter, i.e. the scale space function of the image. and is the scale factor of the two connected images. The scale of the upper image in each group is the scale of the lower image. times, , here Indicates the number of layers necessary for extreme value extraction.

[0042] Construct a Gaussian pyramid:

[0043] a. Use Gaussian kernels of different scales to smooth the underlying images of the same size to build each layer in the group.

[0044] b. Downsample to get the next set of underlying images and repeat step a.

[0045] Gaussian pyramid is composed of multiple (Group) Image composition (According to experience, the number of groups , are the sizes of the two sides), each group has layers, the upper layer in the group is scaled by the lower layer times, The scale of the bottom image of the group For the The third-to-last image of the group The Gaussian difference scale space is formed by subtracting two images close to each other in the Gaussian pyramid, and the points to be analyzed and the corresponding points in the adjacent upper and lower layers are detected. points and the same layer A point, if it has the maximum value, is a potential characteristic extreme point.

[0046] Step 2.1.2: Use a three-dimensional quadratic function to perform curve fitting on discrete potential feature extreme points to achieve sub-pixel accuracy, eliminate low-contrast potential feature extreme points, and obtain accurate positioning of feature points.

[0047] The potential feature extreme points are calculated values ​​of discrete coordinates, which have low accuracy. Three-dimensional quadratic functions are used for curve fitting to achieve sub-pixel accuracy:

[0048] a. Eliminate low-contrast potential feature extreme points.

[0049] b. Downsample to get the next set of underlying images and repeat step a.

[0050] The Gaussian linear kernel of the potential feature extreme points Detecting positions at extreme points Taylor second-order expansion , Indicates exact location Detecting positions relative to extreme points The offset includes the point difference and scale difference. Derivative, when the first derivative is zero, find the exact position : , , and then remove the low-contrast potential feature extreme points.

[0051] Potential feature extreme points The Hessian matrix is , For traces, is the determinant. If it does not meet , take the intermediate parameter , then there is an obvious gap between the principal curvatures of the potential characteristic extreme points in the two directions, which are unstable edge response points that need to be eliminated.

[0052] Step 2.1.3: Use the precise positioning results of feature points to construct a gradient modulus histogram and determine the main direction of the feature points.

[0053] If the scale of the target Gaussian image is , locate the center of the circle where the feature point is located, perform gradient information statistics on the pixels around the feature point, and select a radius of , the gradient information includes the gradient modulus and the gradient direction , For feature points After completing the feature point gradient calculation, a gradient modulus histogram is constructed, and the angle corresponding to the maximum value of the accumulated result is taken as the main direction of the feature point.

[0054] Step 2.1.4, construct the feature point descriptor after accurate positioning;

[0055] Rotate the image to the primary direction On the other hand, the field is divided into sub-regions, calculate the cumulative sum of the 8 main direction gradient values ​​of the neighborhood points, and the SIFT descriptor is expressed as dimensional feature vector. Finally, normalization is performed:

[0056] ,

[0057] in, is the 128-dimensional feature vector of the SIFT descriptor calculated previously, is the normalized feature vector.

[0058] Step 2.2, based on the feature point descriptors, the FLANN matching algorithm is used to dynamically match adjacent frames;

[0059] The FLANN index is constructed based on the image feature descriptors of the precisely located feature points. KD-Tree is selected to accelerate the nearest neighbor matching search and compare the distances between the nearest neighbor and the next nearest neighbor for screening. The feature point descriptors are divided into clusters, assuming the input set T, K-means is implemented by minimizing the square error of the clusters ,in For the points within a cluster, For the Cluster center.

[0060] Step 2.3, based on the feature point descriptor, the pyramid LK optical flow method is used to calculate the motion vector of each pixel in adjacent frames;

[0061] The 0th layer in the pyramid is the original image , the Fth layer will reduce the width and height times , initialize the top-level image optical flow , calculate the optical flow from top to bottom, and infer the initial optical flow of the current layer F in the x and y directions based on the calculation results of the previous layer F+1: , let the adjacent frames of layer F be , solve the optical flow of layer F , defines the image at the Fth level of the pyramid, the amount of light loss The error in the neighborhood can be minimized, and its loss function is ,satisfy ,in is the neighborhood window, is the pixel position. Provides the initial value for optical flow calculation for the next layer , and then get the final optical flow .

[0062] Step 2.4, mutual verification matching is performed based on the FLANN matching algorithm and the pyramid LK optical flow calculation method;

[0063] The first set of matching point groups is obtained by using the pyramid LK optical flow method to initialize the motion vector. The second set of matching point groups is obtained by searching the feature points of two adjacent frames using the FLANN matching algorithm. The intersection of the first and second matching point groups is taken to obtain a set of matching point pairs. The Euclidean distance is used to quantify each set of matching point pairs in the set of matching point pairs. The similarity is calculated and the threshold is set to remove the error points to obtain the optimized strong matching feature points. The SIFT feature point descriptors of two adjacent frames contain information:

[0064] ,

[0065] in, are the feature point coordinates, is the normalized feature vector. Use the pyramid LK optical flow method to query the feature points of the previous frame image The point group is obtained by moving the position in the next frame ; Use the FLANN matching algorithm to search for feature points in two frames of images Successfully matched point groups :

[0066] ,

[0067] The point groups calculated for adjacent video frames and The points are both on the next frame of the image. The intersection of the two points is calculated, and the Euclidean distance of the corresponding points is less than a certain threshold. , it is considered to be a strong matching feature point :

[0068] ,

[0069] Step 3: Calculate the vanishing point and assign weights to the surrounding points within the preset range of the vanishing point. Based on the strong matching points, a robust index of adjacent frame similarity is constructed to obtain the adjacent frame similarity and implement key frame screening of the video stream.

[0070] After completing the pre-classification of the scene, different processing methods are adopted for the feature points. In the case of a straight road, the vanishing point of the image is first calculated. The picture changes little in the area near the vanishing point. With the vanishing point as the center, Gaussian distribution weights are assigned to the surrounding strong matching feature points. In the case of a curved road, since the entire picture is changing, uniform distribution weights are used for all strong matching feature points.

[0071] When calculating the vanishing point, first use the LSD algorithm to detect lines. Usually, in order to generate a vanishing point hypothesis, at least two line segments need to intersect. A vanishing point can be obtained by intersecting any line segment with the vanishing line. Define the geographic distance to determine the inner point. ,in, Assume that the vanishing point is is the distance in radians, is a set of line segments Then, using the dot product, a least squares solution to minimize the cost function is proposed to solve the vanishing point, where is the vanishing point after fine-tuning, , is the acceleration, and the formula is , using parallel line segment elements and vertical line elements Finally, the image vanishing point is calculated.

[0072] On a straight road, the optical flow movement is proportional to the Euclidean distance from the vanishing point. On a curved road, the entire image continues to change, and each feature point moves in a different direction. For a straight road image, with the vanishing point as the center, a Gaussian distribution function is used to weight the optical flow of the feature points: ,in is the Euclidean distance between the current pixel and the vanishing point, is the maximum Euclidean distance between the pixel and the vanishing point is the variance; for the turning road image, the uniform distribution function is used for weighting, and the weight is 1.

[0073] Calculate the weighted optical flow changes of matching points in adjacent frames, and finally construct a robust similarity index :

[0074] ,

[0075] in, Calculated by mutual verification matching of adjacent frames Group strong matching feature points, For the The optical flow calculation weights of the group points, and Respectively The pixel positions of the group points in the previous and next frames.

[0076] The above steps can complete the similarity calculation between a group of adjacent frames, obtain the similarity of all adjacent frames of the self-collected video, and when it is detected that there are still unmatched adjacent video frames in the video frame, repeat the previous operation until the similarity of all adjacent frames is calculated; then set the first frame of the video as the key frame, and obtain the next video frame information in turn. According to the similarity value calculated previously, determine whether it is less than the threshold. If it is less than the threshold, keep the current video frame as the key frame and add it to the video stream key frame set. Otherwise, discard the current frame and repeat the operation until the video frame detection is completed, and finally obtain the final video stream key frame set. Through the above steps, an efficient and reliable key frame extraction process is constructed, making subsequent applications and visual scene reconstruction more accurate and efficient.

[0077] On the other hand, the present invention provides a video 3D reconstruction key frame extraction device based on mutual verification weighted matching, and each unit included in the device can implement the above-mentioned method steps, specifically including:

[0078] A preprocessing unit, used for acquiring an input video stream and pre-classifying image scenes included in the input video stream;

[0079] The feature point matching unit is used to solve the image feature points and descriptors using the SIFT detection algorithm, capture the dynamic changes of the matching points of adjacent frames based on FLANN matching and pyramid LK optical flow calculation, and obtain strong matching feature points based on the intersection of the FLANN matching and pyramid LK optical flow calculation results;

[0080] The key frame extraction unit is used to solve the vanishing point based on the strong matching feature points and assign weights to the surrounding points within the preset range of the vanishing point, construct a robust indicator of the similarity of adjacent frames, obtain the similarity of adjacent frames, and realize the key frame screening of the video stream.

[0081] In a third aspect, the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned video three-dimensional reconstruction key frame extraction method based on mutual verification weighted matching.

[0082] In a fourth aspect, the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the aforementioned method for extracting key frames for three-dimensional reconstruction of video based on mutual verification weighted matching.

[0083] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for extracting key frames for video 3D reconstruction based on mutual verification weighted matching, characterized in that: The method comprises: Step 1: Obtain an input video stream and pre-classify the image scenes included in the input video stream; Step 2: Use the SIFT detection algorithm to solve the image feature points and descriptors, capture the dynamic changes of the matching points of adjacent frames based on FLANN matching and pyramid LK optical flow calculation, and obtain strong matching feature points based on the intersection of FLANN matching and pyramid LK optical flow calculation results; Step 3: Based on the strongly matched feature points, the vanishing point is solved and weights are assigned to the surrounding points within the preset range of the vanishing point, a robust index of adjacent frame similarity is constructed, the adjacent frame similarity is obtained, and the key frame screening of the video stream is realized.

2. The method for extracting key frames for video 3D reconstruction based on mutual verification weighted matching according to claim 1, characterized in that: The step 1 comprises: fitting the input video stream based on the least square method, and dividing the image scenes included in the input video stream into a turning road sequence diagram and a straight road sequence diagram.

3. The method for extracting key frames for video 3D reconstruction based on mutual verification weighted matching according to claim 1, characterized in that: The step 2 comprises: Step 2.1, perform feature point detection based on SIFT detection algorithm to obtain feature point descriptors; Step 2.2, based on the feature point descriptors, the FLANN matching algorithm is used to dynamically match adjacent frames; Step 2.3, based on the feature point descriptor, the pyramid LK optical flow method is used to calculate the motion vector of each pixel in adjacent frames; Step 2.4: Perform mutual verification matching based on the FLANN matching algorithm and the pyramid LK optical flow calculation method.

4. The method for extracting key frames for video 3D reconstruction based on mutual verification weighted matching according to claim 1, characterized in that: The step 3 comprises: after completing the pre-classification of the image scene, respectively performing vanishing point calculation and weighting for the straight road sequence diagram and the turning road sequence diagram.

5. The method for extracting key frames for video 3D reconstruction based on mutual verification weighted matching according to claim 2, characterized in that: The step 2.1 comprises: Step 2.1.1, construct the Gaussian difference scale space and determine the potential feature extreme points in the scale space; Step 2.1.2, use a three-dimensional quadratic function to perform curve fitting on discrete potential feature extreme points to achieve sub-pixel accuracy, remove low-contrast potential feature extreme points, and obtain accurate positioning of feature points; Step 2.1.3, using the precise positioning results of the feature points to construct a gradient modulus histogram and determine the main direction of the feature points; Step 2.1.4: Construct the feature point descriptor after precise positioning.

6. The method for extracting key frames for video 3D reconstruction based on mutual verification weighted matching according to claim 3, characterized in that: The step 2.4 includes using the pyramid LK optical flow method to initialize the motion vector to obtain a first group of matching point groups, using the FLANN matching algorithm to search for feature points of two adjacent frames to obtain a second group of matching point groups, taking the intersection of the first group of matching point groups and the second group of matching point groups to obtain a set of matching point pairs, using the Euclidean distance to quantify each group of matching point pairs in the set of matching point pairs, calculating the similarity and setting a threshold to eliminate erroneous points, and obtaining optimized strong matching feature points.

7. The method for extracting key frames for video 3D reconstruction based on mutual verification weighted matching according to claim 4, characterized in that: The step 3 specifically includes: after completing the pre-classification of the image scene, for the straight road sequence diagram, calculating the vanishing point of the image, taking the vanishing point as the center, weighting the strong matching feature points within a preset range around it with Gaussian distribution; for the turning road sequence diagram, weighting all the strong matching feature points with uniform distribution; calculating the weighted optical flow changes of the matching point pairs of adjacent frames to obtain a similarity robust index.

8. A device for extracting key frames for video 3D reconstruction based on mutual verification weighted matching, characterized in that: include: A preprocessing unit, used for acquiring an input video stream and pre-classifying image scenes included in the input video stream; The feature point matching unit is used to solve the image feature points and descriptors using the SIFT detection algorithm, capture the dynamic changes of the matching points of adjacent frames based on FLANN matching and pyramid LK optical flow calculation, and obtain strong matching feature points based on the intersection of the FLANN matching and pyramid LK optical flow calculation results; The key frame extraction unit is used to solve the vanishing point based on the strong matching feature points and assign weights to the surrounding points within the preset range of the vanishing point, construct a robust indicator of the similarity of adjacent frames, obtain the similarity of adjacent frames, and realize the key frame screening of the video stream.

9. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; Wherein, when one or more programs are executed by the one or more processors, the one or more processors implement the video three-dimensional reconstruction key frame extraction method based on mutual verification weighted matching as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: Executable instructions are stored thereon, and when the instructions are executed by the processor, the processor can implement the video three-dimensional reconstruction key frame extraction method based on mutual verification weighted matching as described in any one of claims 1-7.

Images