Image registration method and device based on multi-frame fusion

Abstract

The invention provides an image registration method and device based on multi-frame fusion. The method provided by the invention comprises the following steps: acquiring a wide-field-of-view reference image and continuous multiple narrow-field-of-view image frames; performing region cutting on the wide-view-field reference image to obtain a cut wide-view-field local region image; key points and descriptors are extracted from the cut wide-view-field local area image and each narrow-view-field image frame; matching the wide-view-field key points and the wide-view-field descriptors with the narrow-view-field key points and the narrow-view-field descriptors of each narrow-view-field image frame to obtain multiple groups of matching point pairs and corresponding confidence scores; recombining and integrating the multiple groups of matching points based on the confidence score to obtain a recombined feature point set; and constructing a fixed-length container to store a recombined feature point set, splicing all recombined feature points in the fixed-length container and feature points of the wide-view-field reference image, solving a homography transformation matrix from the narrow-view-field image to the wide-view-field image, and carrying out image registration based on the homography transformation matrix.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image registration, in particular to a multi-frame fusion-based image registration method and device. BACKGROUND

[0002] In the field of image registration, since the wide-view camera and the narrow-view camera are usually installed at the same position, the images obtained by the two cameras have high consistency in view angle. In order to realize the spatial information correlation of the two types of images, the narrow-view image and the wide-view image need to be spatially aligned through image registration, which is the basis for subsequent utilization of complementary information (such as the global view of the wide-view camera and the local details of the narrow-view camera) of the two types of images.

[0003] At present, the mainstream image registration method is based on feature point matching and homography transformation, that is, feature points (including key points and corresponding descriptors) in the image are extracted and matched, and a homography transformation matrix is solved by using the matched point pairs to complete the spatial mapping. However, these methods have significant limitations: in a sparse texture scene (such as grass or a wall), the number of feature points available for matching is extremely small, resulting in unstable estimation of the homography transformation matrix; the inherent differences in focal length and distortion parameters between the wide-view camera and the narrow-view camera will introduce feature distribution deviation and increase the matching difficulty; when facing a textureless region or an image pair with significant parallax, the single-frame feature matching performance will decrease sharply; and in the multi-frame registration process, jitter phenomenon is easy to occur, which seriously affects the accuracy of subsequent operations such as boundary box mapping.

[0004] Therefore, there is an urgent need for a method to solve the problem of insufficient registration robustness of existing methods in sparse texture, camera parameter difference and complex scene, and to improve the stability and accuracy of narrow-view image and wide-view image registration. SUMMARY

[0005] Therefore, the present application provides a multi-frame fusion-based image registration method and device to solve the problem of insufficient registration robustness of existing methods in sparse texture, camera parameter difference and complex scene, and to improve the stability and accuracy of narrow-view image and wide-view image registration.

[0006] Specifically, the present application is implemented by the following technical solutions:

[0007] The first aspect of the present application provides a multi-frame fusion-based image registration method, which comprises:

[0008] obtaining a wide-view reference image and a plurality of continuous narrow-view image frames;

[0009] regionally cropping the wide-view reference image to obtain a cropped wide-view local region image;

[0010] extract wide field key points and corresponding wide field descriptors from the cropped wide field local region images respectively, and extract narrow field key points and corresponding narrow field descriptors from each frame of the narrow field image frames;

[0011] match the wide field key points, the wide field descriptors, the narrow field key points and the narrow field descriptors of each frame of the narrow field image frames to obtain a plurality of groups of matching point pairs and corresponding confidence scores;

[0012] recombine and integrate the plurality of groups of matching points based on the confidence scores to obtain a recombined feature point set;

[0013] construct a fixed-length container to store the recombined feature point set, splice all recombined feature points in the fixed-length container with feature points of the wide field reference image, solve a homographic transformation matrix of the narrow field image to the wide field image, and perform image registration based on the homographic transformation matrix.

[0014] The second aspect of the application provides an image registration device based on multi-frame fusion, which comprises an acquisition module, a cropping module, an extraction module, a matching module, a recombination module and a solving module;

[0015] The acquisition module is configured to acquire a wide field reference image and a plurality of continuous narrow field image frames.

[0016] The cropping module is configured to crop a region of the wide field reference image to obtain a cropped wide field local region image.

[0017] The extraction module is configured to extract wide field key points and corresponding wide field descriptors from the cropped wide field local region images respectively, and extract narrow field key points and corresponding narrow field descriptors from each frame of the narrow field image frames.

[0018] The matching module is configured to match the wide field key points, the wide field descriptors, the narrow field key points and the narrow field descriptors of each frame of the narrow field image frames to obtain a plurality of groups of matching point pairs and corresponding confidence scores.

[0019] The recombination module is configured to recombine and integrate the plurality of groups of matching points based on the confidence scores to obtain a recombined feature point set.

[0020] The solving module is configured to construct a fixed-length container to store the recombined feature point set, splice all recombined feature points in the fixed-length container with feature points of the wide field reference image, solve a homographic transformation matrix of the narrow field image to the wide field image, and perform multi-frame image registration based on the homographic transformation matrix.

[0021] The image registration method and device based on multi-frame fusion provided in this application utilize the core concept of multi-frame fusion to organically connect the various steps from data acquisition to registration matrix solution, achieving efficient and accurate image registration. First, multiple consecutive narrow-field-of-view image frames are acquired rather than a single frame, laying the foundation for subsequent multi-frame information fusion. Compared to single-frame registration, this method can introduce redundant information in the temporal dimension, reducing matching errors caused by noise, occlusion, or feature loss in a single frame. A wide-field-of-view reference image is simultaneously acquired as a registration benchmark, clarifying the target coordinate system for registration. Second, the wide-field-of-view reference image is cropped to process only the local region relevant to the narrow-field-of-view image. This reduces the feature extraction and matching computational complexity of irrelevant regions, ensuring registration relevance while reducing the system computational load and improving processing efficiency. Next, key points and descriptors are extracted from the wide-field-of-view local region and each narrow-field-of-view frame, providing basic feature information for feature matching. A confidence score is introduced during matching, allowing subsequent reassembly to filter out more reliable matching point pairs from multiple frames based on the score, eliminating low-confidence mismatches and improving matching accuracy from the source. Through confidence-based reassembly and integration, matching point pairs from multiple narrow-FOV image frames are fused, aggregating stable, high-confidence feature matching results across multiple frames. This compensates for the potential shortage of matching points or local mismatches in a single frame, increasing the total number and quality of valid matching point pairs. A fixed-length container is then constructed to store the reassembled feature point set. This fixed-length design allows the number of frames stored to be controlled based on the system's computational load, avoiding computational pressure caused by excessive data volume. Furthermore, chronological storage ensures the use of multiple frames of sequentially continuous information, providing temporally consistent feature data for subsequent stitching. Finally, all reassembled feature points in the fixed-length container are concatenated with the wide-FOV feature points. The homography transformation matrix is ​​solved using the sufficient and reliable matching point pairs obtained from multi-frame fusion. Compared to solving the matrix using matching point pairs from a single frame, the greater number of valid point pairs provides richer constraint information for the matrix solution. Combined with an adaptive solution method, outliers can be further eliminated. The resulting homography matrix more accurately represents the spatial mapping relationship between the narrow-FOV and wide-FOV images, enabling high-precision and robust image registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Flowchart of the image registration method based on multi-frame fusion provided in Example 1 of the present application;

[0023] Figure 2 This is a structural diagram of the image registration device based on multi-frame fusion provided in Example 2 of the present application. DETAILED DESCRIPTION

[0024] The exemplary embodiments will be described in detail herein with reference to several drawings. Descriptions of well-known functions and structures are omitted so as not to unnecessarily obscure the application. The examples described herein represent the best

[0025] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0026] It is to be understood that the singular forms "a", "an", and "the" include plural referents unless the context clearly dictates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0027] The specific embodiments are given below to introduce the technical solutions of the application in detail.

[0028] Figure 1 The flow chart of the image registration method based on multi-frame fusion provided by Embodiment One of the application is shown in FIG. 1. Please refer to Figure 1 The method provided by the embodiment can include:

[0029] S101, acquire a wide field reference image and a plurality of continuous narrow field image frames.

[0030] Specifically, the wide field reference image refers to an image with a large angle range acquired by a wide field camera, which serves as the spatial reference of the entire registration process. The wide field reference image contains global scene information and can provide a unified spatial coordinate framework, which is the "reference standard" for the spatial alignment of the narrow field image. The plurality of continuous narrow field image frames refer to a sequence of images with a small angle but high details acquired by a narrow field camera. The narrow field image frames contain dynamic change information of local scenes and can capture the detailed features and motion trajectories of the target.

[0031] It should be noted that the wide field of view reference image and the continuous multiple frames of narrow field of view image frames are in a "reference and dynamic object" relationship, which is specifically manifested as follows: the two are collected by different cameras installed at the same position, have high consistency in view angle, and provide a basis for spatial alignment through homography transformation; the wide field of view reference image provides a spatial reference of the global scene, and the narrow field of view image frames provide dynamic information of local details, and the two realize information fusion of the global and the local through registration; the continuous multiple frames of narrow field of view image frames need to be spatially aligned based on the wide field of view reference image, and finally the local features of the narrow field of view image are mapped into the global coordinates of the wide field of view through a homography transformation matrix, so as to realize spatial unification of the two types of images and provide a prerequisite for subsequent processing (such as bounding box mapping and multi-frame feature stitching).

[0032] In specific implementation, the wide field of view camera and the narrow field of view camera are started, the consistency of the acquisition timing is ensured through a hardware trigger or a timestamp synchronization mechanism, and it is ensured that the wide field of view camera and the narrow field of view camera are in the same installation position and are in a synchronous working state. The wide field of view camera is controlled to collect a single image covering the global range of the target scene, which is defined as a wide field of view reference image, stored as a digital image file in a preset format (such as JPEG, PNG), and the storage path is associated with a designated reference image database. The narrow field of view camera is controlled to continuously collect local detail images of the target scene at a fixed frame rate, generating a continuous sequence of multiple frames of narrow field of view images, and each image is named according to the acquisition timestamp and sequentially stored in a timing image buffer area.

[0033] S102, regionally cropping the wide field of view reference image to obtain a cropped wide field of view local area image.

[0034] In specific implementation, the regionally cropping the wide field of view reference image to obtain a cropped wide field of view local area image includes: taking four corner points of a narrow field of view image as original corner points, mapping the original corner points to the wide field of view reference image through a preset initial homography transformation relationship to obtain four corresponding mapping corner points in the wide field of view reference image; the narrow field of view image is a certain frame in the continuous multiple frames of narrow field of view images, and the initial homography transformation relationship is used to preliminarily determine the spatial overlap range of the narrow field of view and the wide field of view images; a closed region is defined in the wide field of view reference image based on the four mapping corner points, and the closed region is a region of interest to be cropped; the wide field of view reference image is cropped based on the coordinate range of the four mapping corner points, the image content in the region of interest is retained, the image part outside the region of interest is removed, and a cropped wide field of view local area image is obtained.

[0035] Specifically, a certain frame is selected from continuous multiple narrow field of view image frames as a current processing object, four corner points (i.e. four vertices of the image boundary) of the frame image are automatically recognized by an image processing tool, and pixel coordinate values thereof are obtained as original corner points. An initial homographic transformation relationship is determined by camera calibration or historical registration data, the coordinate values of the four original corner points are input into the transformation relationship, and the mapping coordinates of each original corner point in the wide field of view reference image are calculated to form four mapping corner points. Further, a closed quadrilateral region is drawn on the wide field of view reference image with the four mapping corner points as the vertices. The minimum circumscribed rectangle (i.e. the smallest rectangular region containing all the mapping corner points) of the quadrilateral region is calculated, the coordinates of the upper left corner and the lower right corner of the rectangle are determined, and the region of interest to be cropped is obtained. Using a cropping function provided by an image processing library, the wide field of view reference image is subjected to a pixel-level cropping operation according to the coordinate range of the region of interest, all pixel data within the coordinate range is retained, and pixel data outside the range is discarded, and finally a wide field of view local region image containing only the content of the region of interest is generated. The specific implementation process and implementation principle of determining the initial homographic transformation relationship can be referred to the description in the related art, which will not be described here.

[0036] For example, in an embodiment, the four original corner points of the selected narrow field of view image frame are , and the corresponding four mapping corner points in the wide field of view reference image after mapping are .

[0037] ;

[0038] wherein the four mapping corner points are , the four original corner points are , and the initial homographic transformation relationship is .

[0039] The method provided in this embodiment can focus on the effective region that overlaps with the narrow field of view image frame by regionally cropping the wide field of view reference image, and reduce the interference of irrelevant regions. Since the wide field of view reference image covers the global scene, and the narrow field of view image frame only captures local details, the effective matching region of the two is limited to the overlapping part. By cropping, the redundant region in the wide field of view image that is irrelevant to the narrow field of view image can be removed, the range of subsequent feature point extraction and matching is reduced, the invalid calculation amount is reduced, the efficiency of feature matching is improved, and meanwhile, limiting the matching region can reduce the false matching caused by the dispersed distribution of feature points in the global range, enhance the pertinence of the association of the wide field of view and the narrow field of view feature points, provide a more accurate regional basis for subsequent similarity calculation and matching point pair screening, and further improve the accuracy and robustness of the overall registration process.

[0040] S103, extract wide field key points and corresponding wide field descriptors from the cropped wide field local region images, and extract narrow field key points and corresponding narrow field descriptors from each frame of the narrow field image frames.

[0041] Specifically, the wide field key points are pixel points extracted from the cropped wide field local region images, which have uniqueness and stability, and are usually positions with significant features in the images, such as edge intersection points, corner points, texture mutation points, etc. These points can remain relatively stable in image transformation (such as scaling, rotation, illumination change), and can be used as "feature markers" of the local region images for subsequent matching with the narrow field images. The wide field descriptor is a quantitative expression of the local image information around the wide field key point, and is a set of numerical vectors (or data structures) that can describe the features of the key point neighborhood. It converts visual features into computable and comparable digital forms by extracting the gray value distribution, gradient direction, texture pattern, etc. of the pixels around the wide field key point, so as to perform similarity measurement with the narrow field descriptor in the subsequent steps, and realize the matching of the key points.

[0042] Further, the narrow field key points are pixel points extracted from each frame of the narrow field image frames, which have uniqueness and stability, and are essentially the same as the wide field key points, and are also positions with significant features such as edges, corners, and texture mutations in the images. Since the narrow field images focus on local details, the key points more reflect the subtle features of local regions in the scene, and are the core "feature markers" of the narrow field images. The narrow field descriptor is a quantitative expression of the local image information around the narrow field key point, and has the same logical definition as the wide field descriptor, and is a numerical vector (or data structure) generated based on the gray, gradient, texture, etc. features of the neighborhood of the narrow field key point. Its role is to convert the visual features of the narrow field key points into computable digital forms, so as to compare with the wide field descriptor, and thus determine the correspondence between the wide field local region images and the narrow field image frames.

[0043] In a specific implementation, extracting wide field key points and corresponding wide field descriptors from the cropped wide field local region images, and extracting narrow field key points and corresponding narrow field descriptors from each frame of the narrow field image frames, includes: using a feature point extractor to detect and extract wide field key points from the cropped wide field local region images, and simultaneously generate a wide field descriptor corresponding to each wide field key point; the wide field descriptor represents the local feature information of the wide field key point; using the same feature point extractor to detect and extract narrow field key points from each frame of the narrow field image frames, and simultaneously generate a narrow field descriptor corresponding to each narrow field key point; wherein the wide field key points and the narrow field key points are corresponding points with a mapping relationship in space.

[0044] It should be noted that in the embodiment, the feature point extractor is used for both the extraction of the wide field of view key points and the extraction of the narrow field of view key points. Therefore, only the specific implementation process of the extraction of the wide field of view key points is introduced, and the extraction of the narrow field of view key points is similar, which is not described herein again.

[0045] The feature point extractor is used to detect and extract the wide field of view key points from the cropped wide field of view local region image, and generate a wide field of view descriptor corresponding to each wide field of view key point, including: processing the cropped wide field of view local region image based on a super point algorithm, detecting a target pixel point in the wide field of view local region image as an initial wide field of view key point through a convolutional neural network architecture, and generating a basic descriptor corresponding to the initial wide field of view key point; the gray scale distribution, neighborhood structure and difference from surrounding pixels of the target pixel point are greater than a preset value, and the target pixel point can be stably detected when the image is scaled, rotated or light changed; inputting the initial wide field of view key point and the basic descriptor into a general image matcher, the general image matcher eliminating false key points susceptible to noise interference based on the stability of the initial wide field of view key point, and enhancing the features of the basic descriptor corresponding to the remaining stable initial wide field of view key point to obtain the final wide field of view key point and the corresponding wide field of view descriptor.

[0046] In implementation, the cropped wide-field-of-view local region image is fed into a feature point extractor based on the SuperPoint algorithm, which employs a convolutional neural network (CNN) architecture. The CNN extracts features and reduces dimensionality from the image through multiple convolutional and pooling layers, generating a feature pyramid. At the top of the feature pyramid, the classifier branch performs binary classification on each pixel in the wide-field-of-view local region image. A softmax function is used to output a probability value for each pixel belonging to a keypoint. Based on this probability value, the keypoint is determined. Pixels with probabilities greater than a preset threshold (e.g., 0.01) are selected as initial wide-field-of-view keypoints. Non-maximum suppression is used to ensure spatial uniqueness of keypoints, retaining only those with the highest local probability. Another branch of the CNN (the descriptor branch) generates a basic descriptor for each detected keypoint. The descriptor branch extracts features from the patch surrounding the keypoint (e.g., an 8×8 or 16×16 pixel area) through convolution operations and maps these features into a fixed-dimensional vector (e.g., 256). The resulting vector is normalized to obtain a basic descriptor for each initial wide-field-of-view keypoint. Furthermore, the initial wide-FOV keypoints and basic descriptors are fed into a general image matcher (GIM). After slightly perturbing the image (e.g., rotating or scaling), the GIM redetects keypoints and calculates the repetition rate based on the proportion of points detected before and after the perturbation. The gradient direction histogram of the keypoint neighborhood is analyzed to assess its stability under different lighting or viewing angles. The probability value at the time of keypoint detection is used as the initial score, and points with too low response strength are removed. A stability threshold (e.g., repetition rate > 0.7) is set to remove pseudo keypoints below the stability threshold. The retained basic descriptors are then quadratically normalized and compressed to 64 or 128 dimensions using a linear mapping (e.g., a fully connected layer) to improve matching efficiency. The Transformer architecture is used to analyze the spatial relationships between keypoints. The descriptor vector is adjusted through a self-attention mechanism to enhance its ability to express local context, resulting in the final wide-FOV keypoints and corresponding wide-FOV descriptors.

[0047] For example, in one embodiment, the wide field of view key points and the corresponding wide field of view descriptors are , the narrow field of view key points and the corresponding narrow field of view descriptors are .

[0048] ;

[0049] ;

[0050] Among them, the is a key point of wide field of view; is a wide field of view descriptor; is the cropped wide field of view local area image; is a narrow field key point; the is a narrow field descriptor; the is a narrow field image frame for each frame.

[0051] The method provided by the embodiment extracts wide field and narrow field key points in combination with two-step processing of a Superpoint algorithm (Superpoint) and a general image matcher (GIM). When the Superpoint detects initial key points through a convolutional neural network, target pixel points that are significantly different from the surrounding and stable in detection under scaling, rotation, and illumination change are deliberately selected according to the gray distribution and neighborhood structure, which preliminarily ensures that the initial key points have basic stability and distinguishability, provides potential feature candidates for registration, and avoids redundant calculation caused by extracting features from a large number of meaningless pixels. The GIM further eliminates false key points (such as misdetected points in a sparse texture area and unreal edge points caused by illumination mutation) that are susceptible to noise interference based on stability, and enhances the features of the descriptors. This improves the purity of the remaining key points, ensures that the key points participating in registration are stable points that truly represent the image content, and enhances the distinguishability of the descriptors, so that the descriptors of key points at different positions are more significantly different, and the descriptors at the same position are more consistent under changing conditions. This makes the matching success rate of correct corresponding points higher and the false matching rate lower in the subsequent feature matching process of wide field and narrow field images, and finally greatly improves the accuracy and robustness of image registration. At the same time, the calculation amount in the matching stage is reduced due to the elimination of redundant false key points, and the registration efficiency and effect are taken into account.

[0052] S104, match the wide field key points, wide field descriptors, narrow field key points, and narrow field descriptors of each narrow field image frame, to obtain a plurality of sets of matching point pairs and corresponding confidence scores.

[0053] Specifically, the plurality of sets of matching point pairs refer to a one-to-one correspondence relationship set formed after successful descriptor matching between the wide field key points of the wide field image and the narrow field key points of the narrow field image. Each set of matching point pairs represents that one wide field key point in the wide field image and one narrow field key point in the narrow field image have corresponding association in content. The plurality of sets of matching point pairs include the coordinates of the wide field key points, the coordinates of the corresponding narrow field key points, and the confidence scores corresponding to the matching point pairs.

[0054] In a specific implementation, the matching the wide field key points, the wide field descriptors, the narrow field key points and the narrow field descriptors of each frame of the narrow field image frames to obtain a plurality of groups of matching point pairs and corresponding confidence scores includes: determining feature point pairs in a spatial overlap region of each frame of the narrow field image frames and the cropped wide field local region image; each feature point pair includes a wide field descriptor and a narrow field descriptor; analyzing feature correlation of the wide field descriptors and the narrow field descriptors in the spatial overlap region based on a lightweight glue algorithm, screening candidate point pairs based on spatial positions of the feature point pairs and the feature correlation, and generating similarity of the candidate point pairs based on cosine similarity as initial confidence scores; inputting the candidate point pairs and the corresponding initial confidence scores into a general image matcher, the general image matcher eliminating false matching point pairs caused by differences in camera parameters based on consistency of viewing angles of the wide field and the narrow field images, calibrating initial confidence scores of the retained candidate point pairs to obtain a plurality of groups of matching point pairs and calibrated confidence scores.

[0055] Specifically, according to the spatial mapping relationship between the cropped wide field partial region image and each frame of narrow field image frame, the overlapping region range of the two is calculated; the points in the wide field key points whose coordinates are located in the overlapping region and the corresponding wide field descriptors are screened out, and all narrow field key points and their descriptors in the narrow field image frame form a feature point pair set. Further, the feature point pair set is input into the LightGlue algorithm, and the wide field descriptors and the narrow field descriptors are respectively input into two Transformer encoders to generate context-enhanced descriptors. A bipartite graph is constructed, the nodes are the wide field and narrow field key points, and the edge weight is initialized by the cosine similarity between the descriptors. The LightGlue algorithm iteratively updates the edge weight through the cross-attention mechanism, each wide field key point focuses on all narrow field key points, and the attention score is calculated. Each narrow field key point also focuses on all wide field key points. After multiple iterations (usually 3-5 times), the edge weight converges to a stable value to form a feature association matrix. Further, bidirectional matching is performed on the feature association matrix: for each wide field key point, the narrow field key point with the strongest association is selected as a candidate. If the strongest association of the narrow field key point also points to the same wide field key point, it is retained as a candidate point pair. The projection error of the candidate point pair is calculated, and the point pair with an error exceeding a threshold (such as 5 pixels) is removed. If the matching points in a certain region are too dense, the point pair with the strongest association is retained. For the candidate point pair selected through screening, the final edge weight in the feature association matrix is directly converted into an initial confidence score (range 0-1) as the cosine similarity. Further, the GIM algorithm is used to iteratively sample the candidate point pairs, and 4 pairs are randomly selected each time to calculate the homography matrix. The re-projection error of all point pairs under the current homography matrix is calculated, and the point pairs with an error less than a threshold (such as 3 pixels) are marked as inliers. Repeat the iteration (usually 100-200 times), select the optimal homography matrix with the most inliers, and remove the mismatched point pairs that are not marked as inliers. Calculate the number of other inliers within a radius r (such as 10 pixels) around each inlier as the local consistency score. Multiply the initial confidence score by the normalized value of the local consistency score to enhance the confidence of the matching points in the dense region, and obtain multiple matching point pairs and calibrated confidence scores.

[0056] For example, in an embodiment, the multiple matching point pairs and calibrated confidence scores can be represented as:

[0057] ;

[0058] wherein the is a matching point pair; the is a confidence score; the is a wide field key point; the is a wide field descriptor; the is a narrow field key point; the is a narrow field of view descriptor.

[0059] The method provided in this embodiment first filters feature point pairs by limiting the spatial overlapping area, retaining only feature points that may have a real correspondence between the local area of ​​the wide field of view and the narrow field of view image, and first eliminating completely irrelevant feature points from the spatial range, avoiding the waste of computing resources caused by invalid matching in non-overlapping areas, and focusing the matching on truly valuable areas, narrowing the scope for subsequent matching and improving efficiency; then, the Lightglue algorithm deeply models the feature association of wide field of view and narrow field of view descriptors through the attention mechanism, not only calculating the cosine similarity of a single descriptor, but also capturing the contextual dependency between key points (such as the spatial distribution of adjacent key points and texture association) through cross-attention iteration, making the feature association analysis more comprehensive, and then combining the spatial position constraints to screen candidate point pairs to ensure that the candidate point pairs meet both feature similarity and spatial rationality. The generated initial confidence score can also preliminarily quantify the matching reliability. This step effectively reduces the false matches caused by the similarity of a single feature (such as the false correspondence of repeated texture areas) and provides a high-quality initial matching set for alignment. Finally, GIM combines the perspective consistency of wide-field and narrow-field cameras (such as the inherent correlation between the optical center position and the main direction) and eliminates hidden false matching point pairs caused by camera parameter differences (such as feature projection deviations caused by different focal lengths and distortion) through geometric constraints (these false matches may be retained in Lightglue's initial screening due to similar descriptors). At the same time, the confidence score is calibrated (such as combining the inlier density and the stability adjustment score of multi-frame matching) to make the final confidence score more in line with the actual matching quality. This step further improves the purity of the matching point pairs, ensuring that the retained point pairs meet both feature similarity and geometric rationality. Overall, the three-step process is progressive: first, invalid calculations are reduced through spatial constraints, then the initial accuracy is improved through Lightglue's context-aware matching, and finally, hidden mismatches are eliminated and the confidence is optimized through GIM's geometric and perspective constraints. The final set of matching point pairs is not only moderate in number and low in mismatch rate, but also reliable in confidence, providing high-quality input for the subsequent solution of the homography transformation matrix, reducing external point interference and making the matrix estimation more stable, especially in scenes with sparse textures and large differences in camera parameters. It can significantly improve the accuracy and robustness of image registration, and avoid registration offset or jitter caused by poor matching quality.

[0060] S105 , reorganizing and integrating the multiple groups of matching points based on the confidence scores to obtain a reorganized feature point set.

[0061] In a specific implementation, multiple groups of matching points are sorted from high to low according to the confidence scores, and all matching points with confidence scores higher than a first preset threshold and corresponding descriptors are taken out to form a recombined feature point set.

[0062] For example, in an embodiment, the first preset threshold is 0.95, and in this embodiment, all matching key point pairs and their corresponding descriptors with a confidence score higher than 0.95 are reorganized and integrated to obtain:

[0063] ;

[0064] The reorganized key points in the wide field of view image are obtained. The reorganized descriptors corresponding to the wide field of view image are obtained. The wide field of view key points before reorganization are obtained. The wide field of view descriptors before reorganization are obtained.

[0065] S106, construct a fixed-length container to store the reorganized feature point set, stitch all reorganized feature points in the fixed-length container with feature points of the wide field of view reference image, solve a homographic transformation matrix of the narrow field of view image to the wide field of view image, and perform image registration based on the homographic transformation matrix.

[0066] Specifically, the homographic transformation matrix is a 3x3 matrix used to describe the perspective transformation relationship between two planes. In image registration, the homographic transformation matrix can map points in the narrow field of view image to corresponding positions in the wide field of view image, achieving spatial alignment between images.

[0067] In a specific implementation, the method for constructing a fixed-length container to store the reorganized feature point set comprises: determining the capacity of the fixed-length container based on the frame rate of the narrow field of view image, the feature matching time consumption of the wide field of view and the narrow field of view images, and the system allowed computing load; the capacity is the maximum number of frames of the reorganized feature point set that can be stored; defining the storage structure of the fixed-length container based on the capacity and the composition of the reorganized feature point set; each unit of the storage structure is used to store a complete reorganized feature point set, and the number of units of the storage structure is equal to the capacity; initializing the fixed-length container, assigning an index to each unit of the storage structure based on the time sequence of the acquisition of multiple narrow field of view image frames; the index value is positively correlated with the time sequence; and storing the reorganized feature point set in the storage units of the fixed-length container in sequence according to the corresponding narrow field of view image frame acquisition time, to obtain the fixed-length container.

[0068] ​Specifically, the capacity of the fixed-length container is calculated and determined (such as 20 frames) in combination with the frame rate of the narrow field of view image, the time consumption of the wide field of view and the narrow field of view image feature matching, and the computing load (such as the upper limit of single frame processing power) allowed by the system. According to the determined capacity and the composition of the reorganized feature point set, the storage structure of the fixed-length container is defined: in the form of an array or a queue, each storage unit corresponds to a data structure (such as a structure or a dictionary) for storing all information of a group of reorganized feature point sets completely, and the total number of units of the storage structure is consistent with the capacity. The fixed-length container is initialized, and each unit of the storage structure is assigned a unique index (such as index 0 corresponding to the earliest collected frame, index 1 corresponding to the second earliest collected frame, and so on, and the index value increases with time) according to the time sequence of the collection of the multi-frame narrow field of view image frames. The generated reorganized feature point set is sequentially stored in the storage unit with a matching index in the fixed-length container according to the collection time of the corresponding narrow field of view image frame, and the construction of the fixed-length container is completed. The specific implementation process and implementation principle of the capacity determination of the fixed-length container can be referred to the description in the related art, which will not be repeated here.

[0069] Optionally, after the fixed-length container is constructed to store the reorganized feature point set, the method further includes: monitoring whether the number of the reorganized feature point sets stored in the fixed-length container reaches a capacity threshold value in real time; if the capacity threshold value is reached, identifying and removing the reorganized feature point set stored earliest in the fixed-length container, and adding the newly generated reorganized feature point set to the fixed-length container; if the capacity threshold value is not reached, directly adding the newly generated reorganized feature point set to the fixed-length container; wherein the storage time of the reorganized feature point set is based on the collection time of the corresponding narrow field of view image frame, and the reorganized feature point set stored earliest corresponds to the reorganized feature point generated by the narrow field of view image frame collected earliest in time.

[0070] The method provided in this embodiment monitors the capacity of a fixed-length container in real time to see if it reaches a capacity threshold, and updates the reorganized feature point set in chronological order. When the capacity is full, the oldest stored reorganized feature point set is removed and the most recently generated set is added. If the capacity threshold is not reached, the most recently generated set is directly added. This approach ensures that the container always retains the most recent reorganized feature point set. Fixed-length capacity control avoids the increased computational burden caused by an excessive number of feature points (such as redundant calculations during the subsequent homography matrix solution). A temporal update mechanism also ensures that the feature points involved in registration are more closely aligned with the current scene state (reducing the impact of scene changes caused by large time spans on matching accuracy, such as feature offsets caused by illumination changes and target movement). Furthermore, the time-sequential replacement logic ensures the timeliness and continuity of the feature point set, preventing old feature points (which may have become invalid due to scene changes) from interfering with the current registration. Ultimately, the solved homography transformation matrix more accurately reflects the spatial correspondence between the current wide-field-of-view and narrow-field-of-view images, reduces registration jitter caused by outdated or uncontrolled feature counts in multiple frames, and improves the real-time and stability of registration.

[0071] In a specific implementation, all the recombined feature points in the fixed-length container are spliced ​​with the feature points of the wide-field-of-view reference image to solve the homography transformation matrix from the narrow-field-of-view image to the wide-field-of-view image, including: aligning the timestamp of the narrow-field-of-view image frame with the spatial coordinate range of the wide-field-of-view reference image, integrating the multiple groups of recombined feature point pairs stored in the fixed-length container and the feature points of the matched wide-field-of-view reference image in the order of acquisition timestamps to form a joint matching point set; the feature points include wide-field-of-view key points and wide-field-of-view descriptors; determining the parameters to be solved of the homography transformation matrix; the homography transformation matrix represents the spatial mapping relationship between the narrow-field-of-view image and the wide-field-of-view image; determining the number of matching point pairs based on the number of parameters to be solved; matching the corresponding solution method based on the number of matching point pairs, and solving the homography transformation matrix based on the solution method.

[0072] Optionally, the solution method based on the matching of the number of matching point pairs, and solving the homography transformation matrix based on the solution method, include: when the number of matching point pairs is not greater than a preset number and multiple pairs of matching points are non-collinear, substituting the coordinate information of each pair of matching points into the linear equation group of the homography transformation, and using the matrix inversion operation to solve the parameters to be solved to obtain the homography transformation matrix; when the number of matching point pairs is greater than the preset number, fitting all matching point pairs based on the least squares method, optimizing all parameters to be solved of the homography transformation matrix with the goal of minimizing the sum of squared errors, and obtaining a preliminary homography transformation matrix; inputting the preliminary homography transformation matrix and all matching point pairs into the random sampling consistency algorithm for robust estimation, iteratively screening out matching point pairs that meet the spatial mapping relationship and eliminating erroneous matching point pairs, and finally re-solving the homography transformation matrix based on the screened matching point pairs.

[0073] In a specific implementation, the multiple sets of reorganized feature point pairs stored in the fixed-length container are sorted according to the time stamps of the narrow field of view image frames. According to the initial homographic transformation relationship between the narrow field of view image and the wide field of view reference image, the narrow field of view key point coordinates are mapped to the wide field of view coordinate system to ensure that all feature points are in the same spatial coordinate system. The reorganized feature point pairs in the fixed-length container are traversed in the time stamp order, and the wide field of view key point coordinates and their descriptors in each pair are extracted. The extracted wide field of view key points are merged with the feature points (pre-extracted wide field of view key points and descriptors) of the wide field of view reference image to form a joint matching point set containing multiple frame information. The homographic transformation matrix is a 3x3 matrix containing 8 independent parameters (since it is scale invariant, the last element is usually set to 1). Since each matching point pair can provide two equations (corresponding to x and y coordinate mapping), at least 4 pairs of non-collinear matching points are required to solve 8 parameters. To improve robustness, the actual number of matching point pairs is 3-5 times the minimum required number (such as 12-20 pairs), and the highest confidence point pairs are selected from the joint matching point set. When the number of matching point pairs is 4, the direct linear transformation (DLT) algorithm is used to solve the homographic matrix. When the number of matching point pairs is more than 4, the RANSAC algorithm is used for iterative solution: 4 matching points are randomly selected each time to calculate a candidate matrix. The re-projection error of all point pairs under the candidate matrix is calculated, and the point pairs with an error less than a threshold (such as 3 pixels) are regarded as inliers. Repeat the iteration (such as 1000 times), and select the matrix with the most inliers as the optimal solution. The final matrix is normalized to obtain the homographic transformation matrix.

[0074] For example, in an embodiment, the joint matching point set can be represented as:

[0075] ;

[0076] wherein the is the feature point after splicing; the is the descriptor after splicing; the is the wide field of view key point; the is the wide field of view descriptor; the , is the reorganized feature point set.

[0077] ;

[0078] ;

[0079] wherein the is the wide field of view key point set; the is the narrow field of view key point set; the is the homographic transformation matrix; the 、 are the point coordinates in the wide-field image; 、 are the coordinates of the point in the narrow field of view image.

[0080] In the method provided by this embodiment, firstly, the entire process organically connects the various links from data acquisition to registration through the core logic of multi-frame fusion, forming an efficient and robust processing chain: acquiring multiple consecutive narrow field-of-view images instead of a single frame, introducing redundant information in the time dimension, and compensating for the problem of insufficient feature points in a single frame in scenes with sparse texture and occlusion; cropping the wide field-of-view reference image to focus on the local area overlapping with the narrow field of view, reducing invalid calculations in irrelevant areas and improving processing efficiency; extracting highly stable key points and descriptors through the SuperPoint algorithm combined with a general image matcher, providing a reliable feature basis for matching; then, through multi-frame matching, reorganization, and splicing, finally using sufficient high-quality matching point pairs to solve the homography matrix, the overall solution to the problem of unstable registration in scenes with sparse texture and different camera parameters of existing methods, and achieving high-precision spatial alignment of wide field of view and narrow field of view images. Secondly, the matching process significantly improves the matching quality through layered processing: first, the spatial overlapping areas are limited to filter feature point pairs to avoid redundant matching in irrelevant areas; then the lightweight glue algorithm is used to analyze feature associations and combine spatial position constraints to filter candidate point pairs, and the initial confidence is generated through cosine similarity to capture the contextual association between features to reduce false matches caused by repeated textures; finally, the universal image matcher is combined with perspective consistency to eliminate hidden false matches caused by camera parameter differences, and the confidence is calibrated so that the final matching point pairs meet both feature similarity and geometric rationality, providing high-purity matching data for subsequent alignment. Thirdly, the reassembly and splicing link optimizes the matrix solution through multi-frame information fusion: matching points are reorganized and integrated based on confidence, and stable matching results with high confidence in multiple frames are gathered to make up for the defect of insufficient number of matching points in a single frame; fixed-length containers are used to store reorganized feature points, which not only retains multi-frame information with temporal continuity in chronological order, but also avoids computational pressure caused by excessive data volume through capacity control; all reorganized feature points and wide-field feature points in the splicing container provide sufficient and reliable matching point pairs for homography matrix solution, and the RANSAC robust algorithm is combined to eliminate outliers, making the matrix estimation more stable, and ultimately improving the accuracy and noise resistance of the registration, and reducing the jitter phenomenon in multi-frame registration.

[0081] Corresponding to the aforementioned embodiment of an image registration method based on multi-frame fusion, the present application also provides an embodiment of an image registration device based on multi-frame fusion.

[0082] Figure 2 This is a structural diagram of the image registration device based on multi-frame fusion provided in Example 2 of this application. Figure 2The device provided in the embodiment comprises an acquisition module 210, a clipping module 220, an extraction module 230, a matching module 240, a recombination module 250 and a solving module 260.

[0083] The acquisition module 210 is configured to acquire a wide-view field reference image and a plurality of continuous narrow-view field image frames.

[0084] The clipping module 220 is configured to clip a region of the wide-view field reference image to obtain a clipped wide-view field local region image.

[0085] The extraction module 230 is configured to extract wide-view field key points and corresponding wide-view field descriptors from the clipped wide-view field local region image, and extract narrow-view field key points and corresponding narrow-view field descriptors from each of the narrow-view field image frames.

[0086] The matching module 240 is configured to match the wide-view field key points, the wide-view field descriptors, the narrow-view field key points and the narrow-view field descriptors of each of the narrow-view field image frames to obtain a plurality of groups of matching point pairs and corresponding confidence scores.

[0087] The recombination module 250 is configured to recombine and integrate the plurality of groups of matching points based on the confidence scores to obtain a recombined feature point set.

[0088] The solving module 260 is configured to construct a fixed-length container to store the recombined feature point set, splice all the recombined feature points in the fixed-length container with feature points of the wide-view field reference image, solve a homographic transformation matrix of the narrow-view field image to the wide-view field image, and perform multi-frame image registration based on the homographic transformation matrix.

[0089] The device of the embodiment can be used to execute the steps of the method embodiment, and the specific implementation principle and implementation process are similar, which will not be described here. Figure 1 The functions and roles of the units in the device are achieved in the implementation process of the corresponding steps in the above method, which will not be described here.

[0090] The functions and roles of the units in the device are achieved in the implementation process of the corresponding steps in the above method, which will not be described here.

[0091] For the device embodiment, since it basically corresponds to the method embodiment, the related parts can be referred to the part of the method embodiment. The device embodiment described above is only illustrative, and the units described as separate components can be or can not be physically separated, and the components displayed as units can be or can not be physical units, that is, they can be located in one place or distributed on multiple network units. According to the actual needs, part or all of the modules can be selected to achieve the purpose of the scheme of the present application. Those skilled in the art can understand and implement it without creative labor.

[0092] The above description is only the preferred embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of protection of the present application.

Claims

1. A multi-frame fusion based image registration method, characterized in that, The method comprises: acquiring a wide field of view reference image and a plurality of continuous narrow field of view image frames; regionally cropping the wide field of view reference image to obtain a cropped wide field of view local area image; extracting wide field of view key points and corresponding wide field of view descriptors from the cropped wide field of view local area image, and extracting narrow field of view key points and corresponding narrow field of view descriptors from each of the narrow field of view image frames; matching the wide field of view key points, the wide field of view descriptors, the narrow field of view key points and the narrow field of view descriptors of each of the narrow field of view image frames to obtain a plurality of matching point pairs and corresponding confidence scores; reorganizing and integrating the plurality of matching points based on the confidence scores to obtain a reorganized feature point set; constructing a fixed-length container to store the reorganized feature point set, splicing all the reorganized feature points in the fixed-length container with feature points of the wide field of view reference image, solving a homographic transformation matrix of the narrow field of view image to the wide field of view image, and performing image registration based on the homographic transformation matrix.

2. The method of claim 1, wherein, The matching of the wide field of view key points, the wide field of view descriptors, the narrow field of view key points and the narrow field of view descriptors of each of the narrow field of view image frames to obtain a plurality of matching point pairs and corresponding confidence scores comprises: determining feature point pairs in a spatial overlap region of the cropped wide field of view local area image and each of the narrow field of view image frames; each feature point pair comprises a wide field of view descriptor and a narrow field of view descriptor; analyzing feature correlations of the wide field of view descriptors and the narrow field of view descriptors in the spatial overlap region based on a lightweight glue algorithm, screening candidate point pairs based on spatial positions of the feature point pairs and the feature correlations, and generating similarity of the candidate point pairs based on cosine similarity as initial confidence scores; inputting the candidate point pairs and the corresponding initial confidence scores into a general image matcher, the general image matcher eliminating mismatched point pairs caused by differences in camera parameters based on consistency of viewing angles of the wide field of view and the narrow field of view, and calibrating initial confidence scores of the retained candidate point pairs to obtain a plurality of matching point pairs and calibrated confidence scores.

3. The method of claim 1, wherein, The splicing of all the reorganized feature points in the fixed-length container with feature points of the wide field of view reference image to solve a homographic transformation matrix of the narrow field of view image to the wide field of view image comprises: aligning timestamps of the narrow field of view image frames with a spatial coordinate range of the wide field of view reference image, integrating the plurality of reorganized feature point pairs stored in the fixed-length container and the matched feature points of the wide field of view reference image in chronological order to form a joint matching point set; the feature points include wide field of view key points and wide field of view descriptors; determining to-be-solved parameters of the homographic transformation matrix; the homographic transformation matrix represents a spatial mapping relationship between the narrow field of view image and the wide field of view image; determining a number of matching point pairs based on the number of to-be-solved parameters; matching a corresponding solving method based on the number of matching point pairs, and solving the homographic transformation matrix based on the solving method.

4. The method of claim 3, wherein, The matching of the corresponding solving method based on the number of matching point pairs, and the solving of the homographic transformation matrix based on the solving method comprise: When the number of the matching point pairs is not greater than the preset number and the multiple matching point pairs are not collinear, the coordinate information of each matching point pair is substituted into a linear equation set of homographic transformation, and the parameters to be solved are solved by using matrix inversion operation to obtain the homographic transformation matrix; When the number of the matching point pairs is greater than the preset number, all the matching point pairs are fitted based on a least square method, all the parameters to be solved of the homographic transformation matrix are optimized with the minimum error square sum as the target, and a preliminary homographic transformation matrix is obtained. The preliminary homographic transformation matrix and all the matching point pairs are input into a random sample consensus algorithm for robust estimation, the matching point pairs meeting the spatial mapping relationship are screened out by iteration, and the false matching point pairs are removed, and finally the homographic transformation matrix is re-solved based on the screened matching point pairs.

5. The method of claim 1, wherein, The wide field key points and corresponding wide field descriptors are extracted from the cropped wide field local area images, and the narrow field key points and corresponding narrow field descriptors are extracted from each frame of the narrow field image frames, including: A feature point extractor is used to detect and extract the wide field key points from the cropped wide field local area images, and generate the corresponding wide field descriptors of each wide field key point; the wide field descriptors represent the local feature information of the wide field key points; A same feature point extractor is used to detect and extract the narrow field key points from each frame of the narrow field image frames, and generate the corresponding narrow field descriptors of each narrow field key point; the wide field key points and the narrow field key points are corresponding points having a mapping relationship in space.

6. The method of claim 5, wherein, The wide field key points and corresponding wide field descriptors are extracted from the cropped wide field local area images, and generate the corresponding wide field descriptors of each wide field key point; the wide field descriptors represent the local feature information of the wide field key points; The cropped wide field local area images are processed based on a super point algorithm, and a target pixel point in the wide field local area image is detected as an initial wide field key point by a convolutional neural network architecture, and a basic descriptor corresponding to the initial wide field key point is generated; the gray scale distribution, neighborhood structure and difference from surrounding pixels of the target pixel point are greater than a preset value, and the target pixel point can be stably detected when the image is scaled, rotated or light changed; The initial wide field key points and the basic descriptors are input into a general image matcher, the general image matcher removes false key points susceptible to noise interference based on the stability of the initial wide field key points, and enhances the features of the basic descriptors corresponding to the remaining stable initial wide field key points to obtain the final wide field key points and corresponding wide field descriptors.

7. The method of claim 1, wherein, The wide field reference image is regionally cropped to obtain the cropped wide field local area image, including: Four corner points of a narrow field image are taken as original corner points, the original corner points are mapped to corresponding four mapping corner points in the wide field reference image through a preset initial homographic transformation relationship; the narrow field image is a certain frame in continuous multiple frames of narrow field image frames, and the initial homographic transformation relationship is used to preliminarily determine the spatial overlapping range of the narrow field and wide field images. A closed region is defined in the wide field reference image based on the four mapping corner points, and the closed region is a region of interest to be cropped; The wide field reference image is cropped based on the coordinate range of the four mapping corner points, image content in the region of interest is retained, and an image part outside the region of interest is removed, to obtain a cropped wide field local region image.

8. The method of claim 1, wherein, The method further comprises: The capacity of the fixed-length container is determined based on the frame rate of the narrow field image, the feature matching time consumption of the wide field and narrow field images, and the system allowed computing load; the capacity is the maximum number of frames of the reorganized feature point set that can be stored; The storage structure of the fixed-length container is defined based on the capacity and the composition of the reorganized feature point set; each unit of the storage structure is used to store a complete reorganized feature point set, and the number of units of the storage structure is equal to the capacity; The fixed-length container is initialized, and each unit of the storage structure is assigned an index based on the time sequence of the acquisition of multiple frames of the narrow field image; the index value is positively correlated with the time sequence; The reorganized feature point set is sequentially stored in the storage unit of the fixed-length container according to the corresponding narrow field image frame acquisition time, to obtain the fixed-length container.

9. The method of claim 1, wherein, After the fixed-length container stores the reorganized feature point set, the method further comprises: It is monitored in real time whether the number of frames of the reorganized feature point set stored in the fixed-length container reaches a capacity threshold; If the capacity threshold has been reached, the reorganized feature point set stored earliest in the fixed-length container is identified and removed, and the newly generated reorganized feature point set is added to the fixed-length container; If the capacity threshold has not been reached, the newly generated reorganized feature point set is directly added to the fixed-length container; wherein the storage time of the reorganized feature point set is based on the acquisition time of the corresponding narrow field image frame, and the reorganized feature point set stored earliest corresponds to the reorganized feature point generated by the narrow field image frame collected earliest in time.

10. An image registration apparatus based on multi-frame fusion, characterized by comprising: The device comprises an acquisition module, a cropping module, an extraction module, a matching module, a reorganization module, and a solving module; The acquisition module is configured to acquire a wide field reference image and multiple frames of a narrow field image; The cropping module is configured to perform regional cropping on the wide field reference image to obtain a cropped wide field local region image; The extraction module is configured to extract wide field key points and corresponding wide field descriptors from the cropped wide field local region image, and extract narrow field key points and corresponding narrow field descriptors from each frame of the narrow field image; The matching module is configured to match the wide field key points, the wide field descriptors, the narrow field key points, and the narrow field descriptors of each frame of the narrow field image to obtain multiple groups of matching point pairs and corresponding confidence scores; The reorganization module is configured to reorganize and integrate multiple groups of matching points based on the confidence scores to obtain a reorganized feature point set; and The solving module is configured to solve the reorganized feature point set based on the wide field reference image. The solving module is configured to construct a fixed-length container to store the reorganized feature point set, stitch all the reorganized feature points in the fixed-length container with feature points of a wide-view reference image, solve a homographic transformation matrix of a narrow-view image to the wide-view image, and perform multi-frame image registration based on the homographic transformation matrix.

Images