An image registration method and device based on multi-frame fusion

Abstract

The application provides a multi-frame fusion-based image registration method and device. The method provided by the application comprises the following steps: acquiring a wide-view reference image and a plurality of continuous narrow-view image frames; performing regional cutting on the wide-view reference image to obtain a cut wide-view local area image; extracting key points and descriptors from the cut wide-view local area image and each narrow-view image frame respectively; matching the wide-view key points, the wide-view descriptors, the narrow-view key points and the narrow-view descriptors of each narrow-view image frame to obtain a plurality of groups of matching point pairs and corresponding confidence scores; recombining and integrating the plurality of groups of matching points based on the confidence scores to obtain a recombined feature point set; constructing a fixed-length container to store the recombined feature point set, splicing all the recombined feature points in the fixed-length container with the feature points of the wide-view reference image, solving a homographic transformation matrix of the narrow-view image to the wide-view image, and performing image registration based on the homographic transformation matrix.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image registration, in particular to a multi-frame fusion-based image registration method and device. BACKGROUND

[0002] In the field of image registration, since the wide-view camera and the narrow-view camera are usually installed at the same position, the images obtained by the two cameras have high consistency in view angle. In order to realize the spatial information correlation of the two types of images, the narrow-view image and the wide-view image need to be spatially aligned through image registration, which is the basis for subsequent utilization of complementary information (such as the global view of the wide-view camera and the local details of the narrow-view camera) of the two types of images.

[0003] At present, the mainstream image registration method is based on feature point matching and homography transformation, that is, feature points (including key points and corresponding descriptors) in the image are extracted and matched, and a homography transformation matrix is solved by using the matched point pairs to complete the spatial mapping. However, these methods have significant limitations: in a sparse texture scene (such as grass or a wall), the number of feature points available for matching is extremely small, resulting in unstable estimation of the homography transformation matrix; the inherent differences in focal length and distortion parameters between the wide-view camera and the narrow-view camera will introduce feature distribution deviation and increase the matching difficulty; when facing a textureless region or an image pair with significant parallax, the single-frame feature matching performance will decrease sharply; and in the multi-frame registration process, jitter phenomenon is easy to occur, which seriously affects the accuracy of subsequent operations such as boundary box mapping.

[0004] Therefore, there is an urgent need for a method to solve the problem of insufficient registration robustness of existing methods in sparse texture, camera parameter difference and complex scene, and to improve the stability and accuracy of narrow-view image and wide-view image registration. SUMMARY

[0005] Therefore, the present application provides a multi-frame fusion-based image registration method and device to solve the problem of insufficient registration robustness of existing methods in sparse texture, camera parameter difference and complex scene, and to improve the stability and accuracy of narrow-view image and wide-view image registration.

[0006] Specifically, the present application is implemented by the following technical solutions:

[0007] The first aspect of the present application provides a multi-frame fusion-based image registration method, which comprises:

[0008] obtaining a wide-view reference image and a plurality of continuous narrow-view image frames;

[0009] regionally cropping the wide-view reference image to obtain a cropped wide-view local region image;

[0010] extract wide field key points and corresponding wide field descriptors from the cropped wide field local region images respectively, and extract narrow field key points and corresponding narrow field descriptors from each frame of the narrow field image frames;

[0011] match the wide field key points, the wide field descriptors, the narrow field key points and the narrow field descriptors of each frame of the narrow field image frames to obtain a plurality of groups of matching point pairs and corresponding confidence scores;

[0012] recombine and integrate the plurality of groups of matching points based on the confidence scores to obtain a recombined feature point set;

[0013] construct a fixed-length container to store the recombined feature point set, splice all recombined feature points in the fixed-length container with feature points of the wide field reference image, solve a homographic transformation matrix of the narrow field image to the wide field image, and perform image registration based on the homographic transformation matrix.

[0014] The second aspect of the application provides an image registration device based on multi-frame fusion, which comprises an acquisition module, a cropping module, an extraction module, a matching module, a recombination module and a solving module;

[0015] The acquisition module is configured to acquire a wide field reference image and a plurality of continuous narrow field image frames.

[0016] The cropping module is configured to crop a region of the wide field reference image to obtain a cropped wide field local region image.

[0017] The extraction module is configured to extract wide field key points and corresponding wide field descriptors from the cropped wide field local region images respectively, and extract narrow field key points and corresponding narrow field descriptors from each frame of the narrow field image frames.

[0018] The matching module is configured to match the wide field key points, the wide field descriptors, the narrow field key points and the narrow field descriptors of each frame of the narrow field image frames to obtain a plurality of groups of matching point pairs and corresponding confidence scores.

[0019] The recombination module is configured to recombine and integrate the plurality of groups of matching points based on the confidence scores to obtain a recombined feature point set.

[0020] The solving module is configured to construct a fixed-length container to store the recombined feature point set, splice all recombined feature points in the fixed-length container with feature points of the wide field reference image, solve a homographic transformation matrix of the narrow field image to the wide field image, and perform multi-frame image registration based on the homographic transformation matrix.

[0021] The image registration method and apparatus based on multi-frame fusion provided in this application organically link the steps from data acquisition to registration matrix solution through the core idea of ​​multi-frame fusion, achieving efficient and accurate image registration. First, it acquires multiple consecutive narrow-field-of-view image frames instead of a single frame, laying the foundation for subsequent multi-frame information fusion. Compared to single-frame registration, this introduces redundant information in the temporal dimension, reducing matching errors caused by noise, occlusion, or missing features in a single frame image. Simultaneously, a wide-field-of-view reference image is acquired as the registration benchmark, clarifying the target coordinate system for registration. Second, the wide-field-of-view reference image is cropped, processing only local regions related to the narrow-field-of-view image. This reduces the computational load of feature extraction and matching in irrelevant regions, ensuring registration relevance while reducing system computational load and improving processing efficiency. Next, key points and descriptors of the wide-field-of-view local regions and each narrow-field-of-view frame are extracted, providing basic feature information for feature matching. A confidence score is introduced during matching, enabling subsequent reassembly and integration to filter more reliable matching point pairs from multiple frames based on the score, eliminating low-confidence mismatches and improving matching accuracy from the source. Then, through confidence score-based recombination and integration, matching point pairs from multiple frames of narrow field-of-view images are fused, aggregating stable, high-confidence feature matching results from multiple frames. This compensates for potential issues such as insufficient matching points or local mismatches in a single frame, increasing both the total number and quality of effective matching point pairs. Next, a fixed-length container is constructed to store the recombined feature point set. The fixed-length design allows for control over the number of frames stored based on system computational load, avoiding computational pressure caused by excessive data volume. Simultaneously, storing data in chronological order ensures the use of temporally continuous multi-frame information, providing time-consistent feature data for subsequent stitching. Finally, all recombined feature points in the fixed-length container are stitched together with wide field-of-view feature points. The homography transformation matrix is ​​solved using the sufficient and reliable matching point pairs obtained from multi-frame fusion. Compared to solving with single-frame matching point pairs, more effective point pairs provide richer constraint information for matrix solving. Combined with an adapted solution method, outliers can be further eliminated. The resulting homography matrix more accurately represents the spatial mapping relationship between narrow and wide field-of-view images, thus achieving high-precision and robust image registration. Attached Figure Description

[0022] Figure 1 A flowchart of the image registration method based on multi-frame fusion provided in Embodiment 1 of this application;

[0023] Figure 2 This is a schematic diagram of the structure of the image registration device based on multi-frame fusion provided in Embodiment 2 of this application. Detailed Implementation

[0024] The exemplary embodiments will be described in detail herein with reference to several drawings. Descriptions of well-known functions and structures are omitted so as not to unnecessarily obscure the application. The examples described herein represent the best

[0025] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0026] It is to be understood that the singular forms "a", "an", and "the" include plural referents unless the context clearly dictates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0027] The specific embodiments are given below to introduce the technical solutions of the application in detail.

[0028] Figure 1 The flow chart of the image registration method based on multi-frame fusion provided by Embodiment One of the application is shown in FIG. 1. Please refer to Figure 1 The method provided by the embodiment can include:

[0029] S101, acquire a wide field reference image and a plurality of continuous narrow field image frames.

[0030] Specifically, the wide field reference image refers to an image with a large angle range acquired by a wide field camera, which serves as the spatial reference of the entire registration process. The wide field reference image contains global scene information and can provide a unified spatial coordinate framework, which is the "reference standard" for the spatial alignment of the narrow field image. The plurality of continuous narrow field image frames refer to a sequence of images with a small angle but high details acquired by a narrow field camera. The narrow field image frames contain dynamic change information of local scenes and can capture the detailed features and motion trajectories of the target.

[0031] It should be noted that the wide field of view reference image and the continuous multiple frames of narrow field of view image frames are in a "reference and dynamic object" relationship, which is specifically manifested as follows: the two are collected by different cameras installed at the same position, have high consistency in view angle, and provide a basis for spatial alignment through homography transformation; the wide field of view reference image provides a spatial reference of the global scene, and the narrow field of view image frames provide dynamic information of local details, and the two realize information fusion of the global and the local through registration; the continuous multiple frames of narrow field of view image frames need to be spatially aligned based on the wide field of view reference image, and finally the local features of the narrow field of view image are mapped into the global coordinates of the wide field of view through a homography transformation matrix, so as to realize spatial unification of the two types of images and provide a prerequisite for subsequent processing (such as bounding box mapping and multi-frame feature stitching).

[0032] In specific implementation, the wide field of view camera and the narrow field of view camera are started, the consistency of the acquisition timing is ensured through a hardware trigger or a timestamp synchronization mechanism, and it is ensured that the wide field of view camera and the narrow field of view camera are in the same installation position and are in a synchronous working state. The wide field of view camera is controlled to collect a single image covering the global range of the target scene, which is defined as a wide field of view reference image, stored as a digital image file in a preset format (such as JPEG, PNG), and the storage path is associated with a designated reference image database. The narrow field of view camera is controlled to continuously collect local detail images of the target scene at a fixed frame rate, generating a continuous sequence of multiple frames of narrow field of view images, and each image is named according to the acquisition timestamp and sequentially stored in a timing image buffer area.

[0033] S102, regionally cropping the wide field of view reference image to obtain a cropped wide field of view local area image.

[0034] In specific implementation, the regionally cropping the wide field of view reference image to obtain a cropped wide field of view local area image includes: taking four corner points of a narrow field of view image as original corner points, mapping the original corner points to the wide field of view reference image through a preset initial homography transformation relationship to obtain four corresponding mapping corner points in the wide field of view reference image; the narrow field of view image is a certain frame in the continuous multiple frames of narrow field of view images, and the initial homography transformation relationship is used to preliminarily determine the spatial overlap range of the narrow field of view and the wide field of view images; a closed region is defined in the wide field of view reference image based on the four mapping corner points, and the closed region is a region of interest to be cropped; the wide field of view reference image is cropped based on the coordinate range of the four mapping corner points, the image content in the region of interest is retained, the image part outside the region of interest is removed, and a cropped wide field of view local area image is obtained.

[0035] Specifically, a certain frame is selected from continuous multiple narrow field of view image frames as a current processing object, four corner points (i.e. four vertices of the image boundary) of the frame image are automatically recognized by an image processing tool, and pixel coordinate values thereof are obtained as original corner points. An initial homographic transformation relationship is determined by camera calibration or historical registration data, the coordinate values of the four original corner points are input into the transformation relationship, and the mapping coordinates of each original corner point in the wide field of view reference image are calculated to form four mapping corner points. Further, a closed quadrilateral region is drawn on the wide field of view reference image with the four mapping corner points as the vertices. The minimum circumscribed rectangle (i.e. the smallest rectangular region containing all the mapping corner points) of the quadrilateral region is calculated, the coordinates of the upper left corner and the lower right corner of the rectangle are determined, and the region of interest to be cropped is obtained. Using a cropping function provided by an image processing library, the wide field of view reference image is subjected to a pixel-level cropping operation according to the coordinate range of the region of interest, all pixel data within the coordinate range is retained, and pixel data outside the range is discarded, and finally a wide field of view local region image containing only the content of the region of interest is generated. The specific implementation process and implementation principle of determining the initial homographic transformation relationship can be referred to the description in the related art, which will not be described here.

[0036] For example, in an embodiment, the four original corner points of the selected narrow field of view image frame are , and the corresponding four mapping corner points in the wide field of view reference image after mapping are .

[0037] ;

[0038] wherein the four mapping corner points are , the four original corner points are , and the initial homographic transformation relationship is .

[0039] The method provided in this embodiment can focus on the effective region that overlaps with the narrow field of view image frame by regionally cropping the wide field of view reference image, and reduce the interference of irrelevant regions. Since the wide field of view reference image covers the global scene, and the narrow field of view image frame only captures local details, the effective matching region of the two is limited to the overlapping part. By cropping, the redundant region in the wide field of view image that is irrelevant to the narrow field of view image can be removed, the range of subsequent feature point extraction and matching is reduced, the invalid calculation amount is reduced, the efficiency of feature matching is improved, and meanwhile, limiting the matching region can reduce the false matching caused by the dispersed distribution of feature points in the global range, enhance the pertinence of the association of the wide field of view and the narrow field of view feature points, provide a more accurate regional basis for subsequent similarity calculation and matching point pair screening, and further improve the accuracy and robustness of the overall registration process.

[0040] S103, extract wide field key points and corresponding wide field descriptors from the cropped wide field local region images, and extract narrow field key points and corresponding narrow field descriptors from each frame of the narrow field image frames.

[0041] Specifically, the wide field key points are pixel points extracted from the cropped wide field local region images, which have uniqueness and stability, and are usually positions with significant features in the images, such as edge intersection points, corner points, texture mutation points, etc. These points can remain relatively stable in image transformation (such as scaling, rotation, illumination change), and can be used as "feature markers" of the local region images for subsequent matching with the narrow field images. The wide field descriptor is a quantitative expression of the local image information around the wide field key point, and is a set of numerical vectors (or data structures) that can describe the features of the key point neighborhood. It converts visual features into computable and comparable digital forms by extracting the gray value distribution, gradient direction, texture pattern, etc. of the pixels around the wide field key point, so as to perform similarity measurement with the narrow field descriptor in the subsequent steps and realize the matching of the key points.

[0042] Further, the narrow field key points are pixel points extracted from each frame of the narrow field image frames, which have uniqueness and stability, and are essentially the same as the wide field key points, and are also positions with significant features such as edges, corners, and texture mutations in the images. Since the narrow field images focus on local details, the key points more reflect the subtle features of local regions in the scene, and are the core "feature markers" of the narrow field images. The narrow field descriptor is a quantitative expression of the local image information around the narrow field key point, and has the same logical definition as the wide field descriptor, and is a numerical vector (or data structure) generated based on the gray, gradient, texture, etc. features of the neighborhood of the narrow field key point. Its role is to convert the visual features of the narrow field key points into computable digital forms, so as to compare with the wide field descriptor and determine the correspondence between the wide field local region images and the narrow field image frames.

[0043] In a specific implementation, extracting wide field key points and corresponding wide field descriptors from the cropped wide field local region images, and extracting narrow field key points and corresponding narrow field descriptors from each frame of the narrow field image frames, includes: using a feature point extractor to detect and extract wide field key points from the cropped wide field local region images, and simultaneously generate a wide field descriptor corresponding to each wide field key point; the wide field descriptor represents the local feature information of the wide field key point; using the same feature point extractor to detect and extract narrow field key points from each frame of the narrow field image frames, and simultaneously generate a narrow field descriptor corresponding to each narrow field key point; wherein the wide field key points and the narrow field key points are corresponding points with a mapping relationship in space.

[0044] It should be noted that in the embodiment, the feature point extractor is used for both the extraction of the wide field of view key points and the extraction of the narrow field of view key points. Therefore, only the specific implementation process of the extraction of the wide field of view key points is introduced, and the extraction of the narrow field of view key points is similar, which is not described herein again.

[0045] The feature point extractor is used to detect and extract the wide field of view key points from the cropped wide field of view local region image, and generate a wide field of view descriptor corresponding to each wide field of view key point, including: processing the cropped wide field of view local region image based on a super point algorithm, detecting a target pixel point in the wide field of view local region image as an initial wide field of view key point through a convolutional neural network architecture, and generating a basic descriptor corresponding to the initial wide field of view key point; the gray scale distribution, neighborhood structure and difference from surrounding pixels of the target pixel point are greater than a preset value, and the target pixel point can be stably detected when the image is scaled, rotated or light changed; inputting the initial wide field of view key point and the basic descriptor into a general image matcher, the general image matcher eliminating false key points susceptible to noise interference based on the stability of the initial wide field of view key point, and enhancing the features of the basic descriptor corresponding to the remaining stable initial wide field of view key point to obtain the final wide field of view key point and the corresponding wide field of view descriptor.

[0046] In practice, the cropped wide-field-of-view local region image is input into a feature point extractor based on the SuperPoint algorithm, which employs a Convolutional Neural Network (CNN) architecture. The CNN extracts features and reduces dimensionality through multiple convolutional and pooling layers, generating a feature pyramid. At the top of the feature pyramid, the classifier branch performs binary classification on each pixel of the wide-field-of-view local region image, outputting the probability value of each pixel belonging to a keypoint using the softmax function. Based on the probability value, it determines whether a pixel is a keypoint, selecting pixels with a probability value greater than a preset threshold (e.g., 0.01) as initial wide-field-of-view keypoints. Non-maximum suppression ensures the spatial uniqueness of keypoints, i.e., only retaining points with the highest local probability. Another branch of the CNN (the descriptor branch) generates a corresponding basic descriptor for each detected keypoint. The descriptor branch extracts features from the patches (e.g., 8×8 or 16×16 pixel regions) surrounding the keypoint through convolutional operations, mapping them to fixed-dimensional vectors (e.g., 256-dimensional). The generated vectors are normalized to obtain the basic descriptor for each initial wide-field-of-view keypoint. Furthermore, the initial wide-field-of-view keypoints and basic descriptors are input into a general image matcher (GIM). The GIM performs slight perturbations on the image (e.g., rotation, scaling) and re-detects keypoints, calculating the repetition rate based on the proportion of points detected both before and after the perturbation. The gradient orientation histogram of the keypoint neighborhood is analyzed to evaluate its stability under different lighting or viewing angles. The probability value during keypoint detection is used as the initial score to remove points with excessively low response intensity. A stability threshold (e.g., repetition rate > 0.7) is set to remove pseudo-keypoints below the stability threshold. The retained basic descriptors are then subjected to secondary normalization, and the descriptor dimension is compressed to 64 or 128 dimensions through linear mapping (e.g., fully connected layers) to improve matching efficiency. The spatial relationships between keypoints are analyzed using a Transformer architecture, and the descriptor vectors are adjusted through a self-attention mechanism to enhance their expressive ability of local context, resulting in the final wide-field-of-view keypoints and their corresponding wide-field-of-view descriptors.

[0047] For example, in one embodiment, the wide field of view key points and the corresponding wide field of view descriptors are: The key points of the narrow field of view and the corresponding narrow field of view descriptors are: .

[0048] ;

[0049] ;

[0050] Among them, the For wide field of view key points; the aforementioned For wide field-of-view descriptors; the This is a cropped wide-field-of-view local region image; is a narrow field key point; the is a narrow field descriptor; the is a narrow field image frame for each frame.

[0051] The method provided by the embodiment extracts wide field and narrow field key points in combination with two-step processing of a Superpoint algorithm (Superpoint) and a general image matcher (GIM). When the Superpoint detects initial key points through a convolutional neural network, target pixel points that are significantly different from the surroundings in terms of gray scale distribution and neighborhood structure and that can still be stably detected under scaling, rotation, and illumination changes are deliberately selected, which preliminarily ensures that the initial key points have basic stability and distinguishability, provides potential feature candidates for registration, and avoids redundant computation caused by extracting features from a large number of meaningless pixels. The GIM further eliminates false key points (such as misdetected points in texture sparse areas and unreal edge points caused by illumination mutations) that are susceptible to noise interference based on stability, and simultaneously enhances the features of the descriptors. This improves the purity of the remaining key points, ensures that the key points participating in registration are stable points that truly represent the image content, and enhances the distinguishability of the descriptors, so that the descriptors of key points at different positions are more significantly different, and the descriptors at the same position are more consistent under changing conditions. This makes the matching success rate of correct corresponding points higher and the false matching rate lower in the subsequent feature matching process of wide field and narrow field images, and finally greatly improves the accuracy and robustness of image registration. At the same time, because redundant false key points are eliminated, the amount of computation in the matching stage is reduced, and the registration efficiency and effect are balanced.

[0052] S104, match the wide field key points, the wide field descriptors, the narrow field key points, and the narrow field descriptors of each narrow field image frame, to obtain a plurality of sets of matching point pairs and corresponding confidence scores.

[0053] Specifically, the plurality of sets of matching point pairs refer to a one-to-one correspondence relationship set formed after successful descriptor matching between the wide field key points of the wide field image and the narrow field key points of the narrow field image. Each set of matching point pairs represents that one wide field key point in the wide field image and one narrow field key point in the narrow field image have corresponding associations in content. The plurality of sets of matching point pairs include the coordinates of the wide field key points, the coordinates of the corresponding narrow field key points, and the confidence scores corresponding to the matching point pairs.

[0054] In a specific implementation, the matching the wide field key points, the wide field descriptors, the narrow field key points and the narrow field descriptors of each frame of the narrow field image frames to obtain a plurality of groups of matching point pairs and corresponding confidence scores includes: determining feature point pairs in a spatial overlap region of each frame of the narrow field image frames and the cropped wide field local region image; each feature point pair includes a wide field descriptor and a narrow field descriptor; analyzing feature correlation of the wide field descriptors and the narrow field descriptors in the spatial overlap region based on a lightweight glue algorithm, screening candidate point pairs based on spatial positions of the feature point pairs and the feature correlation, and generating similarity of the candidate point pairs based on cosine similarity as initial confidence scores; inputting the candidate point pairs and the corresponding initial confidence scores into a general image matcher, the general image matcher eliminating false matching point pairs caused by differences in camera parameters based on consistency of viewing angles of the wide field and the narrow field images, calibrating initial confidence scores of the retained candidate point pairs to obtain a plurality of groups of matching point pairs and calibrated confidence scores.

[0055] Specifically, based on the spatial mapping relationship between the cropped wide-field-of-view local region image and each narrow-field-of-view image frame, the overlapping region is calculated. Points within the overlapping region from the wide-field-of-view keypoints, along with their corresponding wide-field-of-view descriptors, are selected and combined with all narrow-field-of-view keypoints and their descriptors from the narrow-field-of-view image frames to form a feature point pair set. Further, the feature point pair set is input into the Light Glue algorithm, and the wide-field-of-view and narrow-field-of-view descriptors are input into two Transformer encoders respectively to generate context-enhanced descriptors. A bipartite graph is constructed, with nodes representing wide-field-of-view and narrow-field-of-view keypoints, and edge weights initialized by the cosine similarity between descriptors. The Light Glue algorithm iteratively updates edge weights through a cross-attention mechanism, with each wide-field-of-view keypoint focusing on all narrow-field-of-view keypoints and calculating an attention score. Each narrow-field-of-view keypoint also focuses on all wide-field-of-view keypoints. This process is repeated multiple times (typically 3-5 times) until the edge weights converge to stable values, forming a feature association matrix. Furthermore, bidirectional matching is performed on the feature association matrix: for each wide field-of-view keypoint, the narrow field-of-view keypoint with the strongest association is selected as a candidate. If the strongest association of the narrow field-of-view keypoint also points to the same wide field-of-view keypoint, it is retained as a candidate point pair. The projection error of the candidate point pairs is calculated, and point pairs with errors exceeding a threshold (e.g., 5 pixels) are discarded. If the matching points in a certain region are too dense, the point pairs with the strongest association are retained. For the candidate point pairs that pass the screening, the final edge weights in their feature association matrix are used as cosine similarity and directly converted into initial confidence scores (range 0-1). Furthermore, the GIM algorithm is used to iteratively sample the candidate point pairs, randomly selecting 4 pairs each time to calculate the homography matrix. The reprojection error of all point pairs under the current homography matrix is ​​calculated, and point pairs with errors less than a threshold (e.g., 3 pixels) are marked as inliers. This process is repeated (usually 100-200 times), and the homography matrix with the most inliers is selected, while mismatched point pairs that are not marked as inliers are discarded. Calculate the number of other inliers within a radius r (e.g., 10 pixels) around each inlier, and use this as the local consistency score. Multiply the initial confidence score by the normalized value of the local consistency score to enhance the confidence of matching points in dense regions, resulting in multiple sets of matching point pairs and calibrated confidence scores.

[0056] For example, in one embodiment, multiple sets of matching point pairs and calibrated confidence scores can be represented as:

[0057] ;

[0058] Among them, the For matching point pairs; the The confidence score; For wide field of view key points; the aforementioned For wide field-of-view descriptors; the Key points for narrow field of view; This is a narrow field-of-view descriptor.

[0059] The method provided in this embodiment first filters feature point pairs by limiting the spatially overlapping region, retaining only feature points in the wide field of view local region and the narrow field of view image that may have a real correspondence. This eliminates completely irrelevant feature points from a spatial perspective, avoiding the waste of computational resources caused by invalid matching in non-overlapping regions. Simultaneously, it focuses matching on truly valuable regions, narrowing the scope and improving efficiency for subsequent matching. Then, the Lightglue algorithm deeply models the feature associations between wide and narrow field of view descriptors through an attention mechanism. It not only calculates the cosine similarity of individual descriptors but also captures the contextual dependencies between key points (such as the spatial distribution and texture associations of adjacent key points) through cross-attention iteration, making the feature association analysis more comprehensive. Finally, it combines spatial location constraints to filter candidate point pairs, ensuring that candidate point pairs meet both feature similarity and spatial rationality requirements. The generated initial confidence score can also preliminarily quantify the reliability of the match. This step effectively reduces mismatches caused by single feature similarity (such as the unrealistic correspondence of repeated texture regions), providing a high-quality initial match set for registration. Finally, GIM combines the viewpoint consistency of wide and narrow field-of-view cameras (such as the inherent correlation of optical center position and principal direction) and uses geometric constraints to eliminate hidden mismatched point pairs caused by differences in camera parameters (such as feature projection deviation caused by different focal lengths and distortions). These mismatches may be retained in Lightglue's initial screening due to descriptor similarity. At the same time, the confidence score is calibrated (such as adjusting the score by combining interior point density and the stability of multi-frame matching), so that the final confidence score is closer to the real match quality. This step further improves the purity of the matching point pairs, ensuring that the retained point pairs meet both feature similarity and geometric rationality. Overall, the three-step processing is progressive: first, spatial constraints reduce invalid computations; then, Lightglue's context-aware matching improves initial accuracy; and finally, GIM's geometric and viewpoint constraints eliminate hidden mismatches and optimize confidence. The resulting set of matching point pairs is not only of moderate quantity and has a low mismatch rate, but also has reliable confidence, providing high-quality input for solving the homography transformation matrix. It reduces out-of-place interference, making matrix estimation more stable. Especially in scenes with sparse textures and large differences in camera parameters, it can significantly improve the accuracy and robustness of image registration, avoiding registration shifts or jitter caused by poor matching quality.

[0060] S105. Based on the confidence score, multiple sets of matching points are recombined and integrated to obtain a recombined feature point set.

[0061] In practice, multiple sets of matching points are sorted from high to low according to their confidence scores. All matching points with confidence scores higher than the first preset threshold and their corresponding descriptors are extracted to form a recombined feature point set.

[0062] For example, in an embodiment, the first preset threshold is 0.95, and in this embodiment, all matching key point pairs and their corresponding descriptors with a confidence score higher than 0.95 are reorganized and integrated to obtain:

[0063] ;

[0064] The reorganized key points in the wide field of view image are obtained. The reorganized descriptors corresponding to the wide field of view image are obtained. The wide field of view key points before reorganization are obtained. The wide field of view descriptors before reorganization are obtained.

[0065] S106, construct a fixed-length container to store the reorganized feature point set, stitch all reorganized feature points in the fixed-length container with feature points of the wide field of view reference image, solve a homographic transformation matrix of the narrow field of view image to the wide field of view image, and perform image registration based on the homographic transformation matrix.

[0066] Specifically, the homographic transformation matrix is a 3x3 matrix used to describe the perspective transformation relationship between two planes. In image registration, the homographic transformation matrix can map points in the narrow field of view image to corresponding positions in the wide field of view image, achieving spatial alignment between images.

[0067] In a specific implementation, the method for constructing a fixed-length container to store the reorganized feature point set comprises: determining the capacity of the fixed-length container based on the frame rate of the narrow field of view image, the feature matching time consumption of the wide field of view and the narrow field of view images, and the system allowed computing load; the capacity is the maximum number of frames of the reorganized feature point set that can be stored; defining the storage structure of the fixed-length container based on the capacity and the composition of the reorganized feature point set; each unit of the storage structure is used to store a complete reorganized feature point set, and the number of units of the storage structure is equal to the capacity; initializing the fixed-length container, assigning an index to each unit of the storage structure based on the time sequence of the acquisition of multiple narrow field of view image frames; the index value is positively correlated with the time sequence; and storing the reorganized feature point set in the storage units of the fixed-length container in sequence according to the corresponding narrow field of view image frame acquisition time, to obtain the fixed-length container.

[0068] ​Specifically, the capacity of the fixed-length container is calculated and determined (such as 20 frames) in combination with the frame rate of the narrow field of view image, the time consumption of the wide field of view and the narrow field of view image feature matching, and the computing load (such as the upper limit of single frame processing power) allowed by the system. According to the determined capacity and the composition of the reorganized feature point set, the storage structure of the fixed-length container is defined: in the form of an array or a queue, each storage unit corresponds to a data structure (such as a structure or a dictionary) for storing all information of a group of reorganized feature point sets completely, and the total number of units of the storage structure is consistent with the capacity. The fixed-length container is initialized, and each unit of the storage structure is assigned a unique index (such as index 0 corresponding to the earliest collected frame, index 1 corresponding to the second earliest collected frame, and so on, and the index value increases with time) according to the time sequence of the collection of the multi-frame narrow field of view image frames. The generated reorganized feature point set is sequentially stored in the storage unit with a matching index in the fixed-length container according to the collection time of the corresponding narrow field of view image frame, and the construction of the fixed-length container is completed. The specific implementation process and implementation principle of the capacity determination of the fixed-length container can be referred to the description in the related art, which will not be repeated here.

[0069] Optionally, after the fixed-length container is constructed to store the reorganized feature point set, the method further includes: monitoring whether the number of the reorganized feature point sets stored in the fixed-length container reaches a capacity threshold value in real time; if the capacity threshold value is reached, identifying and removing the reorganized feature point set stored earliest in the fixed-length container, and adding the newly generated reorganized feature point set to the fixed-length container; if the capacity threshold value is not reached, directly adding the newly generated reorganized feature point set to the fixed-length container; wherein the storage time of the reorganized feature point set is based on the collection time of the corresponding narrow field of view image frame, and the reorganized feature point set stored earliest corresponds to the reorganized feature point generated by the narrow field of view image frame collected earliest in time.

[0070] The method provided by the embodiment can ensure that the reorganized feature point set retained in the container is always relatively new in time, avoid an increase in the calculation burden caused by too many feature points (for example, redundant calculation when a homography matrix is solved later), make the feature points participating in registration closer to the current scene state (reduce the influence of scene changes on matching accuracy caused by a large time span, such as feature offset caused by light changes or target movement), ensure the timeliness and continuity of the feature point set based on the replacement logic in time sequence, avoid interference of old feature points (which may have been invalidated due to scene changes) on current registration, and finally make the solved homography transformation matrix more accurately reflect the spatial correspondence relationship between the current wide-view field image and the narrow-view field image, reduce registration jitter caused by multiple frames of old features or out-of-control quantity, and improve the real-time performance and stability of registration.

[0071] In a specific implementation, the operation of splicing all the reorganized feature points in the fixed-length container with feature points of the wide-view field reference image, and solving a homography transformation matrix of the narrow-view field image to the wide-view field image includes: aligning a timestamp of the narrow-view field image frame with a spatial coordinate range of the wide-view field reference image, integrating the plurality of groups of reorganized feature points and the matched feature points of the wide-view field reference image stored in the fixed-length container in a time-stamp order to form a joint matching point set; the feature points include wide-view field key points and wide-view field descriptors; determining to-be-solved parameters of the homography transformation matrix; the homography transformation matrix represents a spatial mapping relationship between the narrow-view field image and the wide-view field image; determining a number of matching point pairs based on the number of to-be-solved parameters; matching a corresponding solving method based on the number of matching point pairs, and solving the homography transformation matrix based on the solving method.

[0072] Optionally, the operation of matching a corresponding solving method based on the number of matching point pairs, and solving the homography transformation matrix based on the solving method includes: when the number of matching point pairs is not greater than a preset number and the multiple matching point pairs are not collinear, substituting coordinate information of each matching point pair into a linear equation set of the homography transformation, using a matrix inversion operation to solve the to-be-solved parameters, and obtaining the homography transformation matrix; when the number of matching point pairs is greater than the preset number, fitting all the matching point pairs based on a least square method, optimizing all the to-be-solved parameters of the homography transformation matrix to obtain a preliminary homography transformation matrix, inputting the preliminary homography transformation matrix and all the matching point pairs into a random sample consensus algorithm for robust estimation, and finally solving the homography transformation matrix again based on the filtered matching point pairs.

[0073] In a specific implementation, the multiple sets of reorganized feature point pairs stored in the fixed-length container are sorted according to the time stamps of the narrow field of view image frames. According to the initial homographic transformation relationship between the narrow field of view image and the wide field of view reference image, the narrow field of view key point coordinates are mapped to the wide field of view coordinate system to ensure that all feature points are in the same spatial coordinate system. The reorganized feature point pairs in the fixed-length container are traversed in the time stamp order, and the wide field of view key point coordinates and their descriptors in each pair are extracted. The extracted wide field of view key points are merged with the feature points (pre-extracted wide field of view key points and descriptors) of the wide field of view reference image to form a joint matching point set containing multiple frame information. The homographic transformation matrix is a 3x3 matrix containing 8 independent parameters (since it is scale invariant, the last element is usually set to 1). Since each matching point pair can provide two equations (corresponding to x and y coordinate mapping), at least 4 pairs of non-collinear matching points are required to solve 8 parameters. To improve robustness, the actual number of matching point pairs is 3-5 times the minimum required number (such as 12-20 pairs), and the highest confidence point pairs are selected from the joint matching point set. When the number of matching point pairs is 4, the direct linear transformation (DLT) algorithm is used to solve the homographic matrix. When the number of matching point pairs is more than 4, the RANSAC algorithm is used for iterative solution: 4 matching points are randomly selected each time to calculate a candidate matrix. The re-projection error of all point pairs under the candidate matrix is calculated, and the point pairs with an error less than a threshold (such as 3 pixels) are regarded as inliers. Repeat the iteration (such as 1000 times), and select the matrix with the most inliers as the optimal solution. The final matrix is normalized to obtain the homographic transformation matrix.

[0074] For example, in an embodiment, the joint matching point set can be represented as:

[0075] ;

[0076] wherein the is the feature point after splicing; the is the descriptor after splicing; the is the wide field of view key point; the is the wide field of view descriptor; the , is the reorganized feature point set.

[0077] ;

[0078] ;

[0079] wherein the is the wide field of view key point set; the is the narrow field of view key point set; the is the homographic transformation matrix; the , The coordinates of a point in a wide field-of-view image; , Let be the coordinates of the point in the narrow field-of-view image.

[0080] The method provided in this embodiment, in its first aspect, organically links the various stages of data acquisition and registration through the core logic of multi-frame fusion, forming an efficient and robust processing chain: acquiring continuous multi-frame narrow field-of-view images instead of a single frame, introducing redundant information in the temporal dimension to compensate for the lack of feature points in a single frame under sparse texture and occluded scenes; cropping the wide field-of-view reference image to focus on the local area overlapping with the narrow field of view, reducing invalid calculations in irrelevant areas and improving processing efficiency; extracting highly stable key points and descriptors through the SuperPoint algorithm combined with a general image matcher, providing a reliable feature basis for matching; and then through multi-frame matching, recombination, and stitching, finally using sufficient high-quality matching point pairs to solve the homography matrix, thus solving the problem of unstable registration in existing methods under sparse texture and camera parameter differences, and achieving high-precision spatial alignment between wide and narrow field-of-view images. Secondly, the matching process significantly improves matching quality through layered processing: first, feature point pairs are filtered by limiting the spatially overlapping areas to avoid redundant matching in irrelevant areas; then, a lightweight glue algorithm is used to analyze feature associations and filter candidate point pairs by combining spatial location constraints, and an initial confidence score is generated by cosine similarity to capture the contextual association between features and reduce mismatches caused by repeated textures; finally, a general image matcher is used to eliminate hidden mismatches caused by differences in camera parameters by combining viewpoint consistency, and the confidence score is calibrated so that the final matched point pairs meet both feature similarity and geometric rationality, providing high-purity matching data for subsequent registration. Thirdly, the reconstruction and stitching process optimizes the matrix solution through multi-frame information fusion: based on confidence, the reconstructed and integrated matching points aggregate stable matching results with high confidence from multiple frames, compensating for the deficiency of insufficient matching points in a single frame; a fixed-length container stores the reconstructed feature points, preserving temporally continuous multi-frame information in chronological order while avoiding computational pressure caused by excessive data volume through capacity control; all reconstructed feature points and wide-field feature points in the stitching container provide sufficient and reliable matching point pairs for homography matrix solution, and combined with the RANSAC robust algorithm to remove outliers, making the matrix estimation more stable, ultimately improving the registration accuracy and noise resistance, and reducing jitter in multi-frame registration.

[0081] Corresponding to the aforementioned embodiment of an image registration method based on multi-frame fusion, this application also provides an embodiment of an image registration apparatus based on multi-frame fusion.

[0082] Figure 2 This is a schematic diagram of the image registration device based on multi-frame fusion provided in Embodiment 2 of this application. Please refer to... Figure 2The device provided in the embodiment comprises an acquisition module 210, a clipping module 220, an extraction module 230, a matching module 240, a recombination module 250 and a solving module 260.

[0083] The acquisition module 210 is configured to acquire a wide-view field reference image and a plurality of continuous narrow-view field image frames.

[0084] The clipping module 220 is configured to clip a region of the wide-view field reference image to obtain a clipped wide-view field local region image.

[0085] The extraction module 230 is configured to extract wide-view field key points and corresponding wide-view field descriptors from the clipped wide-view field local region image, and extract narrow-view field key points and corresponding narrow-view field descriptors from each of the narrow-view field image frames.

[0086] The matching module 240 is configured to match the wide-view field key points, the wide-view field descriptors, the narrow-view field key points and the narrow-view field descriptors of each of the narrow-view field image frames to obtain a plurality of groups of matching point pairs and corresponding confidence scores.

[0087] The recombination module 250 is configured to recombine and integrate the plurality of groups of matching points based on the confidence scores to obtain a recombined feature point set.

[0088] The solving module 260 is configured to construct a fixed-length container to store the recombined feature point set, splice all the recombined feature points in the fixed-length container with feature points of the wide-view field reference image, solve a homographic transformation matrix of the narrow-view field image to the wide-view field image, and perform multi-frame image registration based on the homographic transformation matrix.

[0089] The device of the embodiment can be used to execute the steps of the method embodiment, and the specific implementation principle and implementation process are similar, which will not be described here. Figure 1 The functions and roles of the units in the device are achieved in the implementation process of the corresponding steps in the above method, which will not be described here.

[0090] The functions and roles of the units in the device are achieved in the implementation process of the corresponding steps in the above method, which will not be described here.

[0091] For the device embodiment, since it basically corresponds to the method embodiment, the related parts can be referred to the part of the method embodiment. The device embodiment described above is only illustrative, and the units described as separate components can be or can not be physically separated, and the components displayed as units can be or can not be physical units, that is, they can be located in one place or distributed on multiple network units. According to the actual needs, part or all of the modules can be selected to achieve the purpose of the scheme of the present application. Those skilled in the art can understand and implement it without creative labor.

[0092] The above description is only the preferred embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of protection of the present application.

Claims

1. An image registration method based on multi-frame fusion, characterized in that, The method includes: Acquire a wide field-of-view reference image and multiple consecutive narrow field-of-view image frames; The wide field of view reference image is cropped to obtain a cropped wide field of view local region image; Wide field of view key points and corresponding wide field of view descriptors are extracted from the cropped wide field of view local region image, and narrow field of view key points and corresponding narrow field of view descriptors are extracted from each of the narrow field of view image frames. The wide field of view key points and wide field of view descriptors are matched with the narrow field of view key points and narrow field of view descriptors of each narrow field of view image frame to obtain multiple sets of matching point pairs and corresponding confidence scores. Based on the confidence score, multiple sets of matching points are recombined and integrated to obtain a recombined feature point set; A fixed-length container is constructed to store the reconstructed feature point set. All reconstructed feature points in the fixed-length container are stitched together with feature points of a wide-field-of-view reference image. The homography transformation matrix from the narrow-field-of-view image to the wide-field-of-view image is solved, and image registration is performed based on the homography transformation matrix. The capacity of the fixed-length container is determined based on the frame rate of the narrow-field-of-view image, the feature matching time between the wide-field-of-view and narrow-field-of-view images, and the computational load allowed by the system. The capacity is the maximum number of frames that can store the reconstructed feature point set.

2. The method according to claim 1, characterized in that, The process of matching the wide field-of-view keypoints and wide field-of-view descriptors with the narrow field-of-view keypoints and narrow field-of-view descriptors of each narrow field-of-view image frame yields multiple sets of matching point pairs and corresponding confidence scores, including: Identify feature point pairs within the spatial overlap region between the cropped wide field-of-view local region image and each frame of the narrow field-of-view image; each feature point pair includes a wide field-of-view descriptor and a narrow field-of-view descriptor; The feature association between wide field-of-view descriptors and narrow field-of-view descriptors in the spatially overlapping region is analyzed based on the lightweight glue algorithm. Candidate point pairs are selected based on the spatial location of the feature point pairs and the feature association. The similarity of the candidate point pairs is generated based on cosine similarity and used as the initial confidence score. The candidate point pairs and their corresponding initial confidence scores are input into a general image matcher. The general image matcher combines the viewpoint consistency between wide and narrow field-of-view images to eliminate mismatched point pairs caused by differences in camera parameters. The initial confidence scores of the retained candidate point pairs are calibrated to obtain multiple sets of matched point pairs and calibrated confidence scores.

3. The method according to claim 1, characterized in that, The step of concatenating all reconstructed feature points in the fixed-length container with feature points of the wide-field-of-view reference image, and solving the homography transformation matrix from the narrow-field-of-view image to the wide-field-of-view image, includes: Align the timestamps of the narrow field-of-view image frames with the spatial coordinate range of the wide field-of-view reference image, and integrate the multiple sets of recombined feature point pairs stored in the fixed-length container and the feature points of the matched wide field-of-view reference image in the order of acquisition timestamps to form a joint matching point set; the feature points include wide field-of-view key points and wide field-of-view descriptors; The parameters to be solved for the homography transformation matrix are determined; the homography transformation matrix represents the spatial mapping relationship between the narrow field of view image and the wide field of view image. The number of matching point pairs is determined based on the number of parameters to be solved; Based on the solution method corresponding to the number of matching point pairs, the homography transformation matrix is ​​solved based on the solution method.

4. The method according to claim 3, characterized in that, The solution method based on the number matching of the matching point pairs, and the solution method for solving the homography transformation matrix, include: When the number of matching point pairs is not greater than the preset number and multiple matching point pairs are not collinear, the coordinate information of each pair of matching points is substituted into the linear equation system of homography transformation, and the parameters to be solved are obtained by matrix inversion operation to obtain the homography transformation matrix. When the number of matching point pairs is greater than the preset number, all matching point pairs are fitted based on the least squares method. With the goal of minimizing the sum of squared errors, all parameters to be solved in the homography transformation matrix are optimized to obtain the preliminary homography transformation matrix. The initial homography transformation matrix and all matching point pairs are input to the random sampling consensus algorithm for robust estimation. Matching point pairs that conform to the spatial mapping relationship are selected through iteration and erroneous matching point pairs are eliminated. Finally, the homography transformation matrix is ​​obtained again based on the selected matching point pairs.

5. The method according to claim 1, characterized in that, Extracting wide field-of-view key points and corresponding wide field-of-view descriptors from the cropped wide field-of-view local region image, and extracting narrow field-of-view key points and corresponding narrow field-of-view descriptors from each of the narrow field-of-view image frames, including: A feature point extractor is used to detect and extract wide field-of-view key points from the cropped wide field-of-view local region image, and at the same time, a wide field-of-view descriptor corresponding to each wide field-of-view key point is generated; the wide field-of-view descriptor represents the local feature information of the wide field-of-view key point. The same feature point extractor is used to detect and extract narrow field-of-view key points from each narrow field-of-view image frame, and a narrow field-of-view descriptor corresponding to each narrow field-of-view key point is generated; wherein, the wide field-of-view key points and the narrow field-of-view key points are corresponding points that have a spatial mapping relationship.

6. The method according to claim 5, characterized in that, The step involves using a feature point extractor to detect and extract wide-field-of-view key points from the cropped wide-field-of-view local region image, and simultaneously generating a wide-field-of-view descriptor corresponding to each key point, including: The cropped wide field-of-view local region image is processed based on the super point algorithm. Target pixels in the wide field-of-view local region image are detected through a convolutional neural network architecture and used as initial wide field-of-view key points. Basic descriptors corresponding to the initial wide field-of-view key points are generated. The difference between the grayscale distribution and neighborhood structure of the target pixel and the surrounding pixels is greater than a preset value. The target pixel can still be stably detected when the image is scaled, rotated, or the illumination changes. The initial wide field-of-view key points and basic descriptors are input into a general image matcher. The general image matcher removes pseudo key points that are susceptible to noise interference based on the stability of the initial wide field-of-view key points, and performs feature enhancement on the basic descriptors corresponding to the retained stable initial wide field-of-view key points to obtain the final wide field-of-view key points and corresponding wide field-of-view descriptors.

7. The method according to claim 1, characterized in that, The step of cropping the wide field-of-view reference image to obtain a cropped wide field-of-view local region image includes: Using the four corner points of the narrow field of view image as the original corner points, the original corner points are mapped to the wide field of view reference image through a preset initial homography transformation relationship, resulting in the four corresponding mapped corner points in the wide field of view reference image; the narrow field of view image is a frame in a series of narrow field of view image frames, and the initial homography transformation relationship is used to initially determine the spatial overlap range between the narrow field of view image and the wide field of view image; Based on the four mapped corner points, a closed region is defined in the wide field of view reference image, and the closed region is the region of interest to be cropped. Based on the coordinate range of the four mapping corner points, the wide field of view reference image is cropped, retaining the image content within the region of interest and removing the image portion outside the region of interest, to obtain the cropped wide field of view local region image.

8. The method according to claim 1, characterized in that, The construction of a fixed-length container to store the recombined feature point set includes: Based on the capacity and the composition of the recombined feature point set, a storage structure for a fixed-length container is defined; each unit of the storage structure is used to store a complete set of recombined feature points, and the number of units in the storage structure is equal to the capacity. The fixed-length container is initialized, and an index is assigned to each unit of the storage structure based on the acquisition time order of multiple narrow field-of-view image frames; the index value is positively correlated with the time order. The recombined feature point set is sequentially stored into the storage unit of a fixed-length container according to the acquisition time of the corresponding narrow field-of-view image frame, thus obtaining a fixed-length container.

9. The method according to claim 1, characterized in that, After constructing a fixed-length container to store the recombined feature point set, the method further includes: Real-time monitoring of whether the number of frames in the set of recombined feature points stored in the fixed-length container has reached the capacity threshold; If the capacity threshold has been reached, identify and remove the set of recombined feature points with the earliest storage time in the fixed-length container, and add the latest generated set of recombined feature points to the fixed-length container. If the capacity threshold is not reached, the newly generated set of recombined feature points is directly added to the fixed-length container; wherein, the storage time of the set of recombined feature points is based on the acquisition time of the corresponding narrow field-of-view image frame, and the earliest stored set of recombined feature points corresponds to the recombined feature points generated by the earliest acquired narrow field-of-view image frame in time.

10. An image registration device based on multi-frame fusion, characterized in that, The device includes an acquisition module, a cropping module, an extraction module, a matching module, a recombination module, and a solution module; The acquisition module is used to acquire a wide field-of-view reference image and multiple consecutive narrow field-of-view image frames. The cropping module is used to crop the wide field of view reference image to obtain a cropped wide field of view local region image. The extraction module is used to extract wide field-of-view key points and corresponding wide field-of-view descriptors from the cropped wide field-of-view local region image, and to extract narrow field-of-view key points and corresponding narrow field-of-view descriptors from each frame of the narrow field-of-view image. The matching module is used to match the wide field of view key points and wide field of view descriptors with the narrow field of view key points and narrow field of view descriptors of each frame of narrow field of view image to obtain multiple sets of matching point pairs and corresponding confidence scores. The recombination module is used to recombine and integrate multiple sets of matching points based on the confidence score to obtain a recombined feature point set. The solution module is used to construct a fixed-length container to store the recombined feature point set, stitch all the recombined feature points in the fixed-length container with the feature points of the wide field of view reference image, solve the homography transformation matrix from the narrow field of view image to the wide field of view image, and perform multi-frame image registration based on the homography transformation matrix; wherein, the capacity of the fixed-length container is determined based on the frame rate of the narrow field of view image, the feature matching time of the wide field of view image and the narrow field of view image, and the computational load allowed by the system, and the capacity is the maximum number of frames that can store the recombined feature point set.

Images