Remote sensing image automatic registration method, device, equipment, medium and program product

Abstract

This application relates to a method, apparatus, device, medium, and program product for automatic registration of remote sensing images. The method includes: determining the effective overlapping and non-overlapping regions of a reference image and an image to be registered; adaptively selecting a global registration process or a block registration process based on the size of the effective overlapping region to obtain corresponding image pairs for registration processing; filtering feature matching point pairs that meet a preset confidence threshold; performing geometric transformation estimation and image correction adapted to the corresponding registration process on the effective overlapping and non-overlapping regions based on the feature matching point pairs; and, in the case of executing the block registration process, performing intelligent stitching through a globally optimal seam fusion algorithm based on dynamic programming to generate a visually seamless second registration result image. This method can improve the robustness and accuracy of feature matching in complex scenes, effectively eliminate visual artifacts in the stitching of registered images, and achieve visually seamless fusion.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, device, medium, and program product for automatic registration of remote sensing images. Background Technology

[0002] With the rapid development of space-based Earth observation technology, the spatiotemporal and spectral resolution of remote sensing satellites has been continuously improved, resulting in a massive increase in the amount of multi-temporal, multi-view, and multi-sensor remote sensing image data that can be acquired. Remote sensing image registration, as a core foundational technology for remote sensing data processing and analysis, is a prerequisite for advanced applications such as multi-source remote sensing image fusion, temporal change detection, target tracking and positioning, land resource surveys, and disaster emergency monitoring. Its registration accuracy and robustness directly determine the reliability of subsequent quantitative remote sensing analysis.

[0003] In traditional techniques, remote sensing image registration mainly employs matching methods based on manually designed features. These methods extract local features invariant to geometric and radiometric changes from the image using predefined operators, and then estimate the transformation relationships between images through feature matching to complete the registration. In recent years, with the development of deep learning technology, registration schemes based on deep neural networks have emerged.

[0004] However, the relevant registration schemes still have significant technical defects: First, traditional manual feature design relies on prior knowledge. In complex scenarios such as weak texture, repetitive texture, large viewpoint changes, seasonal lighting differences, and cloud and fog obstruction common in remote sensing images, the repeatability and discriminability of features decrease significantly, easily leading to insufficient matching point pairs and a surge in mismatch rates. Second, existing deep learning registration schemes are difficult to adapt to the local nonlinear geometric distortions caused by terrain undulations and platform attitude changes in large-frame remote sensing images. Third, existing image stitching after block registration often adopts direct superposition or simple linear fusion methods, which easily produces visual artifacts such as stitching seams, ghosting, and texture misalignment, seriously damaging the geometric consistency and visual continuity of the image and introducing uncontrollable errors into subsequent quantitative analysis of remote sensing. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, device, equipment, medium, and program product for automatic registration of remote sensing images that can improve the robustness and accuracy of feature matching in complex scenarios, effectively eliminate visual artifacts in image stitching after registration, and achieve seamless visual fusion, in order to address the above-mentioned technical problems.

[0006] Firstly, this application provides an automatic registration method for remote sensing images. The method includes:

[0007] Acquire a reference image and an image to be registered and perform coordinate alignment. Determine the effective overlapping and non-overlapping regions of the reference image and the image to be registered. Adaptively select a global registration process or a block registration process based on the size of the effective overlapping region to obtain the corresponding image pair for registration processing.

[0008] Image key points and feature description vectors of the image pairs are extracted by a self-supervised deep network. After feature context enhancement and cross-image information interaction through graph neural networks based on attention mechanism, feature matching point pairs that meet the preset confidence threshold are obtained by optimal matching solution.

[0009] Based on the feature matching point pairs, geometric transformation estimation and image correction for the corresponding registration process are performed on the effective overlapping region and the non-overlapping region, respectively.

[0010] When performing a global registration process, a first registration result image with a coordinate system consistent with the reference image is generated;

[0011] When performing a block registration process, intelligent stitching is performed using a globally optimal seam fusion algorithm based on dynamic programming to generate a visually seamless second registration result image.

[0012] In some embodiments of the method, the step of adaptively selecting a global registration process or a block registration process based on the size of the effective overlapping region to obtain the corresponding registered image pair includes:

[0013] Calculate the width and height of the bounding rectangle of the effective overlapping area, and compare the maximum value of the width and height with a preset size threshold;

[0014] If the maximum value is not greater than a preset size threshold, it is determined that a global registration process will be executed, and the reference image and the image to be registered will be used as an image pair for registration processing.

[0015] If the maximum value is greater than the preset size threshold, the block registration process is executed. According to the preset cropping size and preset overlap rate, the effective overlapping areas of the reference image and the image to be registered are divided into regular grids to generate multiple pairs of reference image blocks and image blocks to be registered. For the boundary image blocks adjacent to the non-overlapping area, the preset pixel expansion processing is performed on the non-overlapping area. Finally, each pair of image blocks is used as the image pair for registration processing.

[0016] In some embodiments of the method, the step of extracting image keypoints and feature description vectors of the image pairs through a self-supervised deep network, performing feature context enhancement and inter-graph cross-attention cross-image information interaction through an attention-based graph neural network, and then filtering feature matching point pairs that meet a preset confidence threshold through optimal matching solution includes:

[0017] Subpixel-level key points of the image pairs are extracted using a self-supervised deep network, and normalized feature description vectors are generated for each key point.

[0018] The key points of the image pairs are constructed into nodes of a fully connected graph using a graph neural network based on an attention mechanism. The position and confidence of the key points are then fused with the feature description vector to generate an initial feature vector.

[0019] Based on the initial feature vector, feature enhancement is performed through alternating propagation of self-attention and cross-attention, and then feature matching point pairs that meet the preset confidence threshold are selected through the optimal transmission matching algorithm.

[0020] In some embodiments of the method, the step of performing geometric transformation estimation and image correction for the effective overlapping region and the non-overlapping region, respectively, based on the feature matching point pairs, to adapt to the corresponding registration process includes:

[0021] When performing a global registration process, the global homography transformation matrix of the effective overlapping region is estimated using a random sampling consistency algorithm based on the feature matching point pairs, and the overall resampling correction is performed on the effective overlapping region using the global homography transformation matrix.

[0022] When performing the block registration process, the local homography transformation matrix is ​​estimated for the feature matching point pairs corresponding to each group of image blocks using the random sampling consistency algorithm. After the inlier number threshold is checked, the valid image blocks are resampled and corrected block by block, while the original pixel data of the invalid image blocks are retained.

[0023] The reliable feature matching point pairs that have passed the verification within the effective overlapping area are aggregated, and after deduplication, a global homography transformation matrix is ​​obtained by least squares fitting. The global homography transformation matrix is ​​then used to perform geometric transformation and resampling correction on the non-overlapping area.

[0024] In some embodiments of the method, the step of generating a visually seamless second registration result image by performing intelligent stitching through a globally optimal seam fusion algorithm based on dynamic programming, while executing a block registration process, includes:

[0025] The corrected image block data and the non-overlapping region correction data are mapped to the unified coordinate space of the reference image according to the corresponding spatial location information to identify the edge overlap region and corner overlap region between image blocks.

[0026] For the edge overlap region and the corner overlap region, the optimal seam finding and feathering fusion processing are performed respectively by the global optimal seam fusion algorithm based on dynamic programming. After all overlap regions have been processed, a visually seamless second registration result image that is consistent with the coordinate system of the reference image is synthesized.

[0027] In some embodiments of the method, the optimal seam finding and feathering fusion processing for the edge overlap region and the corner overlap region respectively is performed using a global optimal seam fusion algorithm based on dynamic programming, including:

[0028] For the edge overlap region, a stitching cost function that fuses pixel color differences and gradient consistency is constructed. The optimal seam line with the least visual trace is searched by a dynamic programming algorithm. After image segmentation is completed along the optimal seam line, linear gradient feathering fusion is performed in the transition zone with a preset width on both sides of the seam line.

[0029] For the corner overlapping area, a multi-round step-by-step splicing strategy is adopted, which first splices vertically and then horizontally, and then performs optimal seam finding and feathering fusion processing on the overlapping areas in the vertical and horizontal directions in turn.

[0030] According to a second aspect of the present disclosure, an automatic registration device for remote sensing images is provided. The device includes:

[0031] The processing module is used to acquire a reference image and an image to be registered and perform coordinate alignment, determine the effective overlapping area and non-overlapping area of ​​the reference image and the image to be registered, and adaptively select a global registration process or a block registration process according to the size of the effective overlapping area to obtain the corresponding image pair for registration processing.

[0032] The feature extraction module is used to extract image key points and feature description vectors of the image pairs through a self-supervised deep network. After feature context enhancement and cross-image information interaction through graph neural networks based on attention mechanism, feature matching point pairs that meet the preset confidence threshold are obtained by optimal matching solution.

[0033] The correction module is used to perform geometric transformation estimation and image correction for the effective overlapping region and the non-overlapping region respectively, based on the feature matching point pairs, to adapt to the corresponding registration process.

[0034] The output module is used to generate a first registration result image with a coordinate system consistent with the reference image when performing a global registration process;

[0035] The stitching module is used to perform intelligent stitching using a globally optimal seam fusion algorithm based on dynamic programming during the block registration process, generating a visually seamless second registration result image.

[0036] According to a third aspect of the present disclosure, a computer device is provided. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the above-described automatic registration method for remote sensing images.

[0037] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the above-described automatic registration method for remote sensing images.

[0038] According to a fifth aspect of the present disclosure, a computer program product is provided. The computer program product includes a computer program that, when executed by a processor, implements the above-described automatic registration method for remote sensing images.

[0039] The automatic remote sensing image registration scheme provided in this application can solve the problems of insufficient robustness in matching complex scenes, weak local distortion correction capability of large-format images, and severe artifacts in block registration and stitching in traditional technologies. The adaptive registration process balances registration efficiency and correction accuracy, the cascaded deep network significantly improves the reliability of feature matching in complex remote sensing scenes, the differentiated correction strategy achieves the unity of local fine distortion correction and global geometric consistency, and the intelligent stitching algorithm eliminates stitching visual artifacts. Finally, it realizes high-precision and high-robust automated registration of remote sensing images in multiple scenes, providing a reliable foundation for subsequent quantitative applications of remote sensing.

[0040] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0041] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0042] Figure 1 This is a flowchart illustrating an automatic registration method for remote sensing images according to an exemplary embodiment;

[0043] Figure 2 This is a schematic diagram illustrating the specific process of an automatic registration method for remote sensing images according to an exemplary embodiment.

[0044] Figure 3 This is a flowchart illustrating an intelligent stitching process according to an exemplary embodiment;

[0045] Figure 4 This is a schematic diagram illustrating the specific process of intelligent splicing steps according to an exemplary embodiment;

[0046] Figure 5 This is a comparison diagram of the automatic registration effect of remote sensing images according to an exemplary embodiment;

[0047] Figure 6 This is a comparison diagram of splicing effects according to an exemplary embodiment;

[0048] Figure 7 This is a structural block diagram of an automatic registration apparatus for remote sensing images according to an exemplary embodiment;

[0049] Figure 8 This is a diagram illustrating the internal structure of a computer device according to an exemplary embodiment. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0051] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure. The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded. For example, the use of terms such as "first," "second," etc., to denote names does not indicate any specific order.

[0052] In some embodiments provided in this disclosure, the execution of the automatic registration method for remote sensing images can be controlled by a unified controller or by multiple controllers. These controllers may include controllers on local terminals or controllers on remote servers. In some embodiments, the controllers on local terminals and the controllers on servers may work together to complete the registration processing of remote sensing images. The local terminal mentioned in this disclosure may include, but is not limited to, various robotic devices, vehicle-mounted devices, personal computers, laptops, smartphones, tablets, wearable devices, medical devices, VR (Virtual Reality) devices, etc. The server may also be a server, server cluster, distributed subsystem, cloud processing platform, server containing blockchain nodes, or a combination thereof. The controllers described in this disclosure may include various control units capable of implementing logic processing functions, including but not limited to CPU (Central Processing Unit), PLC (Programmable Logic Controller), ECU (Electronic Control Unit), MCU (Microcontroller Unit), FPGA (Field Programmable Gate Array), and CPLD (Complex Programmable Logic Device), as well as controllers composed of one or more logic function units, chips, etc.

[0053] In some embodiments of this disclosure, an automatic registration method for remote sensing images is provided, such as... Figure 1 As shown, it includes the following steps:

[0054] S20. Acquire the reference image and the image to be registered and perform coordinate alignment. Determine the effective overlapping area and non-overlapping area of ​​the reference image and the image to be registered. Adaptively select the global registration process or the block registration process according to the size of the effective overlapping area to obtain the corresponding image pair for registration processing.

[0055] A reference image typically refers to an image used as a spatial coordinate reference during remote sensing image registration. After precise geometric correction and geocoding, it possesses a stable and authoritative spatial coordinate system, providing a unified alignment standard for the images to be registered. Reference images can include baseline remote sensing images. In scenarios such as multi-source image fusion, reference images can be selected from remote sensing images that are temporally stable, have clear ground feature characteristics, and whose geometric accuracy meets industry application requirements.

[0056] Images to be registered typically refer to images that need to be spatially aligned with a reference image. These images can originate from different imaging times, observation perspectives, or sensor platforms. They may exhibit geometric distortions, coordinate system differences, or variations in radiometric characteristics, requiring registration operations to correct them to a spatial framework consistent with the reference image. Similarly, images to be registered can include remote sensing images. Aligning the reference and target images with their coordinates can meet the needs of subsequent remote sensing data analysis and applications.

[0057] The effective overlap region usually refers to the image area where the reference image and the image to be registered intersect within the spatial coordinate range. This region contains the ground features shared by the two images and is the core area for realizing feature matching and solving the geometric transformation model. It is also the key carrier for achieving pixel-level alignment during the registration process.

[0058] Non-overlapping regions refer to unique image regions in the image to be registered that extend beyond the spatial coverage of the reference image. These regions do not have common ground features corresponding to the reference image, and geometric correction cannot be completed through direct feature matching. It is necessary to rely on the transformation rules of the overlapping regions to achieve reasonable spatial alignment.

[0059] Global registration typically refers to performing unified feature extraction, matching, and geometric transformation on the entire image. It can correct the entire image to be registered using a single transformation model and is suitable for registration scenarios where the image coverage is small and the overall geometric distortion is uniform.

[0060] The block registration process typically involves dividing a large-format image into multiple overlapping sub-image blocks according to a regular grid, performing feature matching and local transformation model solving for each sub-image block, and completing geometric correction block by block. This process is suitable for registration scenarios where the image coverage is wide and different regions have different nonlinear geometric distortions.

[0061] An image pair typically refers to a combination of a reference image and a target image, generated according to an adaptively selected registration process, used for subsequent feature extraction and matching operations. In a global registration process, a single image pair can be composed of a complete reference image and a complete target image. In a block registration process, multiple pairs of reference image blocks and target image blocks are generated after dividing the effective overlapping area into regular grids, and these can be used as input image pairs.

[0062] S22. Extract the image key points and feature description vectors of the image pair through a self-supervised deep network. After feature context enhancement and cross-image information interaction through graph neural networks based on attention mechanism, feature matching point pairs that meet the preset confidence threshold are obtained by optimal matching solution.

[0063] Self-supervised deep networks (SDNs) typically refer to core deep neural network models used for end-to-end image keypoint detection and feature description vector generation. In some implementations, SDNs can be trained through self-supervised learning, eliminating the need for manually labeled feature point data. They can extract sub-pixel-level keypoints from input remote sensing images that are robust to disturbances such as viewpoint changes, lighting differences, and image blur, and generate feature description vectors, replacing traditional manually designed feature extraction operators. This enables stable feature extraction in complex remote sensing scenarios. Image keypoints generally refer to pixels in an image that possess unique visual features and reflect the contours or textures of ground features.

[0064] Attention-based graph neural networks typically refer to core network models used to achieve accurate cross-image feature matching. This model aggregates contextual information of key points within a single image, enhances each key point's ability to perceive other features within the same image, and also enables cross-image information interaction between two images. This allows feature descriptions to be integrated with global matching association information, completing the enhancement processing of the original features and significantly improving the discriminative power of features and the global consistency of matching.

[0065] Optimal matching solution typically refers to the core computational process of achieving a precise one-to-one matching of key points between two images based on enhanced feature representations. This involves filtering correct matches and eliminating incorrect matches, ultimately finding the globally optimal matching result from a massive pool of candidate matches.

[0066] Preset confidence thresholds typically refer to the criteria used to screen highly reliable matching point pairs. They are critical values ​​for matching confidence that are pre-set based on the accuracy requirements of remote sensing image registration and the characteristics of the application scenario before the registration process begins.

[0067] Feature matching point pairs typically refer to the combinations of key points that correspond one-to-one with the reference image and the image to be registered, obtained after feature extraction, feature enhancement, and optimal matching.

[0068] S24. Based on the feature matching point pairs, perform geometric transformation estimation and image correction for the effective overlapping region and the non-overlapping region respectively to adapt to the corresponding registration process.

[0069] Geometric transformation estimation typically refers to the computational process of solving the spatial coordinate mapping relationship between a reference image and an image to be registered, based on highly reliable feature matching point pairs.

[0070] Image correction typically refers to the calculation process of spatial coordinate transformation and pixel resampling of the image to be registered, based on the obtained geometric transformation matrix.

[0071] S26. In the case of executing the global registration process, a first registration result image with a coordinate system consistent with the reference image is generated.

[0072] The first registration result image usually refers to the final registration result output after performing the global registration process.

[0073] S28. When performing the block registration process, intelligent stitching is performed by a global optimal seam fusion algorithm based on dynamic programming to generate a visually seamless second registration result image.

[0074] Globally optimal seam fusion algorithms typically refer to algorithms used to achieve seamless stitching of multiple corrected images. They fundamentally avoid visual artifacts such as seams, ghosting, and texture misalignment caused by traditional linear fusion.

[0075] The second registration result image usually refers to the final registered image output after the block registration process is executed and the global optimal seam fusion algorithm completes the intelligent stitching of the entire region.

[0076] Intelligent stitching is a refined, seamless stitching technology for registered remote sensing image blocks. It abandons the traditional method of direct overlay and simple linear fusion. It relies on algorithms to achieve this. In the unified coordinate space of the reference image, it uses dynamic programming to find the optimal seam line with the least visual traces. Combined with stitching strategies, it performs adaptation processing on different overlapping areas to eliminate visual artifacts such as stitching seams, ghosting, and texture misalignment. Finally, it synthesizes a geometrically consistent and visually seamless registered image, ensuring the spatial continuity and visual uniformity of the image.

[0077] In some embodiments of this disclosure, the problems of insufficient robustness in matching complex scenes, weak local distortion correction capability of large-format images, and severe artifacts in block registration and stitching that exist in traditional technologies can be solved. The adaptive registration process balances registration efficiency and correction accuracy, the cascaded deep network significantly improves the reliability of feature matching in complex remote sensing scenes, the differentiated correction strategy achieves the unity of local fine distortion correction and global geometric consistency, and the intelligent stitching algorithm eliminates stitching visual artifacts. Finally, high-precision and high-robust automated registration of remote sensing images in multiple scenes is achieved, providing a reliable foundation for subsequent quantitative applications of remote sensing.

[0078] In some embodiments of this disclosure, S20 includes:

[0079] Calculate the width and height of the bounding rectangle of the effective overlapping area, and compare the maximum value of the width and height with a preset size threshold;

[0080] If the maximum value is not greater than a preset size threshold, it is determined that a global registration process will be executed, and the reference image and the image to be registered will be used as an image pair for registration processing.

[0081] If the maximum value is greater than the preset size threshold, the block registration process is executed. According to the preset cropping size and preset overlap rate, the effective overlapping areas of the reference image and the image to be registered are divided into regular grids to generate multiple pairs of reference image blocks and image blocks to be registered. For the boundary image blocks adjacent to the non-overlapping area, the preset pixel expansion processing is performed on the non-overlapping area. Finally, each pair of image blocks is used as the image pair for registration processing.

[0082] The following is in conjunction with the appendix Figure 2 Further explanation is needed.

[0083] In some implementations, the width and height of the bounding rectangle of the effective overlapping region can be calculated, the maximum value of the width and height can be extracted, and this value can be compared with a preset size threshold. If the value is not greater than the preset size threshold, the current image is determined to be suitable for global registration, and the complete reference image and the image to be registered are directly used as image pairs for subsequent registration processing. If the value is greater than the preset size threshold, the current image is determined to be suitable for block registration. According to the preset cropping size and overlap rate, the effective overlapping region of the reference image and the image to be registered is divided into regular grids to generate multiple pairs of reference image blocks and image blocks to be registered. At the same time, for the boundary image blocks adjacent to the non-overlapping region, a certain number of pixels are extended into the non-overlapping region to ensure that there is sufficient overlap between the boundary block and the non-overlapping region. Finally, the divided pairs of image blocks are used as image pairs for subsequent registration processing.

[0084] In some examples, a baseline image is read first. With the image to be registered The image data and corresponding geocoding information (the reference image and the image to be registered can be Geotiff images with geographic coordinates) are used to complete the initial coordinate alignment of the two images. If there is a difference in the coordinate reference system between the two, the image to be registered is reprojected onto the coordinate reference system corresponding to the reference image to ensure that the two have a unified spatial coordinate reference.

[0085] After unifying the coordinate system, calculate the reference image. With the image to be registered The spatial coverage range of the two images is determined, and the intersection of their spatial ranges is extracted as the effective overlapping region A1. Simultaneously, the effective overlapping region A1 is recorded in the reference image. Spatial boundaries in coordinate system, images to be registered The remaining portion after deducting the effective overlapping region A1 is the non-overlapping region A2. After dividing the effective overlapping region A1 and the non-overlapping region A2, the width W and height H of the bounding rectangle of the effective overlapping region A1 are calculated, and the maximum value of the two is taken along with a preset size threshold. (For example, 2000 pixels) are compared, and the appropriate registration process is adaptively selected based on the comparison results, so as to obtain the image pair that needs to be processed under the corresponding registration process.

[0086] In other examples, such as if If the image is large, it is classified as a "large image," and a block-based registration process is executed; otherwise, it is classified as a "small image," and a global registration process is executed. For the global registration process, the complete reference image is directly used. With the image to be registered Image pairs for subsequent registration processing. For the block registration process, cropping is performed according to a preset size. With overlap rate (For example =2000, =0.2), respectively for and middle The corresponding region is divided into regular grids to generate a series of paired image patches. and It is important to note that, There is in and The boundary block, which is The boundary block and is usually less than or equal to the trim size. The boundary block can be extended by 1000 pixels into the non-overlapping area to ensure that the boundary block is aligned with the surrounding area. There is some overlap. The final output is: effective overlapping area. The corresponding image pairs or image patch pairs, and the non-overlapping regions of the images to be registered. .

[0087] In some embodiments of this disclosure, the registration process can be automated and adaptively selected, allowing the optimal processing path to be chosen based on the actual size of the image without manual intervention. This improves the automation level and scene adaptability of the registration method. During the block processing, by setting a reasonable inter-block overlap rate, sufficient common features between adjacent image blocks are ensured for matching and stitching, avoiding the problem of feature loss between blocks. The expansion processing of boundary blocks further ensures the smooth connection between image edge areas and non-overlapping areas, avoiding the problem of insufficient registration accuracy at image edges. This allows the block registration process to adapt to the high-precision registration requirements of the entire area of ​​large-format remote sensing images.

[0088] In some embodiments of this disclosure, S22 includes:

[0089] Subpixel-level key points of the image pairs are extracted using a self-supervised deep network, and normalized feature description vectors are generated for each key point.

[0090] The key points of the image pairs are constructed into nodes of a fully connected graph using a graph neural network based on an attention mechanism. The position and confidence of the key points are then fused with the feature description vector to generate an initial feature vector.

[0091] Based on the initial feature vector, feature enhancement is performed through alternating propagation of self-attention and cross-attention, and then feature matching point pairs that meet the preset confidence threshold are selected through the optimal transmission matching algorithm.

[0092] In some implementations, for input image pairs, sub-pixel-level keypoints can be extracted from the images using a self-supervised deep network. Simultaneously, a normalized feature description vector is generated for each detected keypoint. After keypoint and feature description extraction, a graph neural network based on an attention mechanism is used to construct nodes in fully connected graphs for the two sets of keypoints corresponding to the reference image and the image to be registered. The positional confidence of each keypoint and its corresponding feature description vector are fused to generate an initial feature vector for each node. Based on the generated initial feature vector, feature enhancement is performed through an alternating propagation mechanism of self-attention and cross-attention. Self-attention operations transmit information within the keypoint set of a single image, strengthening each keypoint's contextual awareness of other keypoints within the same image. Cross-attention operations transmit information between the keypoint sets of two images, enabling each keypoint to perceive information from all keypoints in the other image. After feature enhancement, an optimal transmission matching algorithm is used to solve the matching problem, selecting high-reliability feature matching point pairs that meet a preset confidence threshold.

[0093] In some examples, feature extraction and feature matching can be implemented using the SuperPoint and SuperGlue models. The SuperPoint model employs a shared convolutional encoder and a dual-branch decoding structure (detector head and descriptor head) to achieve end-to-end feature point detection and description. Specifically, it uses a VGG (Visual Geometry Group Network)-style encoder to extract multi-scale feature maps, followed by two independent decoder heads. The shared encoder contains four convolutional blocks, three 2×2 max-pooling operations, and an overall downsampling factor of 8, to extract multi-level image features, providing rich feature representations for subsequent detection and description. Its advantages include: shared weights reduce the number of parameters and ensure consistency between detected and described features. The detection head can generate a 65-channel feature map. (in: The first 64 channels correspond to an 8×8 local region, with each channel representing the probability that a specific location within that region is a feature point. The 65th channel is a special "non-feature point" channel, representing the probability that there are no feature points within the 8×8 region. Sub-pixel level keypoint coordinate sets are obtained through Softmax and non-maximum suppression. and its confidence score The descriptor head is used to first compute a dense descriptor graph. Then, based on the precise location of the keypoints, bilinear interpolation is used from the descriptor map to calculate the value of each detected keypoint. Extract an L2-normalized 256-dimensional description vector The SuperPoint model can predict stable feature points from a single image under virtual viewpoint changes without manual annotation through self-supervised training, thus exhibiting robustness to changes in viewpoint and illumination.

[0094] In some examples, the input to the SuperGlue model may include data from... of Feature points and from of Feature points The SuperGlue model can include an attention-map neural network and a differentiable optimal matching layer. It models the feature point matching problem as an optimal matching problem between sets of local feature descriptors, enhances the discriminative power of the feature descriptors through an attention mechanism, and utilizes a differentiable optimization layer to obtain the matching result. Each image's feature point set can be viewed as a fully connected graph node, and a multilayer perceptron is used to locate the position of each point. Confidence level and visual descriptors The features are merged into an initial feature vector; then, an alternating propagation residual connection mechanism using self-attention and cross-attention is employed to generate an enhanced representation. Among them, the self-attention layers are respectively in the set and Information is passed internally, enhancing each point's awareness of the context of other points within the same image. The cross-attention layer is used in the set... and Information is exchanged between images, allowing each point to perceive information from all points in another image. This step enables key point features within an image to be fused with other structural and matching information from within and between images.

[0095] Finally, optimal transport matching can be performed, and a scoring matrix can be calculated. ,in Point and The matching score. An optimal transport problem can be solved using the Sinkhorn algorithm (an iterative method for efficiently computing optimal transport problems through entropy regularization), outputting a normalized probability matrix that considers the possibility of mismatch. This matrix maximizes the overall matching score while satisfying the mutual consistency constraint, meaning that the points in the image are the best matches for each other. Finally, matching points with low scores are filtered out based on a confidence threshold, and only matching points that satisfy both the best match and high confidence are set as true matching point pairs. Finally, a set of high-confidence feature matching point pairs is output.

[0096] In some embodiments of this disclosure, end-to-end extraction of key points and feature descriptions can be achieved through self-supervised deep networks, enabling model training and optimization without the need for manual data annotation. This adapts to the current situation in the remote sensing field where it is difficult to obtain labeled data. At the same time, the generated feature description vectors are highly robust to interferences such as common changes in viewpoint, illumination, and seasons in remote sensing images. Through feature enhancement via attention mechanisms, the feature descriptions integrate spatial context information within the image and matching association information across images, significantly improving the feature discrimination ability. Furthermore, the optimal transmission matching algorithm achieves accurate matching that satisfies mutual consistency constraints, effectively reducing the mismatch rate in complex scenes. It can output a sufficient number of highly reliable matching point pairs even under harsh conditions such as weak texture, repeated texture occlusion, etc., thus improving the anti-interference ability and registration accuracy of the registration method from the core aspects.

[0097] In some embodiments of this disclosure, S24 includes:

[0098] When performing a global registration process, the global homography transformation matrix of the effective overlapping region is estimated using a random sampling consistency algorithm based on the feature matching point pairs, and the overall resampling correction is performed on the effective overlapping region using the global homography transformation matrix.

[0099] When performing the block registration process, the local homography transformation matrix is ​​estimated for the feature matching point pairs corresponding to each group of image blocks using the random sampling consistency algorithm. After the inlier number threshold is checked, the valid image blocks are resampled and corrected block by block, while the original pixel data of the invalid image blocks are retained.

[0100] The reliable feature matching point pairs that have passed the verification within the effective overlapping area are aggregated, and after deduplication, a global homography transformation matrix is ​​obtained by least squares fitting. The global homography transformation matrix is ​​then used to perform geometric transformation and resampling correction on the non-overlapping area.

[0101] In some implementations, the appropriate geometric transformation estimation and image correction operations can be performed according to the selected global registration process or block registration process. If the global registration process is performed, a global homography transformation matrix applicable to the entire effective overlapping region can be estimated based on all feature matching point pairs using a random sampling consistency algorithm. Then, using this global homography transformation matrix, overall resampling correction is performed on the effective overlapping region to complete pixel-level alignment between the effective overlapping region and the reference image.

[0102] In some examples, the input can be feature matching point pairs. ,original Effective overlapping area (or ), and non-overlapping regions For the global registration process, the set of matching point pairs can be used directly. A global homography transformation matrix acting on the entire region A1 is estimated using the RANSAC algorithm. It can be used right The corresponding effective overlapping area Perform overall resampling correction.

[0103] In some implementations, if a block registration process is performed, for each corresponding image block, the local homography transformation matrix corresponding to each image block can be estimated using a random sampling consensus algorithm based on the feature matching point pairs corresponding to that image block. At the same time, the quality of each local transformation matrix is ​​checked by using an inlier number threshold. Valid image blocks whose inlier number meets the threshold requirement are resampled and corrected block by block using the corresponding local transformation matrix. Invalid image blocks whose inlier number does not meet the threshold requirement retain their original image pixel data to avoid correction errors caused by invalid matching.

[0104] In some examples, for the block registration process, for each image block pair and their matching point pairs A local homography transformation matrix can be estimated using the RANSAC algorithm. Homography matrix It is a 3×3 matrix describing the projection transformation between two planes, satisfying... ,in Let be the homogeneous coordinates of the point. Statistical analysis is performed on each... number of interior points Set quality threshold (For example: 35). If If so, the block is considered a registrationable block and the record is made. Otherwise, the block is considered invalid, and its transformation matrix is ​​marked as an identity transformation. For each registerable block, its... For the original The corresponding region is resampled using reverse mapping (bilinear interpolation) to obtain the corrected local image. Invalid blocks retain their original pixel data.

[0105] In some implementations, after the correction of the effective overlapping area is completed, all reliable feature matching point pairs that have passed the quality check within the effective overlapping area can be aggregated. After deduplication, the global homography transformation matrix corresponding to the entire image is obtained by least squares fitting. Then, the geometric transformation and resampling correction of the non-overlapping area are performed using the global homography transformation matrix to achieve spatial connection between the non-overlapping area and the effective overlapping area.

[0106] In some examples, global transformation matrix synthesis involves calculating a reasonable geometric transformation model for the non-overlapping region A2. The strategy is to derive a global transformation using all reliable matching information obtained within the effective overlapping region A1. This can be achieved by collecting all matching point pairs contained in blocks (block paths) or global matches (global paths) that pass quality judgment, forming a deduplicated set of globally reliable matching points. It can be used. All point pairs are fitted with a global homography transformation matrix using the least squares method. This model represents the average or dominant geometric relationship based on reliable observations from the coordinate system of the image to be registered to the reference image. Non-overlapping region correction is performed using the data obtained in the previous step. Matrix, pair Non-overlapping regions Perform geometric transformations and resampling. Because... Without corresponding baseline image content, this correction is intended to be based on... The main geometric deformation trend learned from the region will The corrected parts are placed reasonably and smoothly into their respective positions in the output result. The final output is the corrected image or set of image patches, including the effective overlapping areas. All processing units (blocks or wholes) and non-overlapping regions And its absolute position information on the result canvas.

[0107] In some embodiments of this disclosure, the transformation matrix estimation is performed using a random sampling consensus algorithm, which effectively eliminates out-of-point interference in matching point pairs, ensuring the stability and accuracy of the transformation matrix solution. The quality verification mechanism set for the block registration process avoids local correction errors caused by invalid matching point pairs, guaranteeing the correction accuracy of each image block. By combining local and global transformations, fine-grained correction of nonlinear distortion in different regions of large-format images is achieved through local transformation, solving the problem that a single global transformation cannot adapt to locally differentiated distortions. Simultaneously, reasonable correction of non-overlapping regions is achieved through global transformation, ensuring the global geometric consistency of the entire image and avoiding spatial misalignment and global geometric discontinuity issues caused by block processing, while simultaneously balancing local registration accuracy and global spatial integrity.

[0108] In some embodiments of this disclosure, reference is made to Figure 3 S28 includes:

[0109] S282. The corrected image block data and the non-overlapping area correction data are mapped to the unified coordinate space of the reference image according to the corresponding spatial location information, and the edge overlap area and corner overlap area between the image blocks are identified.

[0110] S284. For the edge overlapping region and the corner overlapping region, respectively, the optimal seam finding and feathering fusion processing are performed by the global optimal seam fusion algorithm based on dynamic programming. After all overlapping regions have been processed, a visually seamless second registration result image consistent with the coordinate system of the reference image is synthesized.

[0111] In some implementations, after completing the geometric correction of all image blocks and non-overlapping regions, the corrected image block data and non-overlapping region correction data are mapped to a unified coordinate space corresponding to the reference image according to their respective spatial location information, forming a preliminary result canvas. Then, the overlapping regions between all image blocks within the canvas are automatically detected, and edge overlapping regions and corner overlapping regions are identified and distinguished based on the intersection characteristics of the overlapping regions. For the identified edge overlapping regions and corner overlapping regions, the corresponding optimal seam finding and feathering fusion processing are performed using a global optimal seam fusion algorithm based on dynamic programming. After all the stitching and fusion processing of all overlapping regions is completed, a visually seamless registered result image consistent with the coordinate system of the reference image is synthesized.

[0112] In some examples, during the patch registration process, multiple corrected image patches can be combined into a single, visually seamless image. In this case, the input can be a set of corrected image patches. as well as Based on the absolute position of each image patch, it is initially placed onto a blank result canvas. The system automatically detects overlapping regions on the canvas and identifies their types, including edge overlap regions and corner overlap regions. Edge overlap regions typically refer to areas where only two image patches intersect; corner overlap regions typically refer to areas where four image patches intersect (usually appearing at the intersections of segmented grids). For the identified edge and corner overlap regions, a globally optimal seam fusion algorithm based on dynamic programming is used to perform corresponding optimal seam finding and feathering fusion processing. After all overlapping regions (edges and corners) have been processed, all pixels on the entire result canvas are uniquely determined. Finally, a complete, geometrically aligned, and visually seamless registered image is output. Its coordinate system is the same as the reference image. Consistent.

[0113] In some embodiments of this disclosure, adaptive stitching strategies are adopted for different types of overlapping regions, ensuring the integrity and rationality of the whole-area stitching process. A globally optimal seam fusion algorithm based on dynamic programming replaces the traditional simple fusion method, fundamentally solving the visual artifact problem caused by stitching after block registration. This achieves seamless stitching of corrected image blocks, ensuring both the geometric accuracy of the entire image and its visual continuity. It also avoids the impact of stitching errors on subsequent quantitative analyses such as remote sensing change detection and land cover classification, significantly improving the quality and usability of large-format remote sensing image registration results.

[0114] In some embodiments of this disclosure, S284 includes:

[0115] For the edge overlap region, a stitching cost function that fuses pixel color differences and gradient consistency is constructed. The optimal seam line with the least visual trace is searched by a dynamic programming algorithm. After image segmentation is completed along the optimal seam line, linear gradient feathering fusion is performed in the transition zone with a preset width on both sides of the seam line.

[0116] For the corner overlapping area, a multi-round step-by-step splicing strategy is adopted, which first splices vertically and then horizontally, and then performs optimal seam finding and feathering fusion processing on the overlapping areas in the vertical and horizontal directions in turn.

[0117] In some implementations, refer to Figure 4For overlapping edge regions, a stitching cost function that integrates pixel color differences and gradient consistency can be constructed. This cost function guides the stitching path to preferentially pass through regions with similar color and texture features. Then, a dynamic programming algorithm searches for the optimal seam line with the least visual trace within the overlapping region. After segmenting and stitching the two images along the found optimal seam line, a linear gradient feathering fusion process is performed within a transition zone of a preset width on both sides of the seam line to achieve a smooth transition at the seam. For overlapping corner regions, a multi-round step-by-step stitching strategy of first vertical stitching and then horizontal stitching is adopted. First, the four image blocks at the corner are divided into left and right groups. Within each group, the optimal seam search and feathering fusion process are performed along the vertical overlapping region to generate two intermediate stitching results in the vertical direction. Then, the optimal seam search and feathering fusion process are performed again along the horizontal overlapping region of the two intermediate stitching results to finally complete the complete stitching of the corner region.

[0118] In some examples, for two overlapping image patches and Define each pixel within its overlapping region The cost of splicing This cost takes into account both color difference and gradient consistency, as shown in equation (1):

[0119]

[0120] In equation (1), For the RGB color vector of a pixel, This represents the gradient magnitude of the pixel. and represents the weighting coefficient. This cost function encourages seams to pass through areas with similar color and texture.

[0121] Next, dynamic programming pathfinding is performed. The overlapping region can be treated as a grid, and the process is recursively performed row by row from top to bottom (for horizontal overlap) or column by column from left to right (for vertical overlap). A cumulative cost matrix is ​​defined. Refer to the following formula (2):

[0122]

[0123] Find the pixel with the minimum cumulative cost in the last row (column) and backtrack its path to obtain the optimal seam line with the minimum visual trace.

[0124] During the feathering blending process, stitching can be performed along the seam. For overlapping areas, the seam line can be used as the boundary, and a portion from each of the two image blocks can be stitched together. During the feathering transition, a fixed width is set on both sides of the seam line. The transition zone. Within the transition zone, pixel values ​​are fused from the two image blocks according to a linearly varying weight, as shown in equation (3):

[0125]

[0126] In equation (3), the weights The value can be smoothly varied from 0.5 at the seam line to 1.0 or 0.0 at the edge of the transition zone.

[0127] In some examples, for overlapping corner areas, a multi-round step-by-step stitching strategy can be adopted, first vertically stitching and then horizontally stitching. The first round (vertical stitching): The four image blocks at the corner are treated as two groups, left and right. Within each group, the overlapping areas in the vertical direction are vertically stitched using the above steps, generating two intermediate result strips. The second round (horizontal stitching): The two vertical strips generated in the first round are horizontally stitched using the above steps again along their horizontal overlapping area, ultimately synthesizing the complete part of the corner area.

[0128] In some embodiments of this disclosure, for overlapping edge areas, a cost function that fuses color and gradient information allows the found optimal seam line to automatically bypass prominent features such as buildings, trees, and vehicles, as well as areas with strong textures. This fundamentally avoids ghosting and structural misalignment caused by the same feature being segmented on both sides of the seam. Furthermore, feathering fusion eliminates harsh boundaries at the seam caused by differences in exposure color, achieving a visually imperceptible natural transition. The multi-round step-by-step stitching strategy for overlapping corner areas resolves stitching conflicts in the intersection of four image blocks, ensuring smoothness and consistency in corner stitching. This avoids stitching misalignment and visual artifacts in the intersection of multiple images, achieving seamless fusion of the entire image and further improving the visual quality and geometric consistency of the registered image.

[0129] In some examples, the SuperPoint network, designed specifically for geometric matching tasks, is employed, utilizing a 256-dimensional descriptor obtained through self-supervised training, and exhibiting strong invariance to illumination, viewpoint, and blur. The SuperGlue matcher, based on an attention-based graph neural network, interacts with cross-image contextual information through self-attention and cross-attention layers, and leverages optimal transport theory for one-to-one matching decisions. (See reference...) Figure 5Compared to traditional hand-crafted features, this model can detect and extract more stable and repeatable feature points in weakly textured areas, repetitive pattern areas, and scenes with seasonal and lighting variations commonly found in remote sensing images. The model can jointly infer the relationships between all feature points, effectively suppressing local interference caused by clouds, shadows, moving targets, etc. Even when the initial matching includes a large number of outliers, it can output highly accurate matching point pairs, significantly improving the anti-interference capability and reliability of the matching stage. It can effectively improve the robustness and registration accuracy of feature matching in complex scenes. For example... Figure 5 Image (a) shows a visualization of the overlay effect of the original images: a 2019 satellite image (lower half) and a 2021 satellite image (upper half). A clear misalignment can be observed between the two images. Figure 5 (b) shows the registration result based on manual feature extraction. There is some alignment adjustment, but there is still about 8 pixels of misalignment. Figure 5 (c) shows the registration results based on a single SuperGlue model and a global transformation model. There is some alignment adjustment, but there is still a misalignment of about 10 pixels. Figure 5 In the middle (d), the technical solution provided by the embodiments of this disclosure is adopted, and the alignment effect is the best.

[0130] In some examples, a single global transformation model cannot handle local distortions within an image; while a simple block-based strategy can correct local deformations, it is prone to inter-block stitching conflicts and global geometric inconsistencies. By introducing an adaptive block-based mechanism based on image size determination, the effective overlapping regions of large-format images are independently divided and registered, and an independent local transformation matrix is ​​estimated for each block. Figure 5 Compared to (d) Figure 5 The middle (c) model can finely characterize and correct the differential geometric distortions that may exist in different regions of a large-scale image (such as edge distortion caused by terrain undulations and slight changes in sensor posture), overcoming the limitations of a single global model's "one-size-fits-all" approach and achieving pixel-level local alignment accuracy.

[0131] In some examples, a dynamic programming-based optimal seam line finding technique can be employed. This involves constructing a cost function that considers color and gradient differences in the overlapping region to find the splicing path with the least visual artifacts. Simultaneously, an adaptive feathering blending technique is used on both sides of the optimal seam line for pixel-level weighted transition. (See reference...) Figure 6 (a) and Figure 6 (b) Figure 6 (a) shows a schematic diagram of the seamless mosaicking tool of a certain remote sensing image professional software, which has obvious misalignment and ghosting phenomena; Figure 6 Figure (b) shows a schematic diagram of the splicing of the technical solution disclosed herein, which achieves a visually seamless integration.

[0132] The automatic registration methods for remote sensing images disclosed herein can solve the problems of insufficient robustness in matching complex scenes, weak local distortion correction capability of large-format images, and severe artifacts in block registration and stitching in traditional technologies. The adaptive registration process balances registration efficiency and correction accuracy, the cascaded deep network significantly improves the reliability of feature matching in complex remote sensing scenes, the differentiated correction strategy achieves the unity of local fine distortion correction and global geometric consistency, and the intelligent stitching algorithm eliminates stitching visual artifacts. Ultimately, it achieves high-precision and high-robust automated registration of remote sensing images in multiple scenes, providing a reliable foundation for subsequent quantitative applications of remote sensing.

[0133] It is understood that the various embodiments of the methods described in this specification are presented in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. Related details can be found in the descriptions of other method embodiments.

[0134] It should be understood that although the steps in the flowcharts shown in the accompanying drawings are displayed sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the accompanying drawings may include multiple steps or stages, which are not necessarily completed at the same time, but may be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but may be performed alternately or in turn with other steps or at least a portion of the steps or stages of other steps.

[0135] Based on the description of the above-described embodiments of the automatic remote sensing image registration method, this disclosure also provides an automatic remote sensing image registration device for implementing the aforementioned automatic remote sensing image registration method. The device may include a system (including a distributed system), software (application), module, component, controller, server, terminal, etc., using the method described in the embodiments of this specification, combined with necessary implementation hardware. Based on the same innovative concept, the devices in one or more embodiments provided in this disclosure are as described in the following embodiments. Since the implementation schemes and methods for solving the problem by the devices are similar, the implementation of specific devices in the embodiments of this specification can refer to the implementation of the foregoing method, and repeated details will not be repeated. As used below, the terms "unit" or "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0136] Figure 7This is a schematic block diagram of an automatic registration device for remote sensing images, according to an exemplary embodiment. The device can be the aforementioned terminal, a server, or a module, component, device, control unit, etc., integrated into the terminal. For details, please refer to... Figure 7 The device 100 may include: a processing module 120, a feature extraction module 140, a correction module 160, an output module 180, and a splicing module 190. The system includes the following components: a processing module 120, which acquires a reference image and an image to be registered, performs coordinate alignment, determines the effective overlapping and non-overlapping regions of the reference image and the image to be registered, and adaptively selects either a global registration process or a block registration process based on the size of the effective overlapping region to obtain corresponding image pairs for registration processing; a feature extraction module 140, which extracts image key points and feature description vectors of the image pairs through a self-supervised deep network, performs feature context enhancement and cross-image information interaction through a graph neural network based on an attention mechanism, and then selects feature matching point pairs that meet a preset confidence threshold through optimal matching solution; a correction module 160, which performs geometric transformation estimation and image correction adapted to the corresponding registration process for the effective overlapping region and the non-overlapping region based on the feature matching point pairs; an output module 180, which generates a first registration result image with a coordinate system consistent with the reference image when performing a global registration process; and a stitching module 190, which performs intelligent stitching through a globally optimal seam fusion algorithm based on dynamic programming when performing a block registration process to generate a visually seamless second registration result image.

[0137] Each module in the aforementioned automatic registration device for remote sensing images can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0138] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements an automatic registration method for remote sensing images.

[0139] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0140] Based on the foregoing description of the relevant methods and apparatus embodiments, this disclosure also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the remote sensing image automatic registration method described in any embodiment of this specification.

[0141] Based on the foregoing description of the relevant methods and apparatus embodiments, this disclosure also provides a computer-readable storage medium that, when the instructions in the computer-readable storage medium are executed by the processor of a computer device, enables the computer device to implement the remote sensing image automatic registration method as described in any embodiment of this disclosure.

[0142] Based on the foregoing description of the relevant methods and apparatus embodiments, this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the remote sensing image automatic registration method described in any embodiment of this specification.

[0143] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, hardware + program embodiments are relatively simple in description because they are fundamentally similar to method embodiments; relevant parts can be referred to the descriptions in the method embodiments.

[0144] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0145] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0146] It should be noted that the apparatus, computer equipment, storage medium, and computer program products described above may also include other implementation methods according to the description of the method embodiments. Specific implementation methods can be found in the description of the relevant method embodiments. Furthermore, new embodiments formed by combinations of features from various methods, apparatuses, devices, and server embodiments still fall within the scope of this disclosure and will not be elaborated upon here.

[0147] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, when implementing one or more of these specifications, the functions of each module can be implemented in the same or different software and / or hardware, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling and communication connections between the devices or units shown or described can be implemented through direct and / or indirect coupling / connection, through standard or custom interfaces or protocols, and can be implemented electrically, mechanically, or in other forms.

[0148] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0149] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A method for automatic registration of remote sensing images, characterized in that, The method includes: Acquire a reference image and an image to be registered and perform coordinate alignment. Determine the effective overlapping and non-overlapping regions of the reference image and the image to be registered. Adaptively select a global registration process or a block registration process based on the size of the effective overlapping region to obtain the corresponding image pair for registration processing. Image key points and feature description vectors of the image pairs are extracted by a self-supervised deep network. After feature context enhancement and cross-image information interaction through graph neural networks based on attention mechanism, feature matching point pairs that meet the preset confidence threshold are obtained by optimal matching solution. Based on the feature matching point pairs, geometric transformation estimation and image correction for the corresponding registration process are performed on the effective overlapping region and the non-overlapping region, respectively. When performing a global registration process, a first registration result image with a coordinate system consistent with the reference image is generated; When performing the block registration process, intelligent stitching is performed by a global optimal seam fusion algorithm based on dynamic programming to generate a visually seamless second registration result image; In the case of executing the block registration process, intelligent stitching is performed through a globally optimal seam fusion algorithm based on dynamic programming to generate a visually seamless second registration result image, including: The corrected image block data and the non-overlapping region correction data are mapped to the unified coordinate space of the reference image according to the corresponding spatial location information to identify the edge overlap region and corner overlap region between image blocks. For the edge overlap region and the corner overlap region, the optimal seam finding and feathering fusion processing are performed respectively by the global optimal seam fusion algorithm based on dynamic programming. After all overlap regions have been processed, a visually seamless second registration result image consistent with the coordinate system of the reference image is synthesized. The overlapping edge regions and the overlapping corner regions are respectively processed using a global optimal seam fusion algorithm based on dynamic programming to perform optimal seam finding and feathering fusion processing, including: For the edge overlap region, a stitching cost function that fuses pixel color differences and gradient consistency is constructed. The optimal seam line with the least visual trace is searched by a dynamic programming algorithm. After image segmentation is completed along the optimal seam line, linear gradient feathering fusion is performed in the transition zone with a preset width on both sides of the seam line. For the corner overlapping area, a multi-round step-by-step splicing strategy is adopted, which first splices vertically and then horizontally, and the optimal seam finding and feathering fusion processing are performed on the overlapping areas in the vertical and horizontal directions in turn.

2. The method of claim 1, wherein, The step of adaptively selecting either a global registration process or a block registration process based on the size of the effective overlapping region to obtain the corresponding registered image pair includes: Calculate the width and height of the bounding rectangle of the effective overlapping area, and compare the maximum value of the width and height with a preset size threshold; If the maximum value is not greater than a preset size threshold, it is determined to execute a global registration process, and the reference image and the image to be registered are used as an image pair for registration processing. If the maximum value is greater than the preset size threshold, the block registration process is executed. According to the preset cropping size and preset overlap rate, the effective overlapping areas of the reference image and the image to be registered are divided into regular grids to generate multiple pairs of reference image blocks and image blocks to be registered. For the boundary image blocks adjacent to the non-overlapping area, the preset pixel expansion processing is performed on the non-overlapping area. Finally, each pair of image blocks is used as the image pair for registration processing.

3. The method of claim 1, wherein, The process involves extracting image keypoints and feature description vectors from the image pairs using a self-supervised deep network, performing feature context enhancement and inter-graph cross-attention cross-image information interaction via an attention-based graph neural network, and then filtering for feature matching point pairs that meet a preset confidence threshold through optimal matching. This includes: Subpixel-level key points of the image pairs are extracted using a self-supervised deep network, and normalized feature description vectors are generated for each key point. The key points of the image pairs are constructed into nodes of a fully connected graph using a graph neural network based on an attention mechanism. The position and confidence of the key points are then fused with the feature description vector to generate an initial feature vector. Based on the initial feature vector, feature enhancement is performed through alternating propagation of self-attention and cross-attention, and then feature matching point pairs that meet the preset confidence threshold are selected through the optimal transmission matching algorithm.

4. The method of claim 1, wherein, The step of performing geometric transformation estimation and image correction for the effective overlapping region and the non-overlapping region according to the feature matching point pairs includes: When performing a global registration process, the global homography transformation matrix of the effective overlapping region is estimated using a random sampling consistency algorithm based on the feature matching point pairs, and the overall resampling correction is performed on the effective overlapping region using the global homography transformation matrix. When performing the block registration process, the local homography transformation matrix is ​​estimated for the feature matching point pairs corresponding to each group of image blocks using the random sampling consistency algorithm. After the inlier number threshold is checked, the valid image blocks are resampled and corrected block by block, while the original pixel data of the invalid image blocks are retained. The reliable feature matching point pairs that have passed the verification within the effective overlapping area are aggregated, and after deduplication, a global homography transformation matrix is ​​obtained by least squares fitting. The global homography transformation matrix is ​​then used to perform geometric transformation and resampling correction on the non-overlapping area.

5. An apparatus for automatic registration of remote sensing images, characterized in that, The device includes: The processing module is used to acquire a reference image and an image to be registered and perform coordinate alignment, determine the effective overlapping area and non-overlapping area of ​​the reference image and the image to be registered, and adaptively select a global registration process or a block registration process according to the size of the effective overlapping area to obtain the corresponding image pair for registration processing. The feature extraction module is used to extract image key points and feature description vectors of the image pairs through a self-supervised deep network. After feature context enhancement and cross-image information interaction through graph neural networks based on attention mechanism, feature matching point pairs that meet the preset confidence threshold are obtained by optimal matching solution. The correction module is used to perform geometric transformation estimation and image correction for the effective overlapping region and the non-overlapping region respectively, based on the feature matching point pairs, to adapt to the corresponding registration process. The output module is used to generate a first registration result image with a coordinate system consistent with the reference image when performing a global registration process; The stitching module is used to perform intelligent stitching using a globally optimal seam fusion algorithm based on dynamic programming during the block registration process, generating a visually seamless second registration result image. The stitching module is also used to map the corrected image block data and the non-overlapping region correction data to the unified coordinate space of the reference image according to the corresponding spatial location information, and to identify the edge overlap region and corner overlap region between the image blocks; it is also used to perform optimal seam finding and feathering fusion processing on the edge overlap region and the corner overlap region respectively through a global optimal seam fusion algorithm based on dynamic programming, and after all the overlapping regions have been processed, to synthesize a visually seamless second registration result image that is consistent with the coordinate system of the reference image; The stitching module is also used to construct a stitching cost function that integrates pixel color differences and gradient consistency for the edge overlapping areas, search for the optimal seam line with the least visual trace using a dynamic programming algorithm, complete image segmentation along the optimal seam line, and perform linear gradient feathering fusion within a transition zone of a preset width on both sides of the seam line; it is also used to employ a multi-round step-by-step stitching strategy of first vertical stitching and then horizontal stitching for the corner overlapping areas, sequentially performing optimal seam search and feathering fusion processing on the vertical and horizontal overlapping areas.

6. A computer device, comprising: It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, It stores a computer program thereon, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 4.

8. A computer program product, characterised in that, Includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 4.

Images