Method and system for registration of optical images and sar images of the moon

Abstract

The application discloses a method and system for registering optical images and SAR images of the moon, and relates to the technical field of image registration. In the method, the same typical lunar object in the images to be registered is determined first, and the corresponding optical and SAR regional images are combined to form a regional image pair. Then, a feature-based registration algorithm is used to perform feature extraction and matching on each regional image pair, to obtain a preliminary matching point pair, and a high-confidence matching point pair is selected. A regional affine transformation matrix of each regional image pair is calculated based on the high-confidence matching point pair, and then a global optimization objective function is constructed according to all the regional affine transformation matrices, to solve an optimal affine transformation matrix, so that the accurate registration of the images to be registered is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image registration, and in particular to a method and system for registering optical images and SAR images of the moon. BACKGROUND

[0002] With the continuous advancement of lunar exploration and manned lunar landing missions, the demand for spatial resolution and height measurement accuracy of large-scale lunar terrain data is increasing, from meter-level to sub-meter level, and the detection means has shifted from optical sensors to multi-sensor joint detection such as optical and SAR. SAR has the ability of all-weather and all-day imaging and height measurement, as well as surface penetration. In recent years, with the continuous development of its miniaturization technology, it has become an important technical means for obtaining high-precision lunar terrain data after optical means.

[0003] The imaging principles of optical sensors and SAR sensors are different, and the observation angles are also different, so there are significant modal differences in the images obtained, but they also provide important information for mutual complementation of lunar terrain data acquisition. Optical image and SAR image registration is an indispensable key technology to achieve geometric positioning consistency of optical image and SAR image, and is also a prerequisite for data fusion processing, which is of great significance for lunar mapping and scientific research using multi-source remote sensing images.

[0004] Due to the special lunar environment, there is a lack of natural features such as vegetation, rivers, lakes, and man-made features such as buildings and roads, and the surface is extremely barren. Coupled with the factors of volcanic activity stopping, long-term weathering, and the like, the lunar dust covering the surface weakens the texture details, and the overall appearance of the moon is highly homogenized, resulting in a lack of structural reference information in lunar images, making it difficult to register optical images and SAR images of the moon.

[0005] The existing optical image and SAR image registration methods can be divided into three categories: region-based registration, deep learning-based registration, and feature-based registration, and the related method research is mainly for earth observation, with less lunar observation. Specifically:

[0006] 1) Region-based image registration methods, also known as template matching methods, include three categories: intensity-based methods, structure-based methods, and phase-correlation methods. Intensity-based methods are simple to implement and suitable for images with small intensity differences, but are sensitive to noise and structural differences. Structure-based methods, by extracting features such as edges, shapes, or local self-similarity, exhibit strong robustness against noise and radiation differences, achieving high registration accuracy, but with high computational complexity. Phase-correlation methods utilize Fourier transform to achieve good invariance to translation, scale, and rotation, resulting in high computational efficiency, but are sensitive to changes in image content and noise, limiting registration accuracy. Region-based registration methods rely on the consistency of grayscale or texture between images; their adaptability and robustness are significantly insufficient when the lunar surface has weak texture, low contrast, and significant modal differences.

[0007] 2) Image registration based on deep learning is a new research direction in recent years. Although related research is increasing, there are still many problems: First, there is a lack of a large number of high-quality visible light and SAR image datasets. Currently, there are few SAR payloads in lunar exploration missions that have been launched, and the spatial resolution of the lunar SAR remote sensing images that have been acquired is low, all on the order of tens of meters, which is far lower than the meter-level resolution of lunar optical remote sensing images; Second, the internal working principle is unclear and the interpretability is poor.

[0008] 3) Feature-based image registration methods extract, describe, and match salient features in images to determine transformation parameters between images, exhibiting good robustness to differences in intensity, scale, and rotation. Salient features in images include points, lines, and surfaces. Point features refer to points with unique properties in the image, including spots and corners; line features are commonly found in natural features such as coastlines and rivers, including curves and straight lines in the image, and are usually extracted using edge detection operators; surface features typically select closed regions with high contrast as registration features, such as lakes, forests, buildings, and farmland, and are mainly extracted using edge detection and image segmentation techniques. Feature-based image registration methods extract salient features such as corners and edges in images for matching, possessing strong robustness and applicability, and are an effective means to cope with the complex imaging conditions and limited observation data of the moon. However, the special environment of the lunar surface with weak texture and high similarity, as well as the modal differences between optical and SAR, result in a general lack of clear and consistent edge and structural information in lunar optical and SAR remote sensing images, leading to problems such as difficulty in feature point extraction, unstable feature matching, and sensitivity to local errors. Summary of the Invention

[0009] The technical problem to be solved by this invention is to address the shortcomings of existing technologies, specifically by providing a method and system for registering optical images and SAR images of the moon, as detailed below:

[0010] 1) In a first aspect, the present application provides a method for registering optical images and SAR images of the moon, and the specific technical solutions are as follows:

[0011] determining the same typical lunar landform objects existing in the optical image to be registered and the SAR image to be registered of the moon;

[0012] performing feature extraction and matching on each regional image pair by using a feature-based registration algorithm, and outputting a plurality of preliminary matching point pairs, wherein the optical regional image of each typical lunar landform object in the optical image to be registered and the SAR regional image in the SAR image to be registered respectively form a regional image pair;

[0013] screening a plurality of high-confidence matching point pairs from all the preliminary matching point pairs;

[0014] calculating a regional affine transformation matrix for representing a geometric transformation relationship of the SAR regional image to the optical regional image in each regional image pair based on all the high-confidence matching point pairs;

[0015] solving an optimal affine transformation matrix according to all the regional affine transformation matrices and a constructed global optimization objective function;

[0016] registering the optical image to be registered and the SAR image to be registered by using the optimal affine transformation matrix.

[0017] The method for registering optical images and SAR images of the moon provided by the present application has the following beneficial effects:

[0018] In view of the problem that the registration method based on regions mentioned in the background art is insufficient in adaptability and robustness in the case of weak texture, low contrast and significant difference in image modalities on the lunar surface, the scheme provides a more stable and reliable reference basis for the registration process by determining the same typical lunar landform objects existing in the optical image to be registered and the SAR image to be registered of the moon. The typical lunar landform objects have relatively unique morphology and distribution characteristics on the lunar surface. Compared with relying on the consistency of gray scale or texture, taking these typical lunar landform objects as the core basis for registration can better cope with the problem of high homogeneity of the overall appearance of the lunar surface, effectively improving the adaptability and robustness of registration. For the problems of lack of large high-quality data sets and poor interpretability faced by the registration method based on deep learning, the scheme adopts a feature-based registration algorithm to extract and match features for each regional image pair, outputs multiple preliminary matching point pairs, and further optimizes and selects high-confidence matching point pairs. The entire process does not require large-scale deep learning training data, and the feature extraction, matching and optimization and selection steps have clear physical meaning and geometric interpretation, overcoming the shortcomings of deep learning methods in this regard, making the registration process more interpretable and stable. In the face of the problems of difficulty in feature point extraction, unstable feature matching and local error sensitivity of the feature-based registration method in the lunar environment, the application extracts and matches features for each regional image pair corresponding to each typical lunar landform object, narrows the range of feature extraction and matching, and improves the accuracy of feature extraction. At the same time, the preliminary matching point pairs are optimized and selected to further eliminate false matches and unstable matches, ensuring the quality of the matching point pairs. On this basis, the regional affine transformation matrix is calculated based on the high-confidence matching point pairs, and the optimal affine transformation matrix is solved by combining the global optimization objective function, realizing accurate registration from local to global, effectively solving the problem of local error sensitivity, improving the accuracy and reliability of the overall registration, thereby providing more accurate registration results for subsequent lunar mapping and scientific research, and promoting the process of multi-sensor data fusion in lunar exploration missions.

[0019] On the basis of the above-mentioned scheme, the registration method of the optical image and the SAR image of the moon of the present application can be further improved as follows.

[0020] Further, according to all regional affine transformation matrices and the constructed global optimization objective function, the optimal affine transformation matrix is solved, comprising:

[0021] Each regional affine transformation matrix is decomposed into a horizontal translation component, a vertical translation component, a horizontal scaling component, a vertical scaling component, a horizontal shear component, a vertical shear component and a rotation component, the mean value on each same component is calculated, a fused affine transformation matrix is generated, and an affine transformation vector corresponding to the fused affine transformation matrix is obtained;

[0022] The affine transformation vector corresponding to the fused affine transformation matrix is taken as an iteration starting point of the Levenberg-Marquardt algorithm, and an optimal affine transformation matrix is determined by using a global optimization objective function based on a Huber loss function.

[0023] The beneficial effect of the further scheme is that: by decomposing each regional affine transformation matrix into a horizontal translation component, a vertical translation component, a horizontal scaling component, a vertical scaling component, a horizontal shear component, a vertical shear component and a rotation component, calculating the mean value on each same component to generate a fused affine transformation matrix, the geometric transformation characteristics of the local region can be considered comprehensively, the transformation characteristics of different local regions are fused, more stable and representative initial transformation information is provided for subsequent global optimization, and the starting point accuracy of global optimization is improved. The affine transformation vector corresponding to the fused affine transformation matrix is taken as an iteration starting point of the Levenberg-Marquardt algorithm, and an optimal affine transformation matrix is determined by using a global optimization objective function based on a Huber loss function. The Levenberg-Marquardt algorithm has the characteristics of fast convergence, can efficiently search for an optimal solution, and the Huber loss function has good robustness to abnormal values, can effectively reduce the influence of local errors or abnormal matching points on the global optimization result, so that the optimal affine transformation matrix finally solved can more accurately represent the geometric transformation relationship between the entire images, improve the registration accuracy, reduce error accumulation and propagation, and thus realize more reliable and accurate lunar optical and SAR image registration.

[0024] Further, a plurality of high-confidence matching point pairs are selected from all the preliminary matching point pairs, including:

[0025] The RANSAC algorithm and the angle constraint algorithm are used to select a plurality of high-confidence matching point pairs from all the preliminary matching point pairs.

[0026] The beneficial effect of the further scheme is that: the random sample consensus of the RANSAC algorithm and the geometric constraint characteristics of the angle constraint algorithm can effectively remove false matching point pairs, retain high-confidence matching point pairs that meet the geometric transformation relationship, improve matching accuracy and reliability, and solve the problems of difficult extraction and unstable matching of lunar image feature points. The RANSAC algorithm reduces the influence of noise and abnormal values on the matching result through iterative sampling and model verification, improves the robustness of matching, and ensures that accurate matching point pairs can still be stably obtained under complex lunar imaging conditions. The angle constraint algorithm geometrically constrains the matching point pairs to ensure that the angle relationship of the matching point pairs meets the actual terrain features, further reduces false matching, enhances the geometric consistency of the matching point pairs, and improves the registration quality.

[0027] Further, each typical lunar feature is characterized by a five-tuple parameter including a label, a longitude of a center point, a latitude of the center point, an east-west span, and a north-south span, to obtain a prior database;

[0028] The same typical lunar feature objects existing in the optical image and the SAR image to be registered of the moon are determined, including:

[0029] The same typical lunar feature objects existing in the optical image and the SAR image to be registered of the moon are determined according to the prior database, the geometric positioning information of the optical image to be registered, and the geometric positioning information of the SAR image to be registered.

[0030] The beneficial effects of the further scheme are: the five-tuple parameter is used to characterize the typical lunar feature, which can accurately describe the position and range, effectively improving the expression ability and discrimination of the lunar feature. The construction of the prior database provides rich reference information for subsequent image registration, facilitating the rapid determination of the same lunar feature objects in the optical image and the SAR image, and improving the registration efficiency. The same lunar feature objects are determined based on the prior database and the geometric positioning information, avoiding errors caused by directly extracting and matching features from the images. This determination method based on prior knowledge can more accurately identify the same lunar features in the two images, providing a more reliable basis for subsequent feature extraction and registration, thereby improving the accuracy of registration. The detailed information of the typical lunar features stored in the prior database enables the registration process to fully utilize existing knowledge and data, enhancing the robustness of the registration process to factors such as image quality changes and imaging condition differences. Even under complex conditions such as weak lunar texture and low contrast, the same lunar features can be determined with the aid of prior information, improving the reliability and stability of the entire registration process.

[0031] 2) In a second aspect, the present application also provides a registration system for optical images and SAR images of the moon, and the specific technical solutions are as follows:

[0032] The registration system comprises a typical lunar feature object determination module, a preliminary matching point pair determination module, a screening module, a regional affine transformation matrix determination module, an optimal affine transformation matrix determination module, and a registration module.

[0033] The typical lunar feature object determination module is configured to determine the same typical lunar feature objects existing in the optical image and the SAR image to be registered of the moon.

[0034] The preliminary matching point pair determination module is configured to perform feature extraction and matching on each regional image pair by using a feature-based registration algorithm, and output a plurality of preliminary matching point pairs, wherein the optical regional image of each typical lunar feature object in the optical image to be registered and the SAR regional image of the same typical lunar feature object in the SAR image to be registered respectively form a regional image pair.

[0035] The screening module is configured to screen a plurality of high-confidence matching point pairs from all the preliminary matching point pairs.

[0036] The regional affine transformation matrix determination module is configured to calculate, based on all the high-confidence matching point pairs, a regional affine transformation matrix for representing a geometric transformation relationship of a SAR regional image to an optical regional image in each regional image pair.

[0037] The optimal affine transformation matrix determination module is configured to solve an optimal affine transformation matrix according to all the regional affine transformation matrices and a constructed global optimization objective function.

[0038] The registration module is configured to register the to-be-registered optical image and the to-be-registered SAR image by using the optimal affine transformation matrix.

[0039] Based on the above scheme, the registration system for the optical image and the SAR image of the moon can be further improved as follows.

[0040] Further, the regional affine transformation matrix determination module is specifically configured to:

[0041] The regional affine transformation matrix is decomposed into a horizontal translation component, a vertical translation component, a horizontal scaling component, a vertical scaling component, a horizontal shear component, a vertical shear component and a rotation component, a mean value on each same component is calculated, a fused affine transformation matrix is generated, and an affine transformation vector corresponding to the fused affine transformation matrix is obtained.

[0042] The affine transformation vector corresponding to the fused affine transformation matrix is taken as an iteration starting point of a Levenberg-Marquardt algorithm, and a global optimization objective function based on a Huber loss function is used to determine the optimal affine transformation matrix.

[0043] Further, the screening module is specifically configured to:

[0044] The RANSAC algorithm and the angle constraint algorithm are used to screen the plurality of high-confidence matching point pairs from all the preliminary matching point pairs.

[0045] Further, the registration system further comprises a database acquisition module, and the database acquisition module is configured to represent each typical lunar feature by using a five-tuple parameter comprising a label, a longitude of a center point, a latitude of the center point, an east-west span and a north-south span, to obtain a priori database.

[0046] The typical lunar feature object determination module is specifically configured to determine the same typical lunar feature object existing in the to-be-registered optical image and the to-be-registered SAR image of the moon according to the priori database, geometric positioning information of the to-be-registered optical image and geometric positioning information of the to-be-registered SAR image.

[0047] 3) In a third aspect, the present invention also provides an electronic device, the electronic device including a processor coupled to a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor, so as to enable the electronic device to implement any of the above-mentioned methods for registering optical images and SAR images of the moon.

[0048] 4) In a fourth aspect, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-described methods for registering optical images and SAR images of the moon.

[0049] It should be noted that the beneficial effects of the technical solutions of the second to fourth aspects of the present invention and their corresponding possible implementations can be found in the above description of the technical effects of the first aspect and its corresponding possible implementations, and will not be repeated here. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments of the present invention will be briefly introduced below:

[0051] Figure 1 This is a schematic flowchart illustrating a method for registering optical images and SAR images of the moon according to an embodiment of the present invention.

[0052] Figure 2 This is a schematic diagram of the structure of a registration system for optical and SAR images of the moon according to an embodiment of the present invention;

[0053] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0054] The principles and features of the present invention are described below. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0055] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0056] like Figure 1 As shown, an embodiment of the present invention provides a method for registering an optical image of the moon and a SAR image, comprising the following steps:

[0057] S1, determining the same typical lunar landform objects existing in the to-be-registered optical image and the to-be-registered SAR image of the moon;

[0058] The to-be-registered optical image refers to an image collected by an optical sensor, and the to-be-registered SAR image refers to an image collected by a synthetic aperture radar.

[0059] S2, performing feature extraction and matching on each regional image pair by using a feature-based registration algorithm, and outputting a plurality of preliminary matching point pairs, wherein the optical regional image and the SAR regional image of each typical lunar landform object in the to-be-registered optical image form a regional image pair.

[0060] The feature-based registration algorithm can be a SIFT (Scale Invariant Feature Transform) algorithm or an ORB (Oriented FAST and Rotated BRIEF) algorithm or other algorithms, and the feature-based registration algorithm is as follows:

[0061] 1) When the SIFT algorithm is used, first, feature extraction is performed on the optical regional image and the SAR regional image in each regional image pair. In the optical image, extreme points in a scale space are detected, and a key point is identified by using a Gaussian difference function, and the key point corresponds to a region with a significant feature in the image, such as a corner point or an edge. At the same time, a direction parameter is associated with each key point, and the direction parameter is determined based on the direction distribution of the image gradient. Then, a descriptor is generated for each key point, and the descriptor is obtained by calculating the gradient amplitude and direction information in the neighborhood of the key point, and a feature vector is constructed, and the feature vector is invariant to scale and rotation. In the SAR regional image, the SIFT algorithm is also used to extract key points and generate descriptors, and although the SAR image has a different imaging mechanism and feature representation, the SIFT algorithm can capture significant feature points therein. Feature matching is performed by using the similarity between the descriptors. The nearest neighbor distance ratio method is used to calculate the distance between a feature descriptor and all feature descriptors in another image, and the two key points corresponding to the two feature descriptors with the smallest distance are selected to form a matching point pair. If the ratio of the nearest distance to the second nearest distance is less than a certain threshold, the matching point pair is considered to be reliable. In this way, a plurality of preliminary matching point pairs are selected from the optical regional image and the SAR regional image.

[0062] 2) When using the ORB algorithm, first use the FAST (Fast Corner Detection) algorithm to detect the corner points in the optical region image and the SAR region image. The FAST algorithm judges whether it is a corner point by detecting the neighborhood pixels around the pixel. It has the characteristic of fast calculation speed. Then, use the BRIEF (Binary Feature Descriptor) method to generate a descriptor for the detected corner points. The BRIEF algorithm selects a series of random point pairs in the neighborhood of the corner point, compares the pixel intensities of these point pairs, and encodes the comparison results as a binary string to form a descriptor. The descriptor has a binary form, which is convenient for fast matching. In the feature matching stage, the Hamming distance is used to measure the similarity between the descriptors. By comparing the Hamming distance of the descriptors in the optical region image and the SAR region image, the matching pairs with smaller distance are selected as the preliminary matching point pairs. The ORB algorithm has high computational efficiency while maintaining a certain matching accuracy, and is suitable for processing large amounts of image data.

[0063] In the entire implementation process, whether it is the SIFT algorithm or the ORB algorithm, or other algorithms, quality evaluation and screening of the extracted features are needed to remove unreliable feature point pairs, thereby improving the accuracy of matching. In addition, the imaging differences between the optical image and the SAR image, such as the brightness, contrast changes of the optical image, and the shadow, speckle noise of the SAR image, etc. factors affecting feature extraction and matching, appropriate image preprocessing and post-processing steps are needed to optimize the matching results, and finally output multiple preliminary matching point pairs, providing a basis for subsequent image registration accuracy improvement and geometric transformation parameter estimation. In addition, in S2, the feature point extraction range is limited by the region constraint, which can effectively improve the success rate of feature point matching.

[0064] Preliminary matching point pairs refer to the feature point pairs extracted and matched from the region image pairs of the optical image and the SAR image to be registered by a feature-based registration algorithm (such as the SIFT or ORB algorithm or other algorithms). These preliminary matching point pairs are matched according to the similarity of the feature descriptors and have not been further optimized and screened, and may contain false matches or low reliability point pairs. They are used as input for subsequent optimization and screening to further output high-confidence matching point pairs, providing a basis for subsequent geometric transformation relationship calculation and image registration.

[0065] S3, screen multiple high-confidence matching point pairs from all preliminary matching point pairs, specifically:

[0066] Use the RANSAC algorithm and the angle constraint algorithm to screen multiple high-confidence matching point pairs from all preliminary matching point pairs, and the number of output high-confidence matching point pairs is at least 3 pairs.

[0067] The RANSAC algorithm and the angle constraint algorithm are explained as follows:

[0068] ①RANSAC algorithm: Based on the principle of random sample consensus, it automatically identifies and removes the mismatched point pairs through iterative method, and retains the high-confidence matching point pairs. As a classic robust estimation algorithm, RANSAC can effectively eliminate the false matching, greatly improve the accuracy and reliability of registration.

[0069] ②Angle constraint algorithm: It further optimizes and filters the matching point pairs by constraining the reasonable range of rotation angle. First, it calculates the rotation angle of all matching point pairs between the optical image and the SAR image; then it takes the mean of the rotation angle as the main direction; finally, it sets the acceptable maximum rotation angle deviation, and removes the matching point pairs deviating from the main direction by more than the maximum rotation angle deviation. Angle constraint can further eliminate the false matching point pairs, and improve the accuracy and stability of registration.

[0070] The process of obtaining at least 3 pairs of high-confidence matching point pairs is as follows:

[0071] First, the preliminary matching point pairs are preprocessed, and the coordinate information of all point pairs is counted, including the point coordinates (x_optical, y_optical) in the optical regional image and the corresponding point coordinates (x_sar, y_sar) in the SAR regional image. Then the related parameters of RANSAC algorithm are initialized, such as the iteration threshold is set to 1000 times, the minimum number of preliminary matching point pairs required for model determination is set to 4 pairs, and the error threshold of consistent matching point pairs is set to 2 pixels. Then the iteration process of RANSAC algorithm starts: randomly select 4 pairs of preliminary matching point pairs as samples, construct the affine transformation model from the optical image to the SAR image, calculate the geometric transformation error of all preliminary matching point pairs to the model, and mark the point pairs with error less than 2 pixels as consistent matching point pairs; record the number of consistent matching point pairs of the current model; repeatedly sample, model construction and consistent point pair evaluation process until the iteration number reaches 1000 times or a consistent model that meets a certain confidence is found (for example, the consistent model does not change significantly for 100 consecutive iterations). Select the model with the most consistent matching point pairs as the current optimal model, and select the first 3 pairs with the smallest error from the consistent matching point pair set corresponding to the model as the candidate high-confidence matching point pairs. At the same time, the angle constraint algorithm is introduced to further verify the candidate matching point pairs. According to the geometric characteristics of typical lunar features, such as the circular symmetry of lunar impact craters, the angle between the line connecting each pair of candidate matching point pairs in the optical image and the SAR image and the main axis of the lunar feature (such as the diameter direction of the impact crater) is calculated as θ_optical and θ_sar. For real homonymic points, it should theoretically satisfy is the allowed angle deviation caused by imaging geometry difference, usually set to 10 degrees). Calculate the angle deviation of each candidate matching point pair​ : , remove point pairs with angle deviation >10 degrees; if less than 3 pairs remain, fill in from the set of RANSAC consistent matching point pairs in order of error from small to large and re-verify the angle constraint until at least 3 pairs of matching point pairs satisfying the angle constraint condition are obtained. Finally, refine the screened matching point pairs: use the least squares method to re-fit the local geometric transformation model of the screened matching point pairs, calculate the residual of each pair of points and perform the final confidence evaluation, output the final at least 3 pairs of high-confidence matching point pairs, which are assigned the highest confidence label, with the residual less than 1 pixel and the angle deviation ≤5 degrees.

[0072] S4, based on all the high-confidence matching point pairs, calculate the regional affine transformation matrix for representing the geometric transformation relationship of the SAR regional image to the optical regional image in each regional image pair, specifically:

[0073] using the least squares algorithm, calculate the regional affine transformation matrix for representing the geometric transformation relationship of the SAR regional image to the optical regional image in each regional image pair, wherein the geometric transformation relationship includes translation, rotation, scaling and other transformation relationships, specifically, for each regional image pair, first construct the following equation:

[0074]

[0075] wherein, , represents the pixel position of the i-th pair of high-confidence matching point pairs in the i-th regional image pair on the to-be-registered optical image, represents the x-axis coordinate of the i-th pair of high-confidence matching point pairs in the i-th regional image pair on the to-be-registered optical image, represents the y-axis coordinate of the i-th pair of high-confidence matching point pairs in the i-th regional image pair on the to-be-registered optical image, , represents the pixel position of the i-th pair of high-confidence matching point pairs in the i-th regional image pair on the to-be-registered SAR image; represents the x-axis coordinate of the i-th pair of high-confidence matching point pairs in the i-th regional image pair on the to-be-registered SAR image, represents the y-axis coordinate of the i-th For the first region in the image pair For the y-axis coordinates of high-confidence matching point pairs on the SAR image to be registered, the constant 1 is an extension term added to adapt to matrix multiplication. , For the first The total number of high-confidence matching point pairs for each region image pair is such that, since the calculation of the affine transformation matrix for each region requires at least 3 pairs of high-confidence matching point pairs, therefore... .

[0076] Next, the least squares algorithm is used to solve the above equations to obtain the affine transformation matrix of the region:

[0077]

[0078] in, Indicates: used to characterize the first The region affine transformation matrix, which represents the geometric transformation relationship from the SAR region image to the optical region image in a region image pair, can also be called the first... Affine transformation matrix for each region;

[0079] It can be used express Vector form:

[0080]

[0081] in, , Indicates: the total number of region image pairs. Indicates: in the first In the geometric transformation from SAR region image to optical region image in each region image pair, parameters controlling horizontal scaling and cropping are used. Indicates: in the first In the geometric transformation from SAR region image to optical region image in each region image pair, parameters for vertical scaling and cropping are controlled. Indicates: in the first In the geometric transformation from SAR region image to optical region image in each region image pair, parameters controlling horizontal scaling and cropping are used. Indicates: in the first In the geometric transformation from SAR region image to optical region image in each region image pair, parameters for vertical scaling and cropping are controlled. Indicates: in the first In the geometric transformation from SAR regional image to optical regional image in each regional image pair, the horizontal translation amount from SAR regional image to optical regional image; denotes: in the first In the geometric transformation of the SAR regional image to the optical regional image in each of the regional image pairs, the vertical translation amount of the SAR regional image to the optical regional image.

[0082] S5, according to all regional affine transformation matrices and the constructed global optimization objective function, the optimal affine transformation matrix is obtained, specifically comprising:

[0083] S50, decompose each regional affine transformation matrix into a horizontal translation component, a vertical translation component, a horizontal scaling component, a vertical scaling component, a horizontal shear component, a vertical shear component and a rotation component, calculate the mean value on each same component, generate a fused affine transformation matrix, and obtain an affine transformation vector corresponding to the fused affine transformation matrix.

[0084] Wherein, the horizontal translation component obtained by decomposing the first regional affine transformation matrix is , the vertical translation component obtained by decomposing the first regional affine transformation matrix is .

[0085] Wherein, the rotation component obtained by decomposing the first regional affine transformation matrix is the rotation angle, denoted as :

[0086]

[0087] Wherein, atan2 is the inverse tangent function, denotes: using the inverse tangent function to obtain the rotation angle between and .

[0088] Wherein, the horizontal scaling component (scaling factor) obtained by decomposing the first regional affine transformation matrix is , the vertical scaling component (scaling factor) obtained by decomposing the first regional affine transformation matrix is , and the norm of the first regional affine transformation matrix is obtained by calculating:

[0089]

[0090] Wherein, the horizontal shear component obtained by decomposing the first regional affine transformation matrix is , the vertical shear component obtained by decomposing the first regional affine transformation matrix is , which is used to describe the degree of image shape deformation from the SAR regional image to the optical regional image, is calculated by the following formula:

[0091]

[0092] wherein the mean value on each same component is calculated as:

[0093]

[0094] wherein, represents the mean value of all horizontal translation components, represents the mean value of all vertical translation components, represents the mean value of all rotation components, represents the mean value of all horizontal scaling components, represents the mean value of all vertical scaling components, represents the mean value of all horizontal shear components, represents the mean value of all vertical shear components.

[0095] S51, the fused regional affine transformation matrix is calculated by the following formula :

[0096]

[0097] The affine transformation vector corresponding to the fused affine transformation matrix is denoted as :

[0098]

[0099] It should be noted that, represents the vector form of the fused regional affine transformation matrix.

[0100] S52, the affine transformation vector corresponding to the fused affine transformation matrix is used as the iteration starting point of the Levenberg-Marquardt algorithm, and a global optimization objective function based on the Huber loss function is used to determine the optimal affine transformation matrix.

[0101] wherein the global optimization objective function is:

[0102]

[0103] wherein, represents the loss value calculated by the global optimization objective function, represents the residual of all high-confidence matching point pairs, represents the affine transformation vector, which is a variable.

[0104] wherein, Huber loss function, specifically defined as:

[0105]

[0106] wherein, is the residual, . The residual is related to the affine transformation vector (or affine transformation matrix), and by minimizing the weighted residual sum of all high-confidence matching point pairs, the globally optimal affine transformation matrix can be solved.

[0107] The Huber loss function combines the advantages of mean square error and mean absolute error by setting the hyperparameter, has high precision and fast convergence of mean square error loss for small residuals, and has strong anti-interference and strong robustness of mean absolute error loss for large residuals, and can simultaneously achieve high precision and high stability with the highest efficiency.

[0108] The present application uses Levenberg-Marquardt (LM) algorithm (Levenberg-Marquardt algorithm is an adaptive iterative algorithm) for iterative optimization, which combines the advantages of Gauss-Newton method and gradient descent method, and can provide an optimization path considering accuracy and stability when solving nonlinear least squares problems. Specifically, when the error is small, the algorithm tends to Gauss-Newton method, which can quickly approach the optimal solution and realize efficient convergence; when the error is large, it shows similar characteristics of gradient descent, has better robustness, and can effectively avoid falling into divergence or unreasonable update.

[0109] When calculating using the Levenberg-Marquardt (LM) algorithm, set the minimum update threshold and the maximum number of iterations, and in the first iteration, perform the following operations:

[0110] ① Set the initial condition as: ;

[0111] ② Under the initial condition, calculate the residual of all high-confidence matching point pairs using the following formula is:

[0112]

[0113] wherein, , represents: the pixel position of the th high-confidence matching point pair on the optical image to be registered, represents: the pixel position of the The x-axis coordinates of high-confidence matching point pairs on the optical image to be registered. Indicates: the The y-axis coordinates of high-confidence matching point pairs on the optical image to be registered. , Indicates: the The pixel locations of high-confidence matching point pairs on the SAR image to be registered; Indicates: the The x-axis coordinates of high-confidence matching point pairs on the SAR image to be registered. Indicates: the For the y-axis coordinates of high-confidence matching point pairs on the SAR image to be registered, the constant 1 is an extension term added to adapt to matrix multiplication. , This represents the total number of all high-confidence matching point pairs, i.e., the sum of the number of high-confidence matching point pairs for each pair of region-image pairs. Since the calculation of the affine transformation matrix for each region requires at least three pairs of high-confidence matching points, therefore, , express The corresponding affine transformation matrix.

[0114] ③ Under the initial conditions, calculate the mean of the residuals of all high-confidence matching pairs using the following formula. :

[0115]

[0116] ④ Under the initial conditions, calculate the standard deviation of the residuals of all high-confidence matching pairs using the following formula. :

[0117]

[0118] ⑤ Set hyperparameters : .

[0119] ⑥ Update the affine transformation vector: based on the residuals of all high-confidence matching point pairs and hyperparameters Calculate the loss value According to the loss value calculate Increment Finally Reassign to .

[0120] Each subsequent iteration of the Levenberg-Marquardt algorithm will utilize steps ② and ⑥ above to calculate... Increment , and is re-assigned to , the above process is iterated until the 2-norm of the increment is smaller than a minimum update threshold, or the iteration number reaches a maximum iteration number, at which time the obtained corresponds to the optimal affine transformation matrix, and the obtained is denoted as , and the optimal affine transformation matrix is denoted as , , wherein is the first component of the affine transformation vector at the iteration stop of the LM algorithm, = 1, 2, …, 6, and specifically, is the second component of the affine transformation vector at the iteration stop, is the third component of the affine transformation vector at the iteration stop, is the fourth component of the affine transformation vector at the iteration stop, is the fifth component of the affine transformation vector at the iteration stop, is the sixth component of the affine transformation vector at the iteration stop, is the seventh component of the affine transformation vector at the iteration stop, .

[0121] S6, registering the optical image to be registered and the SAR image to be registered using the optimal affine transformation matrix, specifically:

[0122] For each pixel point in the SAR image to be registered, the corresponding coordinate position in the optical image to be registered is calculated according to the parameters in the optimal affine transformation matrix, thereby realizing the accurate geometric transformation of the SAR image to the optical image. This step requires resampling the image and other operations to ensure the integrity and accuracy of the transformed image. During the resampling process, methods such as bilinear interpolation can be used to ensure the quality of the transformed image. Finally, by applying the optimal affine transformation matrix, the optical image to be registered and the SAR image to be registered are accurately aligned in geometric position.

[0123] In addition, to further improve the registration accuracy and visualization effect, post-processing techniques such as optical flow can be used to detect and correct possible local errors in the registered images. Optical flow can effectively analyze the motion of pixel points in image sequences, thereby further optimizing the alignment effect of the images. At the same time, in order to verify the accuracy of the registration result, region growing algorithm and other methods can be used to segment and analyze the registered images, and the consistency of the features in different regions is compared to evaluate the registration effect. Region growing algorithm can divide the regions according to the similarity of the pixel points in the image, thereby helping to identify the problem areas in the registration process.

[0124] After completing the above steps, the final registered images can be used for subsequent lunar terrain analysis, lunar mapping, scientific research and other applications. Through this series of operations based on the optimal affine transformation matrix, efficient and accurate registration of optical images and SAR images is achieved, providing reliable data support for lunar exploration missions. This process fully utilizes the technical terms and methods in the technical scheme, ensuring the scientificity and practicality of the registration results.

[0125] Optionally, in the above technical scheme, further comprising:

[0126] S01, each typical lunar feature is characterized by a five-tuple parameter including a label, a longitude of a center point, a latitude of the center point, a span in the east-west direction, and a span in the north-south direction, to obtain a prior database;

[0127] Wherein, the typical lunar features include lunar impact craters, lunar rills, ridges and lunar valleys, etc., which can be set according to actual conditions.

[0128] The determination process of the label, the longitude of the center point, the latitude of the center point, the span in the east-west direction and the span in the north-south direction is explained as follows:

[0129] 1) The label can use a unique English identifier.

[0130] 2) Longitude of the center point, latitude of the center point: using the publicly available lunar optical DOM image, a horizontal rectangular frame covering the edge line of the typical lunar feature region is used to represent the core distribution range of the typical lunar feature region, and the longitude of the center point and the latitude of the center point of each horizontal rectangular frame region are determined using image processing tools. Specifically:

[0131] The published lunar optical DOM images are acquired, which contain rich lunar topographic information. Then, for a typical lunar topographic area to be studied, a horizontal rectangular frame covering the area is drawn according to its edge line on the image, and the frame is used to represent the core distribution range of the typical lunar topographic area. The determined horizontal rectangular frame area is analyzed and processed by using an image processing tool. Through specific algorithms and calculation methods, the center point position of the rectangular frame area is determined. Since the moon has a specific geographic coordinate system, after the center point is determined, the center point is further converted into corresponding longitude and latitude values by using related technical means, so as to obtain the longitude of the center point and the latitude of the center point of each horizontal rectangular frame area, and finally to accurately describe and locate the core distribution range of the typical lunar topographic area, and to provide key data support for constructing a prior database of five-tuple parameters including a label, the longitude of the center point, the latitude of the center point, the east-west span and the north-south span.

[0132] 3) East-west span and north-south span: the east-west span and the north-south span are calculated based on the actual size of the horizontal rectangular frame area, and are appropriately extended to adapt to different spatial resolution images and tolerate certain geometric positioning errors, so as to improve the registration applicability in complex terrain and low-quality images. Specifically:

[0133] It is necessary to determine the specific position and actual size of the horizontal rectangular frame area in the lunar optical DOM image. According to the lunar geographic coordinate system, the longitude interval and the latitude interval corresponding to the range covered by the horizontal rectangular frame area are determined. According to the actual size, the east-west span, i.e. the span of the horizontal rectangular frame area in the longitude direction, is calculated, which can be obtained by calculating the difference between the easternmost longitude and the westernmost longitude of the area. Similarly, the north-south span is the span of the horizontal rectangular frame area in the latitude direction, which is determined by calculating the difference between the northernmost latitude and the southernmost latitude of the area. In actual operation, it is necessary to ensure that the actual size data of the obtained horizontal rectangular frame area is accurate, so as to accurately calculate the east-west span and the north-south span, so that these data can form complete and accurate five-tuple parameters together with the label, the longitude of the center point and the latitude of the center point, for accurately characterizing the typical lunar topography and constructing a prior database.

[0134] In S1, the same typical lunar topographic object existing in the to-be-registered optical image and the to-be-registered SAR image of the moon is determined, including:

[0135] S10, according to the prior database, the geometric positioning information of the to-be-registered optical image and the geometric positioning information of the to-be-registered SAR image, the same typical lunar topographic object existing in the to-be-registered optical image and the to-be-registered SAR image of the moon is determined, specifically including the following steps:

[0136] S100, determine the geometric positioning information of the optical image to be registered and the geometric positioning information of the SAR image to be registered, in particular:

[0137] The geometric positioning information of the optical image to be registered mainly includes the central point longitude and latitude of the image, the pixel resolution, and the attitude angle during imaging, etc. The geometric positioning information of the SAR image to be registered includes the central point longitude and latitude, the radar wavelength, the incidence angle, the azimuth resolution and the range resolution, etc. These geometric positioning information can accurately describe the geographical position range of the image on the lunar surface and the geometric relationship during imaging.

[0138] S101, in combination with the typical lunar feature five-tuple parameters in the prior database, the typical lunar feature objects in the prior database with similar longitude and latitude to the central point longitude and latitude of the optical image to be registered and the SAR image to be registered are first screened out, and the range of the typical lunar feature possibly existing in the image to be registered is preliminarily determined. Then, according to the geometric positioning information of the optical image to be registered and the SAR image to be registered, the range of the typical lunar feature possibly appearing in the respective images is calculated, for example, through the pixel resolution and the imaging attitude angle of the optical image, the east-west span and the north-south span in the prior database can be converted into the pixel range on the optical image; by using the radar wavelength, the incidence angle and other parameters of the SAR image, the span in the prior database is converted into the distance range on the SAR image.

[0139] S102, by comparing the regions where the typical lunar feature possibly appears in the optical image and the SAR image determined by the geometric positioning information, in combination with the image content features, such as the shape, edge features of the typical lunar feature in the optical image and the echo features of the corresponding typical lunar feature in the SAR image, the same typical lunar feature objects existing in the two images to be registered are further verified and determined. When the typical lunar feature in the two images matches in position (the range determined by the geometric positioning information), shape features, etc., it can be determined that the typical lunar feature object is the same object existing in the two images to be registered, thereby providing a basis for the subsequent image registration work.

[0140] Then, the optical region image of each typical lunar feature object in the optical image to be registered and the SAR region image in the SAR image to be registered are respectively composed into a region image pair, in particular:

[0141] ①When the typical lunar feature is a lunar crater, in the optical image to be registered, the possible area of the crater is determined according to the geometric positioning information of the optical image, the circular edge profile and the internal shape of the crater are recognized by using image processing tools, and the pixel range covered by the crater is extracted as an optical area image; in the SAR image to be registered, the corresponding area is determined according to the geometric positioning information of the SAR image, the SAR features of the crater are recognized by analyzing the echo intensity and texture features, and the corresponding area is extracted as a SAR area image, so as to form an area image pair.

[0142] ②When the typical lunar feature is a lunar rille, in the optical image to be registered, the linear extension feature of the lunar rille is found according to the geometric positioning information of the optical image, which is usually manifested as a long and narrow and relatively low reflectivity winding area on the optical image, the profile of the lunar rille is outlined and extracted as an optical area image; in the SAR image to be registered, the corresponding position is located according to the geometric positioning information of the SAR image, the lunar rille presents specific linear extension features on the SAR image due to the influence of surface roughness and slope on echo characteristics, the area is recognized and extracted, and the area image pair is formed with the lunar rille area image in the optical image.

[0143] ③If the typical lunar feature is a wrinkle ridge, in the optical image to be registered, the linear ridge feature of the wrinkle ridge is found according to the geometric positioning information of the optical image, which is manifested as a relatively high reflectivity and steep linear landform, and the range of the wrinkle ridge in the image is extracted as an optical area image; in the SAR image to be registered, the corresponding position is found by using the geometric positioning information of the SAR image, the wrinkle ridge forms a specific high echo intensity area due to the different reflection characteristics of the radar wave, and the area is extracted to form an area image pair with the wrinkle ridge area image in the optical image.

[0144] ④When the typical lunar feature is a lunar valley, in the optical image to be registered, the wide valley bottom and steep valley wall features of the lunar valley are recognized according to the geometric positioning information of the optical image, which are usually manifested as areas with high reflectivity on both sides and low reflectivity in the middle, and the area is extracted as an optical area image; in the SAR image to be registered, the position is determined by using the geometric positioning information of the SAR image, the echo features of the lunar valley on the SAR image are analyzed, such as low echo of the valley bottom and high echo of the valley wall, and the corresponding area is extracted to form an area image pair with the lunar valley area image in the optical image.

[0145] Optionally, the number of the same typical lunar features in the optical image to be registered and the SAR image to be registered of the moon can also be determined.

[0146] Currently, there is no prior art directly related to the present application among the disclosed technical solutions. Through systematic investigation, it is found that the current research on the registration of lunar remote sensing satellite optical and SAR images is relatively limited, and most of the existing methods are designed for earth observation scenes, which are difficult to effectively deal with the special environmental problems such as lack of texture information on the lunar surface and high surface similarity. The registration method of the optical image and the SAR image of the moon in the present application is a method based on a three-stage registration framework of regional coarse matching, regional fine matching and global optimization. By constructing a prior database of typical lunar features such as lunar craters, lunar rivers, lunar ridges and lunar valleys, regional-level geographic information is introduced to provide effective spatial constraints for the regional coarse matching stage; on the basis of coarse matching, combined with the feature-based registration algorithm, regional fine matching is realized; finally, a global optimization model is constructed by fusing the affine transformation parameters of multiple regions of the optical and SAR images, and an adaptive iterative algorithm is used to solve the globally optimal affine transformation matrix, realizing high-precision and strong-robustness optical and SAR remote sensing image registration. Regional guidance can effectively reduce the risk of false matching, solve the problems of difficult feature point extraction and unstable feature matching, and can tolerate large image geometric positioning errors; global optimization can effectively reduce the influence of local matching errors, realize the consistency of overall image registration, and improve the overall image registration accuracy. Taking typical lunar features such as lunar craters, lunar rivers, lunar ridges and lunar valleys as objects, regional-level geographic information is introduced to provide effective spatial constraints for feature extraction and matching, effectively reduce the risk of false matching, tolerate large image geometric positioning errors, effectively solve the problems of difficult feature point extraction and unstable feature matching; on the basis of regional fine matching, a global optimization model is constructed by fusing the matching results of multiple regions, and an adaptive iterative algorithm is used to solve it, realizing the consistency of overall image registration, effectively reducing the influence of local matching errors, and improving the overall image registration accuracy.

[0147] In the above embodiments, although the steps are numbered S1, S2, etc., the execution order of S1, S2, etc. can be adjusted according to the actual situation by those skilled in the art, which is also within the protection scope of the present application. It can be understood that in some embodiments, some or all of the above embodiments can be included.

[0148] As shown in Figure 2 The registration system 200 of the optical image and the SAR image of the moon in the embodiment of the present application includes a typical lunar feature object determination module 201, a preliminary matching point pair determination module 202, a screening module 203, a regional affine transformation matrix determination module 204, an optimal affine transformation matrix determination module 205 and a registration module 206.

[0149] The typical lunar feature object determination module 201 is used to determine the same typical lunar feature objects existing in the to-be-registered optical image and the to-be-registered SAR image of the moon.

[0150] The preliminary matching point pair determination module 202 is configured to perform feature extraction and matching on each regional image pair by using a feature-based registration algorithm, and output a plurality of preliminary matching point pairs, wherein each typical lunar feature object in the optical regional image in the optical image to be registered and the SAR regional image in the SAR image to be registered respectively form a regional image pair;

[0151] The screening module 203 is configured to screen a plurality of high-confidence matching point pairs from all the preliminary matching point pairs;

[0152] The regional affine transformation matrix determination module 204 is configured to calculate a regional affine transformation matrix for representing a geometric transformation relationship of the SAR regional image to the optical regional image in each regional image pair based on all the high-confidence matching point pairs;

[0153] The optimal affine transformation matrix determination module 205 is configured to solve an optimal affine transformation matrix according to all the regional affine transformation matrices and a constructed global optimization objective function;

[0154] The registration module 206 is configured to register the optical image to be registered and the SAR image to be registered by using the optimal affine transformation matrix.

[0155] Optionally, in the above technical solution, the regional affine transformation matrix determination module 205 is specifically configured to:

[0156] decompose each regional affine transformation matrix into a horizontal translation component, a vertical translation component, a horizontal scaling component, a vertical scaling component, a horizontal shear component, a vertical shear component and a rotation component, calculate a mean value of each same component, generate a fused affine transformation matrix, and obtain an affine transformation vector corresponding to the fused affine transformation matrix;

[0157] use the affine transformation vector corresponding to the fused affine transformation matrix as an iteration starting point of a Levenberg-Marquardt algorithm, and determine the optimal affine transformation matrix by using a global optimization objective function based on a Huber loss function.

[0158] Optionally, in the above technical solution, the screening module 203 is specifically configured to:

[0159] screen the plurality of high-confidence matching point pairs from all the preliminary matching point pairs by using a RANSAC algorithm and an angle constraint algorithm.

[0160] Optionally, the above technical solution further includes a database acquisition module, and the database acquisition module is configured to represent each typical lunar feature by using a five-tuple parameter including a label, a longitude of a center point, a latitude of the center point, an east-west span and a north-south span, and obtain a priori database.

[0161] The typical lunar appearance object determination module 201 is specifically configured to determine the same typical lunar appearance object existing in the optical image and the SAR image of the moon to be registered according to the prior database, the geometric positioning information of the optical image to be registered, and the geometric positioning information of the SAR image to be registered.

[0162] It should be noted that the beneficial effects of the optical image and SAR image registration system 200 of the moon provided in the above embodiment are the same as those of the optical image and SAR image registration method of the moon, which will not be repeated here. In addition, when the system provided in the above embodiment implements its functions, only the division of the above functional modules is exemplified, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the system is divided into different functional modules according to actual conditions to complete all or part of the above described functions. In addition, the system and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0163] Among them, the optical image and SAR image registration system of the moon of the present application can be a computer program (including program code) running in a computer device, for example, the optical image and SAR image registration system of the moon of the present application is an application software, which can be used to execute the corresponding steps in the optical image and SAR image registration method of the moon of the present application.

[0164] In some embodiments, the optical image and SAR image registration system of the moon of the present application can be realized in a combination of software and hardware, for example, the optical image and SAR image registration system of the moon of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the optical image and SAR image registration method of the moon of the present application, for example, the processor in the form of a hardware decoding processor can adopt one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs) or other electronic components.

[0165] Among them, the modules involved in the embodiments of the present application can be realized by software or hardware. Among them, the name of the module does not constitute a limitation of the module itself in some cases.

[0166] An electronic device according to an embodiment of the present application includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method for registering the optical image and the SAR image of the moon according to any one of the above embodiments when executing the computer program. That is, an electronic device according to an embodiment of the present application can include, but is not limited to, a processor and a memory, the memory configured to store a computer program, and the processor configured to execute the method for registering the optical image and the SAR image of the moon according to any one of the embodiments of the present application by invoking the computer program.

[0167] An electronic device is provided in an alternative embodiment, as shown in Figure 3 Figure 3 The electronic device 4000 shown in the figure includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, through a bus 4002. Optionally, the electronic device 4000 can further include a transceiver 4004, which can be used for data interaction, such as data transmission and / or data reception, between the electronic device and other electronic devices. It should be noted that the transceiver 4004 is not limited to one in actual application, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0168] The processor 4001 can be a CPU (Central Processing Unit, central processor), a general-purpose processor, a DSP (Digital Signal Processor, digital signal processor), an ASIC (Application Specific Integrated Circuit, application specific integrated circuit), an FPGA (Field Programmable Gate Array, field programmable gate array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logical blocks, modules and circuits described in combination with the disclosure. The processor 4001 can also be a combination of computing functions, such as one or more microprocessor combinations, combinations of DSP and microprocessor, etc.

[0169] The bus 4002 can include a path for transmitting information between the above-mentioned components. The bus 4002 can be a PCI (Peripheral Component Interconnect, peripheral component interconnect) bus or an EISA (Extended Industry Standard Architecture, extended industry standard architecture) bus, etc. The bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For the sake of expression, Figure 3 ​Only one bus 4002 is shown, but it could be comprised of several buses. Bus 4002 is used to transmit information between the various components of the computer.

[0170] The memory 4003 can be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions; a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions; an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, a magnetic disk storage or other magnetic storage devices, or any other medium capable of storing desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto.

[0171] The memory 4003 is used to store application code (computer program) for implementing the scheme of the present application, and is controlled by the processor 4001 to execute. The processor 4001 is used to execute the application code stored in the memory 4003 to realize the content shown in the foregoing method embodiments.

[0172] The electronic device can also be a terminal device, which can be any device that can install an application, including at least one of a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a smart television, and a smart vehicle device.

[0173] It should be noted that, Figure 3 The electronic device shown is only an example and should not limit the functions and use range of the embodiments of the present application.

[0174] The computer readable storage medium of the embodiments of the present application, the computer readable storage medium stores a computer program, and the computer program is executed by the processor to realize the registration method of the optical image and the SAR image of the moon.

[0175] Alternatively, the computer readable storage medium can be a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM), a magnetic tape, a floppy disk and an optical data storage device, etc.

[0176] In an example embodiment, a computer program product or computer program including computer instructions stored in a computer readable storage medium is also provided. A processor of an electronic device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the electronic device to perform any of the above-mentioned methods of registering optical images and SAR images of the Moon.

[0177] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0178] It should be understood that the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of various embodiments of the present application. In this regard, each block in the flowchart and block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the flowchart or block diagrams can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or combinations of hardware and software.

[0179] The computer readable storage medium provided by the embodiments of the present application can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device.

[0180] The computer readable storage medium described above carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the embodiments described above.

[0181] The above description is merely the preferred embodiments of the present application and the explanation of the technical principles used. It should be understood by those skilled in the art that the disclosed scope of the present application is not limited to the technical solutions formed by the specific combinations of the technical features described above, and should also cover other technical solutions formed by any combinations of the technical features described above or their equivalent features without departing from the disclosed concept. For example, the technical solutions formed by replacing the above features with the technical features disclosed in the present application (but not limited to) having similar functions.

[0182] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application are used to distinguish similar objects, and represent no specific order or sequence. The order of use of similar objects can be interchanged in appropriate cases, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described.

[0183] Those skilled in the art know that the present application can be implemented as a system, a method or a computer program product, so the present application can be specifically implemented as follows: it can be a complete hardware, a complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, which is generally referred to as "circuit", "module" or "system" in this paper. In addition, in some embodiments, the present application can also be implemented as a computer program product in one or more computer readable media, which contains computer readable program code.

[0184] Although the embodiments of the present application have been shown and described above, it is understood that the above-described embodiments are exemplary and are not to be construed as limiting the present application, and that variations, modifications, substitutions and changes can be made by those skilled in the art without departing from the scope of the present application.

Claims

1. A method of registering optical images and SAR images of the moon, characterized in that, The method comprises the following steps: determining the same typical lunar landform objects existing in the to-be-registered optical image and the to-be-registered SAR image of the moon; performing feature extraction and matching on each regional image pair by using a feature-based registration algorithm, and outputting a plurality of preliminary matching point pairs, wherein the optical regional image of each typical lunar landform object in the to-be-registered optical image and the SAR regional image of each typical lunar landform object in the to-be-registered SAR image respectively form a regional image pair; screening a plurality of high-confidence matching point pairs from all the preliminary matching point pairs; calculating a regional affine transformation matrix for representing a geometric transformation relationship of the SAR regional image to the optical regional image in each regional image pair based on all the high-confidence matching point pairs; solving an optimal affine transformation matrix according to all the regional affine transformation matrices and a constructed global optimization objective function; registering the to-be-registered optical image and the to-be-registered SAR image by using the optimal affine transformation matrix; The method further comprises the following steps: characterizing each typical lunar landform by using a five-tuple parameter comprising a label, a longitude of a center point, a latitude of the center point, an east-west span and a north-south span, to obtain a priori database; determining the same typical lunar landform objects existing in the to-be-registered optical image and the to-be-registered SAR image of the moon, comprising:

2. The method of registering optical images and SAR images of the Moon according to claim 1, characterized in that, determining the same typical lunar landform objects existing in the to-be-registered optical image and the to-be-registered SAR image of the moon according to the a priori database, geometric positioning information of the to-be-registered optical image and geometric positioning information of the to-be-registered SAR image. Solving an optimal affine transformation matrix according to all the regional affine transformation matrices and a constructed global optimization objective function, comprising: decomposing each regional affine transformation matrix into a horizontal translation component, a vertical translation component, a horizontal scaling component, a vertical scaling component, a horizontal shear component, a vertical shear component and a rotation component, calculating a mean value on each same component, generating a fused affine transformation matrix, and obtaining an affine transformation vector corresponding to the fused affine transformation matrix; 3. The method of registering optical images and SAR images of the moon according to claim 1, wherein, determining the optimal affine transformation matrix based on the affine transformation vector corresponding to the fused affine transformation matrix as an iteration starting point of a Levenberg-Marquardt algorithm, and using a global optimization objective function based on a Huber loss function. Screening a plurality of high-confidence matching point pairs from all the preliminary matching point pairs, comprising:

4. A system for registering optical images and SAR images of the moon, characterized in that, screening a plurality of high-confidence matching point pairs from all the preliminary matching point pairs by using a RANSAC algorithm and an angle constraint algorithm. The method comprises a typical lunar landform object determination module, a preliminary matching point pair determination module, a screening module, a regional affine transformation matrix determination module, an optimal affine transformation matrix determination module and a registration module; The typical lunar landform object determination module is configured to determine the same typical lunar landform objects existing in the to-be-registered optical image and the to-be-registered SAR image of the moon. The preliminary matching point pair determination module is configured to: perform feature extraction and matching on each regional image pair by using a feature-based registration algorithm, and output a plurality of preliminary matching point pairs, wherein each typical lunar feature object in the optical regional image in the optical image to be registered and in the SAR regional image in the SAR image to be registered respectively forms a regional image pair; The screening module is configured to: screen a plurality of high-confidence matching point pairs from all the preliminary matching point pairs; The regional affine transformation matrix determination module is configured to: calculate a regional affine transformation matrix for representing a geometric transformation relationship of the SAR regional image to the optical regional image in each regional image pair based on all the high-confidence matching point pairs; The optimal affine transformation matrix determination module is configured to: solve an optimal affine transformation matrix according to all the regional affine transformation matrices and a constructed global optimization objective function; The registration module is configured to: register the optical image to be registered and the SAR image to be registered by using the optimal affine transformation matrix. The database acquisition module is configured to: represent each typical lunar feature by using a five-tuple parameter including a label, a longitude of a center point, a latitude of the center point, an east-west span, and a north-south span, and obtain a priori database. The typical lunar feature object determination module is specifically configured to: determine the same typical lunar feature object existing in the optical image to be registered and the SAR image to be registered of the moon according to the priori database, geometric positioning information of the optical image to be registered, and geometric positioning information of the SAR image to be registered.

5. The system for registration of optical images and SAR images of the Moon according to claim 4, characterized in that, The regional affine transformation matrix determination module is specifically configured to: decompose each regional affine transformation matrix into a horizontal translation component, a vertical translation component, a horizontal scaling component, a vertical scaling component, a horizontal shear component, a vertical shear component, and a rotation component, calculate a mean value on each same component, generate a fused affine transformation matrix, and obtain an affine transformation vector corresponding to the fused affine transformation matrix; use the affine transformation vector corresponding to the fused affine transformation matrix as an iteration starting point of a Levenberg-Marquardt algorithm, and determine an optimal affine transformation matrix by using a global optimization objective function based on a Huber loss function.

6. The system for registration of optical images and SAR images of the Moon according to claim 4, characterized in that, The screening module is specifically configured to: screen a plurality of high-confidence matching point pairs from all the preliminary matching point pairs by using a RANSAC algorithm and an angle constraint algorithm.

7. An electronic device, comprising: The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the registration method of the optical image and the SAR image of the moon according to any one of claims 1 to 3.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the registration method of the optical image and the SAR image of the moon according to any one of claims 1 to 3.

Images