Synthetic Aperture Radar and Optical Image Registration System and Method Based on Structure Extraction

Abstract

The present invention discloses a synthetic aperture radar and optical image registration system and method based on structure extraction, including a structure extraction module, a feature point extraction module, a feature point description module, a secondary registration module, and a registration result display module; the method includes: performing structure extraction on the synthetic aperture radar and optical images to obtain multi-layer structure diagrams; extracting feature points in each layer of the structure diagrams by using a multi-scale Harris corner extractor; combining the Sobel operator and the maximum moment of phase consistency to construct an enhanced structure descriptor; performing coarse-to-fine registration in combination with the position and angle information of the feature points; and displaying the registered images in a checkerboard format. The present invention extracts feature points and constructs an enhanced descriptor on the structure image after removing the texture, which can overcome the influence of the spot noise on the synthetic aperture radar image and the texture area of the optical image on the registration, promote the distribution of feature points on the structures with significant features, and can also obtain accurate and distinguishable descriptors, effectively improving the registration accuracy and robustness of the system.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and particularly relates to a registration method for synthetic aperture radar images and optical images based on structure extraction. Background Art

[0002] With the development of space technology, more and more artificial satellites carrying various imaging sensors have been sent into space, providing us with a large amount of remote sensing data. Each sensor has different characteristics, providing us with richer ground observation information. Among them, SAR images generated by synthetic aperture radar (SAR) and optical images generated by optical cameras are two very important types of images in remote sensing images, and the two can provide highly complementary ground observation information. The registration of SAR images and optical images is an important part of remote sensing image processing and understanding, and is also a prerequisite for subsequent remote sensing image fusion, change detection, and target detection tasks.

[0003] The registration of SAR images and optical images has always been a difficult problem in the field of computer vision. The huge radiation differences, geometric differences, non-linear transformation of the gray values of the imaged objects, and the inherent multiplicative speckle noise in SAR images are all important problems that need to be faced in the development of SAR and optical image registration technologies.

[0004] The SAR image and optical image registration technology is a technology that aligns the same landmark positions on two images through an optimal mapping transformation. Currently, SAR and optical image registration methods are mainly divided into gray information-based and feature-based image registration methods. In gray information-based image registration algorithms, most use similarity measurement methods such as mutual information (MI) or cross-cumulative residual entropy to achieve template matching. However, these methods are often prone to falling into local optimal solutions and failing, and their iterative solution characteristics make the calculation amount very large. Since the 21st century, image registration has mainly used feature-based methods. The main steps of this method include: feature point extraction, feature description, feature matching, and model parameter estimation. However, existing methods often cannot solve the following problems: due to the inherent multiplicative speckle noise in SAR images, noise is often selected as key points during feature point extraction; feature points distributed in texture regions cannot obtain unique and distinguishable descriptors due to the similar neighborhood structures; on the other hand, due to the non-linear gray value transformation between SAR and optical images, the feature descriptors in the same region of the two images are not repeatable, resulting in the failure of the matching system. Summary of the Invention

[0005] The purpose of the present invention is to provide a synthetic aperture radar and optical image registration system and method based on structure extraction, which overcomes the speckle noise of synthetic aperture radar images, extracts feature points on significant structures, and can obtain unique and repeatable descriptors.

[0006] The technical solution adopted by the present invention is a synthetic aperture radar and optical image registration system based on structure extraction, including:

[0007] A structure extraction module connected to the input image end, which is used to smooth the texture information of the input image at multiple scales, extract structure information, refine edges, and output a structure image with multi-layer texture information smoothed layer by layer;

[0008] A feature point extraction module connected to the structure extraction module, which is used to extract feature points, the layer number, position, and scale information of the feature points in each layer of the structure image;

[0009] A feature point description module connected to the feature point extraction module and the structure extraction module, which is used to describe the main direction of the feature points and construct feature point descriptors;

[0010] A feature point matching module connected to the feature point extraction module and the feature point description module: The nearest neighbor distance ratio method and the random sample consensus algorithm are used to achieve rough registration; after combining the rough registration, the position and direction information of the feature points are used for fine registration to filter out incorrect matching points;

[0011] A registration result display module connected to the feature point matching module and the input image end: The registration effect is displayed in a checkerboard format.

[0012] The characteristics of the present invention also lie in:

[0013] The structure extraction module is specifically used for: using the window relative total transformation iterative optimization algorithm to perform multi-scale smoothing of the texture in the synthetic aperture radar image and the optical image. Each iterative optimization is performed on the image obtained from the previous iterative optimization, and each layer of the image is saved, and the output is a structure image with 6 layers of texture information smoothed layer by layer.

[0014] During the multi-scale smoothing of the texture in the synthetic aperture radar image and the optical image using the window relative total transformation iterative optimization algorithm: The initial smoothing window variance scale of the synthetic aperture radar image is 4 pixels, the initial window scale of the optical image is 2 pixels, the scale window decreases by 0.5 times with the number of iterations, and the number of iterations is 6.

[0015] The feature point extraction module includes:

[0016] A multi-scale Harris function calculation unit, which is used to calculate the Harris function values of each layer of the multi-layer structure image output by the structure extraction module respectively using 6 decreasing Harris corner detection windows. The initial detection window has a scale of 6 pixels, and the window decreases as the number of layers of the structure image increases by decrease to obtain multi-scale Harris function values;

[0017] A feature point information acquisition unit connected to the multi-scale Harris function calculation unit is used to search for the maximum value in the neighborhood of the multi-scale Harris function value. The points that satisfy being greater than the threshold among the searched maximum values are the feature points, as well as the layer number, position, and scale information of the feature points.

[0018] The feature point description module includes:

[0019] An enhanced phase congruency edge construction unit combines the Sobel edge with gradient intensity retained on the phase congruency edge on each layer of the structure image to obtain an enhanced phase congruency edge.

[0020] A descriptor construction unit connected to the enhanced phase congruency edge construction unit and the feature point extraction device: Combining the feature point position information, scale information, and direction information, on the enhanced phase congruency edge, it statistically analyzes the distribution of the gradients and directions of the pixels in 17 fan-shaped ring regions of the circular neighborhood of the feature points to create feature descriptors.

[0021] The feature point matching module includes:

[0022] A rough matching unit uses the nearest neighbor distance ratio and the fast sample consensus algorithm for rough registration to obtain a rough registration transformation matrix H1.

[0023] A fine matching unit connected to the rough matching unit combines the position information and direction information of the feature points, adjusts the Euclidean distance between descriptors, and performs the nearest neighbor distance ratio and the fast sample consensus algorithm for the second time to filter out false matching pairs to obtain a fine registration perspective transformation matrix H2.

[0024] A synthetic aperture radar and optical image registration method based on structure extraction is specifically implemented according to the following steps:

[0025] Step 1: Use the relative total transformation structure extraction algorithm to extract the structures of the synthetic aperture radar and the optical image to obtain a multi-layer structure image with gradually smoothed texture information.

[0026] Step 2: On the multi-layer structure image, use a multi-scale Harris corner detection algorithm with a feature point detection window that gradually decreases layer by layer with the structure image to extract the feature points of the synthetic aperture radar and the optical image, and obtain the layer number, position, and scale information of the feature points.

[0027] Step 3: Construct an enhanced phase congruency edge on each layer of the structure image, and use this edge feature to construct descriptors.

[0028] Step 4: Initially match using the nearest neighbor distance ratio, and use the Random Sample Consensus (RANSAC) algorithm to fit the rough transformation matrix H1. After the initial transformation, adjust the distance between descriptors based on the position distance difference and angle difference of the feature points in the reference image pixel coordinate system, and perform a secondary match to obtain the accurate perspective transformation matrix H2;

[0029] Step 5: Multiply the image to be registered by the transformation matrix H2 to transform it into the pixel coordinate system where the reference image is located, and display the two modal images in the form of a checkerboard.

[0030] The specific process of extracting feature points from synthetic aperture radar and optical images is as follows:

[0031] On the multi-layer structure image, use an initial window scale of 6 pixels, and calculate the Harris function value of each layer of the image with a Harris corner detection window that decreases by times as the number of layers of the structure image increases;

[0032] Search for the maximum value in the neighborhood of the multi-scale Harris function values. The points that satisfy being greater than the threshold are the feature points, and record the layer number, position, and scale information of the feature points; where the threshold τ is a user-defined threshold.

[0033] The specific process of Step 3 is as follows:

[0034] Step 3.1: On each layer of the structure image, calculate the phase congruency edge and the Sobel edge. Perform an AND operation on the corresponding pixel positions of the Sobel edge and the phase congruency edge to obtain the intersection mask mask. Multiply the intersection mask mask by the corresponding elements of the Sobel edge map, perform a spatial domain filter once, and take the larger value of the corresponding bits of the filtered Sobel edge and the phase congruency edge as the enhanced phase congruency edge map M en ;

[0035] Step 3.2: On the enhanced phase congruency edge, calculate the gradient direction histogram at the position of the feature points, assign the main direction to the feature points, and describe the features using the SIFT feature descriptor method in a circular neighborhood that is positively correlated with the scale size of the feature points and is divided into 17 fan-shaped rings. Each feature point finally obtains a 136-dimensional feature point descriptor.

[0036] The specific process of Step 4 is as follows:

[0037] Step 4.1: Initially match using the nearest neighbor distance ratio: Use the nearest neighbor distance ratio to compare the Euclidean distances between the descriptors on the reference image and the image to be registered, select the feature point matching pairs in the two images, and after removing duplicates, use the Random Sample Consensus (RANSAC) algorithm to filter out the incorrect matching pairs and fit the rough transformation matrix H1;

[0038] Step 4.2. Transform the feature point pairs on the image to be registered to the pixel coordinate system of the reference image through the transformation matrix H1, and calculate the position distance difference e between each feature point and all feature points on the other image. p (p i , p′ j ) and the main direction difference e o (p i , p′ j ), and add 1 to the sum of these two differences as the magnification factor to amplify the Euclidean distance between the corresponding descriptors. Then perform the same steps as the initial matching again to obtain the accurate perspective transformation matrix H2.

[0039] The beneficial effects of the present invention are as follows:

[0040] The structure extraction module of the present invention can regard speckle noise as a kind of texture feature and effectively erase the texture, avoiding the influence of noise on the extraction of feature points. The present invention extracts feature points after extracting the image structure, avoiding feature points falling into texture areas where features are not easily distinguishable, so that the feature points extracted from synthetic aperture radar and optical images are mainly distributed on the more prominent structure edges, improving the position space repetition rate of the feature points extracted from the two images and the uniqueness of the descriptors. In addition, the system does not downsample the image, and during structure extraction, the edges become finer and finer, which provides accurate positioning for feature points. When describing feature points, first construct enhanced phase consistency edges on each layer of the structure image to obtain clear, accurate and noise-free edges, and then use the edge information to describe feature points, which can overcome the non-linear gray value transformation between synthetic aperture radar and optical images to obtain distinguishable and similar descriptors for corresponding feature points. The entire system extracts feature points and performs feature description on the structure image where multi-layer texture information is gradually suppressed layer by layer. In this way, not only feature points on prominent structures are extracted, but also feature points on edges with weaker contrast can be extracted from shallow structure images, making the number of extracted feature points more and the distribution more uniform. The invention not only improves the robustness of the registration of synthetic aperture radar and optical images, but also improves the accuracy of image registration. It provides a simple and efficient method for registering synthetic aperture radar and optical images, and has important application value. Description of the Drawings

[0041] Figure 1 Flowchart of the synthetic aperture radar and optical image registration system based on feature points of the present invention.

[0042] Figure 2 Schematic diagram of the principle of the feature point extraction device

[0043] Figure 3 Schematic diagram of the principle of the feature point description device

[0044] Figure 4 Output example diagram of the structure extraction device

[0045] Figure 5 Schematic diagram of the output of the registration result display device;

[0046] Figure 6 Display the registered images of two modalities in the form of a checkerboard. Detailed implementation manners

[0047] The present invention will be described in detail below in conjunction with the specific implementation manners.

[0048] The synthetic aperture radar and optical image registration system based on structure extraction of the present invention includes:

[0049] A structure extraction module connected to the input image end, configured to smooth the texture information of the input image at multiple scales, extract structure information, refine edges, and output a structure image with multi-layer texture information smoothed layer by layer;

[0050] Specifically, the structure extraction module is configured to: perform multi-scale smoothing of the texture in the synthetic aperture radar image and the optical image by using the window relative total transformation iterative optimization algorithm. Each iterative optimization is performed on the image obtained from the previous iterative optimization, and each layer of the image is saved, and a structure image with 6 layers of texture information smoothed layer by layer is output.

[0051] During the multi-scale smoothing process of the texture in the synthetic aperture radar image and the optical image by using the window relative total transformation iterative optimization algorithm: the initial smoothing window variance scale of the synthetic aperture radar image is 4 pixel, the initial window scale of the optical image is 2 pixel, the scale window decreases by 0.5 times with the number of iterations, and the number of iterations is 6.

[0052] A feature point extraction module connected to the structure extraction module, configured to extract feature points, the layer number where the feature points are located, the position, and the scale information thereof in each layer of the structure image;

[0053] As Figure 2 shown, the feature point extraction module includes:

[0054] A multi-scale Harris function calculation unit, configured to calculate the Harris function values of each layer of the multi-layer structure image output by the structure extraction module respectively by using 6 decreasing Harris corner detection windows. The initial detection window has a scale of 6 pixel, and the window decreases with the increase of the layer number of the structure image by decrease, and obtain multi-scale Harris function values;

[0055] A feature point information acquisition unit connected to the multi-scale Harris function calculation unit, which is used to search for the maximum value in the neighborhood of the multi-scale Harris function value. The points that meet the condition of being greater than the threshold among the searched maximum values are the feature points, as well as the layer number, position, and scale information of the feature points. In this method, the threshold is default set to 0.1.

[0056] A feature point description module connected to the feature point extraction module and the structure extraction module, which is used to describe the main direction of the feature points and construct feature point descriptors;

[0057] As Figure 3 shown, the feature point description module includes:

[0058] Construct an enhanced phase congruency edge unit. On each layer of the structure image, combine the Sobel edge that preserves the gradient intensity on the phase congruency edge to obtain the enhanced phase congruency edge;

[0059] A descriptor construction unit connected to the enhanced phase congruency edge unit construction and the feature point extraction device: Combine the feature point position information, scale information, and direction information. On the enhanced phase congruency edge, statistically analyze the distribution of the gradients and directions of the pixels in 17 sector ring regions of the feature point circular neighborhood to create a feature descriptor.

[0060] A feature point matching module connected to the feature point extraction module and the feature point description module: Use the nearest neighbor distance ratio and the random sample consensus algorithm to achieve rough registration; After combining the rough registration, use the position and direction information of the feature points for fine registration to filter out incorrect matching points;

[0061] The feature point matching module includes:

[0062] A rough matching unit that uses the nearest neighbor distance ratio and the random sample consensus algorithm for rough registration to obtain a rough registration transformation matrix H1;

[0063] A fine matching unit connected to the rough matching unit, which combines the position information and direction information of the feature points, adjusts the Euclidean distance between descriptors, and performs the nearest neighbor distance ratio and the random sample consensus algorithm for the second time to filter out incorrect matching pairs to obtain a fine registration perspective transformation matrix H2.

[0064] A registration result display module connected to the feature point matching module and the input image end: Display the registration effect in a checkerboard format.

[0065] The method for registering synthetic aperture radar and optical images based on structure extraction is specifically implemented according to the following steps:

[0066] Step 1: After running the program, the addresses where the synthetic aperture radar and optical images to be registered are located will be opened. Select a pair of synthetic aperture radar and optical images to be registered, and use the relative total transformation structure extraction algorithm to extract the structures of the synthetic aperture radar and optical images, obtaining a multi-layer structure image with 6 layers of texture information smoothed layer by layer;

[0067] The relative total transformation (RTV) algorithm extracts the image structure by iteratively solving the minimum optimization problem, specifically:

[0068]

[0069] Where:

[0070]

[0071]

[0072] are the total window variations (WTV) in the x and y directions respectively, are the internal window variations (WIV) in the x and y directions respectively, i.e., the relative total window variation (RTV), which is used as the penalty term for the structure extraction problem.

[0073] G σ is a two-dimensional Gaussian convolution kernel with variance σ. The convolution window size is generally 5σ. S is the estimated structure image, and are the partial derivatives in the x and y directions respectively. I is the original image, P represents the pixel points in the image, ε is a very small constant to avoid the denominator being 0, and λ is a weight coefficient;

[0074] For the initial calculation of the transformation window scale for the synthetic aperture radar image from GF-3, σ is 4 (pixel), and for the optical image, the initial window scale σ is 2 (pixel), which is used to overcome the influence of speckle noise on the SAR image and texture regions on the optical image on the extraction and description of feature points; the scale window decreases by a factor of 0.5 with the number of iterations, which is used to refine the image structure edges; after 6 iterations, a structure image with 6 layers of texture information smoothed layer by layer is obtained.

[0075] For the initial calculation of the transformation window scale for the synthetic aperture radar image, σ is 0.5 (pixel), and for the optical image, the initial window scale σ is 1 (pixel), which is used to overcome the influence of speckle noise on the synthetic aperture radar image and texture regions on the optical image on the extraction and description of feature points; the scale window decreases by a factor of 0.5 with the number of iterations, which is used to refine the image structure edges; after 6 iterations, a structure image with 6 layers of texture information smoothed layer by layer and edges refined layer by layer is obtained.

[0076] The initial relative total transformation window scale for synthetic aperture radar images is σ = 4 (pixels), and the initial relative total transformation window scale σ for optical images is 2 (pixels). The general window scale is 5σ. The scale window decreases by a factor of 0.5 with the number of iterations. After 6 iterations, a structural image with 6 layers of textures being gradually smoothed and edge localization becoming gradually refined is obtained. The effect of the multi-layer structure diagram is as shown in Figure 4 Figure. The upper and lower rows respectively show the structure diagrams of synthetic aperture radar and optical images at 0, 3, and 6 iterations.

[0077] Among them, the initial window scale, reduction factor, and number of iterations are all empirical values, which are the optimal values obtained through experimental verification.

[0078] Step 2: On the multi-layer structure image, use a multi-scale Harris corner detection algorithm with a feature point detection window that decreases layer by layer with the structure image to extract the feature points of synthetic aperture radar and optical images, and obtain the layer number, position, and scale information of the feature points.

[0079] The specific process of extracting the feature points of synthetic aperture radar and optical images is as follows:

[0080] On the multi-layer structure image, use the Harris window calculation function value with an initial window scale of 6 pixels, and use a Harris corner detection window that decreases by a factor of times with the increase of the structure image layer number to calculate the Harris function value of each layer of the image;

[0081] Search for the maximum value in the neighborhood of the multi-scale Harris function value. The points whose searched maximum values satisfy being greater than the threshold are the feature points, and record the layer number, position, and scale information of the feature points; where the threshold τ is a user-defined threshold, and the default value of this method is set to 0.1.

[0082] Step 3: Construct an enhanced phase congruency edge on each layer of the structure image, and use this edge feature to construct a descriptor; the specific process is as follows:

[0083] Step 3.1: On each layer of the structure image, calculate the phase congruency edge and the Sobel edge, perform an AND operation on the Sobel edge and the phase congruency edge at the corresponding pixel positions to obtain an intersection mask mask, multiply the intersection mask mask by the corresponding elements of the Sobel edge map, perform a spatial domain filtering once, and take the larger value of the corresponding bits of the filtered Sobel edge and the phase congruency edge as the enhanced phase congruency edge map M en ;

[0084] Step 3.2. On the enhanced phase consistency edges, calculate the gradient direction histogram at the positions of the feature points, assign the main direction to the feature points, and describe the features within a circular neighborhood that is positively correlated with the scale size of the feature points and is divided into 17 fan-shaped ring regions according to the GLOH algorithm using the SIFT feature descriptor method. Each feature point finally obtains a 136-dimensional feature point descriptor.

[0085] Step 4. Use the nearest neighbor distance ratio for the initial matching, set the ratio in the method to 0.9, and use the random sample consensus to fit to obtain the rough transformation matrix H1. Then, in combination with the initial transformation, adjust the distance between the descriptors based on the position and angle information of the feature points in the reference image pixel coordinate system, and perform the secondary matching to obtain the accurate perspective transformation matrix H2. The specific process is as follows:

[0086] Step 4.1. Initial matching: Use the nearest neighbor distance ratio for the initial matching: Use the nearest neighbor distance ratio to compare the Euclidean distances between the descriptors on the reference image and the image to be registered, select the matching pairs where the minimum distance is less than 0.9 times the second minimum distance, filter out the repeated matching pairs on the multi-layer structure image, and use the random sample consensus algorithm to filter out the wrong matching pairs and fit the rough transformation matrix H1;

[0087] Step 4.2. Secondary matching: Transform the feature point pairs on the image to be registered to the reference image pixel coordinate system through the transformation matrix H1, and calculate the position distance difference e p (p i , p′ j ) and the main direction difference e o (p i , p′ j ) between every feature point and all the feature points on the other image. The Euclidean distance between the two image descriptors becomes:

[0088] POED(p i , p′ j ) = (1 + e p (p i , p′ j ))(1 + e o (p i , p′ j ))ED(p i , p j )

[0089] where p i is a feature point on the reference image, p j is the corresponding point of p i in the image to be registered, p′ j is the point of p j after being transformed by H1, and ED(p i , p′ j)Representative point p i and p j The Euclidean distance between the descriptors. Then, the descriptors after Euclidean distance transformation are matched using the nearest neighbor distance ratio (NNDR) method to filter out duplicate matching point pairs, and the fast sample consensus (FSC) algorithm is used to fit the exact transformation matrix H2.

[0090] Step 5, Figure 5 Shows the correctly matched feature point pairs on the two-modal images obtained by this method, Figure 6 Shows the registered two-modal images displayed in the form of a checkerboard. It can be seen that the same geographical location of the two images can achieve pixel-level alignment.

[0091] In the synthetic aperture radar and optical image registration system based on structure extraction of the present invention, the structure extraction module can regard speckle noise as a texture feature and effectively erase the texture, avoiding the influence of noise on feature point extraction. The present invention extracts feature points after extracting the image structure, avoiding feature points falling into texture areas where features are not easily distinguishable, so that the feature points extracted on the synthetic aperture radar and optical images are mainly distributed on the more prominent structure edges, improving the spatial repetition rate of the positions of the feature points extracted from the two images and the uniqueness of the descriptors. In addition, this system does not downsample the image, and during structure extraction, the edges become finer and finer, which all provide precise positioning for feature points. When describing feature points, first construct enhanced phase consistency edges on each layer of the structure image to obtain clear, accurate, and noise-free edges, and then use the edge information to describe the feature points, which can overcome the non-linear gray value transformation between the synthetic aperture radar and optical images to obtain distinguishable and similar descriptors for feature points at the same geographical location. The entire system extracts feature points and performs feature description on the structure image where multi-layer texture information is gradually suppressed layer by layer. In this way, not only feature points on prominent structures are extracted, but also feature points on edges with weaker contrast can be extracted on the shallow structure image, making the number of extracted feature points larger and the distribution more uniform. The invention not only improves the robustness of synthetic aperture radar and optical image registration, but also improves the accuracy of image registration. It provides a simple and efficient synthetic aperture radar and optical image registration method, which has important application value.

Claims

1. A synthetic aperture radar and optical image registration system based on structure extraction, characterized in that Including: A structure extraction module connected to the input image end, which is used to smooth the texture information of the input image at multiple scales, extract structure information, refine edges, and output a structure image with multi-layer texture information smoothed layer by layer; Specifically, the structure extraction module is used to: perform multi-scale smoothing of the texture in the synthetic aperture radar image and the optical image by using the window relative total transformation iterative optimization algorithm. Each iterative optimization is performed on the image obtained from the previous iterative optimization, and each layer of the image is saved, and a structure image with 6 layers of texture information smoothed layer by layer is output; A feature point extraction module connected to the structure extraction module, which is used to extract feature points, the layer number where the feature points are located, the position, and their scale information in each layer of the structure image; The feature point extraction module includes: Multi-scale Harris function calculation unit, which is used to calculate the Harris function values of each layer of the multi-layer structure image output by the structure extraction module respectively with 6 decreasing Harris corner detection windows. The initial detection window has a scale of 6 pixels, and the window decreases with the increase of the number of layers of the structure image at a rate of decrease to obtain the multi-scale Harris function values; A feature point information acquisition unit connected to the multi-scale Harris function calculation unit, which is used to search for the maximum value in the neighborhood of the multi-scale Harris function value. The points that satisfy being greater than the threshold among the searched maximum values are the feature points, the layer number where the feature points are located, the position, and their scale information; A feature point description module connected to the feature point extraction module and the structure extraction module, which is used to describe the main direction of the feature points and construct feature point descriptors; The feature point description module includes: Construct an enhanced phase consistency edge unit. On each layer of the structure image, combine the Sobel edge with gradient intensity retained on the phase consistency edge to obtain an enhanced phase consistency edge; A descriptor construction unit connected to the enhanced phase consistency edge construction unit and the feature point extraction device: Combine the feature point position information, scale information, and direction information, and on the enhanced phase consistency edge, statistically analyze the distribution of the gradients and directions of the pixels in 17 sector ring regions of the circular neighborhood of the feature points to create a feature descriptor; A feature point matching module connected to the feature point extraction module and the feature point description module: Use the nearest neighbor distance ratio method and the random sample consensus algorithm to achieve rough registration; After combining the rough registration, perform fine registration based on the position and direction information of the feature points to filter out mis-matched points; A registration result display module connected to the feature point matching module and the input image end: Display the registration effect in a checkerboard format.

2. The synthetic aperture radar and optical image registration system based on structure extraction according to claim 1, wherein In the process of multi-scale smoothing of the texture in the synthetic aperture radar image and the optical image by using the window relative total transformation iterative optimization algorithm: The initial smoothing window variance scale of the synthetic aperture radar image is 4 pixels, the initial window scale of the optical image is 2 pixels, the scale window decreases by 0.5 times with the number of iterations, and the number of iterations is 6.

3. The synthetic aperture radar and optical image registration system based on structure extraction according to claim 1, wherein The feature point matching module includes: Coarse matching unit, which uses the distance ratio of the nearest neighbor method and the random sample consensus algorithm for coarse registration, is used to obtain the coarse registration transformation matrix ; The fine matching unit connected to the coarse matching unit adjusts the Euclidean distance between descriptors by combining the position information and direction information of the feature points, and uses the distance ratio of the second nearest neighbor method and the random sample consensus algorithm to filter out the wrong matching pairs, so as to obtain the fine registration perspective transformation matrix H 2 4. A method for registering synthetic aperture radar and optical images based on structure extraction, characterized in that Specifically implemented according to the following steps: Step 1: Use the relative total transformation structure extraction algorithm to extract the structures of the synthetic aperture radar and the optical image to obtain a multi-layer structure image with the texture information smoothed layer by layer; Step 2: On the multi-layer structure image, use a multi-scale Harris corner detection algorithm with a feature point detection window decreasing layer by layer with the structure image to extract the feature points of the synthetic aperture radar and the optical image, and obtain the layer number where the feature points are located, the position, and their scale information; Among them, the specific process of extracting the feature points of the synthetic aperture radar and the optical image is: On the multi-layer structure image, with an initial window scale of 6 pixels, the Harris function value of each layer of the image is calculated using a Harris corner detection window that decreases by times as the number of layers of the structure image increases; Search for the maximum value in the neighborhood of the multi-scale Harris function value. The points that satisfy the maximum value greater than the threshold are the feature points, and record the layer number, position, and scale information of the feature points; where the threshold is a user-defined threshold; Step 3: Construct enhanced phase consistency edges on each layer of the structural image, and use the edge features to construct descriptors. The specific process is as follows: Step 3.1: Calculate the phase congruency edge and the Sobel edge on each layer of the structural image. Perform an AND operation on the Sobel edge and the phase congruency edge at the corresponding pixel positions to obtain an intersection mask , and multiply the intersection mask by the corresponding elements of the Sobel edge map to perform a spatial domain filtering once. Take the larger value of the corresponding bits of the filtered Sobel edge and the phase congruency edge as the enhanced phase congruency edge map ; Step 3.2: On the enhanced phase consistency edges, statistically calculate the gradient direction histogram at the position of the feature points, assign the main direction to the feature points, and describe the features using the method of SIFT feature descriptors within a circular neighborhood that is positively correlated with the scale size of the feature points and is divided into 17 fan-shaped ring regions. Each feature point finally obtains a 136-dimensional feature point descriptor; Step 4: Perform an initial matching using the ratio of distances in the nearest neighbor method, and obtain a rough transformation matrix by fitting with Random Sample Consensus (RANSAC). Then, combined with the initial transformation, adjust the distance between descriptors based on the position and angle information of the feature points in the pixel coordinate system of the reference image, and perform a secondary matching to obtain an accurate perspective transformation matrix ; Step 5: Multiply the image to be registered by the transformation matrix Transform it to the pixel coordinate system where the reference image is located, and display the two modality images in the form of a checkerboard.

5. The synthetic aperture radar and optical image registration method based on structure extraction according to claim 4, wherein The specific process of Step 4 is as follows: Step 4.

1. Initial matching using the nearest neighbor distance ratio: Use the nearest neighbor distance ratio to compare the Euclidean distances between descriptors on the reference image and the image to be registered, select the feature point matching pairs in the two images, and after removing duplicates, use the random sample consensus algorithm to filter out the incorrect matching pairs and fit the rough transformation matrix ; Step 4.

2. Transform the feature point pairs on the image to be registered to the pixel coordinate system of the reference image through the transformation matrix Calculate the position distance difference between each feature point and all feature points on the other image and the main direction difference , and add 1 to the sum of these two differences as the magnification factor to magnify the Euclidean distance between the corresponding descriptors, and then perform the same steps as the initial matching once again to obtain the accurate perspective transformation matrix .

Images