Optical and SAR image registration method and system based on multi-scale phase antisymmetry

Abstract

The invention discloses an optical and SAR image registration method and system based on multi-scale phase antisymmetry. The method comprises the following steps: acquiring an optical and corresponding SAR image pair as an original image pair; calculating the phase antisymmetry of each scale by using a multi-scale single gene signal to obtain the multi-scale phase antisymmetry of the image, and performing feature point detection on the original image to obtain the feature point of each original image; constructing a multi-scale phase antisymmetry directional diagram by using the phase antisymmetry of each scale, and generating descriptors of the feature points; similarity measurement of descriptors between original images is utilized to carry out approximate matching on feature points of image pairs to obtain matched feature point pairs, and mismatched feature point pairs are eliminated through an FSC method to realize accurate registration of the images; and transforming the image by using a transformation matrix obtained by FSC screening to obtain a registered optical and SAR image pair. According to the method, the registration effect of low-quality optical and SAR images can be effectively improved, and rapid and robust registration is realized.

Description

Background Art

[0002] Multimodal image registration is a basic image analysis technique used to align images from different imaging sensors. Optical synthetic aperture radar (SAR) image registration is one of the typical multimodal image registration tasks and plays an important role in remote sensing applications such as image fusion and visual navigation. There have been a lot of studies on the registration problem of remote sensing images, which can be divided into traditional registration methods and deep learning registration methods. Traditional multi-source remote sensing image registration methods can be roughly divided into feature-based registration methods and region-based registration methods. Deep learning-based registration methods are divided into supervised learning registration methods and unsupervised learning registration methods.

[0003] Region-based matching methods use raw pixel values ​​and specific similarity metrics to find matching relationships between image pairs. Their disadvantages are that they are only applicable to image pairs containing simple changes and the image content within the search window may be relatively smooth and lack significant features. Region-based matching methods can be roughly divided into three subcategories, including correlation-based methods, Fourier-based methods, and mutual information-based methods.

[0004] Compared with region-based techniques, feature-based methods can successfully resolve scale differences and rotation differences by establishing reliable feature matching. Among feature-based methods, SIFT-based methods are the most widely used techniques because of their efficient performance and invariance to scaling, rotation, and illumination changes. However, in multimodal image registration, traditional SIFT methods and their improved methods perform poorly. The intensity or gradient information used in such methods is sensitive to nonlinear grayscale differences and radiation intensity differences between heterogeneous images, resulting in a large number of unreliable feature points being extracted.

[0005] In contrast, phase-consistency features have been shown to be more robust to illumination and contrast changes, a property that makes them insensitive to radiation variations. However, phase-consistency itself has inherent limitations: 1) It is meaningful only when the phase congruence point appears in a wide frequency range; 2) As a normalized measure, it is very sensitive to noise.

[0006] In optical and SAR image registration, phase consistency is not effective for edge characterization of SAR images because SAR images usually have low resolution and are affected by unique multiplicative noise. This results in a large number of repeatable feature points that cannot be detected during the matching process for correct matching, greatly reducing the accuracy and success rate of registration. Summary of the invention

[0007] The technical problem to be solved by the present invention is to provide an optical and SAR image registration method and system based on multi-scale phase antisymmetry in view of the deficiencies in the above-mentioned prior art, so as to realize registration under low resolution and strong noise interference, solve the technical problem of low-quality optical and SAR image registration, and improve the registration effect of low-quality optical and SAR images.

[0008] The present invention adopts the following technical solutions: An optical and SAR image registration method based on multi-scale phase antisymmetry comprises the following steps: Acquire optical and corresponding SAR image pairs as raw image pairs, and form multi-scale single gene signals of the raw image pairs; The multi-scale phase antisymmetry of the image is obtained by using the multi-scale single gene signal, and the feature points of the original image are detected by using the multi-scale phase antisymmetry to obtain the feature points of each original image; A multi-scale phase antisymmetry direction map is constructed using multi-scale phase antisymmetry to generate a descriptor for each feature point of the original image; The feature points of the image pairs are approximately matched using the similarity measurement of the descriptors between the feature points of the original images to obtain the matched feature point pairs, and the mismatched feature point pairs are eliminated using the FSC method; The image is transformed using the transformation matrix obtained by screening the FSC method to obtain the registered optical and SAR image pairs.

[0009] Preferably, the multi-scale single gene signal forming the original image pair is specifically: A set of logarithmic Gabor filters with different center frequencies are constructed and Rize transformed to obtain the single gene signal representation of the logarithmic Gabor filter [ G e ( m ) ,G o1 ( m ) ,G o2 ( m )] T ; The image is convolved with the even and two odd parts of the single gene signal of the logarithmic Gabor filter to obtain a multi-scale single gene signal representation of the image [ f e ( x,y ) ,f o1 ( x,y ) ,f o2 ( x,y )].

[0010] Preferably, the two-dimensional logarithmic Gabor filter Do Rize transformation and get:

[0011] in, is the even part of the single gene signal of the logarithmic Gabor filter, , The two odd parts of the single gene signal of the logarithmic Gabor filter are used to process the two directions of the two-dimensional image. is a two-dimensional frequency vector, and is the frequency vector Components in both directions.

[0012] Preferably, the feature points of the original image are detected by using multi-scale phase antisymmetry, and the feature points of each original image are obtained as follows: use[ f e ( x,y ) ,f o1 ( x,y ) ,f o2 ( x,y )], calculate the antisymmetry of the image at each scale PA λi ; Antisymmetry PA λi Find the average value and regard it as the amplitude of multi-scale phase antisymmetry ; The image The image is divided into m*n blocks, and Harris corner point detection is performed. The first g corner points in each block are sorted from large to small by Harris values ​​as feature points.

[0013] Preferably, the image is antisymmetric at all scales PA λi Specifically:

[0014] in, is the noise threshold, is the local amplitude at a certain scale, is a very small value, typically set to , and represent the odd and even parts of the logarithmic Gabor filter in scale The position below Output.

[0015] Preferably, the magnitude of the multi-scale phase antisymmetry for:

[0016] in, represents the total number of scales, i.e. the total number of filters used.

[0017] Preferably, the descriptor of each feature point of the original image is generated as follows: Defining the multiscale phase antisymmetry direction θ MPA is the direction of phase antisymmetry on multiple scales The multi-scale phase antisymmetry direction is divided into six channels, namely Corresponding channel value i , use these channel values ​​to replace the original intensity values ​​to form the image MPADM ; The descriptor of a feature point is composed of the MPADM The pixel values ​​are expanded into vectors [ v 1 ,v 2 ,...,v n ].

[0018] Preferably, the phase antisymmetry directions on multiple scales θ PAλi Specifically:

[0019] in, and Indicates that the two odd parts of the logarithmic Gabor filter are in scale The position below Response.

[0020] Preferably, the feature points of the image pairs are approximately matched using the similarity measure of the descriptors between the feature points of the original images to obtain matched feature point pairs, and the mismatched feature point pairs are eliminated using the FSC method. Specifically, the following steps are performed: Calculate the Euclidean distance between descriptors and use the Euclidean distance as the similarity measure of feature point pairs; Based on the nearest neighbor to next nearest neighbor distance ratio of the feature point pairs, the feature point pairs are screened to obtain feature point pairs whose distance is less than the registration threshold; The matching feature point pairs are sampled to estimate the transformation model h, and the transformation model H with the most feature point pairs that meet the error threshold E is found through iteration; the feature point pairs that meet the transformation model H are used as the final matching feature points.

[0021] In a second aspect, an embodiment of the present invention provides an optical and SAR image registration system based on multi-scale phase antisymmetry, comprising: A data module, which obtains optical and corresponding SAR image pairs as raw image pairs, and forms multi-scale single gene signals of the raw image pairs; The detection module uses the multi-scale single gene signal to obtain the multi-scale phase antisymmetry of the image, and uses the multi-scale phase antisymmetry to detect the feature points of the original image to obtain the feature points of each original image; The description module uses multi-scale phase antisymmetry to construct a multi-scale phase antisymmetry direction map and generates a descriptor for each feature point of the original image; The matching module uses the similarity measurement of the descriptors between the feature points of the original images to approximately match the feature points of the image pairs, obtains the matched feature point pairs, and removes the mismatched feature point pairs through the FSC method; The screening module transforms the image using the transformation matrix screened by the FSC method to obtain the registered optical and SAR image pairs.

[0022] In a third aspect, a computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the above-mentioned optical and SAR image registration method based on multi-scale phase antisymmetry are implemented.

[0023] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, comprising a computer program, which, when executed by a processor, implements the steps of the above-mentioned optical and SAR image registration method based on multi-scale phase antisymmetry.

[0024] In a fifth aspect, a chip comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned optical and SAR image registration method based on multi-scale phase antisymmetry when executing the computer program.

[0025] In a sixth aspect, an embodiment of the present invention provides an electronic device, including a computer program, which, when executed by the electronic device, implements the steps of the above-mentioned optical and SAR image registration method based on multi-scale phase antisymmetry.

[0026] Compared with the prior art, the present invention has at least the following beneficial effects: An optical and SAR image registration method based on multi-scale phase antisymmetry introduces phase antisymmetry into the registration of optical and SAR images, overcoming the problem in the prior art that the nonlinear intensity difference between the optical image and the SAR image caused by feature point detection using gradient information affects the registration, and the phase consistency in the optical and SAR image registration performs poorly due to the interference of low resolution of SAR images and multiplicative speckle noise; the registration effect of low-quality optical and SAR images is greatly improved.

[0027] Furthermore, the frequency information of each scale of the image can be extracted through multi-scale single gene signals. This comprehensive frequency information contains rich features of the image, allowing the subsequent detection of sufficient feature points that are widely distributed and can provide matching information.

[0028] Furthermore, the Rize transform is used to process the two-dimensional image to obtain the single gene signal of the image, thereby calculating the local amplitude and local direction of the image. This avoids the problem of high computational complexity caused by extending the analytical signal analysis of one-dimensional signals combined with directional variables to the analysis of two-dimensional images in the prior art.

[0029] Furthermore, multiscale phase antisymmetry is the result of phase antisymmetry at all scales of the integrated image, which is robust to multiplicative speckle noise and resolution. It contains the structural information of the image and is not affected by the nonlinear radiation differences between heterogeneous images. Corner detection based on this metric can obtain feature points with high repetition rate on the optical and SAR image pairs to be registered, which helps to find the correct correspondence through matching and screening.

[0030] Furthermore, the multi-scale phase antisymmetry direction is redefined and the multi-scale phase antisymmetry direction map is constructed. MPADM This method enhances the information content of the feature point descriptor and avoids the situation where the original multi-scale phase antisymmetry direction is used to lose information in the region with weak phase antisymmetry direction. This descriptor improves the correct matching rate of feature points and thus improves the success rate of the image pairs to be registered.

[0031] Furthermore, because there are similar structures around the feature points at the same position in the image pair to be registered, the similarity measure of the descriptor describes the similarity in this structure, and a preliminary correspondence can be obtained. Feature points often lead to mismatches due to factors such as scale, angle, and noise. The FSC method can effectively remove these mismatches, further improve the reliability and accuracy of image matching, and obtain an effective transformation model.

[0032] It can be understood that the beneficial effects of the second aspect mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0033] In summary, the present invention can effectively improve the registration effect of low-quality optical and SAR images and achieve fast and robust registration.

[0034] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flow chart for realizing the present invention; Figure 2 is a registration result diagram of the first pair of test images in the simulation experiment of the present invention; Figure 3 is a registration result diagram of the second pair of test images in the simulation experiment of the present invention; Figure 4 is a registration result diagram of the third pair of test images in the simulation experiment of the present invention; Figure 5 A schematic diagram of a computer device provided by an embodiment of the present invention; Figure 6 The block diagram of a chip provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] In the description of the present invention, it should be understood that the terms “include” and “comprises” indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0038] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0039] It should be further understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects are in an "or" relationship.

[0040] It should be understood that, although the terms first, second, third, etc. may be used to describe preset ranges, etc. in the embodiments of the present invention, these preset ranges should not be limited to these terms. These terms are only used to distinguish preset ranges from each other. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0041] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0042] Various structural schematic diagrams of the embodiments disclosed in the present invention are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may additionally design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0043] The present invention provides an optical and SAR image registration method based on multi-scale phase antisymmetry, forming a multi-scale monogenic signal of an optical and corresponding SAR image pair; using the multi-scale monogenic signal to calculate the multi-scale phase antisymmetry of the image, then performing feature point detection on the original image to obtain the feature points of each original image; using the phase antisymmetry of each scale to construct a multi-scale phase antisymmetry direction map (Multi-scale Phase Asymmetry Direction Map, MPADM), generating a descriptor of the feature point; using the similarity measure of the descriptors between the original images, approximate matching of the feature points of the image pair is performed to obtain a matched feature point pair, and the mismatched feature point pair is eliminated by the FSC method to achieve accurate image registration; finally, the image is transformed using the transformation matrix obtained by FSC screening to obtain the registered optical and SAR image pair. The present invention can effectively improve the registration effect of low-quality optical and SAR images and achieve fast and robust registration.

[0044] The present invention provides an optical and SAR image registration method based on multi-scale phase antisymmetry, comprising the following steps: S1, obtaining an optical image pair and a corresponding SAR image pair as an original image pair; S2, forming a multi-scale single gene signal representation of the original image pair; S201, construct a set of logarithmic Gabor filters with different center frequencies, and perform Rize transformation on them to obtain the single gene signal representation of the logarithmic Gabor filter [ G e ( m ) ,G o1 ( m ) ,G o2 ( m )] T ; The dual part of the logarithmic Gabor filter is specifically:

[0045] in, ω 0 is the center frequency of the filter, k is a parameter that adjusts the filter bandwidth. The filter bandwidth is determined by the ratio σ 0= k / ω 0 decision.

[0046] In signal processing, Rize transform is usually used to analyze the phase and amplitude information of the signal, which is a multi-dimensional extension of Hibert transform. In the two-dimensional frequency domain, the Rize transform calculation formula is defined as:

[0047] Perform Rize transform on the even part of the two-dimensional logarithmic Gabor filter and get:

[0048] S202, convolve the image with the even and two odd parts of the single gene signal of the logarithmic Gabor filter, respectively, to obtain a multi-scale single gene signal representation of the image [ f e ( x,y ) ,f o1 ( x,y ) ,f o2 ( x,y )].

[0049] S3, using the single gene signal of step S2 to obtain the multi-scale phase antisymmetry of the image, and using the multi-scale phase antisymmetry to perform feature point detection on the original image to obtain feature points of each original image; S301, use [ f e ( x,y ) ,f o1 ( x,y ) ,f o2 ( x,y )], calculate the antisymmetry of the image at each scale PA λi ; The phase antisymmetry on a single scale is:

[0050] in, A λi ( x,y ) represents the local amplitude at a certain scale; T represents the noise threshold, below which the value is regarded as noise, and the phase antisymmetric is zero to remove the influence of noise; epsilon Introduce a very small value to avoid division by zero.

[0051] S302, Antisymmetry PA λi Find the average value and regard it as the multi-scale phase antisymmetric amplitude MPA; Specifically:

[0052] S303, dividing the MPA map of the image into m*n blocks, performing Harris corner point detection, and taking the first g corner points in descending order of Harris values ​​on each block as feature points.

[0053] Pick m = n =5, g =40.

[0054] S4, constructing a multi-scale phase asymmetry direction map (Multi-scale Phase Asymmetry Direction Map, MPADM) using the multi-scale phase asymmetry obtained in step S3, thereby generating a descriptor of the feature point; S401. Define the multi-scale phase antisymmetry direction θ MPA is the direction of phase antisymmetry on multiple scales θ PAλi The superposition of Phase antisymmetry directions on multiple scales θ PAλi Specifically:

[0055] S402, divide the multi-scale phase antisymmetry direction into six channels, namely [30 ◦ ∗ i, 30 ◦ ∗( i +1)] corresponding channel value i , use these channel values ​​to replace the original intensity values ​​to form the image MPADM ; S403, the descriptor of the feature point is composed of the MPADM The pixel value is expanded into a vector representation as [ v 1 ,v 2 ,...,v n ].

[0056] S5. Using the similarity measurement of the descriptors between the original images, the feature points of the image pairs are approximately matched to obtain matching feature point pairs, and the mismatched feature point pairs are eliminated by the FSC (Fast Sample Consensus) method to achieve accurate image registration; S501, calculating the Euclidean distance between descriptors, and using the Euclidean distance as a similarity measure of feature point pairs; S502, based on the nearest neighbor to next nearest neighbor distance ratio of the feature point pairs, the feature point pairs are screened to obtain feature point pairs whose distance is less than the registration threshold; Let the correct registration threshold thresh = 0.95. When the distance ratio is less than the threshold, the initial matching feature point pair can be obtained.

[0057] S503, sampling the matched feature point pairs to estimate the transformation model h, and finding the transformation model H with the most feature point pairs that meet the error threshold E through iteration; the feature point pairs that meet the transformation model are called the final matched feature points.

[0058] The judgment formula for whether the feature point conforms to the transformation model is:

[0059] in, are the two matching feature points in the image pair to be registered. The formula shows that after the transformation matrix h, the Euclidean distance between the two matching points is small enough.

[0060] S6. Use the transformation matrix obtained by FSC screening to transform the image to obtain the registered optical and SAR image pair.

[0061] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as systems, methods or program products. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "platforms".

[0062] In yet another embodiment of the present invention, a multi-scale phase antisymmetry-based optical and SAR image registration system is provided. The system can be used to implement the above-mentioned multi-scale phase antisymmetry-based optical and SAR image registration method. Specifically, the multi-scale phase antisymmetry-based optical and SAR image registration system includes a data module, a detection module, a description module, a matching module and a screening module.

[0063] Among them, the data module obtains the optical and corresponding SAR image pairs as the original image pairs, and forms the multi-scale single gene signals of the original image pairs; The detection module uses the multi-scale single gene signal to obtain the multi-scale phase antisymmetry of the image, and uses the multi-scale phase antisymmetry to detect the feature points of the original image to obtain the feature points of each original image; The description module uses multi-scale phase antisymmetry to construct a multi-scale phase antisymmetry direction map and generates a descriptor for each feature point of the original image; The matching module uses the similarity measurement of the descriptors between the feature points of the original images to approximately match the feature points of the image pairs, obtains the matched feature point pairs, and removes the mismatched feature point pairs through the FSC method; The screening module transforms the image using the transformation matrix screened by the FSC method to obtain the registered optical and SAR image pairs.

[0064] In another embodiment of the present invention, a terminal device is provided, the terminal device includes a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of the optical and SAR image registration method based on multi-scale phase antisymmetry, including: Acquire an optical and corresponding SAR image pair as an original image pair; form a multi-scale single gene signal of the original image pair; use the multi-scale single gene signal to obtain the multi-scale phase antisymmetry of the image, use the multi-scale phase antisymmetry to detect feature points of the original image, and obtain the feature points of each original image; use the multi-scale phase antisymmetry to construct a multi-scale phase antisymmetry direction map, and generate a descriptor of each original image feature point; use the similarity measure of the descriptors between the original image feature points to approximately match the feature points of the image pair to obtain matched feature point pairs, and use the FSC method to eliminate mismatched feature point pairs; transform the image using the transformation matrix screened by the FSC method to obtain the registered optical and SAR image pairs.

[0065] See also Figure 5 , the terminal device is a computer device, and the computer device 60 of this embodiment includes: a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. When the computer program 63 is executed by the processor 61, the optical and SAR image registration method based on multi-scale phase antisymmetry in the embodiment is implemented. To avoid repetition, it is not described one by one here. Alternatively, when the computer program 63 is executed by the processor 61, the functions of each model / unit in the optical and SAR image registration system based on multi-scale phase antisymmetry in the embodiment are implemented. To avoid repetition, it is not described one by one here.

[0066] The computer device 60 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The computer device 60 may include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art will appreciate that Figure 5 This is only an example of the computer device 60 and does not constitute a limitation of the computer device 60. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the computer device may also include input and output devices, network access devices, buses, etc.

[0067] The processor 61 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0068] The memory 62 may be an internal storage unit of the computer device 60, such as a hard disk or memory of the computer device 60. The memory 62 may also be an external storage device of the computer device 60, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the computer device 60.

[0069] Furthermore, the memory 62 may include both an internal storage unit of the computer device 60 and an external storage device. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 may also be used to temporarily store data that has been output or is to be output.

[0070] See also Figure 6 , the terminal device is a chip, and the chip 600 of this embodiment includes a processor 622, which may be one or more, and a memory 632 for storing a computer program executable by the processor 622. The computer program stored in the memory 632 may include one or more modules, each corresponding to a set of instructions. In addition, the processor 622 may be configured to execute the computer program to perform the above-mentioned optical and SAR image registration method based on multi-scale phase antisymmetry.

[0071] In addition, the chip 600 may further include a power supply component 626 and a communication component 650, wherein the power supply component 626 may be configured to perform power management of the chip 600, and the communication component 650 may be configured to implement communication, for example, wired or wireless communication, of the chip 600. In addition, the chip 600 may further include an input / output interface 658. The chip 600 may operate based on an operating system stored in the memory 632.

[0072] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium, which is a memory device in a terminal device for storing programs and data. It can be understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and the extended storage medium supported by the terminal device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs. It should be noted that the computer-readable storage medium here can be a high-speed RAM memory, or a non-volatile memory (Non-VolatileMemory), such as at least one disk memory.

[0073] One or more instructions stored in a computer-readable storage medium may be loaded and executed by a processor to implement the corresponding steps of the optical and SAR image registration method based on multi-scale phase antisymmetry in the above embodiment; one or more instructions in the computer-readable storage medium are loaded and executed by the processor as follows: Acquire an optical and corresponding SAR image pair as an original image pair; form a multi-scale single gene signal of the original image pair; use the multi-scale single gene signal to obtain the multi-scale phase antisymmetry of the image, use the multi-scale phase antisymmetry to detect feature points of the original image, and obtain the feature points of each original image; use the multi-scale phase antisymmetry to construct a multi-scale phase antisymmetry direction map, and generate a descriptor of each original image feature point; use the similarity measure of the descriptors between the original image feature points to approximately match the feature points of the image pair to obtain matched feature point pairs, and use the FSC method to eliminate mismatched feature point pairs; transform the image using the transformation matrix screened by the FSC method to obtain the registered optical and SAR image pairs.

[0074] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings here can usually be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0075] The technical effects of the present invention are further described below in conjunction with simulation experiments.

[0076] 1. Simulation experiment conditions: The hardware platform of the simulation experiment of the present invention is: The processor is Intel(R) Core(TM) i5-8400 CPU with a main frequency of 2.80GHz and a memory of 16.0 GB.

[0077] The software platform of the simulation experiment of the present invention is: Windows 10 operating system and matlab 2023a.

[0078] 2. Simulation content and results analysis: The registration results of the three methods are evaluated using three evaluation indicators (number of correctly matched feature points, root mean square error and overall running time) for the above three images.

[0079] The correctly matched feature points refer to the points whose Euclidean distance between the transformed and original matched feature points is not greater than the threshold T; The root mean square error is an indicator used to evaluate the accuracy of the image registration method. If there are too few correct matching points, the matching is considered to have failed.

[0080] In the experiment, the number of feature points detected by the three methods on each image is 4000, and the threshold for correct matching is T=2. N correct < 10 is considered a match failure.

[0081] The experimental results are as follows:

[0082] The parts marked with ∗ in the table show that the registration of this method failed. The multi-scale phase antisymmetry method not only successfully matches all three groups of images, but also has a smaller root mean square error, better registration effect, and shorter running time, which verifies the effectiveness of the method of the present invention.

[0083] The simulation experiment of the present invention uses the present invention and two prior art methods, os-sift and Rift, to respectively register three pairs of input test images to obtain a registration result graph.

[0084]

[0085] Compared with the prior art, the present invention has the following advantages: Figure 2 , Figure 3 , Figure 4 This is a chessboard registration result diagram obtained for three pairs of test images in the present invention. It can be seen from the figure that the overlapping areas of the optical and SAR image pairs to be registered in the present invention can be accurately overlapped, and there is almost no misalignment in the results, which well demonstrates that the registration accuracy of the method of the present invention is high.

[0086] In summary, the present invention is an optical and SAR image registration method and system based on multi-scale phase antisymmetry, which can realize the registration of low-resolution SAR images under optical and strong noise interference without being affected by noise and resolution, and has the advantages of low computational complexity and high registration accuracy.

[0087] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0088] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0089] Those skilled in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0090] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0091] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0092] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0093] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0094] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0095] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0096] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0097] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A method for optical and SAR image registration based on multi-scale phase antisymmetry, characterized in that: The following steps are involved: Acquire optical and corresponding SAR image pairs as raw image pairs, and form multi-scale single gene signals of the raw image pairs; The multi-scale phase antisymmetry of the image is obtained by using the multi-scale single gene signal, and the feature points of the original image are detected by using the multi-scale phase antisymmetry to obtain the feature points of each original image; A multi-scale phase antisymmetry direction map is constructed using multi-scale phase antisymmetry to generate a descriptor for each feature point of the original image; The feature points of the image pairs are approximately matched using the similarity measurement of the descriptors between the feature points of the original images to obtain the matched feature point pairs, and the mismatched feature point pairs are eliminated using the FSC method; The image is transformed using the transformation matrix obtained by screening the FSC method to obtain the registered optical and SAR image pairs.

2. The optical and SAR image registration method based on multi-scale phase antisymmetry according to claim 1, characterized in that: The multi-scale single gene signals that form the original image pair are specifically: A set of logarithmic Gabor filters with different center frequencies are constructed and Rize transformed to obtain the single gene signal representation of the logarithmic Gabor filter [ G e ( m ) ,G o1 ( m ) ,G o2 ( m )] T ; The image is convolved with the even and two odd parts of the single gene signal of the logarithmic Gabor filter to obtain a multi-scale single gene signal representation of the image [ f e ( x,y ) ,f o1 ( x,y ) ,f o2 ( x,y )].

3. The optical and SAR image registration method based on multi-scale phase antisymmetry according to claim 2, characterized in that: Two-dimensional logarithmic Gabor filter Do Rize transformation and get: in, is the even part of the single gene signal of the logarithmic Gabor filter, , are the two odd parts of the single gene signal of the logarithmic Gabor filter, which is used to process the two directions of the two-dimensional image; is a two-dimensional frequency vector, and is the frequency vector Components in both directions.

4. The optical and SAR image registration method based on multi-scale phase antisymmetry according to claim 1, characterized in that: The feature points of the original image are detected using multi-scale phase antisymmetry, and the feature points of each original image are obtained as follows: use[ f e ( x,y ) ,f o1 ( x,y ) ,f o2 ( x,y )], calculate the antisymmetry of the image at each scale PA λi ; Antisymmetry PA λi Find the average value and regard it as the amplitude of multi-scale phase antisymmetry ; The image The image is divided into m*n blocks, and Harris corner point detection is performed. The first g corner points in each block are sorted from large to small by Harris values ​​as feature points.

5. The optical and SAR image registration method based on multi-scale phase antisymmetry according to claim 4, characterized in that: Antisymmetry of images at all scales PA λi Specifically: in, is the noise threshold, is the local amplitude at a certain scale, is a very small value, typically set to , and represent the odd and even parts of the logarithmic Gabor filter in scale The position below Output.

6. The optical and SAR image registration method based on multi-scale phase antisymmetry according to claim 4, characterized in that: Magnitude of multiscale phase antisymmetry for: in, represents the total number of scales, i.e. the total number of filters used.

7. The optical and SAR image registration method based on multi-scale phase antisymmetry according to claim 1, characterized in that: The descriptor for each feature point of the original image is generated as follows: Defining the multiscale phase antisymmetry direction θ MPA is the direction of phase antisymmetry on multiple scales The multi-scale phase antisymmetry direction is divided into six channels, namely Corresponding channel value i , use these channel values ​​to replace the original intensity values ​​to form the image MPADM ; The descriptor of a feature point is composed of the MPADM The pixel values ​​are expanded into vectors [ v 1 ,v 2 ,...,v n ].

8. The optical and SAR image registration method based on multi-scale phase antisymmetry according to claim 7, characterized in that: Phase antisymmetry directions on multiple scales θ PAλi Specifically: in, and Indicates that the two odd parts of the logarithmic Gabor filter are in scale The position below Response.

9. The optical and SAR image registration method based on multi-scale phase antisymmetry according to claim 1, characterized in that: The similarity measurement of the descriptors between the feature points of the original images is used to approximate the feature points of the image pairs to obtain matching feature point pairs, and the mismatched feature point pairs are eliminated by the FSC method. Specifically: Calculate the Euclidean distance between descriptors and use the Euclidean distance as the similarity measure of feature point pairs; Based on the nearest neighbor to next nearest neighbor distance ratio of the feature point pairs, the feature point pairs are screened to obtain feature point pairs whose distance is less than the registration threshold; The matching feature point pairs are sampled to estimate the transformation model h, and the transformation model H with the most feature point pairs that meet the error threshold E is found through iteration; the feature point pairs that meet the transformation model H are used as the final matching feature points.

10. An optical and SAR image registration system based on multi-scale phase antisymmetry, characterized in that: include: A data module, which obtains optical and corresponding SAR image pairs as raw image pairs, and forms multi-scale single gene signals of the raw image pairs; The detection module uses the multi-scale single gene signal to obtain the multi-scale phase antisymmetry of the image, and uses the multi-scale phase antisymmetry to detect the feature points of the original image to obtain the feature points of each original image; The description module uses multi-scale phase antisymmetry to construct a multi-scale phase antisymmetry direction map and generates a descriptor for each feature point of the original image; The matching module uses the similarity measurement of the descriptors between the feature points of the original images to approximately match the feature points of the image pairs, obtains the matched feature point pairs, and removes the mismatched feature point pairs through the FSC method; The screening module transforms the image using the transformation matrix screened by the FSC method to obtain the registered optical and SAR image pairs.

Images