Optical and sar image registration method and system based on multi-scale phase antisymmetry

Abstract

The application discloses an optical and SAR image registration method and system based on multi-scale phase antisymmetry, acquires optical and corresponding SAR image pairs as original image pairs; uses multi-scale single gene signals to calculate phase antisymmetry of each scale, obtains multi-scale phase antisymmetry of the image, detects feature points of the original image to obtain feature points of each original image; constructs a multi-scale phase antisymmetry direction map by using phase antisymmetry of each scale to generate a descriptor of the feature points; uses similarity measurement of the descriptors between the original images to approximately match feature points of the image pairs to obtain matched feature point pairs, and removes false matching feature point pairs by using an FSC method to realize accurate image registration; transforms the image by using a transformation matrix obtained by using the FSC screening to obtain the registered optical and SAR image pairs. The application can effectively improve registration effect of low-quality optical and SAR images and realizes fast and robust registration.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to an optical and SAR image registration method and system based on multi-scale phase antisymmetry. BACKGROUND

[0002] Multi-modal image registration, as a basic image analysis technique, is used to align images from different imaging sensors. Optical synthetic aperture radar (SAR) image registration is one of the typical multi-modal image registration tasks, and plays an important role in remote sensing applications such as image fusion and visual navigation. There has been a large amount of research on the registration problem of remote sensing images, which can be divided into traditional registration methods and deep learning registration methods. Traditional multi-source remote sensing image registration methods can be roughly divided into feature-based registration methods and region-based registration methods. Deep learning-based registration methods can be divided into supervised learning registration methods and unsupervised learning registration methods.

[0003] Region-based matching methods use raw pixel values and specific similarity measures to find matching relationships between image pairs. The disadvantage is that it is only suitable for image pairs containing simple changes and the image content within the search window can be relatively smooth, lacking significant features. Region-based matching methods can be roughly divided into three subcategories, including correlation-based methods, Fourier-based methods, and mutual information-based methods.

[0004] Compared with region-based techniques, feature-based methods can successfully address scale and rotation differences by establishing reliable feature matches. In feature-based methods, SIFT-like methods are the most widely used techniques because of their efficient performance and invariance to scaling, rotation, and illumination changes. However, in multi-modal image registration, traditional SIFT methods and their improved methods perform poorly. The intensity or gradient information used in such methods is sensitive to nonlinear gray differences and radiometric intensity differences between heterogeneous images, resulting in a large number of unreliable feature points being extracted.

[0005] In contrast, phase-consistent features have been shown to be more robust to illumination and contrast changes, a property that makes them insensitive to radiometric variations. However, phase consistency has inherent limitations:

[0006] 1) It only makes sense when phase congruency points appear over a wide frequency range;

[0007] 2) As a normalized measure, it is very sensitive to noise.

[0008] In optical and SAR image registration, due to the low resolution and the influence of the unique multiplicative speckle noise of the SAR image, the phase consistency is not good for the edge description of the SAR image. This results in that a large number of repeatable feature points cannot be detected in the matching process to perform correct matching, and the accuracy and success rate of registration are greatly reduced. SUMMARY

[0009] The technical problem to be solved by the present application is to provide an optical and SAR image registration method and system based on multi-scale phase antisymmetry to solve the technical problem of low-quality optical and SAR image registration and improve the registration effect of low-quality optical and SAR images.

[0010] The present application adopts the following technical scheme:

[0011] A method for registering optical and SAR images based on multi-scale phase antisymmetry, comprising the following steps:

[0012] Obtaining an optical and a corresponding SAR image pair as a raw image pair, and forming a multi-scale single gene signal of the raw image pair;

[0013] Obtaining the multi-scale phase antisymmetry of the image using the multi-scale single gene signal, detecting feature points of the raw image using the multi-scale phase antisymmetry, and obtaining feature points of each raw image;

[0014] Constructing a multi-scale phase antisymmetry direction map using the multi-scale phase antisymmetry, and generating a descriptor of each raw image feature point;

[0015] Approximately matching the feature points of the image pair using the similarity measurement of the descriptors between the raw image feature points, obtaining a matched feature point pair, and removing the false matching feature point pair through the FSC method;

[0016] Transforming the image using the transformation matrix obtained through the FSC method to obtain a registered optical and SAR image pair.

[0017] Preferably, the multi-scale single gene signal of the raw image pair is specifically:

[0018] A set of log Gabor filters with different center frequencies is constructed, and a Rize transform is performed on the log Gabor filters to obtain a single gene signal representation of the log Gabor filter G e ( m ) ,G o1 ( m ) ,G o2 ​m )] T ;

[0019] The image is convolved with the even-numbered and two odd-numbered components of the single-gene signal obtained by the logarithmic Gabor filter to obtain a multi-scale single-gene signal representation of the image. f e ( x, y ) ,f o1 ( x, y ) ,f o2 ( x, y )).

[0020] Preferably, for a two-dimensional logarithmic Gabor filter Performing the Rize transformation, we get:

[0021]

[0022] in, For the even-numbered portion of the single-gene signal in the logarithmic Gabor filter, , These are the two odd-numbered parts of a single gene signal from a logarithmic Gabor filter, used to process two directions of a two-dimensional image. It is a two-dimensional frequency vector. and Frequency vector Components in two directions.

[0023] Preferably, feature point detection is performed on the original image using multi-scale phase antisymmetry, and the specific feature points of each original image are as follows:

[0024] use[ f e ( x, y ) ,f o1 ( x, y ) ,f o2 ( x, y The antisymmetry of the image at various scales is calculated. PA λi ;

[0025] Antisymmetry PA λi The average value is considered as the amplitude of a multi-scale phase antisymmetric structure. ;

[0026] Put the image The graph is divided into m*n blocks, and Harris corner detection is performed. The top g corner points with the largest Harris values ​​in each block are selected as feature points.

[0027] Preferably, the antisymmetry in each scale of the image PA λi Specifically, the antisymmetry in each scale of the image is defined as:

[0028]

[0029] wherein, is the noise threshold, is the local amplitude in a certain scale, is a very small value, typically set as , and represent the outputs of the odd and even parts of the log Gabor filter respectively at the position of the scale .

[0030] Preferably, the amplitude of the multiscale phase antisymmetry is:

[0031]

[0032] wherein, is the total number of scales, i.e. the total number of filters used.

[0033] Preferably, the descriptor of each original image feature point is generated specifically as:

[0034] The multiscale phase antisymmetry direction is defined as: θ MPA is the superposition of the phase antisymmetry directions in multiple scales; the multiscale phase antisymmetry direction is divided into six channels, i.e. corresponding to the channel values , the original intensity values are replaced with these channel values to form the i ; the descriptor of the feature point is composed of the blocks within the range around the feature point, and the pixel values thereof are unfolded into a vector MPADM 1 MPADM 2 v ,v ,...,v n ].

[0035] Preferably, the phase antisymmetry directions in multiple scales are: θ PAλi Specifically, the phase antisymmetry directions in multiple scales are:

[0036]

[0037] wherein, and represent the outputs of the two odd parts of the log Gabor filter respectively at the position ​​​the position of the user's head in response.

[0038] Preferably, the feature points of the image pair are approximately matched by using the similarity measure of the descriptors between the feature points of the original images to obtain matched feature point pairs, and the false matched feature point pairs are removed by the FSC method, specifically:

[0039] The Euclidean distance between the descriptors is calculated, and the Euclidean distance is taken as the similarity measure of the feature point pairs.

[0040] Based on the nearest neighbor distance ratio of the feature point pairs, the feature point pairs are screened to obtain the feature point pairs with a distance less than a registration threshold.

[0041] The matched feature point pairs are sampled to estimate the transformation model h, and the transformation model H with the most feature point pairs meeting the error threshold E is found through iteration; the feature point pairs meeting the transformation model H are taken as the final matched feature points.

[0042] In a second aspect, the embodiments of the present application provide an optical and SAR image registration system based on multi-scale phase antisymmetry, comprising:

[0043] A data module acquires an optical and a corresponding SAR image pair as an original image pair, and forms a multi-scale single gene signal of the original image pair.

[0044] A detection module uses the multi-scale single gene signal to obtain the multi-scale phase antisymmetry of the images, and detects feature points of the original images by using the multi-scale phase antisymmetry to obtain the feature points of each original image.

[0045] A description module constructs a multi-scale phase antisymmetry orientation map by using the multi-scale phase antisymmetry, and generates descriptors of the feature points of each original image.

[0046] A matching module approximately matches the feature points of the image pair by using the similarity measure of the descriptors between the feature points of the original images to obtain matched feature point pairs, and removes the false matched feature point pairs by the FSC method.

[0047] A screening module transforms the images by using the transformation matrix obtained by the FSC method to obtain the registered optical and SAR image pair.

[0048] In a third aspect, a computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the above-mentioned optical and SAR image registration method based on multi-scale phase antisymmetry when executing the computer program.

[0049] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium comprising a computer program which, when executed by a processor, implements the steps of the optical and SAR image registration method based on multi-scale phase antisymmetry described above.

[0050] In a fifth aspect, a chip comprises a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the steps of the optical and SAR image registration method based on multi-scale phase antisymmetry described above when executing the computer program.

[0051] In a sixth aspect, an embodiment of the present application provides an electronic device comprising a computer program which, when executed by the electronic device, implements the steps of the optical and SAR image registration method based on multi-scale phase antisymmetry described above.

[0052] Compared with the prior art, the present application has at least the following beneficial effects:

[0053] An optical and SAR image registration method based on multi-scale phase antisymmetry introduces phase antisymmetry into the registration of optical and SAR images, overcomes the influence of nonlinear intensity differences between optical images and SAR images on registration caused by feature point detection based on gradient information in the prior art, and the poor performance of phase consistency in optical and SAR image registration due to the interference of SAR image low resolution and multiplicative speckle noise, and greatly improves the registration effect of low-quality optical and SAR images.

[0054] Further, the multi-scale monogenic signal can extract frequency information at each scale of the image. This comprehensive frequency information contains rich features of the image, so that sufficient feature points widely distributed can be detected and matching information can be provided subsequently.

[0055] Further, the Rize transform is used to process the two-dimensional image to obtain the monogenic signal of the image, so as to calculate the local amplitude and local direction of the image and the like. This avoids the problem of high computational complexity caused by expanding the analysis of the analytic signal of the one-dimensional signal to the analysis of two-dimensional images in combination with the direction variable in the prior art.

[0056] Further, the multi-scale phase antisymmetry is the result of integrating the phase antisymmetry at each scale of the image, and is robust to multiplicative speckle noise and resolution. It contains the structural information of the image and is not affected by the nonlinear radiation differences between heterogeneous images. Angle point detection based on this metric can obtain feature points with high repetition rate on the optical and SAR image pairs to be registered, which helps to find the correct correspondence relationship through matching and screening.

[0057] Further, the multi-scale phase antisymmetry direction is redefined, and a multi-scale phase antisymmetry direction graph is constructed MPADM The method enhances the information amount of the feature point descriptor, and avoids the loss of information in a region with weak phase antisymmetry direction using the original multi-scale phase antisymmetry direction. The descriptor improves the correct matching rate of the feature point, thereby improving the success rate of the image pair to be registered.

[0058] Further, because the surrounding structures of the feature points at the same position on the image pair to be registered are similar, the similarity measurement of the descriptor describes the similarity of the structures, and a preliminary corresponding relationship can be obtained. The feature points are often mismatched due to factors such as scale, angle and noise. The FSC method can effectively remove the mismatching, further improve the reliability and accuracy of image matching, and obtain an effective transformation model.

[0059] It can be understood that the beneficial effects of the second aspect can be referred to the related description in the first aspect, and will not be described here.

[0060] In summary, the present application can effectively improve the registration effect of low-quality optical and SAR images, and realize fast and robust registration.

[0061] The technical solutions of the present application will be further described in detail below with reference to the drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0062] Figure 1 The flowchart is for the implementation of the present application.

[0063] Figure 2 The registration result graph of the first pair of test images in the simulation experiment of the present application is shown.

[0064] Figure 3 The registration result graph of the second pair of test images in the simulation experiment of the present application is shown.

[0065] Figure 4 The registration result graph of the third pair of test images in the simulation experiment of the present application is shown.

[0066] Figure 5 The schematic diagram of the computer device provided by an embodiment of the present application is shown.

[0067] Figure 6 The block diagram of a chip provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0068] Clearly, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments of the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the scope of protection of the present application.

[0069] In the description of the present application, it should be understood that the terms "comprising" and "including" indicate the presence of described features, integers, steps, operations, elements, and / or components, but do not exclude one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0070] It should also be understood that the terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and the appended claims of the present application, unless otherwise clearly indicated by the context, the singular forms "a", "an" and "the" are intended to include the plural forms as well.

[0071] It should be further understood that the term "and / or" used in the specification and the appended claims of the present application means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations, for example, A and / or B can represent three cases of A alone, A and B together, and B alone. In addition, the character " / " in the present application generally represents an "or" relationship between the front and rear associated objects.

[0072] It should be understood that although the terms first, second, third, etc. may be used in the embodiments of the present application to describe preset ranges, etc., these preset ranges should not be limited to these terms. These terms are only used to distinguish the preset ranges from each other. For example, the first preset range can also be referred to as the second preset range, and similarly, the second preset range can also be referred to as the first preset range, without departing from the scope of the embodiments of the present application.

[0073] Depending on the context, the word "if" as used herein can be interpreted as meaning "when" or "while" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "if determined" or "if detecting (a stated condition or event)" can be interpreted as meaning "when determined" or "in response to determining" or "when detecting (a stated condition or event)" or "in response to detecting (a stated condition or event)".

[0074] Various structural schematic diagrams according to the disclosed embodiments of the present application are shown in the drawings. These diagrams are not drawn to scale, in which certain details are exaggerated for the purpose of clarity, and certain details may be omitted. The shapes of various regions, layers shown in the diagrams and their relative sizes and positional relationships may deviate in practice due to manufacturing tolerances or technical limitations, and regions / layers with different shapes, sizes, relative positions may be additionally designed by those skilled in the art according to actual needs.

[0075] The present application provides an optical and SAR image registration method based on multi-scale phase asymmetry, forming multi-scale monogenic signals of optical and corresponding SAR image pairs; using multi-scale monogenic signals to calculate the multi-scale phase asymmetry of the obtained image, and then performing feature point detection on the original image to obtain the feature points of each original image; constructing a multi-scale phase asymmetry direction map (MPADM) using the phase asymmetry of each scale to generate descriptors of the feature points; using the similarity measurement of the descriptors between the original images, approximately matching the feature points of the image pairs to obtain matched feature point pairs, and removing the false matching feature point pairs through the FSC method to realize accurate registration of the images; finally, using the transformation matrix obtained through the FSC screening to transform the images to obtain the registered optical and SAR image pairs. The present application can effectively improve the registration effect of low-quality optical and SAR images and realize fast and robust registration.

[0076] The present application provides an optical and SAR image registration method based on multi-scale phase asymmetry, forming multi-scale monogenic signals of optical and corresponding SAR image pairs; using multi-scale monogenic signals to calculate the multi-scale phase asymmetry of the obtained image, and then performing feature point detection on the original image to obtain the feature points of each original image; constructing a multi-scale phase asymmetry direction map (MPADM) using the phase asymmetry of each scale to generate descriptors of the feature points; using the similarity measurement of the descriptors between the original images, approximately matching the feature points of the image pairs to obtain matched feature point pairs, and removing the false matching feature point pairs through the FSC method to realize accurate registration of the images; finally, using the transformation matrix obtained through the FSC screening to transform the images to obtain the registered optical and SAR image pairs. The present application can effectively improve the registration effect of low-quality optical and SAR images and realize fast and robust registration.

[0077] S1, obtaining an optical and corresponding SAR image pair as an original image pair;

[0078] S2, forming a multi-scale monogenic signal representation of the original image pair;

[0079] S201, constructing a group of log Gabor filters with different center frequencies, and performing Rize transformation to obtain a monogenic signal representation of the log Gabor filter G e m ,G o1 m ,G o2 m T

[0080] The even part of the log Gabor filter is specifically:

[0081] ​​​​​​​

[0082] wherein, ω 0 is the center frequency of the filter, k is a parameter that regulates the bandwidth of the filter. The bandwidth of the filter is given by the ratio σ 0= k / ω 0 is determined.

[0083] In signal processing, the Rize transform is often used to analyze the phase and amplitude information of a signal, which is a multi-dimensional extension of the Hibert transform. In the two-dimensional frequency domain, the Rize transform calculation formula is defined as:

[0084]

[0085] The even part of the two-dimensional log Gabor filter is subjected to Rize transform to obtain:

[0086]

[0087] S202, the even and two odd parts of the single gene signal of the log Gabor filter are respectively convolved with the image to obtain a multi-scale single gene signal representation of the image f e x, y ,f o1 x, y ,f o2 x, y

[0088] S3, using the single gene signal of step S2, obtaining the multi-scale phase antisymmetry of the image, and using the multi-scale phase antisymmetry to detect feature points of the original image to obtain feature points of each original image;

[0089] S301, using f e x, y ,f o1 x, y ,f o2 x, y PA λi

[0090] The phase antisymmetry at a single scale is specifically:

[0091]

[0092] wherein, A λi (​​​​​​​​​​​​​x, y ) represents the local amplitude at a certain scale; T represents the noise threshold, below which it is considered as noise, and the phase anti-symmetry is taken as zero, for removing the influence of noise; epsilon A very small value is introduced to avoid zero division.

[0093] S302, anti-symmetry PA λi Average value, considered as the amplitude of multi-scale phase anti-symmetry MPA;

[0094] Specifically:

[0095]

[0096] S303, divide the MPA map of the image into m*n blocks, perform Harris corner point detection, and take the top g corner points in each block as feature points according to the order from large to small.

[0097] Take m = n = 5, g = 40.

[0098] S4, construct a multi-scale phase anti-symmetry direction map (MPADM) using the multi-scale phase anti-symmetry obtained in step S3, so as to generate a descriptor of the feature point;

[0099] S401, define the multi-scale phase anti-symmetry direction θ MPA The superposition of the phase anti-symmetry directions θ PAλi at multiple scales;

[0100] The phase anti-symmetry directions θ PAλi Specifically:

[0101]

[0102] S402, divide the multi-scale phase anti-symmetry direction into six channels, i.e. ◦ i, 30 ◦ i +1)] corresponding to the channel value i , use these channel values to replace the original intensity values, and form the MPADM of the image;

[0103] S403, the descriptor of the feature point is composed of the MPADM blocks in the range around the feature point, and the pixel values are expanded into a vector represented as​​v 1 ,v 2 ,...,v n ]。

[0104] S5, using the similarity measure between the descriptors of the original images, the feature points of the image pair are approximately matched to obtain the matched feature point pairs, and the FSC (Fast Sample Consensus) method is used to remove the false matching feature point pairs, so as to realize accurate image registration;

[0105] S501, the Euclidean distance between the descriptors is calculated, and the Euclidean distance is used as the similarity measure of the feature point pairs;

[0106] S502, based on the nearest neighbor distance ratio of the feature point pairs, the feature point pairs are screened to obtain the feature point pairs with a distance less than a registration threshold;

[0107] Let the correct registration threshold thresh=0.95, when the distance ratio is less than the threshold, the initial matching feature point pairs can be obtained.

[0108] S503, the matched feature point pairs are sampled to estimate the transformation model h, and the transformation model H with the most feature point pairs meeting the error threshold E is found through iteration; the feature point pairs meeting the transformation model are the final matching feature points.

[0109] The judgment formula of whether the feature points meet the transformation model is:

[0110]

[0111] Wherein, The two matched feature points in the image pair to be registered, the formula shows that the Euclidean distance between the two matched points is small enough after the transformation matrix h.

[0112] S6, the transformation matrix obtained by using FSC screening is used to transform the image, and the registered optical and SAR image pairs are obtained.

[0113] Those skilled in the art can understand that various aspects of the present application can be implemented as a system, a method or a program product. Therefore, various aspects of the present application can be specifically implemented as follows: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, which can be collectively referred to as "circuit", "module" or "platform" here.

[0114] In still another embodiment of the present application, an optical and SAR image registration system based on multi-scale phase antisymmetry is provided, which can be used to implement the optical and SAR image registration method based on multi-scale phase antisymmetry, and specifically, the optical and SAR image registration system based on multi-scale phase antisymmetry comprises a data module, a detection module, a description module, a matching module and a screening module.

[0115] The data module acquires an optical image and a corresponding SAR image pair as a raw image pair, and forms multi-scale single gene signals of the raw image pair.

[0116] The detection module obtains multi-scale phase antisymmetry of the images using the multi-scale single gene signals, and detects feature points of the raw images using the multi-scale phase antisymmetry to obtain feature points of each raw image.

[0117] The description module constructs a multi-scale phase antisymmetry orientation map using the multi-scale phase antisymmetry, and generates a descriptor of the feature points of each raw image.

[0118] The matching module performs approximate matching of the feature points of the image pair using similarity measurement of the descriptors between the feature points of the raw images to obtain matched feature point pairs, and performs elimination of false matching feature point pairs through an FSC method.

[0119] The screening module performs transformation of the images using the transformation matrix obtained through the FSC method to obtain a registered optical and SAR image pair.

[0120] In still another embodiment of the present application, a terminal device is provided, which comprises a processor and a memory, the memory is used to store a computer program, the computer program comprises program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, and are suitable for implementing one or more instructions, and specifically suitable for loading and executing one or more instructions to implement a corresponding method flow or a corresponding function; the processor in the embodiments of the present application can be used for the operation of the optical and SAR image registration method based on multi-scale phase antisymmetry, comprising:

[0121] obtaining an optical and corresponding SAR image pair as a raw image pair; forming a multi-scale single gene signal of the raw image pair; obtaining a multi-scale phase antisymmetry of the image using the multi-scale single gene signal, performing feature point detection on the raw image using the multi-scale phase antisymmetry to obtain feature points of each raw image; constructing a multi-scale phase antisymmetry direction map using the multi-scale phase antisymmetry to generate a descriptor of each raw image feature point; performing approximate matching of the feature points of the image pair using similarity measurement of the descriptors between the raw image feature points to obtain matched feature point pairs, and performing false matching feature point pair elimination through an FSC method; transforming the image using the transformation matrix obtained through the FSC method to obtain a registered optical and SAR image pair.

[0122] Referring to Figure 5 , the terminal device is a computer device, and the computer device 60 of this embodiment includes a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61, which, when executed by the processor 61, implements the multi-scale phase antisymmetry-based optical and SAR image registration method in the embodiment. To avoid repetition, details are not repeated here. Alternatively, the computer program 63, when executed by the processor 61, implements the functions of each model / unit in the multi-scale phase antisymmetry-based optical and SAR image registration system of the embodiment. To avoid repetition, details are not repeated here.

[0123] The computer device 60 can be a desktop computer, a notebook, a palm computer, and a cloud server, etc. The computer device 60 can include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art can understand that Figure 5 is merely an example of the computer device 60 and does not constitute a limitation on the computer device 60, and can include more or fewer components than those shown, or combine certain components, or different components, for example, the computer device can also include an input / output device, a network access device, a bus, etc.

[0124] The processor 61 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0125] The memory 62 can be an internal storage unit of the computer device 60, such as a hard disk or a memory of the computer device 60. The memory 62 can also be an external storage device of the computer device 60, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc.

[0126] Further, the memory 62 can include both an internal storage unit and an external storage device of the computer device 60. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 can also be used to temporarily store data that has been output or will be output.

[0127] Please refer to Figure 6 , the terminal device is a chip, and the chip 600 of the embodiment includes one or more processors 622 and a memory 632 for storing computer programs executable by the processor 622. The computer programs stored in the memory 632 can include one or more modules each corresponding to a set of instructions. In addition, the processor 622 can be configured to execute the computer programs to perform the optical and SAR image registration method based on multi-scale phase antisymmetry described above.

[0128] In addition, the chip 600 can also include a power supply component 626 and a communication component 650, the power supply component 626 can be configured to perform power management of the chip 600, and the communication component 650 can be configured to implement communication of the chip 600, such as wired or wireless communication. In addition, the chip 600 can also include an input / output interface 658. The chip 600 can operate based on an operating system stored in the memory 632.

[0129] In still another embodiment of the present application, the present application also provides a storage medium, specifically a computer readable storage medium, which is a memory device in the terminal device, used for storing programs and data. It can be understood that the computer readable storage medium herein can include the built-in storage medium in the terminal device, and of course can also include the expansion storage medium supported by the terminal device. The computer readable storage medium provides a storage space, which stores the operating system of the terminal. Moreover, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and the instructions can be one or more computer programs. It should be noted that the computer readable storage medium herein can be a high-speed RAM memory, or a non-volatile memory such as at least one disk memory.

[0130] The one or more instructions stored in the computer readable storage medium can be loaded and executed by the processor to realize the corresponding steps of the optical and SAR image registration method based on the multi-scale phase antisymmetry in the above embodiments; the one or more instructions in the computer readable storage medium are loaded and executed by the processor to perform the following steps:

[0131] Obtain an optical and corresponding SAR image pair as an original image pair; form a multi-scale monogenic signal of the original image pair; obtain a multi-scale phase antisymmetry of the image using the multi-scale monogenic signal, and perform feature point detection on the original image using the multi-scale phase antisymmetry to obtain feature points of each original image; construct a multi-scale phase antisymmetry direction map using the multi-scale phase antisymmetry to generate a descriptor of each original image feature point; perform approximate matching on the feature points of the image pair using the similarity measurement of the descriptors between the original image feature points to obtain matched feature point pairs, and perform false matching feature point pair elimination through the FSC method; and transform the image using the transformation matrix obtained through the FSC method to obtain a registered optical and SAR image pair.

[0132] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0133] The technical effects of the present invention will be further explained below with reference to simulation experiments.

[0134] 1. Simulation experimental conditions:

[0135] The hardware platform for the simulation experiment of this invention is as follows:

[0136] The processor is an Intel(R) Core(TM) i5-8400 CPU with a clock speed of 2.80GHz and 16.0 GB of memory.

[0137] The software platform for the simulation experiment of this invention is:

[0138] Windows 10 operating system and MATLAB 2023a.

[0139] 2. Simulation content and result analysis:

[0140] The registration results of the three methods were evaluated using three evaluation metrics (number of correctly matched feature points, root mean square error, and overall running time) for the three images above.

[0141] The correctly matched feature points are those whose Euclidean distance between the transformed and original matched feature points is not greater than the threshold T.

[0142] Root mean square error (RMSE) is a metric used to evaluate the accuracy of image registration methods. If too few correctly matched point pairs are found, the matching is considered a failure.

[0143] In the experiment, the number of feature points detected by the three methods on each image was controlled to be 4000, and the threshold for correct matching was T=2. N correct < 10 indicates a failed match.

[0144] The experimental results are as follows:

[0145]

[0146] The * symbols in the table indicate registration failures. The multi-scale phase antisymmetry method not only successfully matched all three image sets, but also achieved a smaller root mean square error, better registration results, and shorter running time. This verifies the effectiveness of the method presented in this invention.

[0147] The simulation experiment of this invention uses the present invention and two existing technologies, os-sift and Rift, to register three pairs of input test images respectively; and obtains the registration result image.

[0148]

[0149] Compared with the prior art, the present application has the following advantages:

[0150] Figure 2 、 Figure 3 、 Figure 4 The chessboard registration result figures obtained by the present application for three pairs of test images can be seen from the figures, and it can be seen that the overlapping areas of the optical and SAR image pairs to be registered in the present application can be accurately overlapped, and there is almost no misregistration in the results, which well indicates that the registration accuracy of the present application method is high.

[0151] In summary, the present application is an optical and SAR image registration method and system based on multi-scale phase antisymmetry, which can realize the registration of optical and low-resolution SAR images under strong noise interference without being affected by noise and resolution, and has the advantages of low computational complexity and high registration accuracy.

[0152] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or software. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the system can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0153] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0154] Those skilled in the art can realize that the units and method steps of each example described in combination with the embodiments disclosed in the present application can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0155] In the embodiments of the present application, it should be understood that the disclosed apparatus / terminal and method can be implemented in other manners. For example, the embodiments of the apparatus / terminal described above are merely schematic, and the division of the modules or units is merely logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0156] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0157] In addition, each functional unit in the various embodiments of the present application can be integrated into a processing unit, or each unit can be a physically independent unit, or two or more units can be integrated into a unit. The integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0158] The integrated module / unit, if implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the computer readable medium can include or exclude content according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0159] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps in one or more flow or blocks

[0160] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps in one or more flow or blocks

[0161] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps in one or more flow or blocks

[0162] The above merely provides the technical idea of the present application and cannot be used to limit the protection scope of the present application. Any modification made according to the technical idea of the present application on the basis of the technical solutions falls within the protection scope of the claims of the present application.

Claims

1. An optical and SAR image registration method based on multi-scale phase antisymmetry, characterized in that, The method comprises the following steps: Obtain optical and corresponding SAR image pairs as original image pairs, form multi-scale single gene signals of the original image pairs, specifically: A set of log-Gabor filters with different center frequencies are constructed and their Rize transforms are taken to obtain the single gene signal representation of log-Gabor filters G e ( m ) ,G o1 ( m ) ,G o2 ( m )] T ; The even and two odd parts of the single gene signal of the log Gabor filter are respectively convolved with the image to obtain a multi-scale single gene signal representation of the image f e ( x, y ) ,f o1 ( x, y ) ,f o2 ( x, y )] The feature points of the original image are detected by using multi-scale phase antisymmetry, and the feature points of each original image are specifically as follows: Using f e ( x, y ) ,f o1 ( x, y ) ,f o2 ( x, y )] the antisymmetry is computed for each scale of the image PA λi ; antisymmetry PA λi averaged, considered as a magnitude of multiscale phase antisymmetry ; Put the image The graph is divided into m*n blocks, and Harris corner detection is performed. The top g corner points in each block, sorted from largest to smallest by Harris value, are taken as feature points. Obtain multi-scale phase antisymmetry of the images using the multi-scale single gene signals, detect feature points of the original images using the multi-scale phase antisymmetry, and obtain feature points of each original image; Construct a multi-scale phase antisymmetry direction graph using the multi-scale phase antisymmetry, generate descriptors of the feature points of each original image, specifically: Defining multi-scale phase symmetry orientation θ MPA For multi-scale phase symmetry orientation Superimpose; divide the multi-scale phase symmetry orientation into six channels, namely Corresponding channel values i Replace the original intensity values with these channel values to form a multi-scale phase symmetry orientation map of the image; the descriptor of the feature point is composed of multi-scale phase symmetry orientation blocks within a certain range around the feature point, and the pixel values thereof are expanded into a vector v 1 ,v 2 ,...,v n ] Approximately match the feature points of the image pairs using similarity measurement of the descriptors between the feature points of the original images, obtain matched feature point pairs, and remove false matching feature point pairs through the FSC method; Transform the images using the transformation matrix obtained through the FSC method, and obtain registered optical and SAR image pairs.

2. The optical and SAR image registration method based on multi-scale phase antisymmetry according to claim 1, characterized in that, Two-dimensional log-Gabor filter Doing Rize transform, we get: wherein is an even part of the monogenic signal of the log-Gabor filter, , are two odd parts of the monogenic signal of the log-Gabor filter for processing two directions of a two-dimensional image; is a two-dimensional frequency vector, and is a frequency vector in two directions.

3. The optical and SAR image registration method based on multi-scale phase antisymmetry according to claim 1, characterized in that, Anti-symmetry in images at various scales PA λi Specifically: wherein is a noise threshold, is the local amplitude at a certain scale, , is , and denote the output of the odd and even part of the log Gabor filter at the position of the scale .

4. The optical and SAR image registration method based on multi-scale phase antisymmetry according to claim 1, characterized in that, Magnitude of a multi-scale phase antisymmetric is: wherein is the total number of scales, i.e. the total number of filters used.

5. The method for optical and SAR image registration based on multi-scale phase antisymmetry according to claim 1, characterized in that, Phase antisymmetry directions at multiple scales θ PAλi Specifically: wherein and denote the responses of the two odd parts of the log Gabor filter at the positions under the scale .

6. The method for optical and SAR image registration based on multi-scale phase antisymmetry according to claim 1, characterized in that, Approximately match the feature points of the image pairs using similarity measurement of the descriptors between the feature points of the original images, obtain matched feature point pairs, and remove false matching feature point pairs through the FSC method, specifically: Calculate the Euclidean distance between the descriptors, and use the Euclidean distance as the similarity measurement of the feature point pairs; Filter the feature point pairs based on the nearest neighbor distance ratio of the feature point pairs, and obtain feature point pairs with a distance less than a registration threshold; Sample and estimate the transformation model h of the matched feature point pairs, find the transformation model H that has the most feature point pairs that meet the error threshold E through iteration, and use the feature point pairs that meet the transformation model H as the final matched feature points.

7. An optical and SAR image registration system based on multi-scale phase antisymmetry, characterized in that, It comprises: A data module that obtains optical and corresponding SAR image pairs as original image pairs, forms multi-scale single gene signals of the original image pairs, specifically: A set of log-Gabor filters with different center frequencies are constructed and their Rize transforms are taken to obtain the single gene signal representation of log-Gabor filters G e ( m ) ,G o1 ( m ) ,G o2 ( m )] T ; The image is convolved with the even-numbered and two odd-numbered components of the single-gene signal obtained by the logarithmic Gabor filter to obtain a multi-scale single-gene signal representation of the image. f e ( x, y ) ,f o1 ( x, y ) ,f o2 ( x, y The feature points of the original image are detected using multi-scale phase antisymmetry, and the specific feature points of each original image are as follows: Using f e ( x, y ) ,f o1 ( x, y ) ,f o2 ( x, y )] the antisymmetry is computed for each scale of the image PA λi ; antisymmetry PA λi averaged, considered as a multiscale phase antisymmetric magnitude ; Put the image The graph is divided into m*n blocks, and Harris corner detection is performed. The top g corner points in each block, sorted from largest to smallest by Harris value, are taken as feature points. A detection module that obtains multi-scale phase antisymmetry of the images using the multi-scale single gene signals, detects feature points of the original images using the multi-scale phase antisymmetry, and obtains feature points of each original image; A description module that constructs a multi-scale phase antisymmetry direction graph using the multi-scale phase antisymmetry, generates descriptors of the feature points of each original image, specifically: Defining multi-scale phase symmetry orientation θ MPA For multi-scale phase symmetry orientation Superimpose; divide the multi-scale phase symmetry orientation into six channels, namely Corresponding channel values i Replace the original intensity values with these channel values to form a multi-scale phase symmetry orientation map of the image; the descriptor of the feature point is composed of multi-scale phase symmetry orientation blocks within a certain range around the feature point, and the pixel values thereof are expanded into a vector v 1 ,v 2 ,...,v n ] A matching module that approximately matches the feature points of the image pairs using similarity measurement of the descriptors between the feature points of the original images, obtains matched feature point pairs, and removes false matching feature point pairs through the FSC method; A filtering module that transforms the images using the transformation matrix obtained through the FSC method, and obtains registered optical and SAR image pairs.

Images