Railway optical image and SAR image fusion method

Abstract

The invention discloses a railway optical image and SAR image fusion method. The method comprises the following steps: firstly, registering an SAR image and an optical remote sensing image in the same area; decomposing the registered SAR image and optical remote sensing image to obtain a high-frequency sub-image and a low-frequency sub-image of the SAR image and a high-frequency sub-image and a low-frequency sub-image of the optical remote sensing image; fusing the high-frequency sub-image of the SAR image and the high-frequency sub-image of the optical remote sensing image based on the maximum local variance to obtain a high-frequency fused image; fusing the low-frequency sub-image of the SAR image and the low-frequency sub-image of the optical remote sensing image based on the maximum local energy to obtain a low-frequency fused image; and finally, reconstructing the high-frequency fusion image and the low-frequency fusion image to obtain a fusion image. According to the method, the specific scattering characteristic of the SAR image and the high-resolution characteristic of the optical remote sensing image can be reserved, and further more detail information of the image is reserved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and more particularly to a method for fusing railway optical images and SAR images. Background Art

[0002] Synthetic aperture radar (SAR) imaging is an important remote sensing technology that obtains surface information through side-view imaging. However, due to the side-view imaging characteristics of SAR images, there is a certain deviation between the SAR scattering points and the actual position of the surface. In addition, the spatial resolution of SAR images is relatively low, which limits its accuracy in some applications. In order to improve the accuracy of SAR images, researchers began to explore ways to fuse SAR images with other types of images.

[0003] Optical remote sensing images are a common remote sensing technology that has a high spatial resolution and can provide detailed information about the surface. Fusion of optical remote sensing images with SAR images can fully utilize the advantages of both images and improve the accuracy of extracting surface information.

[0004] However, due to the different imaging methods of the two images, there are certain differences between them in the spatial domain.

[0005] Therefore, how to effectively fuse the two images and improve the accuracy of extracting surface information is an urgent problem that technicians in this field need to solve. Summary of the invention

[0006] In view of this, the present invention provides a method for fusing railway optical images and SAR images, which decomposes, fuses and reconstructs high-resolution optical remote sensing images and SAR images in the spatial domain, retains the specific scattering characteristics of SAR images on ground objects and the high spatial resolution of optical remote sensing images, and can improve the accuracy of correspondence between SAR scattering points and the actual positions of ground objects and facilities within a certain range.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A railway optical image and SAR image fusion method, comprising:

[0009] Register SAR images and optical remote sensing images of the same area;

[0010] Decomposing the registered SAR image and optical remote sensing image to obtain a high-frequency sub-image and a low-frequency sub-image of the SAR image, and a high-frequency sub-image and a low-frequency sub-image of the optical remote sensing image;

[0011] The high-frequency sub-image of the SAR image and the high-frequency sub-image of the optical remote sensing image are fused based on the maximum local variance to obtain a high-frequency fused image;

[0012] The low-frequency sub-image of the SAR image and the low-frequency sub-image of the optical remote sensing image are fused based on the local energy maximum to obtain a low-frequency fused image;

[0013] The high-frequency fusion image and the low-frequency fusion image are reconstructed to obtain a fusion image.

[0014] Furthermore, the SAR image and the optical remote sensing image are preprocessed before registration to improve the efficiency of subsequent fusion processing. The preprocessing includes:

[0015] Perform SAR image stitching, data format conversion, geocoding, multi-phase image registration, cropping, and multi-view processing;

[0016] Perform coordinate system transformation, geometric precision correction and cropping on optical remote sensing images.

[0017] Furthermore, the SAR image and the optical remote sensing image of the same area are registered based on the same-name points to reduce the deviation between the SAR scattering points and the actual positions;

[0018] Preferably, the same-name points are encrypted around the object to be extracted according to the root mean square error; including:

[0019] When the root mean square error of the same-name points in the SAR image and the optical remote sensing image exceeds the set threshold, the current same-name points are deleted, otherwise they are retained.

[0020] Furthermore, high-frequency filters and low-frequency filters are used to decompose the registered SAR image and optical remote sensing image; the expression is:

[0021]

[0022] In the formula, is a longitudinal low-frequency filter, is a longitudinal high frequency filter, is a transverse low-frequency filter, is a horizontal high-frequency filter, C is the low-frequency part of the image, D is the high-frequency part of the image, j is the current image decomposition level number, J is the decomposition level, is the SAR image or optical remote sensing image after the j-1th level of decomposition, is the low-frequency sub-image of the image after the j-1th level decomposition; , , They are the high-frequency sub-images in the vertical, horizontal and diagonal directions after the j-1th level decomposition.

[0023] Furthermore, based on the maximum local variance, the fusion is performed to obtain the high-frequency fused image. The fusion formula is:

[0024]

[0025] in, and are the high-frequency sub-images in the k-th direction of the j-th layer decomposition of the SAR image and the optical remote sensing image respectively; and They are and In The local variance in the window area centered at .

[0026] Furthermore, the local variance is obtained by:

[0027]

[0028] In the formula, is the weight coefficient matrix, is the weight coefficient matrix in the window area of ​​size a×a The pixel value at , a is an odd number not less than 3, is the high-frequency sub-image decomposed at the jth layer. The pixel value at for In The mean pixel value in the a×a window area centered at .

[0029] Furthermore, based on the maximum local energy, the fusion is performed to obtain the low-frequency fusion image. The fusion formula is:

[0030]

[0031] In the formula, is the low-frequency sub-image decomposed at the jth level of the SAR image, is the low-frequency sub-image decomposed from the jth layer of the optical remote sensing image, is the local energy of the SAR image, is the local energy of the optical remote sensing image.

[0032] Furthermore, the local energy is obtained by:

[0033]

[0034] In the formula, is the weight coefficient matrix in the window area of ​​size a×a The pixel value at , a is an odd number not less than 3, For the low frequency sub-image The pixel value at .

[0035] By fusing high-frequency sub-images and low-frequency sub-images separately, the image details can be better preserved, making the fused image have better continuity and consistency in the spatial domain. At the same time, fusion based on the principles of local maximum variance and local maximum energy helps to improve the quality of the fused image.

[0036] Further, the high-frequency fused image and the low-frequency fused image are reconstructed according to the following method;

[0037]

[0038] In the formula, To reconstruct the image, and They are and The conjugate transposed matrix of is the low-frequency fusion image, and They are and The conjugate transposed matrix of is the high-frequency fusion image in the vertical direction, is the high-frequency fusion image in the horizontal direction, is the high-frequency fused image in the diagonal direction.

[0039] Compared with the prior art, the present invention provides a method for fusing railway optical images and SAR images, which has the following effects:

[0040] 1. The present invention encrypts the same-name points around the ground objects and facilities, thereby improving the correspondence accuracy between the SAR scattering points and the actual positions of the ground objects and facilities within a certain range, and reducing the position deviation caused by side-view imaging. It should be noted that the present application is particularly suitable for the extraction of specific ground objects such as railway facilities. By encrypting the same-name points and fusing the images, it can provide more accurate data support for the monitoring and management of railway facilities. It can also be extended to other types of ground objects, providing an efficient and accurate means of information extraction for remote sensing applications.

[0041] 2. The present invention can retain the high-resolution characteristics of optical remote sensing images and the specific scattering characteristics of SAR images to ground objects and facilities by decomposing, fusing and reconstructing the high-frequency and low-frequency parts of optical remote sensing images and SAR images, thereby making the surface details clearer and helping to improve the accuracy of surface information extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0043] Figure 1 A flow chart of the method for fusing railway optical images and SAR images provided by the present invention;

[0044] Figure 2 The optical remote sensing image of a railway provided by the present invention;

[0045] Figure 3 The SAR image provided by the present invention;

[0046] Figure 4 This is the fused image provided by the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] The embodiment of the present invention discloses a railway optical image and SAR image fusion method, which can be used for railway facility extraction and facility deformation monitoring.

[0049] The image fusion method provided by the present invention can simultaneously retain the specific scattering characteristics of the SAR image and the high spatial resolution of the optical image, thereby improving the correspondence accuracy between the SAR scattering points and the actual positions.

[0050] In one embodiment, referring to Figure 1 , Figure 1 This is a flow chart of the railway optical image and SAR image fusion method of this application; in this embodiment, the fusion method includes:

[0051] Register SAR images and optical remote sensing images of the same area;

[0052] Decomposing the registered SAR image and optical remote sensing image to obtain a high-frequency sub-image and a low-frequency sub-image of the SAR image, and a high-frequency sub-image and a low-frequency sub-image of the optical remote sensing image;

[0053] The high-frequency sub-image of the SAR image and the high-frequency sub-image of the optical remote sensing image are fused based on the maximum local variance to obtain a high-frequency fused image;

[0054] The low-frequency sub-image of the SAR image and the low-frequency sub-image of the optical remote sensing image are fused based on the local energy maximum to obtain a low-frequency fused image;

[0055] The high-frequency fusion image and the low-frequency fusion image are reconstructed to obtain a fusion image.

[0056] According to an embodiment of the present invention,

[0057] Step 1: Register the SAR image and the optical remote sensing image of the same area;

[0058] In this embodiment, SAR images and optical remote sensing images of the same area are first obtained; in one embodiment, optical remote sensing images and SAR images of the test area that meet the preset date and spatial resolution are collected according to the monitoring plan and the monitoring object; the acquisition time difference between the optical remote sensing image and the SAR image is better than 1 month, the spatial resolution of the optical remote sensing image is better than 0.5m, and BJ3 (0.5m) is selected; the spatial resolution of the SAR image is better than 3m, and TerraSAR (3m) is selected;

[0059] Then the optical remote sensing images and SAR images of the test area are preprocessed, including:

[0060] Perform SAR image stitching, data format conversion, geocoding, multi-phase image registration, cropping, and multi-view processing;

[0061] And perform coordinate system transformation, geometric precision correction and cropping on optical remote sensing images.

[0062] Furthermore, the SAR image and the optical remote sensing image of the same area are registered based on the same-name points. In this embodiment, the registration is performed based on the optical remote sensing image, and during the registration process, the selection of the same-name points must meet the requirement of uniform full frame.

[0063] As a preferred implementation scheme, on the basis of uniform full-frame distribution of homonymous points in the entire image, homonymous points are encrypted around the objects to be extracted, such as railway facilities; during the encryption process, both the root mean square error value of the homonymous points and the predicted position of the homonymous points are referenced;

[0064] The root mean square error calculation formula is:

[0065]

[0066] Where N is the number of points with the same name, is the horizontal coordinate value of the i-th point of the same name in the optical remote sensing image, is the horizontal coordinate value of the i-th point with the same name in the SAR image, is the ordinate value of the i-th point of the same name in the optical remote sensing image, is the ordinate value of the i-th point with the same name in the SAR image.

[0067] In this embodiment, the maximum limit of the root mean square error is within 2.0. If the maximum limit is exceeded, the same-name points with large errors need to be deleted and reselected, otherwise they are retained until the requirements are met.

[0068] At the same time, according to the root mean square error of the existing same-name points, the predicted position of the new same-name points in the image to be registered is obtained. According to the predicted position of the new same-name points in the image to be registered and combined with the ground object information, the position of the same-name points is adjusted.

[0069] Step 2: Decomposing the registered SAR image and the optical remote sensing image to obtain a high-frequency sub-image and a low-frequency sub-image of the SAR image, and a high-frequency sub-image and a low-frequency sub-image of the optical remote sensing image;

[0070] In one embodiment, a high-frequency filter and a low-frequency filter are used to decompose the registered SAR image and the optical remote sensing image; the expression is:

[0071]

[0072] In the formula, is a longitudinal low-frequency filter, is a longitudinal high frequency filter, is a transverse low-frequency filter, is a horizontal high-frequency filter, C is the low-frequency part of the image, D is the high-frequency part of the image, j is the sequence number of the current image decomposition level, and J is the decomposition level. In this embodiment, the decomposition level is set to 3 levels. is the SAR image or optical remote sensing image after the j-1th level of decomposition, is the low-frequency sub-image of the image after the j-1th level decomposition; , , They are the high-frequency sub-images in the vertical, horizontal and diagonal directions after the j-1th level decomposition.

[0073] Step 3: Fuse the high-frequency sub-image of the SAR image and the high-frequency sub-image of the optical remote sensing image based on the maximum local variance to obtain a high-frequency fused image. The fusion formula is:

[0074]

[0075] in, and are the high-frequency sub-images in the k-th direction of the j-th layer decomposition of the SAR image and the optical remote sensing image respectively; and They are and In The local variance within the window area centered at .

[0076] The local variance is obtained as follows:

[0077]

[0078] In the formula, is the weight coefficient matrix, is the weight coefficient matrix in the window area of ​​size a×a The pixel value at , a is an odd number not less than 3, is the high-frequency sub-image decomposed at the jth layer. The pixel value at for In The mean pixel value in the a×a window area centered at .

[0079] When the window area is 3×3,

[0080]

[0081]

[0082] and are the high-frequency images in k directions (horizontal, vertical, and diagonal) decomposed at the jth layer of the SAR and optical images, respectively; and They are and In The local variance within the 3×3 window area centered on and They are and In The mean pixel value in the 3×3 window area centered at .

[0083] Step 4: Fuse the low-frequency sub-image of the SAR image and the low-frequency sub-image of the optical remote sensing image based on the maximum local energy to obtain a low-frequency fused image; the fusion formula is:

[0084]

[0085] In the formula, is the low-frequency sub-image decomposed at the jth level of the SAR image, is the low-frequency sub-image decomposed from the jth layer of the optical remote sensing image, is the local energy of the SAR image, is the local energy of the optical remote sensing image.

[0086] The local energy is obtained as follows:

[0087]

[0088] In the formula, is the weight coefficient matrix, is the weight coefficient matrix in the window area of ​​size a×a The pixel value at , a is an odd number not less than 3, For the low frequency sub-image The pixel value at .

[0089] In one embodiment, a is set to 3, that is, the window area is 3×3, to provide a reasonable compromise point that can retain certain texture details without significantly increasing the complexity of the calculation.

[0090] At this time, the local energy calculation formula of the optical remote sensing image is:

[0091]

[0092] The local energy calculation formula of SAR image is:

[0093]

[0094] Step 5: Reconstruct the high-frequency fusion image and the low-frequency fusion image to obtain a fusion image.

[0095] This embodiment reconstructs the high-frequency fusion image and the low-frequency fusion image according to the following method;

[0096]

[0097] In the formula, To reconstruct the image, and They are and The conjugate transposed matrix of is the low-frequency fusion image, and They are and The conjugate transposed matrix of is the high-frequency fusion image in the vertical direction, is the high-frequency fusion image in the horizontal direction, is the high-frequency fused image in the diagonal direction.

[0098] In one embodiment, Figure 2 The optical remote sensing image of a railway provided by the present invention; Figure 3 The SAR image provided by the present invention; Figure 4 This is the fused image provided by the present invention.

[0099] Obviously, the high-resolution optical image and SAR image are decomposed, fused and reconstructed in the spatial domain. Figure 4 Compared with the pre-fusion Figure 3 It not only retains the specific scattering characteristics of SAR images for railway facilities, but also retains the high spatial resolution of optical images. The fused image obtained in this application can help improve the accuracy of the correspondence between SAR scattering points and the actual positions of railway facilities within a certain range.

[0100] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0101] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A railway optical image and SAR image fusion method, characterized in that: Register SAR images and optical remote sensing images of the same area; Decomposing the registered SAR image and optical remote sensing image to obtain a high-frequency sub-image and a low-frequency sub-image of the SAR image, and a high-frequency sub-image and a low-frequency sub-image of the optical remote sensing image; The high-frequency sub-image of the SAR image and the high-frequency sub-image of the optical remote sensing image are fused based on the maximum local variance to obtain a high-frequency fused image; The low-frequency sub-image of the SAR image and the low-frequency sub-image of the optical remote sensing image are fused based on the local energy maximum to obtain a low-frequency fused image; The high-frequency fusion image and the low-frequency fusion image are reconstructed to obtain a fusion image.

2. The railway optical image and SAR image fusion method according to claim 1, characterized in that: Before registration, the SAR image and the optical remote sensing image are preprocessed, including: Perform SAR image stitching, data format conversion, geocoding, multi-phase image registration, cropping, and multi-view processing; Perform coordinate system transformation, geometric precision correction and cropping on optical remote sensing images.

3. The railway optical image and SAR image fusion method according to claim 1, characterized in that: Based on the same-name points, the SAR image and the optical remote sensing image of the same area are registered, and the same-name points are encrypted around the objects to be extracted according to the root mean square error; including: When the root mean square error of the same-name points in the SAR image and the optical remote sensing image exceeds the set threshold, the current same-name points are deleted, otherwise they are retained.

4. The railway optical image and SAR image fusion method according to claim 1, characterized in that: The high-frequency filter and low-frequency filter are used to decompose the registered SAR image and optical remote sensing image; the expression is: ； In the formula, is a longitudinal low-frequency filter, is a longitudinal high frequency filter, is a transverse low-frequency filter, is a horizontal high-frequency filter, C is the low-frequency part of the image, D is the high-frequency part of the image, j is the current image decomposition level number, J is the decomposition level, is the SAR image or optical remote sensing image after the j-1th level decomposition, is the low-frequency sub-image of the image after the j-1th level decomposition; , , They are the high-frequency sub-images in the vertical, horizontal and diagonal directions after the j-1th level decomposition.

5. The railway optical image and SAR image fusion method according to claim 1, characterized in that: Based on the maximum local variance, the fusion formula for the high-frequency fused image is: ； in, and are the high-frequency sub-images in the k-th direction of the j-th layer decomposition of the SAR image and the optical remote sensing image respectively; and They are and In The local variance in the window area centered at .

6. The railway optical image and SAR image fusion method according to claim 5, characterized in that: The local variance is obtained as follows: ； In the formula, is the weight coefficient matrix, is the weight coefficient matrix in the window area of ​​size a×a The pixel value at , a is an odd number not less than 3, is the high-frequency sub-image decomposed at the jth layer. The pixel value at for In The mean pixel value in the a×a window area centered at .

7. The railway optical image and SAR image fusion method according to claim 1, characterized in that: Based on the maximum local energy, the fusion formula for the low-frequency fusion image is: ； In the formula, is the low-frequency sub-image decomposed at the jth level of the SAR image, is the low-frequency sub-image decomposed from the jth layer of the optical remote sensing image, is the local energy of the SAR image, is the local energy of the optical remote sensing image.

8. The railway optical image and SAR image fusion method according to claim 7, characterized in that: The local energy is obtained as follows: ； In the formula, is the weight coefficient matrix in the window area of ​​size a×a The pixel value at , a is an odd number not less than 3, For the low frequency sub-image The pixel value at .

9. The railway optical image and SAR image fusion method according to claim 1, characterized in that: Reconstructing the high-frequency fused image and the low-frequency fused image according to the following method; ； In the formula, To reconstruct the image, and They are and The conjugate transposed matrix of is the low-frequency fusion image, and They are and The conjugate transposed matrix of is the high-frequency fusion image in the vertical direction, is the high-frequency fusion image in the horizontal direction, is the high-frequency fused image in the diagonal direction.

Images