Ka-SAR remote sensing image fusion method and equipment based on texture feature extraction

Abstract

The present invention relates to a Ka‑SAR remote sensing image fusion method and device based on texture feature extraction, and belongs to the field of image processing technology. First, principal component analysis is performed on the optical remote sensing image; the first principal component and the Ka‑SAR remote sensing image are respectively transformed by wavelet to obtain low-frequency detail components and high-frequency detail components; the texture features of the Ka‑SAR remote sensing image are extracted by combining the cuckoo algorithm optimized by genetic factors with the local binary pattern; the high-frequency detail components corresponding to the optical remote sensing image are enhanced by the extracted texture features, and the enhanced high-frequency components and the low-frequency components of the Ka‑SAR remote sensing image are subjected to inverse wavelet transformation to obtain new principal components; the principal component and the remaining principal components corresponding to the optical remote sensing image are subjected to inverse transformation of principal component analysis to obtain the fused image. The present invention introduces the extracted texture features into the fusion algorithm with the detail components after wavelet transformation, adapts to the imaging characteristics of Ka‑SAR, and has better effects.

Description

Technical Field

[0001] The invention relates to a Ka-SAR remote sensing image fusion method and equipment based on texture feature extraction, belonging to the technical field of image processing. Background Art

[0002] In recent years, satellite remote sensing images can receive electromagnetic wave information reflected, radiated, and scattered from the target object itself over a large range and with high timeliness, and perform multi-angle analysis and processing based on the characteristics of the returned electromagnetic wave information. Remote sensing satellites based on different types of sensors, different application purposes, and different imaging methods are constantly being put into use in various fields. However, single-purpose remote sensing satellite data is often restricted by various conditions and is difficult to be used in practice. Therefore, multi-source remote sensing data based on remote sensing image fusion has received widespread attention in various application fields, such as flood and other disaster monitoring fields.

[0003] For Ka-band SAR remote sensing satellites, the wavelength of satellite signals in this band is shorter than that of other SAR satellites, which makes its images clearer while containing rich surface object information. Ka-SAR images have near-optical imaging characteristics, which not only have the strong penetration of SAR images in extreme weather, but also have higher resolution and a large number of texture features than other band SAR images. However, there are still few application studies in the field of image fusion based on Ka-band SAR images. There are still few suitable algorithms for Ka-SAR in the field of image fusion, and it is difficult for the fusion results to retain the details of the original Ka-SAR images. Patent CN117408902 A discloses a method for colorizing Ka-band SAR images based on the combination of BEMD and IHS transformation. It is the inventor's previous research. The patent uses HIS transformation to decompose optical images and extract brightness components. At the same time, the Ka-SAR image is decomposed based on BEMD to obtain its low-frequency and high-frequency components, and then the brightness component and low-frequency component are decomposed by BEMD. The low-frequency and high-frequency components are respectively subjected to BEMD inverse transformation according to the fusion rules, and the results are then subjected to HIS inverse transformation to finally realize the colorization of Ka-SAR images. The purpose of this patent is to realize the colorization of Ka-SAR images. Although the method of this patent is mainly based on the principle of image fusion, this patent does not use a specific method to achieve fusion based on the characteristics of Ka-SAR images. Secondly, this patent is mainly for the purpose of improving the visual interpretation ability of Ka-SAR and colorizing it. There is no research on fusion algorithms for Ka-SAR images to improve the fusion effect of Ka-SAR. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a Ka-SAR remote sensing image fusion method based on texture feature extraction. In the method, a genetic factor optimized cuckoo algorithm combined with a local binary pattern algorithm (GCS-LBP) is proposed to extract texture features, and then the texture features are used to fuse the Ka-band SAR image with the optical image. The method can adapt to the imaging characteristics of Ka-SAR and has a better fusion effect.

[0005] The technical solution adopted by the present invention is:

[0006] The Ka-SAR remote sensing image fusion method based on texture feature extraction includes the following steps:

[0007] S1. Preprocess the acquired Ka-band SAR remote sensing image and its corresponding optical remote sensing image, and perform principal component analysis on the preprocessed optical remote sensing image to obtain the first principal component PC 1 and the remaining principal components;

[0008] S2. Decompose the first principal component corresponding to the optical remote sensing image and the Ka-band SAR remote sensing image using wavelet transform to obtain the low-frequency detail component W φ opt , W φ SAR High frequency detail components W ω opt , W ω SAR ;

[0009] S3. Use the genetic factor optimized cuckoo algorithm combined with local binary patterns to extract texture features from Ka-band SAR remote sensing images;

[0010] S4. Use the extracted texture features to enhance the high-frequency detail components of optical remote sensing images after wavelet transform decomposition W ω opt , the high-frequency component after texture feature enhancement is obtained, and then it is combined with the low-frequency detail component obtained by wavelet transform decomposition of Ka-band SAR remote sensing image W ω SAR Perform inverse wavelet transform to obtain new principal components PC 1_new ;

[0011] S5. The new principal component PC 1_newThe remaining principal component components corresponding to the optical remote sensing image in step S1 are subjected to principal component analysis inverse transformation to obtain a fused image.

[0012] In the above method, the preprocessing described in step S1 includes orthorectification and image registration.

[0013] In step S3, the method of extracting texture features by combining the genetic factor optimized cuckoo algorithm with the local binary pattern is as follows: the initial parameters of the genetic factor optimized cuckoo algorithm are set, the local binary pattern algorithm (LBP algorithm) for Ka-band SAR remote sensing images is used as the objective function, and the information entropy is used as the evaluation index for initialization setting. The threshold result obtained by each iteration of the genetic factor optimized cuckoo algorithm is passed into the LBP algorithm as an input parameter, and the result obtained by the LBP algorithm is calculated by finding the pixel point corresponding to the threshold and its floating interval as the center, that is, the LBP value and its histogram, and connecting the normalized histograms of all sub-blocks to obtain the texture features of the entire image, and then the information entropy of the texture feature image is calculated as an iterative evaluation index. If it does not meet the requirements, it enters the next iteration and finally outputs the optimal texture features. Among them, information entropy is usually used to measure the complexity or information content of the image in image processing. The higher the entropy value, the more information there is in the image and the richer the texture features are. The calculation formula is as follows:

[0014] ,

[0015] in H ( X ) represents the value of entropy, n is a random variable X the number of possible values, p ( x i ) is a random variable X The value is x i The probability of b The logarithm base, usually 2 or natural logarithm e .

[0016] The LBP algorithm is to divide the image into several N × N The image sub-block is centered on a certain pixel point. The pixel value of the pixel point is used as the threshold to compare with the 8 points in its annular neighborhood. The LBP value of each pixel in each sub-block is calculated. The calculation formula of the LBP algorithm is as follows:

[0017] ,

[0018] in( x c , y c) is the center pixel, i c is its pixel value, i p are the neighboring pixel values, s Then it is a symbolic function, expressed as:

[0019] .

[0020] The genetic factor optimized cuckoo algorithm (GCS) introduces the selection, crossover and mutation operations of the genetic algorithm, uses the crossover factor to generate a new population, and in each iteration, the population after the crossover is mutated according to the mutation probability, so as to find the global optimal solution.

[0021] The principal component analysis inverse transformation formula described in step S5 is as follows:

[0022] ,

[0023] In the formula V k T is the transpose of the feature matrix, Y is the data after dimension reduction.

[0024] Another object of the present invention is to provide a Ka-SAR remote sensing image fusion computing device based on texture feature extraction, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the fusion method as described above when executing the program.

[0025] A storage device is a computer-readable storage device storing a computer program, and the steps of the above method are implemented when the computer program is executed.

[0026] The beneficial effects of the present invention are:

[0027] (1) The present invention aims at the characteristics of Ka-band SAR images, such as high definition and high texture feature richness. Ka-SAR image fusion is performed based on texture features, and the extracted texture features are introduced into the fusion algorithm with the detail components after wavelet transformation, which adapts to the imaging characteristics of Ka-SAR, so that the method has a better effect in Ka-SAR fusion;

[0028] (2) Combining the cuckoo algorithm based on genetic factor optimization with the local binary pattern, the optimal threshold can be adaptively obtained and applied to the local binary pattern algorithm, which can more accurately extract the texture features of SAR images;

[0029] (3) The principal component analysis method is used to decompose the optical image and implement the final inverse transformation in the fusion method. This method can retain the important information in the optical image and thus better fuse it with the Ka-SAR texture features. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flow chart of the method of the present invention;

[0031] Figure 2 The flowchart of the GCS-LBP algorithm combining the genetic factor optimized cuckoo algorithm and the local binary pattern in the present invention;

[0032] Figure 3 This is a schematic diagram of Ka-SAR and optical remote sensing images of the study area;

[0033] Figure 4 Ka-SAR texture features extracted for the study area;

[0034] Figure 5 This is the image of the study area after fusion based on texture features;

[0035] Figure 6 The Ka-SAR fusion results of different methods in the study area and their detailed area comparison. DETAILED DESCRIPTION

[0036] The present invention is further described below with reference to the accompanying drawings and specific embodiments.

[0037] Embodiment A Ka-SAR remote sensing image fusion method based on texture feature extraction comprises the following steps:

[0038] S1. Preprocess the acquired Ka-band SAR remote sensing image and its corresponding optical remote sensing image, and perform principal component analysis on the preprocessed optical remote sensing image to obtain the first principal component PC 1 and the remaining principal components:

[0039] First, the acquired Ka-band SAR image is orthorectified and then registered with the corresponding optical image data to ensure that the number of rows and columns of the two images is consistent.

[0040] Perform principal component analysis on the optical remote sensing image and obtain the principal component PC 1 ,PC 2 , … PC n , extract the first principal component PC Component 1 is used for subsequent processing, and its main formula is as follows:

[0041] ,

[0042] In the formula X is the original data matrix, V k It is a k The matrix of eigenvectors, Y is the transformed data matrix.

[0043] S2. Decompose the first principal component corresponding to the optical remote sensing image and the Ka-band SAR remote sensing image using wavelet transform to obtain the low-frequency detail component W φ opt , W φ SAR High frequency detail components W ω opt , W ω SAR :

[0044] The first principal component is transformed by wavelet transform. PC 1 and Ka-band SAR remote sensing images to obtain the corresponding low-frequency components W φ opt , Wφ SAR With high frequency components W ω opt , W ω SAR , the image decomposition principle of wavelet transform is as follows:

[0045] First, construct a two-dimensional scaling function φ ( x,y ) and the two-dimensional wavelet function ω ( x,y ), where the two-dimensional wavelet basis function includes three basis functions: horizontal, vertical and diagonal. Then, the two-dimensional scaling function and the wavelet basis function are combined to construct the scaling basis function and the translation basis function. The formulas are shown as follows:

[0046] ,

[0047] ,

[0048] In the formula denote the horizontal, vertical and diagonal two-dimensional wavelet basis functions, respectively. j Represents the scale parameter, which determines the performance of the function at different scales. m and n is the translation parameter, used to control the functionx and y The translation position in the direction.

[0049] Based on the scale basis function and the translation basis function, the formula for discrete wavelet transform is:

[0050] ,

[0051] ,

[0052] In the formula Corresponding to the horizontal, vertical and diagonal directions respectively, , , M and N Represent the width and height of the image respectively. Through the above formula, the image is decomposed into low-frequency detail components representing areas with small grayscale changes in the image. W φ and horizontal, vertical, and diagonal high-frequency detail components representing obvious grayscale changes in the image W ω H , W ω V and W ω D , the high frequency detail component W ω opt , W ω SAR All include these three high-frequency detail components.

[0053] S3. Using the genetic factor optimized cuckoo algorithm combined with local binary patterns to extract texture features from Ka-band SAR remote sensing images:

[0054] (1) Set the initial parameters and objective function of the cuckoo algorithm optimized by genetic factors, use the LBP value calculation of Ka-band SAR remote sensing images as the objective function, and use information entropy as the evaluation index for initialization;

[0055] (2) The traditional LBP algorithm divides the image into several N × N The image sub-block is centered on a certain pixel point. The pixel value of the pixel point is used as the threshold to compare with the 8 points in its annular neighborhood. The LBP value of each pixel in each sub-block is calculated. The calculation formula of the LBP algorithm is as follows:

[0056] ,

[0057] in( xc , y c ) is the center pixel, i c is its pixel value, i p is the adjacent pixel value, indicating the sequence number of the pixel in the neighborhood, and s is the sign function, expressed as:

[0058] .

[0059] (3) The genetic factor optimized cuckoo algorithm and local binary pattern combined algorithm (GCS-LBP) proposed in the present invention is to pass the threshold result obtained by each iteration of GCS as an input parameter to the LBP algorithm, and calculate the LBP value and its histogram by finding the pixel point corresponding to the threshold and its floating interval as the center, and connect the normalized histograms of all sub-blocks to obtain the texture features of the entire image. Then, the information entropy of the texture feature image is calculated as an iterative evaluation index. If it does not meet the requirements, it will enter the next iteration and finally output the optimal texture features. Among them, the cuckoo algorithm (GCS) based on genetic factor optimization introduces the selection, crossover, and mutation operations of the genetic algorithm, uses the crossover factor to generate a new population, and in each iteration, the population after crossover is mutated according to the mutation probability, which is conducive to finding the global optimal solution.

[0060] Information entropy is usually used to measure the complexity or amount of information in an image in image processing. The higher the entropy value, the more information there is in the image and the richer the texture features. The calculation formula is as follows:

[0061] ,

[0062] in H ( X ) represents the value of entropy, n is a random variable X the number of possible values, P ( x i ) is a random variable X The value is x i The probability of b The logarithm base, usually 2 or natural logarithm e .

[0063] S4. Use the extracted texture features to enhance the high-frequency detail components of optical remote sensing images after wavelet transform decomposition W ω opt The high-frequency component after texture feature enhancement is obtained, and then it is combined with the low-frequency detail component obtained by wavelet transform decomposition of Ka-band SAR remote sensing image. Wω SAR Perform inverse wavelet transform to obtain new principal components PC 1_new :

[0064] The Ka-SAR texture features obtained by combining the genetic factor cuckoo algorithm with the local binary pattern (GCS-LBP) are used to enhance the high-frequency detail components of the optical image using a weighted fusion method. W ω opt , get the high-frequency component after texture feature enhancement,

[0065] Then it is combined with the low-frequency detail component obtained by decomposing the Ka-band SAR remote sensing image W φ SAR Perform inverse wavelet transform to obtain new principal components PC 1_new ,

[0066] The formula for the high-frequency component after texture feature enhancement is as follows:

[0067] ,

[0068] In the formula α and β is the weight parameter, T represents the texture feature image, Corresponding to the horizontal, vertical and diagonal directions respectively.

[0069] S5. The new principal component PC 1_new The remaining principal component components corresponding to the optical remote sensing image in step S1 are subjected to principal component analysis inverse transformation to obtain the fused image:

[0070] The inverse transformation formula is as follows:

[0071] ,

[0072] In the formula V k T is the transpose of the feature matrix, Y is the data after dimension reduction.

[0073] Obtain Ka-band SAR images (Ka-SAR images) and optical images of a certain area. Figure 3 As shown,

[0074] Extract texture features from Ka-SAR images, set initial parameters of GCS algorithm, and randomly generate n The initial position of the bird nest, set the population size n, variable dimension D, and maximum discovery probabilityP a , step factor α and the maximum number of iterations N Then, the threshold value result of each iteration is input into the LBP algorithm to find the corresponding pixel point and calculate the LBP value, generate the texture feature map, calculate its mutual information value as the iterative judgment condition for iterative calculation, and finally obtain the optimal texture feature. The extraction result is as follows: Figure 4 shown.

[0075] The extracted texture features are used to enhance the high-frequency detail components of optical remote sensing images after wavelet transform decomposition. W ω opt , the high-frequency component after texture feature enhancement is obtained, and then it is combined with the low-frequency detail component obtained by wavelet transform decomposition of Ka-band SAR remote sensing image W φ SAR Perform inverse wavelet transform to obtain new principal components PC 1_new , the new principal component PC 1_new The remaining principal component components corresponding to the optical remote sensing image in step S1 are subjected to principal component analysis inverse transformation to obtain the fused image, such as Figure 5 shown.

[0076] In order to further compare the advantages of the method used in the present invention in the application of Ka-SAR images, different methods are used to compare the fusion results and their detail areas, such as Figure 6 As shown in the figure, (a), (b), and (c) are the fusion results without texture feature enhancement, and (d), (e), and (f) are the fusion results of the method used in the present invention, where regions ①, ②, and ③ are roads, farmland, and urban areas, respectively. It can be seen that the fusion method based on texture detail features has achieved better fusion effects in the fusion of Ka-SAR images. Compared with traditional algorithms, it retains more texture detail features, and can better retain the detail features of street lights on roads in Ka-SAR images and highlights in towns and farmland, giving full play to the characteristics and advantages of Ka-SAR images.

[0077] In order to further evaluate the fusion results comprehensively and objectively, five objective evaluation indicators, including Information Entropy (IE), Standard Deviation (SD), Mutual Information (MI), Structural Similarity (SSIM) and Peak Signal-to-Noise Ratio (PSNR), are used to evaluate the accuracy of the above fusion results. The evaluation indicator data are shown in the following table:

[0078] Table 1 Objective evaluation index data of fusion images

[0079]

[0080] It can be seen that the method used in the present invention is superior to the fusion method based on HIS transformation (the method in patent CN117408902 A) in all indicators, which verifies the feasibility and advantages of the method of the present invention in Ka-SAR image fusion.

[0081] The above is a further description of the present invention in combination with the embodiments, and the protection scope of the present invention is not limited thereto.

Claims

1. The Ka-SAR remote sensing image fusion method based on texture feature extraction is characterized by: The steps include: S1. Preprocess the acquired Ka-band SAR remote sensing image and its corresponding optical remote sensing image, and perform principal component analysis on the preprocessed optical remote sensing image to obtain the first principal component PC 1 and the remaining principal components; S2. Decompose the first principal component corresponding to the optical remote sensing image and the Ka-band SAR remote sensing image using wavelet transform to obtain the low-frequency detail component W φ opt , W φ SAR High frequency detail components W ω opt , W ω SAR ; S3. The texture features of Ka-band SAR remote sensing images are extracted by combining the genetic factor optimized cuckoo algorithm with the local binary pattern. The method for extracting texture features by combining the genetic factor optimized cuckoo algorithm with the local binary pattern is as follows: the initial parameters of the genetic factor optimized cuckoo algorithm are set, the LBP algorithm for Ka-band SAR remote sensing images is used as the objective function, and the information entropy is used as the evaluation index for initialization setting. The threshold result obtained by each iteration of the genetic factor optimized cuckoo algorithm is passed into the LBP algorithm as an input parameter, and the result of the LBP algorithm is calculated by finding the pixel point corresponding to the threshold and its floating interval as the center, that is, the LBP value and its histogram, and connecting the normalized histograms of all sub-blocks to obtain the texture features of the entire image. Then, the information entropy of the texture feature image is calculated as the iterative evaluation index. If it does not meet the requirements, it will enter the next iteration and finally output the optimal texture features. S4. Use the extracted texture features to enhance the high-frequency detail components of optical remote sensing images after wavelet transform decomposition W ω opt , the high-frequency component after texture feature enhancement is obtained, and then it is combined with the low-frequency detail component obtained by wavelet transform decomposition of Ka-band SAR remote sensing image W ω SAR Perform inverse wavelet transform to obtain new principal components PC 1_new ; S5. The new principal component PC 1_new The remaining principal component components corresponding to the optical remote sensing image in step S1 are subjected to principal component analysis inverse transformation to obtain a fused image.

2. The Ka-SAR remote sensing image fusion method based on texture feature extraction according to claim 1, characterized in that: The preprocessing described in step S1 includes orthorectification and image registration.

3. The Ka-SAR remote sensing image fusion method based on texture feature extraction according to claim 1, characterized in that: The LBP algorithm is to divide the image into several N × N The image sub-block is centered on a certain pixel point. The pixel value of the pixel point is used as the threshold to compare with the 8 points in its annular neighborhood. The LBP value of each pixel in each sub-block is calculated. The calculation formula of the LBP algorithm is as follows: ， in( x c , y c ) is the center pixel, i c is its pixel value, < i>i p are the neighboring pixel values, s Then it is a symbolic function, expressed as: 。 4. The Ka-SAR remote sensing image fusion method based on texture feature extraction according to claim 1, characterized in that: The genetic factor optimized cuckoo algorithm introduces the selection, crossover and mutation operations of the genetic algorithm, uses the crossover factor to generate a new population, and in each iteration, the population after the crossover is mutated according to the mutation probability, so as to find the global optimal solution.

5. The Ka-SAR remote sensing image fusion method based on texture feature extraction according to claim 1, characterized in that: The principal component analysis inverse transformation formula described in step S5 is as follows: ， In the formula V k The principal component analysis includes the previous k The matrix of eigenvectors, V k T is the transpose of the feature matrix, Y The data after dimensionality reduction obtained after principal component analysis.

6. A Ka-SAR remote sensing image fusion computing device based on texture feature extraction, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the fusion method according to any one of claims 1 to 5 is implemented.

7. A storage device, which is a computer-readable storage device and stores a computer program, characterized in that: When the computer program is executed, the steps of the method according to any one of claims 1 to 5 are implemented.

Images