Airborne dual-frequency high-resolution SAR image fusion method, device, equipment and medium

Abstract

The application relates to an airborne dual-frequency high-resolution SAR image fusion method, device, equipment and medium. The method comprises the following steps: performing multi-scale transformation on a first source image and a second source image which have been accurately registered to obtain low-frequency sub-band coefficients and high-frequency sub-band coefficients in each direction corresponding to the first source image and the second source image; obtaining an observation matrix in the form of additive noise according to the low-frequency sub-band coefficients corresponding to the first source image and the second source image, decomposing the observation matrix by using a non-negative matrix decomposition algorithm to obtain a feature matrix, performing non-negative matrix decomposition on the feature matrix to obtain low-frequency fusion coefficients; combining the high-frequency sub-band coefficients in each direction by using an improved Laplace energy sum to obtain high-frequency fusion coefficients; and performing inverse transformation on the low-frequency fusion coefficients and the high-frequency fusion coefficients to obtain a fusion image. The method can obtain a high-frequency and low-frequency SAR fusion image with higher resolution, more comprehensive information and smaller distortion.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a dual-frequency high-resolution SAR (Synthetic Aperture Radar) image fusion method, device, equipment and medium. BACKGROUND

[0002] In a radar system, the transmission characteristics and the backscattering characteristics of targets of SARs of different frequency bands are different, which results in different limitations and advantages of SAR images of different frequency bands. For the same area, a high-frequency SAR image can obtain clearer scene contours and target detail features, but it has poor penetration and poor imaging effect on hidden targets. A low-frequency SAR image is relatively dull and has poor description ability for target detail features, but it has strong penetration and can better image hidden targets, such as scenes and hidden targets under the ground in a forest. If SAR image data of high and low frequency bands can be fused to realize information complementation, it is beneficial to subsequent SAR image interpretation and target detection and identification.

[0003] A traditional image fusion method generally includes the following steps of processing an image: performing multi-scale transformation on a source image to obtain high-frequency sub-band coefficients and low-frequency sub-band coefficients, respectively adopting a suitable fusion rule to merge the high-frequency sub-band coefficients and the low-frequency sub-band coefficients, and performing an inverse transformation operation on the merged high-frequency and low-frequency fusion coefficients to obtain a fused image. This process involves two key technologies, one is a multi-scale transformation method, and the other is a fusion rule. At present, there are various multi-scale transformation methods such as wavelet transformation, curvelet transformation and nonsubsampled contourlet transform (NSCT), and various fusion rules such as weighted average, absolute value taking the maximum value and region matching.

[0004] However, since a high-resolution SAR image contains a large amount of detail and texture information, how to obtain more sufficient information of a source image in a multi-scale transformation process and reduce information loss, and how to set a more suitable fusion rule to effectively merge high-frequency and low-frequency sub-band coefficients and then obtain better fusion effect are technical problems to be solved. SUMMARY

[0005] Therefore, it is necessary to provide an airborne dual-frequency high-resolution SAR image fusion method, device, equipment and medium in view of the above technical problems.

[0006] An airborne dual-frequency high-resolution SAR image fusion method, the method comprising:

[0007] performing multi-scale transformation on the first source image and the second source image respectively to obtain low frequency sub-band coefficients and high frequency sub-band coefficients in each direction corresponding to the first source image and the second source image, wherein the first source image is a high frequency band high resolution SAR two-dimensional real image, and the second source image is a low frequency band high resolution SAR two-dimensional real image;

[0008] obtaining an observation matrix in the form of additive noise according to the low frequency sub-band coefficients corresponding to the first source image and the second source image, decomposing the observation matrix by using a non-negative matrix factorization algorithm to obtain a first decomposition result, obtaining a feature matrix according to the observation matrix and the first decomposition result, and performing non-negative matrix factorization on the feature matrix to obtain low frequency fusion coefficients;

[0009] performing merging on the high frequency sub-band coefficients in each direction corresponding to the first source image and the second source image by using an improved Laplace energy sum method to obtain high frequency fusion coefficients;

[0010] performing inverse transformation on the low frequency fusion coefficients and the high frequency fusion coefficients to obtain a fusion image.

[0011] In one of the embodiments, the method further comprises: obtaining an observation image according to the low frequency sub-band coefficients corresponding to the first source image and the second source image, wherein the observation image is:

[0012]

[0013] wherein Y(x, y) is the observation image, Y(x, y) = X(x, y)N(x, y), X(x, y) is a real image, and N(x, y) is speckle noise, is the low frequency sub-band coefficient corresponding to the first source image, is the low frequency sub-band coefficient corresponding to the second source image, performing logarithmic transformation on the observation image to obtain an observation matrix in the form of additive noise:

[0014]

[0015]

[0016] wherein L Y is the observation matrix,

[0017] In one of the embodiments, the method further comprises: performing non-negative matrix factorization on the observation matrix to obtain a first decomposition result:

[0018]

[0019] wherein, is a base matrix, is a coefficient matrix, is noise; according to the observation matrix and the first decomposition result, a feature matrix is obtained as:

[0020]

[0021] wherein V X is a feature matrix.

[0022] In one of the embodiments, further comprising: performing non-negative matrix decomposition on the feature matrix to obtain a second decomposition result as:

[0023] V X = WH + ε

[0024] wherein W is a feature base, H is a coefficient base, and ε is an error; according to the second decomposition result, a feature base is obtained; and according to the feature base, a low-frequency fusion coefficient is obtained.

[0025] In one of the embodiments, further comprising: obtaining an improved Laplacian energy sum of each scale and each direction on the first source image according to the improved Laplacian energy of the high-frequency sub-band coefficient of each direction corresponding to the first source image; obtaining an improved Laplacian energy sum of each scale and each direction on the second source image according to the improved Laplacian energy of the high-frequency sub-band coefficient of each direction corresponding to the second source image; and judging the Laplacian energy sum of the first source image and the second source image at the corresponding position by using a pre-set high-frequency sub-band coefficient fusion rule to obtain a high-frequency fusion coefficient.

[0026] In one of the embodiments, further comprising: obtaining an improved Laplacian energy sum of each scale and each direction on the first source image according to the improved Laplacian energy of the high-frequency sub-band coefficient of each direction corresponding to the first source image as:

[0027]

[0028]

[0029] wherein s represents a variable interval, represents a directional sub-band coefficient at position (i, j) in the l-scale k-direction, represents an improved Laplacian energy, represents an improved Laplacian energy sum corresponding to position (i, j) in the l-scale k-direction, and parameters M and N are window sizes of (2M+1)×(2N+1).

[0030] In one of the embodiments, the multi-scale transform is a non-subsampled shearlet transform.

[0031] An airborne dual-frequency high-resolution SAR image fusion device, the device comprising:

[0032] a multi-scale transform module configured to perform multi-scale transform on the first source image and the second source image respectively to obtain low frequency sub-band coefficients and high frequency sub-band coefficients in each direction corresponding to the first source image and the second source image, wherein the first source image is a high frequency band high resolution SAR two-dimensional real image, and the second source image is a low frequency band high resolution SAR two-dimensional real image;

[0033] a low frequency sub-band coefficient fusion module configured to obtain an observation matrix in the form of additive noise according to the low frequency sub-band coefficients corresponding to the first source image and the second source image, decompose the observation matrix by using a non-negative matrix factorization algorithm to obtain a first decomposition result, obtain a feature matrix according to the observation matrix and the first decomposition result, and perform non-negative matrix factorization on the feature matrix to obtain low frequency fusion coefficients;

[0034] a high frequency sub-band coefficient fusion module configured to perform merging on the high frequency sub-band coefficients in each direction corresponding to the first source image and the second source image by using an improved Laplacian energy sum method to obtain high frequency fusion coefficients;

[0035] an image fusion module configured to perform inverse transform on the low frequency fusion coefficients and the high frequency fusion coefficients to obtain a fusion image.

[0036] A terminal device comprises a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0037] perform multi-scale transform on the first source image and the second source image respectively to obtain low frequency sub-band coefficients and high frequency sub-band coefficients in each direction corresponding to the first source image and the second source image, wherein the first source image is a high frequency band high resolution SAR two-dimensional real image, and the second source image is a low frequency band high resolution SAR two-dimensional real image;

[0038] obtain an observation matrix in the form of additive noise according to the low frequency sub-band coefficients corresponding to the first source image and the second source image, decompose the observation matrix by using a non-negative matrix factorization algorithm to obtain a first decomposition result, obtain a feature matrix according to the observation matrix and the first decomposition result, and perform non-negative matrix factorization on the feature matrix to obtain low frequency fusion coefficients;

[0039] perform merging on the high frequency sub-band coefficients in each direction corresponding to the first source image and the second source image by using an improved Laplacian energy sum method to obtain high frequency fusion coefficients;

[0040] perform inverse transform on the low frequency fusion coefficients and the high frequency fusion coefficients to obtain a fusion image.

[0041] A terminal readable storage medium, having stored thereon a computer program, which, when executed by a processor, implements the following steps:

[0042] Respectively, the first source image and the second source image are subjected to multi-scale transformation to obtain low-frequency sub-band coefficients and high-frequency sub-band coefficients in each direction corresponding to the first source image and the second source image; the first source image is a high-frequency band high-resolution SAR two-dimensional real image; and the second source image is a low-frequency band high-resolution SAR two-dimensional real image;

[0043] According to the low-frequency sub-band coefficients corresponding to the first source image and the second source image, an observation matrix in the form of additive noise is obtained, the observation matrix is decomposed by using a non-negative matrix decomposition algorithm to obtain a first decomposition result, a feature matrix is obtained according to the observation matrix and the first decomposition result, and the feature matrix is subjected to non-negative matrix decomposition to obtain low-frequency fusion coefficients;

[0044] The high-frequency sub-band coefficients in each direction corresponding to the first source image and the second source image are subjected to merging by using an improved Laplace energy sum method to obtain high-frequency fusion coefficients;

[0045] The low-frequency fusion coefficients and the high-frequency fusion coefficients are subjected to inverse transformation to obtain a fusion image.

[0046] The above-mentioned airborne dual-frequency high-resolution SAR image fusion method, device, equipment and medium can retain all features of an image and remove redundant information by fusing low-frequency sub-band coefficients by using a non-negative matrix decomposition algorithm. Specifically, according to the low-frequency sub-band coefficients corresponding to the first source image and the second source image, an observation matrix in the form of additive noise is obtained, which can convert a multiplicative noise model of a SAR image into an additive noise form. Then, the observation matrix is decomposed by using a non-negative matrix decomposition algorithm to obtain a first decomposition result. The observation matrix can be represented by a real image and a speckle noise. The first decomposition result contains feature information of the real image corresponding to the low-frequency sub-band coefficients. According to the observation matrix and the first decomposition result, a feature matrix can be obtained. Then, the feature matrix is subjected to non-negative matrix decomposition to obtain low-frequency fusion coefficients. The high-frequency sub-band coefficients are fused by using an improved Laplace energy sum method to retain edge and texture information of an image, thereby obtaining a high-quality fusion image. The embodiment of the present application can obtain a high-frequency band SAR fusion image and a low-frequency band SAR fusion image with higher resolution, more comprehensive information and less distortion. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 It is a flowchart of the airborne dual-frequency high-resolution SAR image fusion method in one embodiment;

[0048] Figure 2A flowchart of a process of NSST (Non-Subsampled Shearlet Transform) in an embodiment;

[0049] Figure 3 Ku-band CSAR (Circular SAR) two-dimensional imaging results of an imaging scene and local area zoom-in views in an embodiment, wherein (a) is a Ku-band CSAR two-dimensional imaging result, (b) is a Ku-band CSAR two-dimensional imaging result of area 1, and (c) is a Ku-band CSAR two-dimensional imaging result of area 2;

[0050] Figure 4 L-band CSAR two-dimensional imaging results of an imaging scene and local area zoom-in views in an embodiment, wherein (a) is an L-band CSAR two-dimensional imaging result, (b) is an L-band CSAR two-dimensional imaging result of area 1, and (c) is an L-band CSAR two-dimensional imaging result of area 2;

[0051] Figure 5 Fusion results of area 1 in an embodiment, wherein (a) is a Ku-band CSAR two-dimensional imaging result of area 1, (b) is an L-band CSAR two-dimensional imaging result of area 1, and (c) is a fusion result of area 1;

[0052] Figure 6 Fusion results of area 2 in an embodiment, wherein (a) is a Ku-band CSAR two-dimensional imaging result of area 2, (b) is an L-band CSAR two-dimensional imaging result of area 2, and (c) is a fusion result of area 2;

[0053] Figure 7 A flowchart of a process of an airborne dual-frequency high-resolution SAR image fusion method in a specific embodiment;

[0054] Figure 8 A structural block diagram of an airborne dual-frequency high-resolution SAR image fusion device in an embodiment;

[0055] Figure 9 An internal structural diagram of a terminal device in an embodiment. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0057] In an embodiment, as shown in FIG. 1, the present application provides an airborne dual-frequency high-resolution SAR image fusion method. Figure 1As shown, an airborne dual-frequency high-resolution SAR image fusion method is provided, comprising the following steps:

[0058] In step 102, multi-scale transformation is performed on the precisely registered first source image and second source image respectively to obtain low-frequency sub-band coefficients and high-frequency sub-band coefficients in each direction corresponding to the first source image and the second source image.

[0059] The first source image is a high-frequency high-resolution SAR two-dimensional real image; and the second source image is a low-frequency high-resolution SAR two-dimensional real image. In the embodiment of the application, NSST is used as a multi-scale decomposition tool to perform multi-scale decomposition on the first source image and the second source image respectively to obtain low-frequency sub-band coefficients and high-frequency sub-band coefficients in each direction The NSST transformation has the advantages of multi-scale, shift invariance and multi-direction tight support structure, and can perform more fine representation on the image, so as to more fully obtain a large amount of detail and texture information contained in the source image.

[0060] In step 104, an observation matrix in the form of additive noise is obtained according to the low-frequency sub-band coefficients corresponding to the first source image and the second source image, the observation matrix is decomposed by using a non-negative matrix decomposition algorithm to obtain a first decomposition result, a feature matrix is obtained according to the observation matrix and the first decomposition result, and the feature matrix is subjected to non-negative matrix decomposition to obtain low-frequency fusion coefficients.

[0061] Compared with some traditional algorithms, the non-negative matrix decomposition (NMF) algorithm has the advantages of simplicity in implementation, interpretability in decomposition form and decomposition result, and less storage space occupation. In order to efficiently process data stored by a matrix, a key necessary step is to perform decomposition operation on the matrix. Through matrix decomposition, on the one hand, the dimension of the matrix describing the problem is reduced, and on the other hand, a large amount of data can be compressed and summarized.

[0062] Since the low-frequency sub-band coefficients are the main and approximate contents of the source image, they correspond to a non-negative matrix, therefore, the non-negative matrix decomposition algorithm is used to calculate the low-frequency sub-band coefficients to obtain low-frequency fusion coefficients after fusion The traditional non-negative matrix decomposition algorithm is based on additive noise, and for SAR images, the noise model is multiplicative noise, therefore, it is necessary to convert the multiplicative noise into additive noise to adapt to the non-negative matrix decomposition.

[0063] The observation matrix is obtained by converting the observation image into additive noise and form, the observation image includes low frequency sub-band coefficients corresponding to the first source image and the second source image, the observation image can be expressed by the product of a real image and noise, the first decomposition result containing feature information corresponding to the real image is obtained by performing non-negative matrix decomposition on the observation matrix, the feature matrix can be obtained from the observation matrix and the first decomposition result, the feature information of the image can be retained to the greatest extent and the clutter background can be effectively suppressed by extracting the feature matrix, and the low frequency fusion coefficient is obtained by further performing non-negative matrix decomposition on the feature information.

[0064] In step 106, the high frequency sub-band coefficients corresponding to each direction of the first source image and the second source image are merged by using the improved Laplacian energy sum method to obtain the high frequency fusion coefficient.

[0065] The high frequency sub-band coefficients reflect the degree of gray scale transformation of the source image, and correspond to strong edges, textures and other significant detail features. The SML (Sum of Modified Laplacian) reflects the edge feature information of the image, and can appropriately reflect the focusing characteristics and clarity of the image to a certain extent. The traditional SML algorithm only calculates the modified Laplacian (ML) value of each pixel point in the horizontal and vertical directions, while the improved Laplacian energy sum (New Sum of Modified Laplacian, NSML) algorithm calculates the modified Laplacian value of each pixel point in eight directions including horizontal, vertical and diagonal lines. Four directions on the diagonal line are added, which effectively suppresses the interference of noise while fully utilizing the contribution of the current pixel point of the image to the current clarity. The NSML fusion algorithm has excellent clarity discrimination ability, and the higher the clarity of the image, the larger the corresponding NSML value. The NSML value is larger, and the NSML is used as the high frequency sub-band coefficient fusion rule to select the larger high frequency fusion coefficient.

[0066] In step 108, the low frequency fusion coefficient and the high frequency fusion coefficient are inversely transformed to obtain the fusion image.

[0067] In the airborne dual-frequency high-resolution SAR image fusion method, the low-frequency sub-band coefficients are fused through the non-negative matrix decomposition algorithm, the whole characteristics of the image can be retained and the redundant information can be removed, specifically, the observation matrix in the form of additive noise is obtained according to the low-frequency sub-band coefficients corresponding to the first source image and the second source image, the noise model of the SAR image in the form of multiplicative noise can be converted into the form of additive noise, then the observation matrix is decomposed by using the non-negative matrix decomposition algorithm to obtain the first decomposition result, the observation matrix can be represented by the real image and the speckle noise, the real image characteristic information corresponding to the low-frequency sub-band coefficients is contained in the first decomposition result, the characteristic matrix can be obtained from the observation matrix and the first decomposition result, then the low-frequency fusion coefficient is obtained by performing the non-negative matrix decomposition on the characteristic matrix, the edge and texture information of the image is retained by fusing the high-frequency sub-band coefficients through the improved Laplace energy, so that the high-quality fused image is obtained. The embodiment of the application can obtain the high-frequency and low-frequency SAR fused image with higher resolution, more comprehensive information and smaller distortion.

[0068] In one embodiment, as shown in FIG. Figure 2 A flowchart of the NSST transformation is provided, and the method further includes: multi-scale transformation into a non-subsampled shearlet transformation. In this embodiment, the NSST includes two steps: multi-scale decomposition based on a non-subsampled pyramid (NSP) and directional localization based on an improved shearlet filter bank (SF). Figure 2 In this embodiment, the number of NSP decomposition layers is 2, in the NSP decomposition, the source image is decomposed by the first layer NSP to obtain the low-frequency sub-band coefficient and the high-frequency sub-band coefficient of the image, then the low-frequency sub-band coefficient is decomposed by the second layer NSP to obtain the low-frequency sub-band coefficient and the high-frequency sub-band coefficient After the NSP decomposition is completed, directional localization is started, that is, the high-frequency coefficient matrix is respectively subjected to directional localization by using the improved shearlet filter, and the specific process is as follows: first, a window function is constructed by using the “Meyer” wavelet to generate a shearlet filter; second, the generated shearlet filter is mapped from the pseudo-polarized coordinate system to the Cartesian coordinate system; then, the high-frequency coefficient matrix is respectively subjected to convolution operation with the shearlet filter to obtain respective directional sub-band coefficients; finally, the high-frequency sub-band coefficients of each direction are obtained through inverse Fourier transform. Through the above operations, the low-frequency sub-band coefficient and the high-frequency sub-band coefficient of each direction of the first source image The low-frequency sub-band coefficient and the high-frequency sub-band coefficient of each direction of the second source image

[0069] In one embodiment, according to the low-frequency sub-band coefficients corresponding to the first source image and the second source image, an observation matrix in additive noise form is obtained, a non-negative matrix factorization algorithm is used to decompose the observation matrix to obtain a first decomposition result, according to the observation matrix and the first decomposition result, a feature matrix is obtained, and the feature matrix is subjected to non-negative matrix factorization to obtain the low-frequency fusion coefficient.

[0070]

[0071] wherein Y(x, y) is the observation image, Y(x, y) = X(x, y)N(x, y), X(x, y) is the real image, N(x, y) is the speckle noise, X(x, y) and N(x, y) are considered to be independent random processes, N(x, y) is an independent identically distributed noise image, and is subject to Gamma distribution, is the low-frequency sub-band coefficient corresponding to the first source image, is the low-frequency sub-band coefficient corresponding to the second source image, the observation image is subjected to logarithmic transformation to obtain the observation matrix in additive noise form:

[0072]

[0073]

[0074] wherein L Y is the observation matrix, the observation matrix is subjected to non-negative matrix factorization to obtain the first decomposition result:

[0075]

[0076] wherein, is the base matrix, is the coefficient matrix, is the noise, in the embodiment of the application, the number of columns of is set to 1; according to the observation matrix and the first decomposition result, the the logarithm of both sides of the formula can be obtained:

[0077]

[0078] thereby, the feature matrix is obtained:

[0079]

[0080] wherein V X is the feature matrix; the feature matrix is subjected to non-negative matrix factorization to obtain a second decomposition result:

[0081] VX = WH + ε

[0082] wherein, W is a feature base, H is a coefficient base, ε is an error, the column number of W is set to R = 1; the feature base is obtained according to the second decomposition result; and the low-frequency fusion coefficient is obtained according to the feature base.

[0083] In the embodiment, when the non-negative matrix decomposition of the observation matrix is performed, the algorithm converges when the noise is reduced to a preset first threshold; when the non-negative matrix decomposition of the feature matrix is performed, the iteration stops when the error is reduced to a preset second threshold, and a unique feature base W is obtained, which contains complete features of the low-frequency sub-band coefficient , i.e., W is the merged low-frequency fusion coefficient The low-frequency sub-band coefficient of the dual-frequency high-resolution SAR image is fused by using the non-negative matrix decomposition algorithm, the feature information in the observation image is extracted, the noise is suppressed, more useful information of the source image can be retained, and more redundant information can be removed.

[0084] In one embodiment, the step of merging the high-frequency sub-band coefficients of each direction corresponding to the first source image and the second source image to obtain the high-frequency fusion coefficient by using the improved Laplace energy sum method comprises: obtaining the improved Laplace energy sum of each scale and each direction on the first source image according to the improved Laplace energy of the high-frequency sub-band coefficients of each direction corresponding to the first source image, i.e.,

[0085]

[0086]

[0087] wherein, s represents a variable interval, represents the directional sub-band coefficient at position (i, j) in the l scale k direction, represents the improved Laplace energy, represents the improved Laplace energy sum corresponding to the position (i, j) in the l scale k direction, the parameters M and N are the window size (2M+1)×(2N+1); the improved Laplace energy sum of each scale and each direction on the second source image is obtained according to the improved Laplace energy of the high-frequency sub-band coefficients of each direction corresponding to the second source image; the high-frequency fusion coefficient is obtained by using the preset high-frequency sub-band coefficient fusion rule to judge the Laplace energy sum of the first source image and the second source image at the corresponding position. The NSML is taken as the high-frequency sub-band coefficient fusion rule, and the definition is as follows:

[0088]

[0089] wherein, represents a high-frequency sub-band coefficient with a scale of l, a direction of k, and coordinates of (i, j) in the set; represents a high-frequency sub-band coefficient with a scale of l, a direction of k, and coordinates of (i, j) in the set; represents a fused high-frequency sub-band coefficient at a scale of l, a direction of k, and coordinates of (i, j), and a high-frequency fusion coefficient is obtained according to the fused high-frequency sub-band coefficient

[0090] In the embodiment, the improved Laplacian energy sum method has a strong ability to capture high-frequency sub-band edge coefficients, and the fusion of the high-frequency sub-band coefficients of the dual-frequency high-resolution SAR image by the improved Laplacian energy sum method can help to obtain a fused image with richer texture and higher visual quality, so that the fused image has stronger contrast and clearer texture.

[0091] Finally, the NSST inverse transform operation is performed on the calculated low-frequency fusion coefficient and high-frequency fusion coefficient, and the parameters are set corresponding to the NSST transform, so that the dual-frequency high-resolution SAR fused image can be obtained.

[0092] In one specific embodiment, as shown in FIG. 1, first, the NSST transform is used to perform multi-scale decomposition on the precisely registered first source image and second source image respectively to obtain corresponding low-frequency sub-band coefficients and high-frequency sub-band coefficients in each direction. Figure 7 Since the low-frequency sub-band coefficients can reflect the general outline of the source image, can retain the regional characteristics of the source image, and can concentrate most of the energy of the source image, the low-frequency fusion rule based on the non-negative matrix factorization algorithm is used to fuse the low-frequency sub-band coefficients of the first source image and the second source image to obtain a low-frequency fusion coefficient, and the low-frequency fusion rule based on the non-negative matrix factorization algorithm can more truly and accurately reflect the outline of the initial image. Since the high-frequency sub-band coefficients can reflect the detail and edge component of the image, and the distribution of the high-frequency sub-band coefficients has obvious directionality, the high-frequency fusion rule based on the improved Laplacian energy sum method is used to calculate the Laplacian energy sum of each position in each scale and each direction, and then determine the high-frequency fusion coefficient, which can effectively extract the outline information of the image. Finally, the NSST inverse transform is used to reconstruct the fused high-frequency and low-frequency fusion coefficients to obtain a fused image F.

[0093] In one embodiment, the present application is verified by a measured data experiment, and the measured data experiment result proves the effectiveness of the present application. In the measured data experiment, airborne L-band (low frequency band) and Ku-band (high frequency band) CSAR two-dimensional real images are used, the imaging scene is an 800m×800m (X-axis direction×Y-axis direction) urban intersection, the center of the imaging scene is the origin of the Cartesian coordinate system, Figure 3are Ku-band CSAR two-dimensional imaging results of an imaging scene and local area zoom-in pictures, wherein (a) is a Ku-band CSAR two-dimensional imaging result, (b) is a Ku-band CSAR two-dimensional imaging result of area 1, (c) is a Ku-band CSAR two-dimensional imaging result of area 2, wherein the horizontal direction is the X-axis direction and the vertical direction is the Y-axis direction. Figure 3 It can be seen from (a), (b) and (c) that the Ku-band CSAR image presents the appearance of the imaging scene, which is somewhat similar to the optical image, and the scene contour and target detail features can be clearly seen. Figure 4 are L-band CSAR two-dimensional imaging results of an imaging scene and local area zoom-in pictures, wherein (a) is an L-band CSAR two-dimensional imaging result, (b) is an L-band CSAR two-dimensional imaging result of area 1, (c) is an L-band CSAR two-dimensional imaging result of area 2, wherein the horizontal direction is the X-axis direction and the vertical direction is the Y-axis direction. Figure 4 It can be seen from (a), (b) and (c) that the L-band CSAR image is relatively dim and has poor description ability for target detail features, but it has strong penetration and can better image hidden targets, such as scenes and hidden targets in the forest or under the ground. Figure 5 are fusion results of area 1, wherein (a) is a Ku-band CSAR two-dimensional imaging result of area 1, (b) is an L-band CSAR two-dimensional imaging result of area 1, (c) is a fusion result of area 1, and (c) represents the fusion result of (a) and (b). Figure 5 In (a), only the overall contour of the green belt can be seen, and the scenes covered by the vegetation in the green belt are not well presented in the figure, Figure 5 In (b), the scenes covered by the vegetation can be clearly presented, but the overall contour of the green belt is not shown, and the demarcation line between the green belt and the road surface cannot be determined. After the fusion of (a) and (b) by the method of the present application, it can be seen from (c) that not only the scenes covered by the vegetation in the green belt area can be presented, but also the demarcation line between the green belt and the road surface can be clearly seen, and the information of the area is fully presented. Figure 6The display is the fusion result of region 2. Among them, (a) is the Ku-band CSAR two-dimensional imaging result of region 2, (b) is the L-band CSAR two-dimensional imaging result of region 2, (c) is the fusion result of region 2, and (c) represents the fusion result of (a) and (b). In the figure, the blue circle region is a certain artificial ground object. In (a), it can only be seen that the artificial ground object is a polygon object. As for whether there is an object in the middle region and the distribution, it cannot be presented in the figure. In (b), the respective situation of the middle region can be clearly seen. However, the overall contour is relatively dark and cannot be clearly distinguished. After the fusion of (a) and (b) by the method, it can be seen from (c) that the overall contour of the artificial ground object and the respective situation of the internal object can be well presented.

[0094] It can be known from the fusion result of the L-band and Ku-band CSAR images that the method can realize the fusion of high and low frequency band SAR images, and better fusion effect can be obtained, which is beneficial to subsequent SAR image interpretation and target detection and recognition, and has good practicability.

[0095] It should be understood that although each step in the flowcharts of Figure 1 and Figure 7 is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, Figure 1 and Figure 7 at least part of the steps can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these sub-steps or stages is not necessarily sequential, but can be alternately executed with at least part of other steps or sub-steps or stages of other steps.

[0096] In one embodiment, as shown in Figure 8 , an airborne dual-frequency high-resolution SAR image fusion device is provided, comprising: a multi-scale transformation module 802, a low-frequency sub-band coefficient fusion module 804, a high-frequency sub-band coefficient fusion module 806 and an image fusion module 808, wherein:

[0097] The multi-scale transformation module 802 is configured to perform multi-scale transformation on the first source image and the second source image which have been precisely registered, to obtain low-frequency sub-band coefficients and high-frequency sub-band coefficients in each direction corresponding to the first source image and the second source image; the first source image is a high-frequency band high-resolution SAR two-dimensional real image; and the second source image is a low-frequency band high-resolution SAR two-dimensional real image.

[0098] The low-frequency sub-band coefficient fusion module 804 is configured to obtain an observation matrix in the form of additive noise according to the low-frequency sub-band coefficients corresponding to the first source image and the second source image, decompose the observation matrix by using a non-negative matrix factorization algorithm to obtain a first decomposition result, obtain a feature matrix according to the observation matrix and the first decomposition result, and perform non-negative matrix factorization on the feature matrix to obtain low-frequency fusion coefficients.

[0099] The high-frequency sub-band coefficient fusion module 806 is configured to perform merging on the high-frequency sub-band coefficients of each direction corresponding to the first source image and the second source image by using an improved Laplacian energy sum method to obtain high-frequency fusion coefficients.

[0100] The image fusion module 808 is configured to perform inverse transformation on the low-frequency fusion coefficients and the high-frequency fusion coefficients to obtain a fusion image.

[0101] In one of the embodiments, the low-frequency sub-band coefficient fusion module 804 is further configured to obtain an observation image according to the low-frequency sub-band coefficients corresponding to the first source image and the second source image.

[0102]

[0103] wherein Y(x, y) is the observation image, Y(x, y) = X(x, y)N(x, y), X(x, y) is a real image, and N(x, y) is coherent speckle noise, is the low-frequency sub-band coefficient corresponding to the first source image, is the low-frequency sub-band coefficient corresponding to the second source image, the observation image is logarithmically transformed to obtain an observation matrix in the form of additive noise as follows:

[0104]

[0105]

[0106] wherein L Y is the observation matrix,

[0107] In one of the embodiments, the low-frequency sub-band coefficient fusion module 804 is further configured to perform non-negative matrix factorization on the observation matrix to obtain a first decomposition result as follows:

[0108]

[0109] wherein is a basis matrix, is a coefficient matrix, is noise; and an expression of the basis matrix is obtained according to the observation matrix and the first decomposition result as follows:

[0110]

[0111] According to the expression of the basis matrix, the characteristic matrix is obtained as follows:

[0112]

[0113] wherein V X is the characteristic matrix.

[0114] In one of the embodiments, the low-frequency sub-band coefficient fusion module 804 is further configured to perform non-negative matrix factorization on the characteristic matrix, and obtain a second factorization result as follows:

[0115] V X = WH + ε

[0116] wherein W is a characteristic basis, H is a coefficient basis, and ε is an error; the characteristic basis is obtained according to the second factorization result; and the low-frequency fusion coefficient is obtained according to the characteristic basis.

[0117] In one of the embodiments, the high-frequency sub-band coefficient fusion module 806 is further configured to obtain the improved Laplacian energy sum of each scale and each direction on the first source image according to the improved Laplacian energy of the high-frequency sub-band coefficient of each direction corresponding to the first source image, obtain the improved Laplacian energy sum of each scale and each direction on the second source image according to the improved Laplacian energy of the high-frequency sub-band coefficient of each direction corresponding to the second source image, and determine the Laplacian energy sum of the first source image and the second source image at the corresponding position by using a pre-set high-frequency sub-band coefficient fusion rule, to obtain the high-frequency fusion coefficient.

[0118] In one of the embodiments, the high-frequency sub-band coefficient fusion module 806 is further configured to obtain the improved Laplacian energy sum of each scale and each direction on the first source image according to the improved Laplacian energy of the high-frequency sub-band coefficient of each direction corresponding to the first source image, obtain the improved Laplacian energy sum of each scale and each direction on the second source image according to the improved Laplacian energy of the high-frequency sub-band coefficient of each direction corresponding to the second source image, and determine the Laplacian energy sum of the first source image and the second source image at the corresponding position by using a pre-set high-frequency sub-band coefficient fusion rule, to obtain the high-frequency fusion coefficient.

[0119]

[0120]

[0121] wherein s represents a variable interval, represents a directional sub-band coefficient at position (i, j) in the l-scale k-direction, represents an improved Laplacian energy, represents an improved Laplacian energy sum corresponding to position (i, j) in the l-scale k-direction, and parameters M and N are window sizes of (2M+1)×(2N+1).

[0122] In one of the embodiments, the multi-scale transform module 802 is further configured to perform multi-scale transform as non-subsampled shearlet transform.

[0123] The specific limitations of the airborne dual-frequency high-resolution SAR image fusion device can be referred to the limitations of the airborne dual-frequency high-resolution SAR image fusion method in the foregoing, which will not be repeated here. Each module in the airborne dual-frequency high-resolution SAR image fusion device described above can be realized by software, hardware and combinations thereof, in whole or in part. The above-mentioned modules can be embedded in or independent of the processor in the terminal device in hardware form, or can be stored in the memory in the terminal device in software form, so as to be called and executed by the processor to perform the operations corresponding to each of the above modules.

[0124] In one embodiment, a terminal device, which can be a terminal, has an internal structure diagram as shown in Figure 9 The terminal device includes a processor, a memory, a network interface, a display screen and an input device connected by a system bus. The processor of the terminal device is used to provide computing and control capabilities. The memory of the terminal device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the terminal device is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement an airborne dual-frequency high-resolution SAR image fusion method. The display screen of the terminal device can be a liquid crystal display screen or an electronic ink display screen. The input device of the terminal device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the terminal device shell, or an external keyboard, touchpad or mouse, etc.

[0125] Those skilled in the art can understand that Figure 9 The structure shown in the foregoing is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the terminal device to which the scheme of the present application is applied. The specific terminal device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0126] In one embodiment, a computer device is provided, which includes a memory and a processor. The memory stores a computer program. The processor executes the computer program to implement the steps of the method in the above-mentioned embodiments.

[0127] In one embodiment, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the steps of the method in the above-mentioned embodiments.

[0128] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, databases, or other media in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0129] The technical features of the above embodiments can be combined in any way. In order to make the description simple, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the present application.

[0130] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of the patent of the present application should be subject to the appended claims.

Claims

1. An airborne dual-frequency high-resolution SAR image fusion method, characterized in that, The method comprises: respectively performing multi-scale transformation on the precisely registered first source image and second source image to obtain low-frequency sub-band coefficients and high-frequency sub-band coefficients in each direction corresponding to the first source image and the second source image; the first source image is a high-frequency band high-resolution SAR two-dimensional real image; and the second source image is a low-frequency band high-resolution SAR two-dimensional real image; obtaining an observation matrix in the form of additive noise according to the low-frequency sub-band coefficients corresponding to the first source image and the second source image, decomposing the observation matrix by using a non-negative matrix decomposition algorithm to obtain a first decomposition result, obtaining a feature matrix according to the observation matrix and the first decomposition result, and performing non-negative matrix decomposition on the feature matrix to obtain low-frequency fusion coefficients; performing merging on the high-frequency sub-band coefficients in each direction corresponding to the first source image and the second source image by using an improved Laplace energy sum method to obtain high-frequency fusion coefficients; performing inverse transformation on the low-frequency fusion coefficients and the high-frequency fusion coefficients to obtain a fusion image; the step of performing merging on the high-frequency sub-band coefficients in each direction corresponding to the first source image and the second source image by using an improved Laplace energy sum method to obtain high-frequency fusion coefficients comprises: obtaining improved Laplace energy sums in each scale and each direction on the first source image according to improved Laplace energies of the high-frequency sub-band coefficients in each direction corresponding to the first source image; obtaining improved Laplace energy sums in each scale and each direction on the second source image according to improved Laplace energies of the high-frequency sub-band coefficients in each direction corresponding to the second source image; judging Laplace energy sums of the first source image and the second source image at corresponding positions by using a pre-set high-frequency sub-band coefficient fusion rule to obtain high-frequency fusion coefficients; the step of obtaining improved Laplace energy sums in each scale and each direction on the first source image according to improved Laplace energies of the high-frequency sub-band coefficients in each direction corresponding to the first source image comprises: the step of obtaining improved Laplace energy sums in each scale and each direction on the first source image according to improved Laplace energies of the high-frequency sub-band coefficients in each direction corresponding to the first source image is: wherein, denotes a variable interval, denotes a scale a position in a direction is a directional subband coefficient, denotes an improved Laplacian energy, denotes a scale a position in a direction is a corresponding improved Laplacian energy and a parameter and a window size is .

2. The method of claim 1, wherein, the step of obtaining an observation matrix in the form of additive noise according to the low-frequency sub-band coefficients corresponding to the first source image and the second source image comprises: obtaining an observation image according to the low-frequency sub-band coefficients corresponding to the first source image and the second source image is: wherein, is an observed image, , is a real image, is a speckle noise, is a low frequency subband coefficient corresponding to the first source image, is a low frequency subband coefficient corresponding to the second source image, , ; performing logarithmic transformation on the observation image to obtain an observation matrix in the form of additive noise is: wherein is an observation matrix, .

3. The method according to any of claims 1 or 2, characterized in that, the step of decomposing the observation matrix by using a non-negative matrix decomposition algorithm to obtain a first decomposition result, and obtaining a feature matrix according to the observation matrix and the first decomposition result comprises: performing non-negative matrix decomposition on the observation matrix to obtain a first decomposition result is: wherein is a base matrix, is a coefficient matrix, is noise; obtaining a feature matrix according to the observation matrix and the first decomposition result is: wherein is a feature matrix.

4. The method of claim 3, wherein, the step of performing non-negative matrix decomposition on the feature matrix to obtain low-frequency fusion coefficients comprises: performing non-negative matrix decomposition on the feature matrix to obtain a second decomposition result is: wherein, is a feature base, is a coefficient base, is an error; According to the second decomposition result, a characteristic matrix is obtained, and according to the characteristic matrix, a low-frequency fusion coefficient is obtained.

5. The method of claim 1, wherein, The multi-scale transform is a non-subsampled shearlet transform.

6. An airborne dual-frequency high-resolution SAR image fusion device applied to the method according to any one of claims 1-5, characterized in that, The device comprises: A multi-scale transform module is configured to perform multi-scale transform on the precisely registered first source image and second source image respectively to obtain low-frequency sub-band coefficients and high-frequency sub-band coefficients in each direction corresponding to the first source image and the second source image; the first source image is a high-frequency high-resolution SAR two-dimensional real image; and the second source image is a low-frequency high-resolution SAR two-dimensional real image. A low-frequency sub-band coefficient fusion module is configured to obtain an observation matrix in the form of additive noise according to the low-frequency sub-band coefficients corresponding to the first source image and the second source image, decompose the observation matrix by using a non-negative matrix decomposition algorithm to obtain a first decomposition result, obtain a characteristic matrix according to the observation matrix and the first decomposition result, and perform non-negative matrix decomposition on the characteristic matrix to obtain a low-frequency fusion coefficient. A high-frequency sub-band coefficient fusion module is configured to perform merging on the high-frequency sub-band coefficients in each direction corresponding to the first source image and the second source image by using an improved Laplacian energy sum method to obtain a high-frequency fusion coefficient. An image fusion module is configured to perform inverse transform on the low-frequency fusion coefficient and the high-frequency fusion coefficient to obtain a fused image. 7.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-6 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the method in any one of claims 1 to 5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 5.

Images