A multispectral image panchromatic sharpening method, system, device, and medium

Abstract

The application belongs to the technical field of remote sensing image processing, and discloses a multispectral image panchromatic sharpening method, system, device and medium; wherein the multispectral image panchromatic sharpening method comprises the following steps: performing IHS transformation decomposition on a to-be-processed multispectral image to obtain an I component representing image spatial information and H and S components representing spectral information; performing linear stretching on a panchromatic image corresponding to the to-be-processed multispectral image according to the obtained I component to obtain an equalized panchromatic image; performing multi-resolution decomposition on the obtained I component and the equalized panchromatic image respectively using NSST, and fusing low-frequency coefficients of the I component and low-frequency coefficients of the equalized panchromatic image based on a regional low-rank representation method to obtain fused low-frequency coefficients; fusing high-frequency coefficients of the I component and high-frequency coefficients of the equalized panchromatic image to obtain fused high-frequency coefficients; performing NSST inverse transformation based on the fused high-frequency coefficients and the fused low-frequency coefficients to obtain a fused I component; and performing IHS inverse transformation based on the fused I component, the H component and the S component to obtain a panchromatic sharpened image. The application combines the advantages of NSST, multi-resolution analysis and regional low-rank representation, and can accurately retain overall structural information of a source image, thereby significantly improving spatial and spectral quality of a sharpened image.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing technology, and specifically relates to a method, system, device and medium for panchromatic sharpening of multispectral images. Background Technology

[0002] Panchromatic sharpening refers to sharpening a low-spatial-resolution multi-channel multispectral image (MS) using a single-channel panchromatic image (PAN) with high spatial resolution, in order to generate a multi-channel multispectral image with high spatial resolution. Currently, commonly used multispectral image panchromatic sharpening methods are divided into spatial domain-based and transform domain-based methods.

[0003] Classical spatial domain-based methods include weighted average, principal component analysis (PCA), intensity-hue-saturation (IHS) transform, and Brovy transform (BT). PCA-based image fusion methods analyze the image from a statistical perspective, extracting the first principal component which represents the most spatially rich part of the image; however, these methods often lead to spectral distortion. IHS-based methods improve spatial resolution and enhance color, but limit the number of bands in the source image. BT-based methods distribute the spectral information of each band of the multispectral image to the fused image according to a simple proportional relationship, typically resulting in the loss of spectral information.

[0004] Classical transformation-domain-based methods mainly include pyramid transform-based methods, wavelet transform-based methods, and other multi-resolution transform-based methods. Among them, wavelet transform-based methods solve the redundancy problem between decomposed sub-images and have good time-frequency characteristics, but lack anisotropy. The subsequently proposed Non-Subsample Shearlet Transform (NSST) is computationally efficient, avoids the pseudo-Gibbs effect, and has translation invariance, providing good approximation of curve edges. This type of method uses NSST to decompose the source image and then fuses the obtained low-frequency and high-frequency coefficients respectively. This type of method makes full use of the multi-resolution decomposition capability of NSST and injects the extracted panchromatic image detail information into the sharpened image, which can improve the spatial quality of the sharpened image to a certain extent. However, this type of method is not good at preserving the structural information contained in the low-frequency coefficients and the detail information contained in the high-frequency coefficients, which leads to spectral distortion and insufficient spatial detail. Summary of the Invention

[0005] The purpose of this invention is to provide a multispectral image panchromatic sharpening method, system, device, and medium to solve one or more of the aforementioned technical problems. Specifically, the technical solution provided by this invention is a multispectral image panchromatic sharpening scheme based on NSST and regional low-rank representation. It combines the advantages of NSST's multi-resolution analysis and regional low-rank representation's ability to accurately preserve the overall structural information of the source image. Compared to existing traditional methods, it can significantly improve the spatial and spectral quality of the sharpened image.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a method for panchromatic sharpening of a multispectral image, comprising the following steps:

[0008] The multispectral image to be processed is decomposed by IHS transform to obtain the I component representing the spatial information of the image and the H and S components representing the spectral information.

[0009] Based on the obtained I component, the panchromatic image corresponding to the multispectral image to be processed is linearly stretched to obtain the equalized panchromatic image.

[0010] The obtained I component and the equalized panchromatic image were decomposed into multi-resolution values ​​using NSST to obtain the low-frequency and high-frequency coefficients of the I component and the low-frequency and high-frequency coefficients of the equalized panchromatic image.

[0011] Based on the regional low-rank representation method, the low-frequency coefficients of the I component and the low-frequency coefficients of the equalized panchromatic image are fused to obtain the fused low-frequency coefficients. The high-frequency coefficients of the I component and the high-frequency coefficients of the equalized panchromatic image are classified into texture high-frequency coefficients and salient high-frequency coefficients, and fused using high-frequency fusion rules based on visual sensitivity coefficients and energy matching measurement coefficients, respectively, to obtain the fused high-frequency coefficients.

[0012] The NSST inverse transform is performed based on the fused high-frequency coefficients and the fused low-frequency coefficients to obtain the fused I component; the IHS inverse transform is performed based on the fused I component, the H component and the S component to obtain the panchromatic sharpened image.

[0013] A further improvement to the method of the present invention is that the step of fusing the low-frequency coefficients of the I component and the low-frequency coefficients of the equalized panchromatic image based on the regional low-rank representation method to obtain the fused low-frequency coefficients specifically includes:

[0014] Clustering of multispectral images yields several image regions;

[0015] The low-frequency coefficients of the I component and the low-frequency coefficients of the equalized panchromatic image are represented by low-rank formulas to obtain the low-rank matrix of the low-frequency coefficients of the I component and the low-rank matrix of the low-frequency coefficients of the equalized panchromatic image.

[0016] The low-frequency coefficients and low-rank coefficient matrix of the I component are fused with the low-frequency coefficients and low-rank coefficient matrix of the equalized panchromatic image by adopting the rule of taking the larger l1 norm, so as to obtain the fused low-frequency coefficients.

[0017] The expression for the formula used in low-rank representation is:

[0018]

[0019] stX = XZ + E;

[0020] In the formula, X = [x1, x2, ..., x p [For a given space R] D X is a data matrix where each vector represents a data sample; each data sample in X can be represented by a dictionary D = [d1, d2, ..., dn]. q The atoms in the array are represented linearly, X = DZ; Z = [z1, z2, ..., z]. p [] is the low-rank matrix corresponding to X; E is the noise matrix; ||·|| * The nuclear norm is represented by λ; λ > 0 is the equilibrium parameter. for Norm.

[0021] A further improvement of the method of the present invention is that a simple linear iterative clustering method is used when clustering multispectral images.

[0022] A further improvement to the method of the present invention is that the expression for fusing the low-frequency coefficient low-rank coefficient matrix of the I component and the low-frequency coefficient low-rank coefficient matrix of the equalized panchromatic image using the rule of taking the larger l1 norm is as follows:

[0023]

[0024] In the formula, This represents the j-th column of the fused low-rank coefficient matrix; Z represents the low-rank matrix of the low-frequency coefficients of the panchromatic image after equalization. PAN The j-th column; Z represents the low-frequency sparse low-rank matrix of the I component. I The j-th column; This represents the l1 norm of the column vectors of a matrix.

[0025] A further improvement to the method of the present invention is that the step of classifying the high-frequency coefficients of the I component and the high-frequency coefficients of the equalized panchromatic image into texture high-frequency coefficients and salient high-frequency coefficients, and fusing them respectively using high-frequency fusion rules based on visual sensitivity coefficients and energy matching measurement coefficients to obtain the fused high-frequency coefficients includes:

[0026] Based on the richness of texture information in the high-frequency coefficients, they are divided into texture high-frequency coefficients and significant high-frequency coefficients;

[0027] A fusion rule based on visual sensitivity coefficients is used to fuse high-frequency coefficients of texture; a fusion rule based on energy matching measurement coefficients is used to fuse significantly high-frequency coefficients.

[0028] A further improvement of the method of the present invention is that, in the step of fusing high-frequency coefficients of texture using a fusion rule based on visual sensitivity coefficients, the fusion rule is expressed as follows:

[0029] in,

[0030]

[0031]

[0032]

[0033]

[0034]

[0035]

[0036] In the formula, Let I be the subband coefficient located at (x,y) with a decomposition direction number of l for the I component at the k-th decomposition. For the panchromatic image at the k-th level decomposition, the directional sub-band coefficients with a decomposition direction number of l are given. To sharpen the image at the k-th decomposition, the directional subband coefficients located at (x,y) with decomposition direction number l; VSC I (x,y) represents the visual sensitivity coefficient of the I component at position (x,y); VSC PAN (x,y) represents the visual sensitivity coefficient of the panchromatic image, VSC. F (x,y) represents the visual sensitivity coefficient of the sharpened image; The average value of the low-frequency subband coefficients of the I component at (x,y) under the k-th level decomposition is given. This represents the average value of the low-frequency subband coefficients located at (x,y) in the k-th level decomposition of the panchromatic image. The average value of the low-frequency subband coefficients at (x,y) in the k-th level decomposition of the sharpened image is given; M×N is the size of the adjacent region, and r,c is the offset distance of each point in the region from the center (x,y).

[0037] A further improvement of the method of the present invention is that, in the step of fusing significantly high-frequency coefficients using a fusion rule based on energy matching measurement coefficients, the fusion rule is expressed as follows:

[0038]

[0039]

[0040] In the formula, Let I be the directional subband coefficient with a decomposition direction number of l under the k-th level decomposition of the I component; For the panchromatic image at the k-th level decomposition, the directional sub-band coefficients with a decomposition direction number of l are given. To sharpen the image, the directional subband coefficients with l decomposition directions at the k-th decomposition level are used; R is the weight coefficient; T is the threshold; Ex I k,l (x,y) represents the neighborhood coefficient difference of the region corresponding to the directional sub-band with l decomposition directions under the k-th level decomposition of the I component; Ex PAN k,l (x,y) represents the neighborhood coefficient difference of the region corresponding to the directional sub-band with l decomposition directions in the panchromatic image at the k-th level decomposition; Ex F k,l (x,y) represents the neighborhood coefficient difference of the region corresponding to the direction sub-band with l decomposition directions in the k-th level decomposition of the sharpened image; M×N represents the size of the adjacent region; r,c represents the offset distance of each point in the region from the center (x,y).

[0041] In a second aspect, the present invention provides a multispectral image panchromatic sharpening system, comprising:

[0042] The IHS transform decomposition module is used to perform IHS transform decomposition on the multispectral image to be processed to obtain the I component representing the spatial information of the image and the H and S components representing the spectral information.

[0043] The linear stretching module is used to linearly stretch the panchromatic image corresponding to the multispectral image to be processed based on the obtained I component, so as to obtain the equalized panchromatic image.

[0044] The multi-resolution decomposition module is used to perform multi-resolution decomposition on the obtained I component and the equalized panchromatic image using NSST, to obtain the low-frequency coefficients and high-frequency coefficients of the I component and the low-frequency coefficients and high-frequency coefficients of the equalized panchromatic image.

[0045] The fusion module is used to fuse the low-frequency coefficients of the I component and the low-frequency coefficients of the equalized panchromatic image based on the region low-rank representation method to obtain the fused low-frequency coefficients; the high-frequency coefficients of the I component and the high-frequency coefficients of the equalized panchromatic image are classified into texture high-frequency coefficients and salient high-frequency coefficients, and fused using high-frequency fusion rules based on visual sensitivity coefficients and energy matching measurement coefficients respectively to obtain the fused high-frequency coefficients.

[0046] The inverse transform module is used to perform NSST inverse transform based on the fused high-frequency coefficients and fused low-frequency coefficients to obtain the fused I component; and to perform IHS inverse transform based on the fused I component, the H component and the S component to obtain a panchromatic sharpened image.

[0047] A third aspect of the present invention provides an electronic device comprising:

[0048] At least one processor; and,

[0049] A memory communicatively connected to the at least one processor; wherein,

[0050] The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the multispectral image panchromatic sharpening method as described in any one of the first aspects of the present invention.

[0051] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the multispectral image panchromatic sharpening method described in any one aspect of the present invention.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] The method provided by this invention is a multispectral image panchromatic sharpening method based on NSST and regional low-rank representation. It combines the advantages of NSST in multi-resolution analysis and regional low-rank representation in accurately preserving the overall structural information of the source image. Compared with existing traditional methods, it can significantly improve the spatial and spectral quality of the sharpened image. Specifically, (1) in response to the problem that the NSST-based sharpening method does not retain the structural information contained in the low-frequency coefficients, the method of this invention proposes a regional low-rank representation method in the low-frequency coefficient fusion process, which can better preserve the overall structural information of the source image and improve the spectral quality of the sharpened image; (2) in response to the problem that the NSST-based sharpening method does not retain the detail information contained in the high-frequency coefficients, this invention divides them into texture high-frequency coefficients and significant high-frequency coefficients, and designs a fusion rule based on visual sensitivity coefficients and energy matching measurement coefficients according to their characteristics, which can improve the spatial quality of the sharpened image and enhance the detail information of the sharpened image. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art are briefly introduced below; obviously, the drawings described below are some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0055] Figure 1 This is a flowchart illustrating a multispectral image panchromatic sharpening method in an embodiment of the present invention;

[0056] Figure 2 This is a schematic diagram of NSST; where, Figure 2 (a) is a schematic diagram of NSST decomposition. Figure 2 (b) is a schematic diagram of the ideal NSST spectrum partitioning;

[0057] Figure 3 This is a schematic diagram of a multispectral image panchromatic sharpening method in a specific embodiment of the present invention;

[0058] Figure 4 This is a schematic diagram of the SLIC clustering partitioning results of a multispectral image in an embodiment of the present invention;

[0059] Figure 5 This is a schematic diagram of the fusion rules in an embodiment of the present invention;

[0060] Figure 6 This is a partial image diagram of the dataset in an embodiment of the present invention; wherein, Figure 6 Image (a) in the middle is an MS image from the first set of sharpened image data sources. Figure 6 Image (b) is the PAN image from the first set of sharpened image data sources. Figure 6 Image (c) in the middle is an MS image from the second set of sharpened image data sources. Figure 6 Image (d) is the PAN image from the second set of sharpened image data sources. Figure 6 Image (e) in the middle is an MS image from the third group of sharpened image data sources. Figure 6 Image (f) in the middle is the PAN image from the third group of sharpened image data sources. Figure 6 Image (g) is the MS image from the fourth group of sharpened image data sources. Figure 6 The image in the middle (h) is the PAN image from the fourth group of sharpened image data sources. Figure 6 In the middle (i), the image is an MS image from the fifth group of sharpened image data sources. Figure 6 (j) is the PAN image from the fifth group of sharpened image data sources. Figure 6 (k) represents the MS image from the sixth group of sharpened image data sources. Figure 6 (l) is the PAN image from the sixth group of sharpened image data sources;

[0061] Figure 7 This is a schematic diagram of the experimental results of the first group of sharpened images in an embodiment of the present invention; wherein, Figure 7 (a) shows a schematic diagram of the experimental results of the GS method. Figure 7 (b) is a schematic diagram of the experimental results of the PCA method. Figure 7 (c) is a schematic diagram of the experimental results of the BT method. Figure 7 (d) is a schematic diagram of the experimental results of the IHS method. Figure 7 (e) is a schematic diagram of the experimental results of the NSST method. Figure 7 (f) is a schematic diagram of the experimental results of the LRR method. Figure 7 In the middle (g), a schematic diagram of the experimental results of the NSST+SR method is shown. Figure 7 The diagram in Figure (h) shows the experimental results of the NSST+LRR method. Figure 7 Image (i) is a schematic diagram of the experimental results of the method of the present invention;

[0062] Figure 8 This is a schematic diagram of the experimental results of the second group of sharpened images in an embodiment of the present invention; wherein, Figure 8 (a) shows a schematic diagram of the experimental results of the GS method. Figure 8 (b) is a schematic diagram of the experimental results of the PCA method. Figure 8 (c) is a schematic diagram of the experimental results of the BT method. Figure 8 (d) is a schematic diagram of the experimental results of the IHS method. Figure 8 (e) is a schematic diagram of the experimental results of the NSST method. Figure 8 (f) is a schematic diagram of the experimental results of the LRR method. Figure 8 In the middle (g), a schematic diagram of the experimental results of the NSST+SR method is shown. Figure 8 The diagram in Figure (h) shows the experimental results of the NSST+LRR method. Figure 8 Image (i) is a schematic diagram of the experimental results of the method of the present invention;

[0063] Figure 9 This is a line graph illustrating the evaluation metrics of the dataset experiment in this embodiment of the invention; wherein, Figure 9 (a) is a line graph diagram of the evaluation index based on information entropy. Figure 9 (b) is a line graph diagram of the evaluation index based on standard deviation. Figure 9 (c) is a line graph diagram of the evaluation index based on root mean square error. Figure 9 (d) is a line graph diagram of the evaluation index based on the average gradient. Figure 9 (e) is a line graph diagram of the evaluation index based on spatial frequency. Figure 9(f) is a line graph illustrating the evaluation index based on the general image quality index. Figure 9 (g) is a line graph diagram of the evaluation index based on spectral angle plotting. Figure 9 The graph in the middle (h) is a broken line diagram of the evaluation index based on spectral distortion. Figure 9 (i) is a line graph diagram of the evaluation index based on the relative dimensionless global error;

[0064] Figure 10 This is a schematic diagram of a multispectral image panchromatic sharpening system in an embodiment of the present invention. Detailed Implementation

[0065] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0066] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0067] The present invention will now be described in further detail with reference to the accompanying drawings:

[0068] Please see Figure 1 The present invention provides a multispectral image panchromatic sharpening method, comprising the following steps:

[0069] Step 1: Perform IHS transform decomposition on the multispectral image to be processed to obtain the I component representing the spatial information of the image and the H and S components representing the spectral information.

[0070] Step 2: Based on the I component obtained in Step 1, linearly stretch the panchromatic image corresponding to the multispectral image to be processed to obtain an equalized panchromatic image; interpretably, the panchromatic image corresponds to the same region as the multispectral image to be processed.

[0071] Step 3: Use NSST to perform multi-resolution decomposition on the I component obtained in Step 1 and the equalized panchromatic image obtained in Step 2, respectively, to obtain the low-frequency coefficients and high-frequency coefficients of the I component and the low-frequency coefficients and high-frequency coefficients of the equalized panchromatic image; interpretably, NSST not only retains the good approximation effect of ST on image edges, but also has translation invariance, achieving the most effective representation of image edges.

[0072] Step 4: Based on the region low-rank representation method, the low-frequency coefficients of the I component and the low-frequency coefficients of the equalized panchromatic image are fused to obtain the fused low-frequency coefficients; the high-frequency coefficients of the I component and the high-frequency coefficients of the equalized panchromatic image are classified into texture high-frequency coefficients and salient high-frequency coefficients, and fused using high-frequency fusion rules based on visual sensitivity coefficients and energy matching measurement coefficients respectively to obtain the fused high-frequency coefficients.

[0073] Step 5: Perform NSST inverse transform based on the fused high-frequency coefficients and fused low-frequency coefficients to obtain the fused I component; perform IHS inverse transform based on the fused I component and the H and S components obtained in Step 1 to obtain the panchromatic sharpened image.

[0074] The technical solution provided in this embodiment of the invention performs panchromatic sharpening of multispectral images based on NSST and regional low-rank representation. This combines the advantages of NSST having multi-resolution analysis and regional low-rank representation being able to accurately preserve the overall structural information of the source image, which can significantly improve the spatial and spectral quality of the sharpened image.

[0075] In a specific exemplary embodiment of the present invention, further explanation of step 1 includes extracting the luminance, chromaticity, and saturation components of a multispectral image with low spatial resolution. This involves performing an IHS transform on the multispectral image, effectively separating the standard RGB image composed of the three bands of the multispectral image into an I component representing the spatial characteristics of the image and H and S components representing the spectral characteristics. This facilitates subsequent fusion, and the IHS color model better matches the human eye's description and interpretation of color. Furthermore, in step 2, linearly stretching the panchromatic image based on the I component reduces spectral distortion.

[0076] Please see Figure 2 and Figure 3 In a further preferred embodiment of the present invention, in step 3,

[0077] The NSST discretization process typically includes two parts: multi-resolution decomposition and directional localization; among them,

[0078] 1) Multi-resolution decomposition: NSST employs a non-subsampled pyramid filter bank (NSP) to implement the multi-resolution decomposition process. Since it eliminates the downsampling step, this process ensures the translation invariance of the transform, thereby suppressing the pseudo-Gibbs phenomenon. Performing k decompositions on the source image yields k high-frequency subbands and one low-frequency subband of the same size as the source image.

[0079] 2) Directional localization: This process is completed by a shearlet filter (SF), which also avoids the downsampling process and thus has translation invariance; this process decomposes each sub-band along l different directions to obtain l directional sub-bands with the same size as the source image.

[0080] Please see Figures 3 to 5 In a further preferred embodiment of the present invention, step 4, the step of obtaining the fused low-frequency coefficients, specifically includes:

[0081] 1) Definition of low-rank representation of a region:

[0082] Low-rank representation (LRR) originates from compressed sensing theory and can represent data as a linear combination of dictionary vectors. By jointly finding the lowest rank among all data, LRR obtains the overall structural information of the data. It can accurately describe the correlation between vectors in the data matrix. LRR has the ability to describe the global information of an image. Introducing LRR into the low-frequency coefficient fusion process can preserve the overall structural information of the source image.

[0083] Specifically, given in space R, D A data matrix X = [x1, x2, ..., x...] p ], where each vector represents a data sample; each data sample can be represented by a dictionary D = [d1, d2, ..., d... q The atoms in ] are represented linearly, that is,

[0084] X=DZ (1)

[0085] In the formula, Z = [z1, z2, ..., z p [] is the low-rank matrix corresponding to the data matrix X;

[0086] Finding the lowest-rank representation of the original data matrix can be described as a constrained minimization problem.

[0087]

[0088] stX=DZ (2)

[0089] Assume that the optimal solution of equation (2) is Z. * Then Z is called * Let X be the low-rank representation of the data matrix X with respect to the dictionary D;

[0090] Due to the discrete nature of the rank function, equation (3) is difficult to solve directly. It is often replaced by the following convex optimization problem, expressed as follows:

[0091] In the formula, ||·|| * The nuclear norm is the sum of the singular values ​​of a matrix.

[0092] In practice, the data matrix X is often used as a dictionary D, and the noise contained in the data is further considered to optimize the above model into,

[0093] In the formula, λ>0 is the balancing parameter, and E is the noise matrix. For l 2,1 Norm, a constraint that causes the value of each column of E to tend to zero;

[0094] The above optimization problem can be solved using various methods. For example, to improve the solution efficiency, the embodiment of the present invention can use the Alternating Direction Method of Multipliers (ADMM) for solution.

[0095] Traditional low-rank representation often uses a sliding window method to process images. When the differences between the structural information in each window are too large, that is, when the dictionary atom D = [d1, d2, ..., d] in equation (2) is large, the results may be problematic. q When the differences between the elements are too large, the optimal solution matrix Z = [z1, z2, ..., z] obtained by equation (3) is... p The rank of the low-rank representation is artificially increased, which disrupts the correlation of information within each window, reducing the accuracy of solving the optimization problem and the accuracy of the low-rank representation in describing global information. This can easily lead to spectral distortion and other problems. Furthermore, for multispectral images with diverse ground features, the rectangular window used in low-rank representation often contains multiple ground features. Therefore, the phenomenon of low similarity of structural information within the window (i.e., a high rank in the matrix after image sliding window vectorization reconstruction) is very common and unavoidable.

[0096] In view of the above, this invention proposes a region-based low-rank representation method, abbreviated as Regional Low-Rank Representation (RLRR). The specific steps include: firstly, using Simple Linear Iteration Clustering (SLIC) to cluster the multispectral image to obtain several image regions E = [e1, e2, ..., e...]. r This clustering process classifies pixels into visually meaningful spatial regions, ensuring the consistency of structural information among pixels within a region. An example of the clustering partitioning results is shown below. Figure 4 As shown; then, formula (4) is used to perform low-rank representation of the low-frequency coefficients of the I component and the low-frequency coefficients of the equalized panchromatic image, and the low-rank matrix Z of the low-frequency coefficients of the I component is obtained. I The low-frequency coefficient low-rank matrix Z of the panchromatic image after equalization PAN .

[0097] Interpretively, using regional low-rank representation for low-frequency coefficient fusion allows for the full preservation of overall structural information in the source image, thereby improving the spectral quality of the sharpened image.

[0098] 2) Integration rule design:

[0099] Low-frequency coefficients reflect the overall information of the source image, mainly containing its structural information. Applying the proposed low-rank regional representation method to low-frequency coefficient fusion can fully extract the overall structural information of the source image.

[0100] Specifically, the l1 norm measures the structural information content of each column of pixels in a low-rank coefficient matrix. A larger l1 norm value indicates a more discrete distribution of pixels and richer structural information. Therefore, this embodiment of the invention uses the rule of maximizing the l1 norm to optimize the low-frequency coefficient low-rank coefficient matrix Z of the I component. I The low-frequency coefficient and low-rank coefficient matrix Z of the panchromatic image after equalization PAN The fusion process can be specifically represented as follows:

[0101]

[0102] In the formula, This represents the j-th column of the fused low-rank coefficient matrix; The j-th column of the low-frequency coefficient low-rank matrix of the panchromatic image after equalization is represented. Represents the j-th column of a low-frequency sparse low-rank matrix of the I component; This represents the l1 norm of the column vectors of a matrix.

[0103] Please see Figures 3 to 5In a further preferred embodiment of the present invention, step 4, the step of obtaining the fused high-frequency coefficients, specifically includes:

[0104] The high-frequency coefficients obtained by NSST decomposition mainly contain information such as details and edges of the source image. NSST decomposition is from coarse to fine to obtain high-frequency coefficients at multiple levels. According to the richness of texture information in the high-frequency coefficients, the present invention divides them into texture high-frequency coefficients and significant high-frequency coefficients, and designs a high-frequency fusion rule based on visual sensitivity coefficients and energy matching measurement coefficients.

[0105] Specifically, the high-frequency texture coefficients contain rich texture detail information. This invention employs a rule based on the visual sensitivity coefficient (VSC), a local contrast coefficient based on the human visual system. It characterizes the grayscale difference level between pixels and reflects the richness of texture information in the image. At position (x,y), the I-component visual sensitivity coefficient VSC... I (x,y), panchromatic image visual sensitivity coefficient (VSC) PAN (x,y) and the visual sensitivity coefficient VSC of the sharpened image F The definitions of (x, y) are as follows:

[0106]

[0107]

[0108]

[0109] In the formula, Let I be the subband coefficient located at (x,y) with a decomposition direction number of l for the I component at the k-th decomposition. For the panchromatic image at the k-th level decomposition, the directional sub-band coefficients with a decomposition direction number of l are given. To sharpen the image at the k-th level decomposition, the directional subband coefficients located at (x,y) with decomposition direction number l; The average value of the low-frequency subband coefficients of the I component at (x,y) under the k-th level decomposition is given. This represents the average value of the low-frequency subband coefficients located at (x,y) in the k-th level decomposition of the panchromatic image. To sharpen the image, the average value of the low-frequency subband coefficients at (x, y) in the k-th level decomposition is defined as follows:

[0110]

[0111]

[0112]

[0113] In the formula, M×N is the size of the adjacent region; r,c is the offset distance of each point in the region from the center point (x,y).

[0114] According to the definition of VSC, the larger the VSC value, the richer the texture detail information of the corresponding image. Therefore, the high-frequency texture coefficients adopt the fusion rule of taking the larger visual sensitivity coefficient, which is expressed as follows:

[0115]

[0116] In the formula, Let I be the subband coefficient located at (x,y) with a decomposition direction number of l for the I component at the k-th decomposition. For the panchromatic image at the k-th level decomposition, the directional sub-band coefficients with a decomposition direction number of l are given. To sharpen the image at the k-th decomposition, the directional subband coefficients located at (x,y) with decomposition direction number l; VSC I (x,y) represents the visual sensitivity coefficient of the I component at position (x,y); VSC PAN (x,y) is the visual sensitivity coefficient of the panchromatic image at position (x,y).

[0117] Furthermore, compared to texture high-frequency coefficients, significant high-frequency coefficients lack sufficient texture detail information, and the similarity between significant high-frequency coefficients at different levels varies. If a uniform linear weighting or extreme value fusion rule is used, the independent characteristics of this coefficient will be lost, affecting the spatial sharpness of the sharpened image. Therefore, this embodiment of the invention first designs a Regional Energy Matching Measurement Coefficient (REMMC) to measure the similarity between significant high-frequency coefficients of the source image, as defined in equation (13). Based on this, different fusion rules are designed for coefficients with high and low similarity; wherein,

[0118]

[0119] In the formula, MΔN is the size of the adjacent region; r,c is the offset distance of each point in the region from the center (x,y); Let I be the directional subband coefficient with a decomposition direction number of l under the k-th level decomposition of the I component; For the panchromatic image at the k-th level decomposition, the directional sub-band coefficients with a decomposition direction number of l are given. To sharpen the image, the direction subband coefficients with a decomposition direction number of l are defined at the k-th decomposition level.

[0120] When the region energy matching measurement coefficient values ​​are small, the similarity between significant high-frequency coefficients in the source images is low and the differences are large. Therefore, a coefficient weighting method is used to utilize complementary detail information between the source images. When the region energy matching measurement coefficient values ​​are large, the similarity between source images is high and the differences are small. In this case, the Neighborhood Coefficient Difference (NCD) is used to highlight the differences, and a larger value rule is adopted to preserve detail information. In summary, the fusion rule for significant high-frequency coefficients is summarized as follows:

[0121]

[0122] In the formula, Ex I k,l (x,y) represents the neighborhood coefficient difference of the region corresponding to the directional sub-band with l decomposition directions under the k-th level decomposition of the I component; Ex PAN k,l (x,y) represents the neighborhood coefficient difference of the region corresponding to the directional sub-band with l decomposition directions in the panchromatic image at the k-th level decomposition; Ex F k,l (x,y) represents the neighborhood coefficient difference of the region corresponding to the direction sub-band with l decomposition directions in the sharpened image at the k-th level decomposition. Let I be the directional subband coefficient with a decomposition direction number of l under the k-th level decomposition of the I component; For the panchromatic image at the k-th level decomposition, the directional sub-band coefficients with a decomposition direction number of l are given. To sharpen the image, the direction subband coefficients with the number of decomposition directions is l at the k-th level decomposition; R is the weight coefficient; and T is the threshold.

[0123] The verification process of this invention includes:

[0124] To verify the good characteristics of the panchromatic sharpening method proposed in this embodiment of the invention, 20 pairs of representative panchromatic and multispectral images containing various types of land features were selected from remote sensing images acquired by the IKONOS satellite. Their resolutions were 1m and 4m, and their sizes were 1024×1024 and 256×256, respectively. The image content included urban areas, hydrology, vegetation, and composite scenes.

[0125] To improve computational efficiency, this embodiment of the invention employs a bicubic decimation operator based on the Wald criterion to degrade the original ground-based PAN and MS images, resulting in PAN and MS images with resolutions of 4 meters and 16 meters, respectively, and sizes of 256×256 and 64×64. The original MS image is used as a reference image. Figure 6 This invention demonstrates a subset of images from the dataset of an embodiment of the invention;

[0126] The experiment used images from the dataset constructed in the embodiments of the present invention for sharpening, and selected several traditional panchromatic sharpening methods for comparison with the methods in the embodiments of the present invention. These methods are: Gram-Schmidt transform-based method (GS method), Principal Component Analysis (PCA)-based method (PCA method), Brovey transform-based method (BT method), Intensity-Hue-Saturation (IHS) transform-based method (IHS method), Non-Subsample Shearlet Transform (NSST)-based method (NSST method), Low Rank Representation (LRR)-based method (LRR method), NSST+SR method based on NSST and Sparse Representation (SR) method, and NSST+LRR method based on NSST and LRR method.

[0127] Furthermore, in this embodiment of the invention, the operating system is Windows 10, and the programming environment is MATLAB R2018b; the number of decomposition layers of NSST is l = 4, and the number of directions corresponding to these layers are respectively [2 3 ,2 3 ,2 4 ,2 5 The local support of the shearing filter is [8,8,16,16]. Based on experimental experience, the number of superpixel blocks k = 30; the low-rank representation coefficient λ = 2.4; and the threshold T = 1.5 in high-frequency coefficient fusion.

[0128] The analysis of panchromatic sharpening effect of multispectral images is usually based on spectral and spatial features. In order to better evaluate the sharpening effect, this invention presents an experimental analysis of the sharpened multispectral images from both subjective and objective perspectives.

[0129] Subjective evaluation specifically refers to the qualitative evaluation of image quality through visual observation, which is simple and quick. Figure 7 and Figure 8 They were displayed respectively Figure 6 Experimental results after sharpening the first two sets of images; from Figure 7 , Figure 8Experimental results show that all sharpening methods can fuse spectral and spatial information extracted from the source image's multispectral and panchromatic images and inject it into the sharpened image. However, the sharpened images obtained by traditional methods all have some shortcomings. For example, the sharpened image obtained by the LRR-based method has insufficient sharpness; the sharpened image obtained by the BT-based method has large spectral distortion and appears dark; the sharpened image obtained by the IHS-based method has severe spectral distortion and is prone to false color in some areas due to directly replacing the luminance component; the sharpened images obtained by the GS-based and PCA-based methods have rich spatial information, but have a certain degree of spectral distortion; the sharpened image obtained by the NSST+SR method retains detail information well, but lacks spectral information; the sharpened images obtained by the NSST-based and NSST+LRR-based methods have rich spectral information and high spatial resolution, but there is still room for improvement. Compared with the above methods, the method of the embodiments of the present invention further improves the visual effect of the image. The sharpened image is not only clearer, but also has less spectral distortion.

[0130] Objective evaluation refers to using quantitative analysis tools and statistical parameters to evaluate the quality of sharpened images, avoiding interference from non-objective factors in the evaluation process. This invention, based on visual evaluation, uses nine commonly used evaluation indicators in the field of multispectral image sharpening to analyze the experimental results: Entropy (EN), Standard Deviation (SD), Root Mean Square Error (RMSE), Average Gradient (AG), Spectral Frequency (SF), Universal Image Quality Index (UIQI), Spectral Angle Mapper (SAM), Spectral Distortion Degree (SDD), and Erreur Relative Global Adimensionnelle De Synthese (ERGAS). Table 1 shows the mean values ​​of the objective evaluation indicators for 20 pairs of experimental data. Figure 9 The fluctuations in the indicator are presented in the form of a line chart.

[0131] Table 1. Average values ​​of objective evaluation indicators for sharpened images

[0132]

[0133] From Table 1 and Figure 9It can be seen that the sharpened image obtained using the method proposed in this embodiment of the invention has the best quality and the best performance across all evaluation metrics. The standard deviation, entropy, and average gradient are optimal, indicating that the sharpened image obtained by this method is clearer and contains richer spatial information. The spectral distortion is also optimal, demonstrating that the method of this invention can better preserve the spectral information of multispectral images. The optimal spectral angle mapping and general image quality index indicate that the sharpened image is closest to the reference image. A comparison of the experimental results of the NSST+LRR-based method and the method of this invention clearly demonstrates the effectiveness of the proposed low-rank region representation method in improving the quality of sharpened images.

[0134] In summary, this invention discloses a multispectral image panchromatic sharpening method based on Non-subsampled Shearlet Transform (NSST) and Regional Low Rank Representation (RLRR) to improve the spectral and spatial quality of the sharpened image. Specifically, the low-frequency coefficient fusion process utilizes a newly designed RLRR method, performing low-rank representation on each region generated after clustering based on pixel spectral features to fully capture the structural information of the low-frequency coefficients, thereby improving the spectral quality of the sharpened image. Simultaneously, the high-frequency coefficients decomposed by NSST are divided into texture high-frequency coefficients and salient high-frequency coefficients. Based on their characteristics, a fusion rule based on visual sensitivity coefficients and energy matching measurement coefficients is designed to improve the spatial quality of the sharpened image. Experimental results show that the method proposed in this invention can better preserve the spatial and spectral information of the source image, resulting in a higher quality sharpened image.

[0135] The following are embodiments of the apparatus of the present invention, which can be used to execute embodiments of the method of the present invention. For details not disclosed in the apparatus embodiments, please refer to the embodiments of the method of the present invention.

[0136] Please see Figure 10 In another embodiment of the present invention, a multispectral image panchromatic sharpening system is provided, comprising:

[0137] The IHS transform decomposition module is used to perform IHS transform decomposition on the multispectral image to be processed to obtain the I component representing the spatial information of the image and the H and S components representing the spectral information.

[0138] The linear stretching module is used to linearly stretch the panchromatic image corresponding to the multispectral image to be processed based on the obtained I component, so as to obtain the equalized panchromatic image.

[0139] The multi-resolution decomposition module is used to perform multi-resolution decomposition on the obtained I component and the equalized panchromatic image using NSST, to obtain the low-frequency coefficients and high-frequency coefficients of the I component and the low-frequency coefficients and high-frequency coefficients of the equalized panchromatic image.

[0140] The fusion module is used to fuse the low-frequency coefficients of the I component and the low-frequency coefficients of the equalized panchromatic image based on the region low-rank representation method to obtain the fused low-frequency coefficients; the high-frequency coefficients of the I component and the high-frequency coefficients of the equalized panchromatic image are classified into texture high-frequency coefficients and salient high-frequency coefficients, and fused using high-frequency fusion rules based on visual sensitivity coefficients and energy matching measurement coefficients respectively to obtain the fused high-frequency coefficients.

[0141] The inverse transform module is used to perform NSST inverse transform based on the fused high-frequency coefficients and fused low-frequency coefficients to obtain the fused I component; and to perform IHS inverse transform based on the fused I component, the H component and the S component to obtain a panchromatic sharpened image.

[0142] In another embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in the computer storage medium to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used for the operation of a multispectral image panchromatic sharpening method.

[0143] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the multispectral image panchromatic sharpening method in the above embodiments.

[0144] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0145] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0146] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0147] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0148] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A method for panchromatic sharpening of multispectral images, characterized in that, Includes the following steps: The multispectral image to be processed is decomposed by IHS transform to obtain the I component representing the spatial information of the image and the H and S components representing the spectral information. Based on the obtained I component, the panchromatic image corresponding to the multispectral image to be processed is linearly stretched to obtain the equalized panchromatic image. The obtained I component and the equalized panchromatic image were decomposed into multi-resolution values ​​using NSST to obtain the low-frequency and high-frequency coefficients of the I component and the low-frequency and high-frequency coefficients of the equalized panchromatic image. Based on the regional low-rank representation method, the low-frequency coefficients of the I component and the low-frequency coefficients of the equalized panchromatic image are fused to obtain the fused low-frequency coefficients. The high-frequency coefficients of the I component and the high-frequency coefficients of the equalized panchromatic image are classified into texture high-frequency coefficients and salient high-frequency coefficients, and then fused using high-frequency fusion rules based on visual sensitivity coefficients and energy matching measurement coefficients, respectively, to obtain the fused high-frequency coefficients. The NSST inverse transform is performed based on the fused high-frequency coefficients and the fused low-frequency coefficients to obtain the fused I component. Based on the fused I component, H component, and S component, an inverse IHS transform is performed to obtain a full-color sharpened image; in, The steps of classifying the high-frequency coefficients of the I component and the high-frequency coefficients of the equalized panchromatic image into texture high-frequency coefficients and significant high-frequency coefficients, and fusing them using high-frequency fusion rules based on visual sensitivity coefficients and energy matching measurement coefficients respectively, to obtain the fused high-frequency coefficients include: classifying them into texture high-frequency coefficients and significant high-frequency coefficients according to the richness of texture information in the high-frequency coefficients; fusing the texture high-frequency coefficients using a fusion rule based on visual sensitivity coefficients; and fusing the significant high-frequency coefficients using a fusion rule based on energy matching measurement coefficients. In the step of fusing significantly high-frequency coefficients using a fusion rule based on energy-matching measurement coefficients, the fusion rule is expressed as follows: ； ； In the formula, For the I component in the th k Under level decomposition, the number of decomposition directions is l Directional subband coefficient; For the panchromatic image in the 1st k Under level decomposition, the number of decomposition directions is l Directional subband coefficient; To sharpen the image at the first k Under level decomposition, the number of decomposition directions is l Directional subband coefficient; These are the weighting coefficients; For the threshold; For the I component in the th k Under level decomposition, the number of decomposition directions is l The difference in the domain coefficients of the corresponding regions of the directional sub-bands; For the panchromatic image in the 1st k Under level decomposition, the number of decomposition directions is l The difference in the domain coefficients of the corresponding regions of the directional sub-bands; To sharpen the image at the first k Under level decomposition, the number of decomposition directions is l The difference in the domain coefficients of the corresponding regions of the directional sub-bands; The dimensions of adjacent regions; From each point in the region to the center The offset distance.

2. The multispectral image panchromatic sharpening method according to claim 1, characterized in that, The method based on regional low-rank representation, which fuses the low-frequency coefficients of the I component with the low-frequency coefficients of the equalized panchromatic image to obtain the fused low-frequency coefficients, specifically includes the following steps: Clustering of multispectral images yields several image regions; The low-frequency coefficients of the I component and the low-frequency coefficients of the equalized panchromatic image are represented by low-rank formulas to obtain the low-rank matrix of the low-frequency coefficients of the I component and the low-rank matrix of the low-frequency coefficients of the equalized panchromatic image. use l The rule of taking the larger 1-norm is used to fuse the low-frequency coefficient low-rank matrix of the I component and the low-frequency coefficient low-rank matrix of the equalized panchromatic image to obtain the fused low-frequency coefficients. The expression for the formula used in low-rank representation is: ； In the formula, For a given space A data matrix in which each vector represents a data sample; X Each data sample can be used with a dictionary The atoms in the equation are represented linearly. X=DZ ; for The corresponding low-rank matrix; E This is the noise matrix; Represents the nuclear norm; For balance parameters; for Norm.

3. The multispectral image panchromatic sharpening method according to claim 2, characterized in that, When clustering multispectral images, a simple linear iterative clustering method is used.

4. The multispectral image panchromatic sharpening method according to claim 2, characterized in that, The use l The expression for fusing the low-frequency coefficient low-rank matrix of the I component and the low-frequency coefficient low-rank matrix of the equalized panchromatic image, based on the rule of taking the larger 1-norm, is as follows: ； In the formula, Denotes the first rank of the fused low-rank matrix j List; This represents the low-rank matrix of the low-frequency coefficients of the panchromatic image after equalization. The j List; Low-rank matrix representing the low-frequency coefficients of the I component The j List; Represents the column vector of a matrix l 1-norm.

5. The multispectral image panchromatic sharpening method according to claim 1, characterized in that, In the step of fusing high-frequency texture coefficients using a fusion rule based on visual sensitivity coefficients, the fusion rule is expressed as follows: ;in, ； ； ； ； ； ； In the formula, For the I component in the th k Under level decomposition, the number of decomposition directions is l The location of Subband coefficient at the location; For the panchromatic image in the 1st k Under level decomposition, the number of decomposition directions is l Directional subband coefficient; To sharpen the image at the first k Under level decomposition, the number of decomposition directions is l The location of The direction sub-band coefficient at that location; In order to be in Visual sensitivity coefficient of component I at location; The visual sensitivity coefficient of the panchromatic image. To sharpen the image's visual sensitivity coefficient; For the I component in the th k The low-frequency subband coefficients in the first-order decomposition are located at The regional average at that location; For the panchromatic image k The low-frequency subband coefficients in the first-order decomposition are located at The regional average at that location; To sharpen the image at the first k The low-frequency subband coefficients in the first-order decomposition are located at The regional average at that location; The size of the adjacent region. From each point in the region to the center The offset distance.

6. A multispectral image panchromatic sharpening system, characterized in that, To implement the multispectral image panchromatic sharpening method of claim 1, the method includes: The IHS transform decomposition module is used to perform IHS transform decomposition on the multispectral image to be processed to obtain the I component representing the spatial information of the image and the H and S components representing the spectral information. The linear stretching module is used to linearly stretch the panchromatic image corresponding to the multispectral image to be processed based on the obtained I component, so as to obtain the equalized panchromatic image. The multi-resolution decomposition module is used to perform multi-resolution decomposition on the obtained I component and the equalized panchromatic image using NSST, to obtain the low-frequency coefficients and high-frequency coefficients of the I component and the low-frequency coefficients and high-frequency coefficients of the equalized panchromatic image. The fusion module is used to fuse the low-frequency coefficients of the I component and the low-frequency coefficients of the equalized panchromatic image based on the region low-rank representation method to obtain the fused low-frequency coefficients; the high-frequency coefficients of the I component and the high-frequency coefficients of the equalized panchromatic image are classified into texture high-frequency coefficients and salient high-frequency coefficients, and fused using high-frequency fusion rules based on visual sensitivity coefficients and energy matching measurement coefficients respectively to obtain the fused high-frequency coefficients. The inverse transform module is used to perform NSST inverse transform based on the fused high-frequency coefficients and fused low-frequency coefficients to obtain the fused I component; and to perform IHS inverse transform based on the fused I component, the H component and the S component to obtain a panchromatic sharpened image.

7. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the multispectral image panchromatic sharpening method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the multispectral image panchromatic sharpening method according to any one of claims 1 to 5.

Images