A Spectral Data Fusion Method and System Based on Dictionary Learning

Abstract

The present invention belongs to the technical field of data processing, and specifically discloses a spectral data fusion method and system based on dictionary learning. The method includes the following steps: Step S1, perform adaptive pixel clustering on hyperspectral data and multispectral data to obtain the ideal number of clusters; Step S2, use the online dictionary learning algorithm to calculate the spectral dictionary of each cluster of hyperspectral data; Step S3, calculate the sparse coefficients of each cluster of multispectral data after clustering in combination with the spectral dictionary; Step S4, calculate the corresponding fusion data for each cluster according to the spectral dictionary and the sparse coefficients, and integrate the fusion data of each cluster to obtain a fusion image. The present invention can obtain more accurate and stable clustering results when processing complex data sets; at the same time, the present invention imposes a non-negative constraint condition on the coefficient matrix, strengthening the robustness of the results.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly relates to a spectral data fusion method and system based on dictionary learning. Background Art

[0002] Optical remote sensing systems are limited by imaging metrics such as the optical diffraction limit, modulation transfer function, and signal-to-noise ratio, and it is difficult to obtain remote sensing images with both high spatial resolution and high spectral resolution. Therefore, a large number of studies are dedicated to combining the advantages of different optical data in terms of spatial resolution and spectral resolution to obtain data with high spatial and spectral resolutions. This process is also known as data fusion.

[0003] Dictionary learning is widely used in multispectral and hyperspectral data fusion. The core idea is to obtain the coefficient matrix of multispectral data and the spectral basis (dictionary) matrix of hyperspectral data by solving an optimization problem, and multiply the two to obtain the final fusion result.

[0004] Currently, most dictionary learning methods use hyperspectral data to learn a general spectral dictionary, and then solve the coefficient matrix based on the dictionary and multispectral data to obtain the fusion result. Learning a single dictionary for the entire hyperspectral data ignores the spatial self-similarity of hyperspectral data and cannot achieve precise fusion for each region in the image. Moreover, large-scale dictionary sparse coding for the entire hyperspectral data will bring a serious computational burden. In addition, the dictionary learning data fusion method assumes that spectral information exists in a low-dimensional subspace (dictionary). However, when the number of atoms in the dictionary of the entire hyperspectral data is much larger than the number of bands of multispectral data, due to the severe ill-posedness of the optimization problem, the accuracy of the sparse representation method will deteriorate. The coefficient matrix of multispectral data is usually solved using the greedy algorithm simultaneous orthogonal matching pursuit (SOMP), which updates a single dictionary atom in each iteration, with a long optimization time and low efficiency. And the solution process uses the standard least squares, i.e., the least squares method approximation, without considering the non-negativity of the solution, and the result is easily affected by noise in the data, with poor robustness.

[0005] In summary, there is a need for a spectral data fusion method and system based on dictionary learning to solve the above problems in the prior art. Summary of the Invention

[0006] The present invention provides a spectral data fusion method and system based on dictionary learning, which solves the problem that most dictionary learning data fusion methods cannot perform precise fusion for each region in the image, and the reduction of the data volume of each class after clustering avoids the serious computational burden caused by large-scale dictionary sparse coding and the problem of accuracy degradation caused by the number of atoms in the dictionary of hyperspectral data being more than the number of bands of multispectral data.

[0007] To achieve the purpose of solving the above technical problems, the present invention adopts the following technical solutions:

[0008] A spectral data fusion method based on dictionary learning, comprising the following steps:

[0009] Step S1: Perform adaptive pixel clustering on hyperspectral data and multispectral data to obtain the ideal number of clusters;

[0010] Step S2: Calculate the spectral dictionary of each cluster of hyperspectral data using the online dictionary learning algorithm;

[0011] Step S3: Calculate the sparse coefficients of each cluster of multispectral data after clustering in combination with the spectral dictionary;

[0012] Step S4: Calculate the fusion data corresponding to each cluster according to the spectral dictionary and the sparse coefficients, and integrate the fusion data of each cluster to obtain a fusion image;

[0013] Among them, the adaptive pixel clustering in the step S1 specifically includes:

[0014] Determine the ideal cluster center through iterative calculation;

[0015] Calculate the Calinski-Harabasz index according to the data within the cluster;

[0016] Preset the number of clusters, and repeat the above steps within its range to obtain multiple Calinski-Harabasz indices. When the Calinski-Harabasz index is the largest, select the corresponding number of clusters as the ideal number of clusters.

[0017] In some embodiments of the present invention, the determining the ideal cluster center through iterative calculation includes:

[0018] Determine the initial cluster center according to the local density of the hyperspectral data and the multispectral data;

[0019] Using the initial cluster center, calculate the mean value of the data points in each cluster and update the cluster center;

[0020] Repeat the above steps until the number of iterations is reached, and then determine the ideal cluster center.

[0021] In some embodiments of the present invention, the update formula of the cluster center is:

[0022] ;

[0023] Among them, k is the number of the cluster, C k is the center of the kth cluster, S k is the set of data points belonging to the kth cluster, d k is the number of data points in the kth cluster, x iRepresents the value of the data point.

[0024] In some embodiments of the present invention, the process of determining the initial cluster center includes the following steps:

[0025] Data preprocessing to eliminate the dimensional difference between different features;

[0026] The following formula is used to calculate the local density:

[0027] ,

[0028] where ρ i is the local density of data point i, d ij is the distance between data points i and j, and σ is the width parameter of the Gaussian kernel;

[0029] Select the point with the highest local density as the initial cluster center.

[0030] In some embodiments of the present invention, the process of calculating the spectral dictionary in step S2 includes the following steps:

[0031] Each cluster of hyperspectral data after clustering in step S1 is denoted as:

[0032] ;

[0033] where k is the cluster number, R is used to represent the set of real numbers; L is the number of hyperspectral data bands; d k is the number of data points in the k-th cluster;

[0034] Calculate the spectral dictionary by solving the following constrained problem ;

[0035] ;

[0036] where, represents the D that minimizes the objective function k , B k values; B k is the sparse coefficient of the k-th cluster of hyperspectral data; represents the L1 norm of matrix B k ; represents the square of the L2 norm of; s.t. means subject to; ε is a number infinitely close to 0; h k is the number of endmembers of the spectral dictionary D k ;

[0037] In some embodiments of the present invention, the calculation of the sparse coefficient in step S3 includes the following steps:

[0038] Each cluster of multi-spectral data after clustering in step S1 is denoted as:

[0039] ;

[0040] where k is the cluster number, R is used to represent the set of real numbers; l is the number of bands of the multi-spectral data; d k is the number of data points in the k-th cluster of hyperspectral data; c is the spatial resolution multiple of the multi-spectral data compared to the hyperspectral data;

[0041] The multi-spectral data is divided into multiple non-overlapping small blocks P of size (n*n) km , denoted as ;

[0042] By using the generalized non-negative simultaneous orthogonal matching pursuit algorithm to solve the following inequality, the sparse coefficient A corresponding to each small block P can be obtained km : km :

[0043] ;

[0044] represents the value of A that minimizes the objective function , km is the hyperspectral to multi-spectral conversion matrix, L is the number of bands of the hyperspectral data, l is the number of bands of the multi-spectral data, is the spectral dictionary after conversion, m represents the total number of small blocks in the k-th cluster; k represents the row sparsity constraint on the coefficient matrix A ; km represents the square of the L2 norm of ; s.t. means subject to; ε is a number infinitely close to 0;

[0045] By summing up the sparse coefficients A of each small block in each cluster, the sparse coefficient A of the current cluster of multi-spectral data can be obtained km : k .

[0046] In some embodiments of the present invention, step S4 specifically includes the following steps:

[0047] According to the spectral dictionary and sparse coefficients of each cluster, calculate the fused image of each cluster using the following formula:

[0048] ;

[0049] Then map the fused image X of each cluster k to the corresponding area to obtain the fused image.

[0050] ​In some embodiments of the present invention, a spectral data fusion system based on dictionary learning is provided, including:

[0051] An adaptive clustering module, which is used to perform adaptive pixel clustering using the Kmeans algorithm according to the density of hyperspectral data and multispectral data;

[0052] A calculation module, which is used to calculate a spectral dictionary through an online dictionary learning algorithm; and is also used to calculate sparse coefficients through a generalized non-negative simultaneous orthogonal matching pursuit algorithm;

[0053] A fusion module, which is used to perform image fusion according to the sparse coefficients and spectral dictionary of each cluster;

[0054] A communication module, used for communicating with external devices.

[0055] In some embodiments of the present invention, an electronic device is provided, including:

[0056] A processor, and a memory and a transceiver communicatively connected to the processor;

[0057] The memory stores computer-executable instructions; the transceiver is used for sending and receiving data;

[0058] The processor executes the computer-executable instructions stored in the memory to implement the above spectral data fusion method.

[0059] In some embodiments of the present invention, a computer-readable storage medium is provided, characterized in that

[0060] The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the above spectral data fusion method.

[0061] The technical solution of the present invention has the following technical effects compared with the prior art:

[0062] The present invention uses a density-based adaptive Kmeans algorithm to cluster multispectral and hyperspectral data, making use of the non-local self-similarity of remote sensing data, and improving the fusion accuracy compared with global dictionary learning. Compared with the traditional Kmeans algorithm, it has significant advantages in aspects such as the selection of the initial cluster center, the robustness to outliers, the selection of the adaptive number of clusters, and the adaptability to clustering of complex shapes. These advantages enable the algorithm to obtain more accurate and stable clustering results when processing complex data sets.

[0063] Meanwhile, the present invention uses the Generalized Nonnegative Simultaneous Orthogonal Matching Pursuit (GSOMP+) algorithm to calculate sparse coefficients, which improves the computational efficiency compared to the Simultaneous Orthogonal Matching Pursuit (SOMP) algorithm used in most data fusion algorithms, imposes non - negative constraint conditions on the coefficient matrix, and enhances the robustness of the results. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0065] Figure 1 It is a flowchart of a spectral data fusion method based on dictionary learning according to the present invention.

[0066] Figure 2 It is a fusion process diagram of a spectral data fusion method based on dictionary learning according to the present invention.

[0067] Figure 3 It is a comparison schematic diagram between the fusion result obtained by the data fusion method of the present invention and the true value.

[0068] Figure 4 It is a comparison schematic diagram between the fusion result obtained by using the ordinary Kmeans algorithm and the true value.

[0069] Figure 5 It is a comparison schematic diagram between the fusion result obtained by performing data fusion using the detail injection method GLP and the true value.

[0070] Figure 6 It is a comparison schematic diagram between the fusion result obtained by performing data fusion using the detail injection method SFIM and the true value.

[0071] Figure 7 It is a comparison schematic diagram between the fusion result obtained by performing data fusion using the matrix decomposition algorithm CNMF and the true value.

[0072] Figure 8 It is a schematic diagram of the structure of the spectral data fusion system.

[0073] Figure 9 It is a schematic diagram of the structure of the electronic device.

[0074] Reference numerals: 100, spectral data fusion system; 110, adaptive clustering module; 120, calculation module; 130, fusion module; 140, communication module; 200, electronic device; 210, processor; 220, memory; 230, transceiver. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0075] Next, in combination with the accompanying drawings in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0076] In the description of the present invention, it should be noted that unless otherwise clearly defined and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0077] Example 1: Refer to Figure 1 and Figure 2 As shown, this embodiment provides a spectral data fusion method based on dictionary learning, including the following steps:

[0078] Step S1: Perform adaptive pixel clustering on hyperspectral data and multispectral data to obtain the ideal number of clusters;

[0079] S11: Data preprocessing: For example, standardization or normalization to eliminate the dimensional difference between different features.

[0080] S12: Calculate the local density: The local density can be determined by calculating the distance between each data point and its neighbor points.

[0081] Density calculation formula: ,

[0082] where ρ i is the local density of data point i, d ij is the distance between data points i and j, and σ is the width parameter of the Gaussian kernel.

[0083] S13: Select the initial cluster center: Select the point with the highest local density as the initial cluster center.

[0084] S14: Execute the clustering algorithm: Using the initial cluster center selected in step S13, allocate the data points to the nearest cluster center according to the distance metric, calculate the mean of all data points in each cluster as the new cluster center, and repeat the steps of allocating clusters and updating the cluster center until the cluster center no longer changes significantly or reaches a predetermined number of iterations.

[0085] Cluster center update formula: ;

[0086] Among them, k is the cluster number, C k is the center of the k-th cluster, S k is the set of data points belonging to the k-th cluster, d k is the number of data points in the k-th cluster, x i represents the value of the data point.

[0087] S15. Calculate the Calinski-Harabasz index: For each cluster, calculate the sum of the squares of the distances from the data points within the cluster to the cluster center, then find the average to obtain the within-cluster variance, calculate the square of the average distance between all cluster centers to obtain the between-cluster variance, and calculate the Calinski-Harabasz index based on the within-cluster variance and the between-cluster variance.

[0088] Within-cluster variance ;

[0089] Total within-cluster variance ;

[0090] where K is the number of clusters;

[0091] Between-cluster variance ;

[0092] where C is the global center of all data points;

[0093] The Calinski-Harabasz index is specifically Calinski-Harabasz = ;

[0094] where N is the total number of data points.

[0095] S16. Select the optimal number of clusters: Preset the number of clusters, and within its preset range of the number of clusters, for example, the range can be selected as 3 - 6, repeat steps S14 and S15, and select the number of clusters that maximizes the Calinski-Harabasz index as the optimal number of clusters, that is, the ideal number of clusters K.

[0096] The present invention uses a density-based adaptive Kmeans algorithm to cluster multi-spectral and hyperspectral data, utilizes the non-local self-similarity of remote sensing data, and improves the fusion accuracy compared with global dictionary learning. The density-based adaptive Kmeans algorithm has significant advantages over the traditional Kmeans algorithm in aspects such as the selection of the initial cluster center, the robustness to outliers, the selection of the adaptive number of clusters, and the adaptability to clustering of complex shapes. These advantages enable the algorithm to obtain more accurate and stable clustering results when processing complex data sets.

[0097] In step S1, considering the non-local self-similarity of remote sensing data, there is high similarity between the spectra of adjacent data points, while the spectral similarity of data points far apart is low. The hyperspectral data is divided into K clusters according to the density-based adaptive Kmeans clustering algorithm, and the data points in each cluster have extremely high similarity. After clustering, each cluster of hyperspectral data can be expressed as , representing the reflectance spectrum of a typical ground object in the imaging area, where d k is the number of data points in the k-th cluster, and L is the number of bands of the hyperspectral data.

[0098] Performing dictionary learning and coefficient estimation separately for each cluster of pixels can accurately fuse the data in each region, make full use of the advantage of the non-local self-similarity of remote sensing data, and at the same time avoid the problem of reduced accuracy caused by the number of atoms in the dictionary of hyperspectral data being larger than the number of bands of multispectral data. Specifically, the following steps are adopted:

[0099] Step S2: Use the online dictionary learning algorithm to calculate the spectral dictionary of each cluster of hyperspectral data;

[0100] Online dictionary learning is a machine learning algorithm aimed at dynamically learning a dictionary (or called a base dictionary, codebook) from data streams or real-time data. This dictionary can efficiently represent the signals or features in the data set. This learning method is different from traditional batch dictionary learning (such as the K-singular value decomposition algorithm), which usually assumes that all data is available before training and learns the entire dictionary at once. Online dictionary learning is more suitable for processing large-scale data streams or real-time data because it can gradually update the dictionary as new data arrives without having to recalculate the entire data set.

[0101] In this step S2, the spectral dictionary is calculated by solving the following constrained problem ;

[0102] ;

[0103] where represents the D that minimizes the objective function k and B k values; B k is the sparse coefficient of the k-th cluster of hyperspectral data; represents the L1 norm of matrix B k ; represents the square of the L2 norm of ; s.t. means subject to; ε is a number infinitely close to 0; h k is the number of endmembers of the spectral dictionary D k .

[0104] Step S3: Calculate the sparse coefficients of each cluster of multi-spectral data after clustering in combination with the spectral dictionary;

[0105] Use the Generalized Nonnegative Simultaneous Orthogonal Matching Pursuit algorithm (GSOMP+) to calculate the sparse coefficients of each cluster of multi-spectral data.

[0106] Obtain the spectral dictionary D of each cluster k After that, it is necessary to calculate the sparse coefficients corresponding to each cluster of high-resolution multi-spectral data corresponding sparse coefficients , where c is the spatial resolution multiple of the multi-spectral data compared to the hyperspectral data.

[0107] Since adjacent pixels in the high-spatial-resolution image represent the same type of ground object in the scene and can be represented by a group of the same atoms, the high-resolution multi-spectral data can be divided into multiple non-overlapping small blocks of size (n*n). Therefore, each small block can be approximately represented by the spectral dictionary obtained previously. (k and m represent the m-th small block in the k-th cluster), and the sparse coefficient A corresponding to each small block is obtained by solving the following problem km .

[0108] ;

[0109] represents the A that minimizes the objective function minimum value of A km value, is the conversion matrix from hyperspectral to multi-spectral, L is the number of bands of the hyperspectral data, l is the number of bands of the multi-spectral data, is the spectral dictionary after conversion, m k represents the total number of small blocks in the k-th cluster; represents the row sparsity constraint on the coefficient matrix A km ; represents the square of the L2 norm of; s.t. means subject to; ε is a number infinitely close to 0;

[0110] Sum the sparse coefficients A of each small block in each cluster km to obtain the sparse coefficient A of the current cluster of multi-spectral data k .

[0111] The problem is solved using an improved version of the Greedy Algorithm Orthogonal Matching Pursuit (OMP), the Generalized Nonnegative Simultaneous Orthogonal Matching Pursuit (GSOMP+). Compared with the original Orthogonal Matching Pursuit algorithm, it adds the constraints of non-negativity of the solution space and allowing multiple dictionary atoms to be updated in each iteration, improving the computational efficiency and enhancing the robustness of the results.

[0112] Step S4. Calculate the fusion data corresponding to each cluster according to the spectral dictionary and the sparse coefficients, and integrate the fusion data of each cluster to obtain a fused image;

[0113] Specifically, calculate the fused image according to the sparse coefficients and the spectral dictionary of each cluster

[0114] ;

[0115] After obtaining the spectral dictionary and the sparse coefficients of each cluster, calculate the fusion data corresponding to each cluster according to the above formula, map the data of each cluster to the corresponding region, and obtain the final fused result.

[0116] Refer to Figure 3 shown, which is a comparison diagram of the fused result obtained by the data fusion method of the present invention with the ground truth, as well as the peak signal-to-noise ratio (PSNR) and the universal image quality index (UIQI). It can be clearly seen that the fused result obtained by using the fusion method of the present invention is highly fitted with the ground truth, proving that the fusion effect of the present invention is good.

[0117] Refer to Figure 4 shown, which is a comparison diagram of the fused result and the ground truth, as well as the peak signal-to-noise ratio (PSNR) and the universal image quality index (UIQI) obtained after clustering by using the ordinary Kmeans algorithm and then performing data fusion; the coincidence effect is poor, and it can be concluded that: the fusion effect obtained by performing fusion after using the density-based adaptive Kmeans algorithm in step S1 of the present invention is better.

[0118] In addition, Figure 5 is a comparison diagram of the fused result obtained by performing data fusion using the detail injection method GLP, as well as the peak signal-to-noise ratio (PSNR) and the universal image quality index (UIQI). Figure 6 is a comparison diagram of the fused result obtained by performing data fusion using the detail injection method SFIM, as well as the peak signal-to-noise ratio (PSNR) and the universal image quality index (UIQI). Figure 7 is a comparison diagram of the fused result obtained by performing data fusion using the matrix factorization algorithm CNMF, as well as the peak signal-to-noise ratio (PSNR) and the universal image quality index (UIQI). The fused results of the above three methods all have large differences from the ground truth.

[0119] The present invention uses the generalized non-negative simultaneous orthogonal matching pursuit (GSOMP+) algorithm to calculate the sparse coefficients, which improves the calculation efficiency compared with the simultaneous orthogonal matching pursuit algorithm (SOMP) used by most data fusion algorithms, and imposes a non-negative constraint condition on the coefficient matrix, strengthening the robustness of the result.

[0120] Example 2: This example will be based on Figure 8 andFigure 9 Describe a spectral data fusion system 100 and an electronic device 200 based on dictionary learning.

[0121] Among them, referring to Figure 8 As shown, a spectral data fusion system 100 based on dictionary learning is provided, including:

[0122] An adaptive clustering module 110, which is used to perform adaptive pixel clustering using the Kmeans algorithm according to the density of hyperspectral data and multispectral data;

[0123] A calculation module 120, which is used to calculate a spectral dictionary through an online dictionary learning algorithm; and is also used to calculate sparse coefficients through a generalized non - negative simultaneous orthogonal matching pursuit algorithm;

[0124] A fusion module 130, which is used to perform image fusion according to the sparse coefficients and spectral dictionary of each cluster;

[0125] A communication module 140, which is used to communicate with external devices.

[0126] It should be understood that the spectral data fusion system 100 here is embodied in the form of functional modules. The term "module" here may refer to an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a proprietary processor, or a group of processors, etc.) for executing one or more software or firmware programs, and a memory, a combined logic circuit, and / or other suitable components that support the described functions. In an alternative example, those skilled in the art can understand that the spectral data fusion system 100 can specifically be the electronic device 200 in the above - mentioned embodiment, or the functions of the electronic device 200 in the above - mentioned embodiment can be integrated in the spectral data fusion system 100. The spectral data fusion system 100 can be used to execute the respective processes and / or steps corresponding to the electronic device 200 in the above - mentioned method embodiments. To avoid repetition, it will not be elaborated here.

[0127] The above - mentioned spectral data fusion system 100 has the function of implementing the corresponding steps executed by the electronic device 200 in the spectral data fusion method in Embodiment 1; the above - mentioned function can be implemented by hardware or by hardware executing corresponding software. This hardware or software includes one or more modules corresponding to the above - mentioned functions. For example, the above - mentioned acquisition module can be a communication interface, such as a transceiver interface.

[0128] Referring to Figure 9 As shown, in this embodiment, an electronic device 200 is provided, including:

[0129] A processor 210, as well as a memory 220 and a transceiver 230 communicatively connected to the processor;

[0130] The memory 220 stores computer-executable instructions; the transceiver 230 is used for transmitting and receiving data;

[0131] The processor 210 executes the computer-executable instructions stored in the memory 220 to implement the spectral data fusion method in Embodiment 1.

[0132] It should be understood that the electronic device 200 can be used to execute the corresponding steps and / or processes in the above method embodiments. Optionally, the memory 220 may include a read-only memory and a random access memory, and provide instructions and data to the processor. A part of the memory 220 may also include a non-volatile random access memory. For example, the memory 220 may also store information about the device type. The processor 210 can be used to execute the instructions stored in the memory 220, and when the processor 210 executes the instructions, the processor 210 can execute the corresponding steps and / or processes in the above method embodiments.

[0133] It should be understood that in the embodiments of the present application, the processor 210 may be a central processing unit (CPU), and the processor 210 may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0134] In the implementation process, each step of the above method may be completed by the integrated logic circuit of the hardware in the processor 210 or the instructions in the form of software. The steps of the method disclosed in combination with the embodiments of the present application may be directly embodied as being executed by the hardware processor, or completed by a combination of the hardware and software modules in the processor 210. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory, and the processor executes the instructions in the memory and combines its hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0135] Embodiment 3: In this embodiment, a computer-readable storage medium is provided. Computer-executable instructions are stored in the computer-readable storage medium, and when the computer-executable instructions are executed by a processor, they are used to implement the spectral data fusion method in Embodiment 1.

[0136] In several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the devices or units can be in electrical, mechanical, or other forms.

[0137] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0138] In addition, in each embodiment of this application, each functional unit can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0139] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0140] In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0141] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A spectral data fusion method based on dictionary learning, characterized in that: The following steps are involved: Step S1, performing adaptive pixel clustering on hyperspectral data and multispectral data to obtain an ideal number of clusters; Step S2, using an online dictionary learning algorithm to calculate the spectral dictionary of each cluster of hyperspectral data; Step S3, calculating the sparse coefficient of each cluster of multispectral data after clustering in combination with the spectral dictionary; Step S4, calculating the fusion data corresponding to each cluster according to the spectral dictionary and the sparse coefficient, and integrating the fusion data of each cluster to obtain a fusion image; The adaptive pixel clustering in step S1 specifically includes: Determine the ideal cluster center through iterative calculation; The Calinski-Harabasz index was calculated based on the intra-cluster data; The number of clusters is preset, and the above steps are repeated within the range to obtain multiple Calinski-Harabasz indices. When the Calinski-Harabasz index is the largest, the corresponding number of clusters is selected as the ideal number of clusters; The calculation of the sparse coefficient in step S3 includes the following steps: Each cluster of multispectral data clustered in step S1 is recorded as: ； Where k is the cluster number, R is used to represent the real number set; l is the number of bands of multispectral data; d k is the number of hyperspectral data points in the kth cluster; c is the spatial resolution multiple of multispectral data compared to hyperspectral data; Divide the multispectral data into multiple non-overlapping blocks P of size n×n km , denoted as ; Using the generalized non-negative synchronous orthogonal matching pursuit algorithm to solve the following inequality, we can get each small block P km The corresponding sparse coefficient A km : ； Represents the objective function The smallest A km Value, is the conversion matrix from hyperspectral to multispectral, L is the number of bands of hyperspectral data, l is the number of bands of multispectral data, is the spectral dictionary after conversion, m k Indicates the total number of small blocks in the kth cluster; Represents the coefficient matrix A km The row sparsity constraint of express The square of the L2 norm of ; st means restricted by ; ε is a number infinitely close to 0; The sparse coefficient A of each small block in each cluster km The sparse coefficient A of the multispectral data of the current cluster can be obtained by summing k .

2. The spectral data fusion method based on dictionary learning according to claim 1, characterized in that: Determining the ideal cluster center by iterative calculation includes: Determine the initial cluster center based on the local density of hyperspectral data and multispectral data; Using the initial cluster centers, calculate the mean of the data points in each cluster and update the cluster centers; Repeat the above steps until the number of iterations is reached and determine the ideal cluster center.

3. The spectral data fusion method based on dictionary learning according to claim 2, characterized in that: The update formula of the cluster center is: ； Where k is the cluster number, C k is the center of the kth cluster, S k is the set of data points belonging to the kth cluster, d k is the number of data points in the kth cluster, x i Represents the value of a data point.

4. The spectral data fusion method based on dictionary learning according to claim 2 is characterized in that: The process of determining the initial cluster center includes the following steps: Data preprocessing to eliminate dimensional differences between different features; The local density is calculated using the following formula: ， where ρ i is the local density of data point i, d ij is the distance between data points i and j, σ is the width parameter of the Gaussian kernel; The point with the highest local density is selected as the initial cluster center.

5. The spectral data fusion method based on dictionary learning according to claim 1, characterized in that: The calculation process of the spectral dictionary in step S2 includes the following steps: Each cluster of hyperspectral data clustered in step S1 is recorded as: ； Where k is the cluster number, R is used to represent the real number set; L is the number of hyperspectral data bands; d k is the number of data points in the kth cluster; The spectral dictionary is calculated by solving the constraint problem shown below ; ； in, Represents the objective function The smallest D k , B k Value; B k is the sparse coefficient of the kth cluster of hyperspectral data; Represents the matrix B k The L1 norm of express The square of the L2 norm of ; st means restricted by ; ε is a number infinitely close to 0; h k is the spectral dictionary D k The number of end members.

6. The spectral data fusion method based on dictionary learning according to claim 1, characterized in that: The step S4 specifically comprises the following steps: The fused image of each cluster is calculated using the following formula based on the spectral dictionary and sparse coefficient of each cluster: ； Then each cluster fused image X k Corresponding to the corresponding area, a fused image is obtained.

7. A spectral data fusion system based on dictionary learning, characterized in that: To implement the method according to any one of claims 1 to 6, comprising: An adaptive clustering module, which is used to perform adaptive pixel clustering using the Kmeans algorithm according to the density of hyperspectral data and multispectral data; A calculation module, which is used to calculate the spectral dictionary through an online dictionary learning algorithm; and is also used to calculate the sparse coefficients through a generalized non-negative synchronous orthogonal matching pursuit algorithm; A fusion module, which is used to fuse images according to the sparse coefficients of each cluster and the spectral dictionary; The communication module is used to communicate with external devices.

8. An electronic device, characterized in that: include: A processor, and a memory and a transceiver communicatively connected to the processor; The memory stores computer-executable instructions; the transceiver is used to send and receive data; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 6 when executed by a processor.

Images