Fusion method and system based on multi-model deep prior, storage medium and equipment
Patent Information
- Application Number
- CN202311442415.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-01
- Publication Date
- 2026-09-22
- Estimated Expiration
- 2043-11-01
AI Technical Summary
[0003]为此，本发明所要解决的技术问题在于克服现有技术中高光谱图像无法保证良好的信噪比、空间分辨率低和丢失大量的空间细节的问题；为解决上述技术问题，本发明提供了一种基于多模型深度先验的融合方法与系统、存储介质和设备，在不增加硬件成本的基础上提高高光谱图像的质量和提升网络模型输出图像的精度
[0034]本发明所述的一种基于多模型深度先验的融合方法，在设计可变残差混合注意力网络的过程中，加入了可变残差的结构，使得模型能够适应任务的特征，复原出高精度的高分辨率高光谱图像；通过改变可变残差混合注意力网络的外圈空间注意力和通道注意力的方式，以低分辨率高光谱图像和高分辨率多光谱图像为可变残差混合注意力网络的输入，获得三个不同的模型输出的深度先验结果；然后，将可变残差混合注意力网络输出的先验结果在光谱维度上进行重构，使用重构结果作为正则化项加入到机理模型之中；将寻找模型最优解的问题转换为求解西尔维斯特方程的问题，使用西尔维斯特方程的快速解法重建出高光谱超分辨率图片，并且通过遗传算法确定不同深度先验结果的最优超参数，从而得到高质量的高光谱图像；本申请运用机理知识将多种超分辨结果融合至一个算法之中，使得在不增加硬件成本的基础上，得到高质量的高光谱图像，有效降低高光谱图像采集设备的硬件成本，显著提升超分辨图像的修复质量。
Smart Images
Figure CN117455788B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of hyperspectral and multispectral image generation technology, and in particular to a fusion method and system, storage medium and device based on multi-model deep prior. Background Technology
[0002] Hyperspectral images are images acquired across many different spectral bands. Compared to traditional digital photography or color images, they can capture more subtle changes in objects and obtain more details and feature information. Therefore, hyperspectral images are widely used in agriculture, medicine, remote sensing, and other fields, and have great application value. However, compared to color images, hyperspectral images require a narrower spectral window to maintain the diversity of spectral channels. Without increasing hardware costs, this can lead to the system being unable to acquire enough photons in the same amount of time to ensure a good signal-to-noise ratio, resulting in reduced spatial resolution and loss of a large amount of spatial detail, making it impossible to obtain high-quality hyperspectral images. To solve this problem without increasing hardware costs, a number of image super-resolution algorithms have been proposed, but these algorithms perform inconsistently on different datasets. Summary of the Invention
[0003] Therefore, the technical problem to be solved by the present invention is to overcome the problems of existing technologies, such as the inability of hyperspectral images to guarantee a good signal-to-noise ratio, low spatial resolution, and loss of a large amount of spatial details. In order to solve the above technical problems, the present invention provides a fusion method and system, storage medium and device based on multi-model deep prior, which improves the quality of hyperspectral images and enhances the accuracy of network model output images without increasing hardware costs.
[0004] To achieve the above objectives, the main technical solutions adopted by the present invention include:
[0005] In a first aspect, the present invention provides a fusion method based on multi-model deep prior, comprising: acquiring a sample set; establishing a variable residual hybrid attention network model based on the sample set; generating a multi-model output image based on the sample set through the variable residual hybrid attention network model; establishing a multi-model regularization term based on the multi-model output image through spectral reconstruction; generating an optimal regularization coefficient based on the multi-model regularization term through a genetic algorithm; establishing a fusion equation based on the optimal regularization coefficient and the multi-model regularization term; and generating a hyperspectral image based on the fusion equation through a fast solution method.
[0006] In one embodiment of the present invention, a dataset is obtained; the dataset includes high-resolution hyperspectral images; based on the dataset, a sample set is obtained through normalization and degradation processing; the sample set includes high-resolution multispectral images and low-resolution hyperspectral images.
[0007] In one embodiment of the present invention,
[0008] The degradation process is as follows:
[0009] Y = XBS
[0010] Z = RX
[0011] B = FDF H
[0012] Wherein, Y is the high-resolution multispectral image; Z is the low-resolution hyperspectral image; X is the high-resolution hyperspectral image; B is the fuzzy matrix; F is the discrete Fourier transform matrix; R is the spectral response matrix; S is the downsampling matrix; F H is the inverse of the Fast Fourier Transform; D is a diagonal matrix.
[0013] In one embodiment of the present invention, the specific steps of establishing a variable residual hybrid attention network model based on the sample set and generating a multi-model output image using the variable residual hybrid attention network model based on the sample set include: generating a training set and a dataset by random allocation based on the sample set; establishing an initial variable residual hybrid attention network model based on the training set, the dataset, the variable residual structure, and the outer ring attention method; the initial variable residual hybrid attention network model includes: a first variable residual hybrid attention network model, a second variable residual hybrid attention network model, and a third variable residual hybrid attention network model; training the initial variable residual hybrid attention network model using the L1 loss function to obtain the variable residual hybrid attention network model; and generating the multi-model output image using the variable residual hybrid attention network model based on the dataset and the sample set.
[0014] In one embodiment of the present invention, the genetic algorithm is:
[0015]
[0016]
[0017]
[0018] in, Let be the fitness function of the genetic algorithm; This is the first mutation iteration step of the genetic algorithm;
[0019] This is the second mutation iteration step of the genetic algorithm; t is the current generation; T is the maximum generation; r is a random number, r∈[0,1]; ran is the direction of mutation; Let i be the i-th hyperparameter that needs to be optimized in generation t.
[0020] In one embodiment of the present invention, the specific steps of generating a hyperspectral image based on the fusion equation using a fast solution method include: generating the Sylvester equation based on the fusion equation; and generating a hyperspectral image based on the Sylvester equation using a fast solution method for the Sylvester equation.
[0021] In one embodiment of the present invention, the fusion equation is:
[0022]
[0023] Where J(X) is the fusion equation; α is the coefficient of the multi-model regularization term; Network priors for multiple model outputs;
[0024] The Sylvester equation is:
[0025] H1X + XH2 = H3
[0026] H1 = BS(BS) T
[0027] H2 = R T R+αI C
[0028]
[0029] Where H1 is the first Sylvester equation term; H2 is the second Sylvester equation term; H3 is the third Sylvester equation term; C is the number of channels in the hyperspectral image; I C It is an identity matrix of size C×C.
[0030] Secondly, this invention provides a fusion system based on multi-model deep priors, comprising: a data acquisition module for acquiring a sample set; a first image generation module for establishing a variable residual hybrid attention network model based on the sample set; generating a multi-model output image based on the sample set using the variable residual hybrid attention network model; a coefficient acquisition module for establishing a multi-model regularization term based on the multi-model output image through spectral reconstruction; generating optimal regularization coefficients based on the multi-model regularization term using a genetic algorithm; and a second image generation module for establishing a fusion equation based on the optimal regularization coefficients and the multi-model regularization term; and generating a hyperspectral image based on the fusion equation using a fast solution method.
[0031] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed, implements the fusion method based on multi-model deep prior as described in any of the first aspects above.
[0032] Fourthly, the present invention provides a storage device, including a storage medium and a processor, wherein the storage medium stores a computer program, and the program, when executed by the processor, implements the fusion method based on multi-model deep prior as described in any of the first aspects above.
[0033] The technical solution of the present invention has the following advantages compared with the prior art:
[0034] This invention discloses a fusion method based on multi-model deep priors. In the design of the variable residual hybrid attention network, a variable residual structure is incorporated, enabling the model to adapt to task characteristics and reconstruct high-precision, high-resolution hyperspectral images. By altering the outer spatial attention and channel attention methods of the variable residual hybrid attention network, and using low-resolution hyperspectral images and high-resolution multispectral images as inputs, three different model outputs of deep priors are obtained. Then, the priors output by the variable residual hybrid attention network are reconstructed in the spectral dimension, and the reconstructed results are used as regularization terms added to the mechanistic model. The problem of finding the optimal solution is transformed into solving the Sylvester equation. A fast solution to the Sylvester equation is used to reconstruct the hyperspectral super-resolution image, and a genetic algorithm is used to determine the optimal hyperparameters for different depth priors, thereby obtaining high-quality hyperspectral images. This application utilizes mechanistic knowledge to fuse multiple super-resolution results into a single algorithm, enabling the acquisition of high-quality hyperspectral images without increasing hardware costs, effectively reducing the hardware cost of hyperspectral image acquisition equipment, and significantly improving the restoration quality of super-resolution images. Attached Figure Description
[0035] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein...
[0036] Figure 1 This is a flowchart illustrating the fusion method based on multi-model deep prior provided in a preferred embodiment of the present invention.
[0037] Figure 2 yes Figure 1 A schematic diagram illustrating the process of obtaining the sample set using the multi-model deep prior fusion method described above;
[0038] Figure 3 yes Figure 1 A schematic diagram illustrating the process of generating multi-model output images using the multi-model deep prior fusion method described above;
[0039] Figure 4 yes Figure 1 A schematic diagram of the process for generating hyperspectral images using the multi-model deep prior fusion method described above;
[0040] Figure 5 yes Figure 1 The flowchart illustrates the process of generating hyperspectral images using a fast solution method based on the fusion equation of the multi-model deep prior fusion method.
[0041] Figure 6 A block diagram of a fusion system based on multi-model deep priors provided in an embodiment of the present invention.
[0042] Explanation of reference numerals in the accompanying drawings: 300, Fusion system based on multi-model deep prior; 301, Data acquisition module; 302, First image generation module; 303, Coefficient acquisition module; 304, Second image generation module. Detailed Implementation
[0043] To better explain and facilitate understanding of the present invention, it will be described in detail below with reference to the accompanying drawings and specific embodiments. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a clearer and more thorough understanding of the invention and to fully convey the scope of the invention to those skilled in the art.
[0044] Firstly, referring to Figure 1-5 A preferred embodiment of the present invention provides a fusion method based on multi-model deep priors, comprising:
[0045] S1, Obtain the sample set;
[0046] The specific steps include:
[0047] S11, Obtain the dataset; the dataset includes high-resolution hyperspectral images;
[0048] S12, Based on the dataset, a sample set is obtained through normalization and degradation processing; the sample set includes high-resolution multispectral images and low-resolution hyperspectral images;
[0049] In this embodiment, a dataset is constructed using high-resolution hyperspectral images, which are then normalized. Corresponding high-resolution multispectral images and low-resolution hyperspectral image pairs are obtained through spectral degradation and spatial degradation. The image pairs are then randomly rotated, sliced, and data augmented to construct a sample set. The specific degradation formula is as follows:
[0050] Y = XBS (1)
[0051] Z = RX (2)
[0052] B = FDF H (3)
[0053] Where Y is a high-resolution multispectral image; Z is a low-resolution hyperspectral image; X is a high-resolution hyperspectral image; B is a fuzzy matrix, representing a block cyclic matrix with cyclic blocks, which can be decomposed into equation (3); F is a discrete Fourier transform matrix; R is a spectral response matrix, the value of which can be obtained from the spectral response function of the camera or satellite image; S is a downsampling matrix; F H is the inverse of the Fast Fourier Transform; D is a diagonal matrix.
[0054] S2, Based on the sample set, establish a variable residual hybrid attention network model; Based on the sample set, generate multi-model output images through the variable residual hybrid attention network model;
[0055] The specific steps include:
[0056] S21, Based on the sample set, generate the training set and dataset through random allocation;
[0057] S22. Based on the training set, dataset, variable residual structure, and outer ring attention method, establish an initial variable residual hybrid attention network model; the initial variable residual hybrid attention network model includes: a first variable residual hybrid attention network model, a second variable residual hybrid attention network model, and a third variable residual hybrid attention network model;
[0058] S23, the initial variable residual hybrid attention network model is trained using the L1 loss function to obtain the variable residual hybrid attention network model;
[0059] S24, based on the dataset and sample set, generates multi-model output images through a variable residual hybrid attention network model.
[0060] In this embodiment, the sample set is randomly divided into a training set and a dataset, and an initial variable residual hybrid attention network model is constructed based on the training set and the dataset. During the design of the initial variable residual hybrid attention network model, a variable residual structure is incorporated, enabling the model to adapt to the characteristics of the task and reconstruct high-precision, high-resolution hyperspectral images. Furthermore, by changing the outer ring attention method, three different modes of the initial variable residual hybrid attention network model are obtained. The initial variable residual hybrid attention network model includes a first variable residual hybrid attention network model, a second variable residual hybrid attention network model, and a third variable residual hybrid attention network model. Specifically, the first variable residual hybrid attention network model corresponds to mode one, where the channel attention in mode one uses a compressed excitation module, and the spatial attention input is a low-resolution hyperspectral image. The second variable residual hybrid attention network model corresponds to mode two, where the spatial attention input in mode two is changed to the difference between a high-resolution multispectral image and a low-resolution hyperspectral image. The third variable residual hybrid attention network model corresponds to mode three, where the channel attention in mode three uses an ECA module.
[0061] Then, low-resolution hyperspectral images and high-resolution multispectral images are used as input variables of the initial variable residual hybrid attention network model, and high-resolution hyperspectral images are used as output variables of the initial variable residual hybrid attention network model. The initial variable residual hybrid attention network model under different modes is trained using the L1 loss function to obtain the depth prior results of the output of the initial variable residual hybrid attention network model under three different modes, i.e., the multi-model output images.
[0062] S3: Based on the multi-model output image, establish a multi-model regularization term through spectral reconstruction; based on the multi-model regularization term, generate the optimal regularization coefficient through a genetic algorithm.
[0063] In this embodiment, the multi-model output images are spectrally reconstructed, and the obtained multi-model output images are redistributed according to the degree of restoration of different spectral channels to reconstruct images with different effect levels, thereby improving the quality of the output images; the reconstructed images are used to construct multi-model regularization terms, and the optimized genetic algorithm is used to optimize the regularization coefficients to determine the optimal regularization coefficients;
[0064] Spectral reconstruction methods are network priors of multiple model outputs. The Y spectrum was compared channel by channel, and the channel with the best reduction performance in each of the three depth priors was used to construct the model. The second-best channel is used for construction. The worst channel is used for construction
[0065] The regularization coefficient optimization based on genetic algorithm is to determine the optimal coefficient by calculating the difference between the reconstructed image and the input image pair using an unsupervised method, and to limit the mutation range. The higher the number of generations, the smaller the mutation range.
[0066] Genetic algorithms include:
[0067]
[0068]
[0069]
[0070] Equation (4) is the fitness function of the genetic algorithm for optimization. The smaller the value, the better the hyperparameters of the group. Equations (5) and (6) are the mutation iteration steps of the genetic algorithm. t represents the current generation, T represents the maximum generation, r is a random number that controls the mutation amplitude between [0, 1], and ran represents the direction of mutation, which is a randomly generated 0 or 1 value. Let i be the i-th hyperparameter that needs to be optimized in generation t.
[0071] The mutation algorithm optimized from the above genetic algorithm gradually narrows the mutation range as the population continues to evolve, and mutations only occur near the superior individuals of the previous generation, which can effectively reduce the time for the algorithm to find the optimal hyperparameters.
[0072] S4. Based on the optimal regularization coefficient and the multi-model regularization term, a fusion equation is established; based on the fusion equation, a hyperspectral image is generated using a fast solution method.
[0073] Specifically, it includes:
[0074] S41, Based on the optimal regularization coefficient and the multi-model regularization term, establish the fusion equation;
[0075] The fusion equation is:
[0076]
[0077] Where J(X) is the fusion equation; α is the coefficient of the multi-model regularization term; This is the i-th spectral reconstructed image output by the multi-model approach. (No labels are shown in the above equation.)
[0078] S42, Based on the fusion equation, a hyperspectral image is generated using a fast solution method;
[0079] Specifically, it includes:
[0080] S421, Based on the fusion equation, generate the Sylvester equation;
[0081] S422, based on the Sylvester equation, a hyperspectral image is generated using a fast solution method for the Sylvester equation.
[0082] In this embodiment, a multispectral and hyperspectral image fusion equation containing multiple model regularization terms is constructed. This equation is then differentiated and transformed into the Sylvester equation form. Finally, a fast solution method for the Sylvester equation is used to solve the equation, outputting a high-quality hyperspectral image.
[0083] H1X + XH2 = H3 (8)
[0084] H1 = BS(BS) T (9)
[0085] H2 = R T R+αI C (10)
[0086]
[0087] Wherein, equation (7) is the fusion equation; equation (8) is the derivative of equation (7) and its conversion into the Sylvester equation form; H1 is the first Sylvester equation term; H2 is the second Sylvester equation term; H3 is the third Sylvester equation term; H1, H2, and H3 represent the terms constituting the Sylvester equation, and their values can be obtained by differentiating equation (7) and converting the result to 0; C is the number of channels of the hyperspectral image; I C It is an identity matrix of size C×C.
[0088] Specifically, the steps for solving the Sylvester equation using a fast solution method include:
[0089] Perform eigenvalue decomposition on matrix B:
[0090] B = FDF H
[0091]
[0092] Perform eigenvalue decomposition on matrix H1:
[0093] H1=Q∧Q -1
[0094]
[0095] Channel-by-channel calculation of auxiliary matrix
[0096] For 1=1 to C do
[0097]
[0098] End for
[0099]
[0100] In the above algorithms, d represents the number of diagonal blocks in matrix D, where D i The size is n×n; I is the intermediate matrix variable of the algorithm. n I d Let n and d be identity matrices, 1 d Let H1 be a column vector of length d consisting entirely of 1s, Q be the eigenvector matrix of H1, and Λ be the diagonal matrix of eigenvalues of H1. λ i Let H1 be the i-th eigenvalue; for The i-th channel value; This is the final fused hyperspectral image.
[0101] In this embodiment, the performance of the algorithm proposed in this invention and other similar algorithms are compared on the hyperspectral datasets CAVE and Chikusei. The performance in different super-resolution tasks is compared, and the performance comparison is shown in Tables 1 and 2.
[0102] Table 1
[0103]
[0104] Table 2
[0105]
[0106]
[0107] Table 1 shows the average Qls results of 11 test images on the CAVE dataset at three different scaling ratios (s = 8, 16, 32); Table 2 shows the average Qls results of 8 test images on the Chikusei dataset at three different scaling ratios (s = 8, 16, 32).
[0108] In Tables 1 and 2, the column containing "Ours" represents the performance of the method described in this invention, while the other columns represent the performance of other methods in the prior art. As can be seen from Tables 1 and 2, the method proposed in this invention has a performance improvement of more than 10% compared to other similar algorithms.
[0109] Secondly, refer to Figure 6 As shown, this embodiment provides a fusion system 300 based on multi-model deep priors, including:
[0110] Data acquisition module 301 is used to acquire a sample set;
[0111] The first image generation module 302 is used to establish a variable residual hybrid attention network model based on the sample set; and to generate a multi-model output image based on the sample set through the variable residual hybrid attention network model.
[0112] The coefficient acquisition module 303 is used to establish a multi-model regularization term based on the multi-model output image through spectral reconstruction; and to generate the optimal regularization coefficient based on the multi-model regularization term through a genetic algorithm.
[0113] The second image generation module 304 is used to establish a fusion equation based on the optimal regularization coefficient and the multi-model regularization term; and to generate a hyperspectral image based on the fusion equation using a fast solution method.
[0114] The fusion system based on multi-model deep prior provided in this embodiment is used to implement the steps of the fusion method based on multi-model deep prior provided in the first aspect embodiment of the present invention. Therefore, the fusion system based on multi-model deep prior has all the technical effects of the fusion method based on multi-model deep prior, which will not be repeated here.
[0115] Thirdly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed, implements the fusion method based on multi-model deep prior as described in any of the first aspects above.
[0116] Fourthly, embodiments of the present invention provide a storage device, including a storage medium and a processor, wherein the storage medium stores a computer program, and when the program is executed by the processor, it implements the fusion method based on multi-model deep prior as described in any of the first aspects above.
[0117] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0118] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, then this invention should also include these modifications and variations.
[0119] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make modifications, alterations, substitutions and variations to the above embodiments within the scope of the present invention.
Claims
1. A fusion method based on multi-model deep priors, characterized in that, include: Obtain the sample set; Based on the sample set, a variable residual hybrid attention network model is established; Based on the sample set, a multi-model output image is generated using the variable residual hybrid attention network model; Based on the multi-model output image, a multi-model regularization term is established through spectral reconstruction; based on the multi-model regularization term, an optimal regularization coefficient is generated through a genetic algorithm. A fusion equation is established based on the optimal regularization coefficient and the multi-model regularization term; a hyperspectral image is generated based on the fusion equation using a fast solution method. Based on the sample set, a variable residual hybrid attention network model is established; The specific steps for generating multi-model output images using the variable residual hybrid attention network model based on the sample set include: dividing the sample set into a training set and a dataset by random allocation, and constructing an initial variable residual hybrid attention network model based on the training set and the dataset; incorporating a variable residual structure during the design of the initial variable residual hybrid attention network model to enable the model to adapt to the characteristics of the task and restore a high-precision, high-resolution, hyperspectral image; obtaining three different modes of the initial variable residual hybrid attention network model by changing the outer ring attention method; training the initial variable residual hybrid attention network model in different modes using the L1 loss function to obtain the depth prior results of the output of the initial variable residual hybrid attention network model in the three different modes, and generating multi-model output images; Spectral reconstruction methods are network priors of multiple model outputs. , per spectral channel By comparison, the channel with the best reduction effect in each spectral channel from the three depth priors is used to construct... The second-best channel is used for construction. The worst channel is used for construction. ; The genetic algorithm is as follows: ； ； ； in, Let be the fitness function of the genetic algorithm; This is the first mutation iteration step of the genetic algorithm; This represents the second mutation iteration step of the genetic algorithm; t is the current generation; T is the maximum generation; and r is a random number. [0,1]; ran is the direction of mutation; For the first The first generation that needs to be optimized One super parameter; The fusion equation is: in, The fusion equation is as follows; The coefficients of the multi-model regularization term; The network priors are the outputs of multiple models; X is a high-resolution hyperspectral image, Y is a high-resolution multispectral image, Z is a low-resolution hyperspectral image; B is a fuzzy matrix; S is a downsampling matrix; R is a spectral response matrix. Denotes the Frobenius norm; Let be the regularization coefficient corresponding to the i-th spectral reconstructed image.
2. The fusion method based on multi-model deep priors according to claim 1, characterized in that, The specific steps for obtaining the sample set include: Obtain the dataset; the dataset includes high-resolution hyperspectral images; Based on the dataset, a sample set is obtained through normalization and degradation processing; the sample set includes high-resolution multispectral images and low-resolution hyperspectral images.
3. The fusion method based on multi-model deep priors according to claim 2, characterized in that: The degradation process is as follows: Wherein, Y is the high-resolution multispectral image; Z is the low-resolution hyperspectral image; X is the high-resolution hyperspectral image; B is the fuzzy matrix; F is the discrete Fourier transform matrix; R is the spectral response matrix; and S is the downsampling matrix. is the inverse of the Fast Fourier Transform; D is a diagonal matrix.
4. The fusion method based on multi-model deep priors according to claim 1, characterized in that: The specific steps for generating a hyperspectral image based on the fusion equation using a fast solution method include: Based on the fusion equation, the Sylvester equation is generated; Hyperspectral images are generated based on the Sylvester equation and using a fast solution method for the Sylvester equation.
5. The fusion method based on multi-model deep priors according to claim 4, characterized in that: The Sylvester equation is: ； ； ； ； in, This is a term in the first Sylvester equation; This is a term in the second Sylvester equation; This is the third Sylvester equation term; The number of channels in the hyperspectral image; for × A unit matrix of a certain size.
6. A fusion system based on multi-model deep priors, characterized in that, include: The data acquisition module is used to acquire the sample set; The first image generation module is used to establish a variable residual hybrid attention network model based on the sample set. Based on the sample set, a multi-model output image is generated using the variable residual hybrid attention network model; The coefficient acquisition module is used to establish a multi-model regularization term based on the multi-model output image through spectral reconstruction; and to generate the optimal regularization coefficient based on the multi-model regularization term through a genetic algorithm. The second image generation module is used to establish a fusion equation based on the optimal regularization coefficient and the multi-model regularization term; and to generate a hyperspectral image based on the fusion equation using a fast solution method. Based on the sample set, a variable residual hybrid attention network model is established; The specific steps for generating multi-model output images using the variable residual hybrid attention network model based on the sample set include: dividing the sample set into a training set and a dataset by random allocation, and constructing an initial variable residual hybrid attention network model based on the training set and the dataset; incorporating a variable residual structure during the design of the initial variable residual hybrid attention network model to enable the model to adapt to the characteristics of the task and restore a high-precision, high-resolution, hyperspectral image; obtaining three different modes of the initial variable residual hybrid attention network model by changing the outer ring attention method; training the initial variable residual hybrid attention network model in different modes using the L1 loss function to obtain the depth prior results of the output of the initial variable residual hybrid attention network model in the three different modes, and generating multi-model output images; Spectral reconstruction methods are network priors of multiple model outputs. , per spectral channel By comparison, the channel with the best reduction effect in each spectral channel from the three depth priors is used to construct... The second-best channel is used for construction. The worst channel is used for construction. ; The genetic algorithm is as follows: ； ； ； in, Let be the fitness function of the genetic algorithm; This is the first mutation iteration step of the genetic algorithm; This represents the second mutation iteration step of the genetic algorithm; t is the current generation; T is the maximum generation; and r is a random number. [0,1]; ran is the direction of mutation; For the first The first generation that needs to be optimized One super parameter; The fusion equation is: in, The fusion equation is as follows; The coefficients of the multi-model regularization term; The network priors are the outputs of multiple models; X is a high-resolution hyperspectral image, Y is a high-resolution multispectral image, Z is a low-resolution hyperspectral image; B is a fuzzy matrix; S is a downsampling matrix; R is a spectral response matrix. Denotes the Frobenius norm; Let be the regularization coefficient corresponding to the i-th spectral reconstructed image.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the fusion method based on multi-model deep prior as described in any one of claims 1 to 5.
8. A storage device comprising a storage medium and a processor, wherein the storage medium stores a computer program, characterized in that, When the processor executes the computer program, it implements the fusion method based on multi-model deep prior as described in any one of claims 1 to 5.