Quaternion-based remote sensing hyperspectral image super-resolution fusion algorithm and system

Abstract

The invention relates to the technical field of remote sensing image processing, and particularly discloses a remote sensing hyperspectral image super-resolution fusion algorithm and system based on quaternion, and the method comprises the steps: employing a panchromatic image quaternion feature extraction module, a hyperspectral image quaternion feature extraction module and a self-attention fusion module, constructing a double-U-Net structure network based on a multi-scale residual link and a jump link; and inputting a panchromatic image and a low-resolution hyperspectral image, and calculating to obtain a high-spatial-resolution hyperspectral image. According to the method, a plurality of spectral channels of a hyperspectral image and high-resolution spatial information of a panchromatic image are embedded into four components of a quaternion, and spectral and spatial characteristics are modeled uniformly by means of algebra of the quaternion; and then feature extraction, fusion and reconstruction are carried out through a deep learning network, and finally a fusion image with both high spectral fidelity and high spatial resolution is generated, so that the spatial resolution of the hyperspectral image can be significantly improved, and the spectral consistency of the hyperspectral image can be maintained at the same time.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a remote sensing hyperspectral image super-resolution fusion algorithm and system based on quaternion. Background Art

[0002] Hyperspectral remote sensing images are widely used in agricultural monitoring, land object classification, environmental protection and other fields due to the rich spectral information they contain. However, due to the physical limitations of imaging sensors, hyperspectral images usually have low spatial resolution and are difficult to meet the needs of refined analysis. Therefore, super-resolution fusion technology is used to combine the rich spectral information of hyperspectral images with the high spatial resolution information of panchromatic images or multispectral images to generate images with both high spectral fidelity and high spatial resolution, which has become a hot topic in current research.

[0003] Traditional hyperspectral image super-resolution fusion algorithm methods mostly rely on technical frameworks based on matrix decomposition, sparse representation and deep learning. These methods have significant limitations in modeling the complex spectral-spatial characteristics of hyperspectral data: on the one hand, matrix methods are difficult to fully capture the nonlinear coupling relationship between multi-spectral channels, which easily leads to distortion of spectral information; on the other hand, existing deep learning methods are often based on scalar convolution, which has limited ability to express high-dimensional spectral information, making it difficult for the fusion results to simultaneously ensure spectral consistency and spatial resolution. In addition, traditional methods perform poorly in terms of adaptability to multi-sensor and multi-scene data, making it difficult to meet the needs of actual applications.

[0004] Existing technologies still have significant deficiencies in spectral-spatial coupling modeling, multi-dimensional characteristic expression, cross-scenario adaptability and computational efficiency.

[0005] 1. Insufficient spectral-spatial characteristic modeling capabilities Linear assumption limitation: Traditional matrix decomposition methods (such as non-negative matrix factorization and dictionary learning) are mostly based on linear assumptions and can only handle the linear correlation between spectral and spatial characteristics. It is difficult to effectively capture the nonlinear coupling relationship that is prevalent in hyperspectral images.

[0006] Distortion of spectral information: During the fusion process, sparse representation or scalar-based deep learning methods often overemphasize the improvement of spatial details and ignore the complete preservation of spectral characteristics, resulting in deviations in the spectral fidelity of the fusion results.

[0007] 2. Limited multi-channel high-dimensional information processing capabilities Limitations of scalar computation: Existing deep learning frameworks (such as CNN and GAN) usually use scalar convolution to process multispectral channels. This method models each channel independently, ignoring the global correlation between spectral channels, and it is difficult to express the multidimensional characteristics of hyperspectral images as a whole.

[0008] Information redundancy and fragmentation: The scalar method needs to process each channel separately, resulting in high computational complexity. In addition, the information correlation between channels is easily fragmented, which limits the overall performance of the fusion result.

[0009] 3. Lack of adaptability and robustness Hyperspectral images usually come from different sensors or scenes, and their resolution, spectral range and noise characteristics vary greatly. However, existing methods have the following shortcomings in terms of adaptability and robustness: Poor cross-sensor compatibility: Most methods rely on the image characteristics of specific sensors and are difficult to achieve generality on multi-sensor data.

[0010] Insufficient adaptability to complex scenarios: Existing models often show reduced performance when faced with complex terrain distribution or multi-scale features and cannot meet actual application requirements.

[0011] 4. Computational complexity and efficiency issues Hyperspectral images have a large number of spectral channels, so the computational complexity of data processing is high: Efficiency bottleneck of large-scale data processing: Methods such as matrix decomposition and sparse representation usually involve complex iterative optimization processes with high computational costs and are difficult to meet real-time processing requirements.

[0012] Redundant calculation of deep learning models: Existing deep learning models based on scalar convolution have a large number of parameters, high resource requirements for training and inference processes, and high hardware performance requirements, making them difficult to apply in resource-constrained scenarios. Summary of the invention

[0013] The present invention aims to solve the problem that the spectral correlation between channels cannot be effectively utilized when processing multispectral data, and the integrity of spectral information cannot be guaranteed while improving spatial resolution. To this end, the present invention provides a remote sensing hyperspectral image super-resolution fusion algorithm and system based on quaternions, which embeds multiple spectral channels of hyperspectral images and high-resolution spatial information of panchromatic images into four components of quaternions, and uses quaternion algebra to uniformly model spectral and spatial characteristics; then, feature extraction, fusion and reconstruction are performed through a deep learning network, and finally a fused image with both high spectral fidelity and high spatial resolution is generated. The present invention can significantly improve the spatial resolution of hyperspectral images while maintaining their spectral consistency, providing technical support for the refined analysis and practical application of hyperspectral images.

[0014] The present invention provides a remote sensing hyperspectral image super-resolution fusion algorithm based on quaternion, and the technical scheme adopted is as follows: Constructing a full-color image quaternion feature extraction module, wherein the full-color image quaternion feature extraction module is used to extract features from the full-color image to obtain a full-color image quaternion feature map; Constructing a hyperspectral image quaternion feature extraction module, wherein the hyperspectral image quaternion feature extraction module is used to represent the hyperspectral image in the form of quaternion, perform feature extraction, and obtain a hyperspectral image quaternion feature map; Constructing a self-attention fusion module, wherein the self-attention fusion module is used to fuse and enhance the features of the hyperspectral image quaternion feature map and the panchromatic image quaternion feature map through a self-attention mechanism; Using the panchromatic image quaternion feature extraction module, the hyperspectral image quaternion feature extraction module and the self-attention fusion module, a dual U-Net structure network based on multi-scale residual links and jump connections is constructed, wherein the dual U-Net structure network includes a first U-Net branch and a second U-Net branch, wherein each layer of the first U-Net branch includes a residual module and a panchromatic image quaternion feature extraction module, and inter-layer jump connections; each layer of the second U-Net branch includes a residual module, a hyperspectral image quaternion feature extraction module and a self-attention fusion module, and inter-layer jump connections; The panchromatic image and the low-resolution hyperspectral image are input into the dual U-Net structure network to calculate the hyperspectral image with high spatial resolution.

[0015] Furthermore, the hyperspectral image quaternion feature extraction module represents the hyperspectral image in the form of quaternion and performs feature extraction. The specific process of obtaining the hyperspectral image quaternion feature map is as follows: The three adjacent channels of the low-resolution hyperspectral image are combined to serve as the three components of the imaginary part of the quaternion, and the average of the pixel values ​​at the same position is taken as the real part of the quaternion to obtain the hyperspectral image represented in the form of quaternion; The hyperspectral image represented in the form of quaternion is input into the quaternion convolution kernel for feature extraction to obtain a hyperspectral image quaternion feature map.

[0016] Furthermore, the representation of the hyperspectral image in quaternion form is: k in, Represents the Hth, H+1th, and H+2th channels of the low-resolution hyperspectral image The pixel value of the hyperspectral image represented by quaternion corresponding to the position; Represents the Hth channel of the low-resolution hyperspectral image The pixel value of the position; Represents the H+1th channel of the low-resolution hyperspectral image The pixel value of the position; Represents the H+2th channel of the low-resolution hyperspectral image The pixel value of the position; express The average value of the pixel value at the location; , k is three imaginary units, satisfying the following relationship: ; Quaternion convolution kernel for: in, is the first component of the quaternion convolution kernel, is the second component of the quaternion convolution kernel, is the third component of the quaternion convolution kernel, is the fourth component of the quaternion convolution kernel; Feature extraction based on quaternion multiplication, quaternion feature map of hyperspectral image The calculation process is: in, Hyperspectral imagery represented in quaternion form.

[0017] Furthermore, the full-color image quaternion feature extraction module extracts features from the full-color image, and the specific process of obtaining the full-color image quaternion feature map is as follows: The full-color image is passed through convolution kernels with kernels of k1×k1, 1×k1, and k1×1, respectively, to extract the features of the full-color image spatial scale in different directions, and the three output feature maps are expressed in the form of quaternions to obtain a full-color image expressed in the form of quaternions; The full-color image represented in the form of quaternion is input into the quaternion convolution kernel for feature extraction to obtain a full-color image quaternion feature map.

[0018] Furthermore, the representation of the full-color image in quaternion form is: k in, Represents a full-color image represented in quaternion form The pixel value of the position; Represents the feature map output after the full-color image passes through the k1×k1 convolution kernel The pixel value of the position; Represents the feature map output after the full-color image passes through the 1×k1 convolution kernel The pixel value of the position; Represents the feature map output after the full-color image passes through the k1×1 convolution kernel The pixel value of the position; express The average value of the pixel value at the location; , k is three imaginary units, satisfying the following relationship: .

[0019] Furthermore, the specific process of the self-attention fusion module to fuse and enhance the features of the hyperspectral image quaternion feature map and the panchromatic image quaternion feature map through the self-attention mechanism is as follows: Perform linear transformation on the quaternion feature map of the hyperspectral image to obtain the Query vector and the Key vector, and perform linear transformation on the quaternion feature map of the panchromatic image to obtain the Value vector; Calculate the attention weight A: Among them, Q represents the Query vector, K represents the Key vector, and T represents the transposition. represents the dimension of the Key vector, Represents the Softmax operation; Calculate fusion features : in, Represents the Value vector.

[0020] Furthermore, the first U-Net branch and the second U-Net branch each include five layers, wherein the first layer and the fifth layer are jump connected, and the second layer and the fourth layer are jump connected; the low-level features are fused with the high-level features through the jump connection.

[0021] The present invention also provides a remote sensing hyperspectral image super-resolution fusion system based on quaternion, and the technical scheme adopted is as follows: comprising: a module construction unit, a network construction unit and an input and output unit, The module construction unit is used to construct a full-color image quaternion feature extraction module, a hyperspectral image quaternion feature extraction module and a self-attention fusion module. The full-color image quaternion feature extraction module is used to extract features from the full-color image to obtain a full-color image quaternion feature map; the hyperspectral image quaternion feature extraction module is used to represent the hyperspectral image in the form of quaternions, perform feature extraction, and obtain a hyperspectral image quaternion feature map; the self-attention fusion module is used to perform feature fusion and enhancement on the hyperspectral image quaternion feature map and the full-color image quaternion feature map through a self-attention mechanism; The network construction unit is used to construct a dual U-Net structure network based on multi-scale residual links and jump connections by using the panchromatic image quaternion feature extraction module, the hyperspectral image quaternion feature extraction module and the self-attention fusion module, wherein the dual U-Net structure network includes a first U-Net branch and a second U-Net branch, wherein each layer of the first U-Net branch includes a residual module and a panchromatic image quaternion feature extraction module, and inter-layer jump connections; each layer of the second U-Net branch includes a residual module, a hyperspectral image quaternion feature extraction module and a self-attention fusion module, and inter-layer jump connections; The input-output unit is used to input the full-color image and the low-resolution hyperspectral image into the dual U-Net structure network, and output the high-spatial-resolution hyperspectral image calculated by the dual U-Net structure network.

[0022] The above one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects: The present invention captures the relationship between adjacent channels of low-resolution hyperspectral images through quaternion convolution, without losing important information of each channel, so that the features of the spectral channel of each hyperspectral image are fully extracted; the feature map obtained by multi-directional feature extraction using quaternions is integrated on the full-color image to more effectively extract the spatial detail information of the PAN image. The present invention has the characteristics of high efficiency, strong robustness, and good spectral fidelity, and can significantly improve the spatial resolution of hyperspectral images while maintaining their spectral consistency, providing technical support for the refined analysis and practical application of hyperspectral images.

[0023] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 It is a flow chart of the method provided by the present invention.

[0026] Figure 2 It is a structural schematic diagram of a full-color image quaternion feature extraction module provided by the present invention.

[0027] Figure 3 It is a structural schematic diagram of the hyperspectral image quaternion feature extraction module provided by the present invention.

[0028] Figure 4 It is a structural schematic diagram of the self-attention fusion module provided by the present invention.

[0029] Figure 5 It is a structural diagram of the dual U-Net structure network provided by the present invention.

[0030] Figure 6 This is a diagram of subjective comparison experiment results of the Botswana data set provided by the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present invention. The following embodiments are used to illustrate the present invention, but cannot be used to limit the scope of the present invention.

[0032] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0033] Combine the following Figures 1 to 6 The present invention is further described in detail, and a quaternion-based remote sensing hyperspectral image super-resolution fusion algorithm and system of the present invention are described: In this embodiment, Figure 1 As shown, a remote sensing hyperspectral image super-resolution fusion algorithm based on quaternion is provided, comprising the following steps: Step 1: construct a full-color image quaternion feature extraction module, which is used to extract features from a full-color image to obtain a full-color image quaternion feature map.

[0034] like Figure 2 As shown, the full-color image quaternion feature extraction module extracts features from the full-color image, and the specific process of obtaining the full-color image quaternion feature map is as follows: The full-color image is passed through three convolution kernels of different kernel sizes and shapes, and the LeakyReLU activation function. In this embodiment, three convolution kernels with kernels of k1×k1, 1×k1, and k1×1 are selected to extract the features of the full-color image spatial scale in different directions. The three output feature maps are transformed into four elements and expressed in the form of quaternions to obtain a full-color image expressed in the form of quaternions. .

[0035] Among them, the representation of the full-color image in quaternion form is: k in, Represents a full-color image represented in quaternion form The pixel value of the position; Represents the feature map output after the full-color image passes through the k1×k1 convolution kernel The pixel value of the position; Represents the feature map output after the full-color image passes through the 1×k1 convolution kernel The pixel value of the position; Represents the feature map output after the full-color image passes through the k1×1 convolution kernel The pixel value of the position; express The average value of the pixel value at the location; , k is three imaginary units, satisfying the following relationship: .

[0036] The full-color image represented in the form of quaternion is input into the quaternion convolution kernel for feature extraction to obtain a full-color image quaternion feature map.

[0037] Step 2: Construct a hyperspectral image quaternion feature extraction module, which is used to represent the hyperspectral image in the form of quaternions, perform feature extraction, and obtain a hyperspectral image quaternion feature map.

[0038] like Figure 3 As shown in the figure, the hyperspectral image quaternion feature extraction module represents the hyperspectral image in the form of quaternion, performs feature extraction, and obtains the specific process of the hyperspectral image quaternion feature map as follows: The three adjacent channels of the low-resolution hyperspectral image are combined to serve as the three components of the imaginary part of the quaternion. The average of the pixel values ​​at the same position is taken as the real part of the quaternion. A four-element transformation is performed to obtain a hyperspectral image represented in the form of a quaternion.

[0039] The representation of a hyperspectral image in quaternion form is: k in, Represents the Hth, H+1th, and H+2th channels of the low-resolution hyperspectral image The pixel value of the hyperspectral image represented by quaternion corresponding to the position; Represents the Hth channel of the low-resolution hyperspectral image The pixel value of the position; Represents the H+1th channel of the low-resolution hyperspectral image The pixel value of the position; Represents the H+2th channel of the low-resolution hyperspectral image The pixel value of the position; express The average value of the pixel value at the location; , k is three imaginary units, satisfying the following relationship: .

[0040] The hyperspectral image represented in the form of quaternion is input into the quaternion convolution kernel for feature extraction to obtain a hyperspectral image quaternion feature map.

[0041] Among them, the quaternion convolution kernel for: in, is the first component of the quaternion convolution kernel, is the second component of the quaternion convolution kernel, is the third component of the quaternion convolution kernel, It is the fourth component of the quaternion convolution kernel.

[0042] For quaternion convolution, this dot multiplication is done based on quaternion multiplication. Feature extraction based on quaternion multiplication, quaternion feature map of hyperspectral image The calculation process is: in, Hyperspectral imagery represented in quaternion form.

[0043] Quaternion feature extraction is performed, and independent addition and multiplication operations are performed for each channel (real and imaginary parts). Each convolution calculation produces a quaternion feature map output, which contains four components (one real part and three imaginary parts).

[0044] Step 3: Construct a self-attention fusion module, which is used to fuse and enhance the features of the hyperspectral image quaternion feature map and the panchromatic image quaternion feature map through a self-attention mechanism.

[0045] like Figure 4 As shown in the figure, the specific process of the self-attention fusion module to fuse and enhance the features of the hyperspectral image quaternion feature map and the panchromatic image quaternion feature map through the self-attention mechanism is as follows: The hyperspectral image quaternion feature map The Query vector and Key vector are obtained by reshaping and linear transformation through the reshaping layer and linear layer respectively, and the quaternion feature map of the full-color image is transformed into The Value vector is obtained by reshaping and linear transformation through the reshaping layer and the linear layer in turn. Thus, the three vectors required by the self-attention block are obtained.

[0046] The attention score is obtained by calculating the element-by-element product of the Query vector and the Key vector, and then normalized by the Softmax function to calculate the attention weight A. The calculation formula is: Among them, Q represents the Query vector, K represents the Key vector, and T represents the transposition. represents the dimension of the Key vector, Represents the Softmax operation. The Softmax operation ensures that each row of the resulting weight matrix represents the correlation between different channels.

[0047] Apply the attention weight A to the Value vector, and then calculate the weighted sum to get the fusion feature : in, Represents the Value vector.

[0048] Fusion Features After reshaping, the fusion graph is obtained.

[0049] Step 4: Using the panchromatic image quaternion feature extraction module, the hyperspectral image quaternion feature extraction module and the self-attention fusion module, a dual U-Net structure network based on multi-scale residual links and skip connections is constructed, as shown in Figure 5 shown.

[0050] The dual U-Net structure network includes two parallel branches, namely the first U-Net branch and the second U-Net branch. The first U-Net branch is used to process full-color images, and the second U-Net branch is used to process hyperspectral images. Each layer of the first U-Net branch includes a residual module and a full-color image quaternion feature extraction module, and inter-layer jump connections; each layer of the second U-Net branch includes a residual module, a hyperspectral image quaternion feature extraction module and a self-attention fusion module, and inter-layer jump connections.

[0051] In this embodiment, the first U-Net branch and the second U-Net branch each include five layers, wherein the first layer and the fifth layer are jump connected, and the second layer and the fourth layer are jump connected; the low-level features are fused with the high-level features through the jump connection.

[0052] Step 5: Input the panchromatic image and the low-resolution hyperspectral image into the dual U-Net structure network, and calculate the high spatial resolution hyperspectral image through the calculation of the dual U-Net structure network.

[0053] In this embodiment, the full-color image is input into the first U-Net branch. In each layer of the first U-Net branch, the full-color image quaternion feature map is calculated through the residual module and the full-color image quaternion feature extraction module in turn, and is input into the next layer, as well as the self-attention fusion module of the corresponding layer of the second U-Net branch. If the layer is jump-connected, the full-color image quaternion feature map also needs to be spliced ​​to the output of the full-color image quaternion feature extraction module of the layer jump-connected to it. The low-resolution hyperspectral image is input into the second U-Net branch. In each layer of the second U-Net branch, the residual module, the hyperspectral image quaternion feature extraction module and the self-attention fusion module are sequentially passed through the output result to the next layer. If the layer is jump-connected, the output result also needs to be spliced ​​to the corresponding output. The full-color image and the low-resolution hyperspectral image are calculated through the entire dual U-Net structure network to obtain a high-spatial-resolution hyperspectral image.

[0054] In order to verify the effect of this method, this embodiment conducted quantitative indicator result comparison experiments and subjective comparison experiments on the public data set Botswana with multiple most advanced algorithms in the world. The most advanced algorithms in the world include GT, CNMF, DARN, DSNet, FPFNet, Hyperkite, Refiner and TreeNet. In order to better observe the difference between the fusion results, the mean absolute error (MAE) diagram between the fusion results and the true value (GTs) is calculated and displayed, and the local area is magnified in the lower right corner of the result, and the corresponding RGB image is displayed in the lower left corner, as shown in Figure 1. Figure 6 As shown in the figure. These images show that the dark blue area in the MAE graph of this method (ous) is larger, indicating that the fusion error is the smallest. As shown in Table 1, the quantitative index results also show that the quantitative result index of this method is better than the index of the international advanced algorithm in recent years.

[0055] Table 1. Comparison of quantitative indicators of Botswana dataset experimental results

[0056] In the table, ↑ means the larger the result, the better, and ↓ means the smaller the result, the better. The indicators selected are linear correlation coefficient (CC), spectral angle mapping (SAM), root mean square error (RMSE), global relative error (ERGAS) and peak signal-to-noise ratio (PSNR).

[0057] This embodiment also provides a quaternion-based remote sensing hyperspectral image super-resolution fusion system, which adopts the following technical solution: comprising: a module construction unit, a network construction unit and an input-output unit connected in sequence.

[0058] The module construction unit is used to construct a full-color image quaternion feature extraction module, a hyperspectral image quaternion feature extraction module and a self-attention fusion module. The full-color image quaternion feature extraction module is used to extract features from the full-color image to obtain a full-color image quaternion feature map; the hyperspectral image quaternion feature extraction module is used to represent the hyperspectral image in the form of quaternions, perform feature extraction, and obtain a hyperspectral image quaternion feature map; the self-attention fusion module is used to perform feature fusion and enhancement on the hyperspectral image quaternion feature map and the full-color image quaternion feature map through a self-attention mechanism.

[0059] The network construction unit is used to construct a dual U-Net structure network based on multi-scale residual links and jump connections by using the full-color image quaternion feature extraction module, the hyperspectral image quaternion feature extraction module and the self-attention fusion module. The dual U-Net structure network includes a first U-Net branch and a second U-Net branch. Each layer of the first U-Net branch includes a residual module and a full-color image quaternion feature extraction module, and inter-layer jump connections; each layer of the second U-Net branch includes a residual module, a hyperspectral image quaternion feature extraction module and a self-attention fusion module, and inter-layer jump connections.

[0060] The input-output unit is used to input the full-color image and the low-resolution hyperspectral image into the dual U-Net structure network, and output the high-spatial-resolution hyperspectral image calculated by the dual U-Net structure network.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A quaternion-based remote sensing hyperspectral image super-resolution fusion algorithm, characterized in that: include: Constructing a full-color image quaternion feature extraction module, wherein the full-color image quaternion feature extraction module is used to extract features from the full-color image to obtain a full-color image quaternion feature map; Constructing a hyperspectral image quaternion feature extraction module, wherein the hyperspectral image quaternion feature extraction module is used to represent the hyperspectral image in the form of quaternion, perform feature extraction, and obtain a hyperspectral image quaternion feature map; Constructing a self-attention fusion module, wherein the self-attention fusion module is used to fuse and enhance the features of the hyperspectral image quaternion feature map and the panchromatic image quaternion feature map through a self-attention mechanism; Using the panchromatic image quaternion feature extraction module, the hyperspectral image quaternion feature extraction module and the self-attention fusion module, a dual U-Net structure network based on multi-scale residual links and jump connections is constructed, wherein the dual U-Net structure network includes a first U-Net branch and a second U-Net branch, wherein each layer of the first U-Net branch includes a residual module and a panchromatic image quaternion feature extraction module, and inter-layer jump connections; each layer of the second U-Net branch includes a residual module, a hyperspectral image quaternion feature extraction module and a self-attention fusion module, and inter-layer jump connections; The panchromatic image and the low-resolution hyperspectral image are input into the dual U-Net structure network to calculate the hyperspectral image with high spatial resolution.

2. The quaternion-based remote sensing hyperspectral image super-resolution fusion algorithm according to claim 1, characterized in that: The hyperspectral image quaternion feature extraction module represents the hyperspectral image in the form of quaternion and extracts the features. The specific process of obtaining the hyperspectral image quaternion feature map is as follows: The three adjacent channels of the low-resolution hyperspectral image are combined to serve as the three components of the imaginary part of the quaternion, and the average of the pixel values ​​at the same position is taken as the real part of the quaternion to obtain the hyperspectral image represented in the form of quaternion; The hyperspectral image represented in the form of quaternion is input into the quaternion convolution kernel for feature extraction to obtain a hyperspectral image quaternion feature map.

3. The quaternion-based remote sensing hyperspectral image super-resolution fusion algorithm according to claim 2, characterized in that: The representation of a hyperspectral image in quaternion form is: k in, Represents the Hth, H+1th, and H+2th channels of the low-resolution hyperspectral image The pixel value of the hyperspectral image represented by quaternion corresponding to the position; Represents the Hth channel of the low-resolution hyperspectral image The pixel value of the position; Represents the H+1th channel of the low-resolution hyperspectral image The pixel value of the position; Represents the H+2th channel of the low-resolution hyperspectral image The pixel value of the position; express The average value of the pixel values ​​at the location; , k is three imaginary units, satisfying the following relationship: ； Quaternion convolution kernel for: in, is the first component of the quaternion convolution kernel, is the second component of the quaternion convolution kernel, is the third component of the quaternion convolution kernel, is the fourth component of the quaternion convolution kernel; Feature extraction based on quaternion multiplication, quaternion feature map of hyperspectral image The calculation process is: in, Hyperspectral imagery represented in quaternion form.

4. The quaternion-based remote sensing hyperspectral image super-resolution fusion algorithm according to claim 1, characterized in that: The full-color image quaternion feature extraction module extracts features from the full-color image, and the specific process of obtaining the full-color image quaternion feature map is as follows: The full-color image is passed through convolution kernels with kernels of k1×k1, 1×k1, and k1×1, respectively, to extract the features of the full-color image spatial scale in different directions, and the three output feature maps are expressed in the form of quaternions to obtain a full-color image expressed in the form of quaternions; The full-color image represented in the form of quaternion is input into the quaternion convolution kernel for feature extraction to obtain a full-color image quaternion feature map.

5. The quaternion-based remote sensing hyperspectral image super-resolution fusion algorithm according to claim 4, characterized in that: The representation of a full-color image in quaternion form is: k in, Represents a full-color image represented in quaternion form The pixel value of the position; Represents the feature map output after the full-color image passes through the k1×k1 convolution kernel The pixel value of the position; Represents the feature map output after the full-color image passes through the 1×k1 convolution kernel The pixel value of the position; Represents the feature map output after the full-color image passes through the k1×1 convolution kernel The pixel value of the position; express The average value of the pixel values ​​at the location; , k is three imaginary units, satisfying the following relationship: 。 6. The quaternion-based remote sensing hyperspectral image super-resolution fusion algorithm according to claim 1, characterized in that: The specific process of the self-attention fusion module to fuse and enhance the features of the hyperspectral image quaternion feature map and the panchromatic image quaternion feature map through the self-attention mechanism is as follows: Perform linear transformation on the quaternion feature map of the hyperspectral image to obtain the Query vector and the Key vector, and perform linear transformation on the quaternion feature map of the panchromatic image to obtain the Value vector; Calculate the attention weight A: Among them, Q represents the Query vector, K represents the Key vector, and T represents the transposition. represents the dimension of the Key vector, Represents the Softmax operation; Calculate fusion features : in, Represents the Value vector.

7. The quaternion-based remote sensing hyperspectral image super-resolution fusion algorithm according to claim 1, characterized in that: The first U-Net branch and the second U-Net branch each include five layers, wherein the first layer and the fifth layer are jump connected, and the second layer and the fourth layer are jump connected; the low-level features are fused with the high-level features through the jump connection.

8. A quaternion-based remote sensing hyperspectral image super-resolution fusion system, characterized in that: The method is used to execute the quaternion-based remote sensing hyperspectral image super-resolution fusion algorithm according to any one of claims 1 to 7, comprising: a module construction unit, a network construction unit and an input and output unit, The module construction unit is used to construct a full-color image quaternion feature extraction module, a hyperspectral image quaternion feature extraction module and a self-attention fusion module. The full-color image quaternion feature extraction module is used to extract features from the full-color image to obtain a full-color image quaternion feature map; the hyperspectral image quaternion feature extraction module is used to represent the hyperspectral image in the form of quaternions, perform feature extraction, and obtain a hyperspectral image quaternion feature map; the self-attention fusion module is used to perform feature fusion and enhancement on the hyperspectral image quaternion feature map and the full-color image quaternion feature map through a self-attention mechanism; The network construction unit is used to construct a dual U-Net structure network based on multi-scale residual links and jump connections by using the panchromatic image quaternion feature extraction module, the hyperspectral image quaternion feature extraction module and the self-attention fusion module, wherein the dual U-Net structure network includes a first U-Net branch and a second U-Net branch, wherein each layer of the first U-Net branch includes a residual module and a panchromatic image quaternion feature extraction module, and inter-layer jump connections; each layer of the second U-Net branch includes a residual module, a hyperspectral image quaternion feature extraction module and a self-attention fusion module, and inter-layer jump connections; The input-output unit is used to input the full-color image and the low-resolution hyperspectral image into the dual U-Net structure network, and output the high-spatial-resolution hyperspectral image calculated by the dual U-Net structure network.

Images