Multispectral and hyperspectral image fusion methods based on cross-scale Octave convolutional networks

Abstract

This invention discloses a multispectral and hyperspectral image fusion method based on a cross-scale Octave convolutional network, comprising the following steps: High spatial resolution hyperspectral images used for training are spatially downsampled and spectrally downsampled to simulate multispectral images (HR-MSI) and hyperspectral images (LR-HSI) captured by a sensor in the same region at the same time; a certain proportion of the resulting simulated multispectral and hyperspectral images are selected as the training set. For the training set, tensors corresponding to each training pair are generated, and each pair of training tensors is input into the convolutional network to obtain the optimal model parameters for multispectral and hyperspectral image fusion; the trained model is used to fuse low spatial resolution hyperspectral images and high spatial resolution multispectral images captured by the sensor to obtain a hyperspectral image with both high spatial and high spectral resolution.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a multispectral and hyperspectral image fusion method based on cross-scale Octave convolutional networks. Background Technology

[0002] Remote sensing technology allows us to observe things on the Earth's surface that are invisible to the naked eye. Remote sensing images are categorized into PAN images, multispectral images (MSI), and hyperspectral images (HSI), among others. In recent years, with the development of satellite sensing technology, hyperspectral images have been widely used in various fields due to their rich spatial and spectral information, attracting increasing attention. However, due to physical and technological limitations, the information provided by a single sensor is incomplete or inaccurate. The resolution of remote sensing images acquired from a single sensor can only be a compromise between high spatial and high spectral resolution. For example, acquired hyperspectral images typically have high spectral resolution but coarse spatial resolution; while multispectral images have high spatial resolution but low spectral resolution. Therefore, how to simultaneously obtain hyperspectral images with both high spatial and high spectral resolution has become a hot topic. Data fusion from sensors is an important method to improve the spatial resolution of hyperspectral images. Its goal is to integrate complementary and non-redundant information to provide an image that better understands the entire scene. Remote sensing images have strong applications in target recognition, classification, and land exploration.

[0003] With the increasing demand for hyperspectral data, many remote sensing image fusion methods have been proposed in recent years. These methods can be broadly classified into three different strategies: component substitution-based methods, multi-resolution analysis methods, and model-based methods.

[0004] In component-replacement-based methods, a spectral transformation is performed on the MS image, replacing one or all components with a histogram-matched PAN image. Commonly used component-replacement-based methods include HIS and PCA. The advantage of component-replacement-based methods is that they preserve geometric structure well, resulting in output images with fine spatial details. However, they also suffer from spectral or color distortion due to the additional spectral information introduced during component replacement. Multi-resolution analysis-based methods posit that spatial details missing in the MS image can be inferred from the PAN image and injected into the MS image. To improve the effectiveness of detail injection, wavelets and other methods have been proposed to perform data fusion tasks. Model-based methods reconstruct the HR-HS image by building a model based on the assumed relationship between LR-HS and HR-MS images, transforming the image reconstruction problem into minimizing the objective function. Although model-based methods trade off between spatial resolution enhancement and spectral preservation, they remain ill-posed and have high computational complexity.

[0005] In response to the shortcomings of traditional remote sensing image fusion methods, in recent years, with the rapid development of deep learning technology, many deep learning-based remote sensing image fusion methods have been proposed to address the super-resolution problem of hyperspectral images. Convolutional neural networks have gained increasing popularity due to their superior performance, and a series of methods have been proposed to fuse observed high spatial resolution multispectral images (HR-MSI) with low spatial resolution high image (LR-HSI) to obtain images with both high spatial resolution and high spectral resolution. Summary of the Invention

[0006] The purpose of this invention is to overcome the problem that existing sensors struggle to directly capture hyperspectral images with both high spatial and spectral resolution, and to solve this problem efficiently and accurately. This invention presents a multispectral and hyperspectral image fusion method based on a cross-scale Octave convolutional network. This method utilizes deep learning theory to process high-dimensional tensors from different directions, fully extracting and integrating spatial and spectral features from both images. It extracts useful information while minimizing information clutter and redundancy, thus obtaining a good fusion result. To achieve the above objective, the technical solution adopted by this invention is as follows:

[0007] A multispectral and hyperspectral image fusion method based on cross-scale Octave convolutional networks includes the following steps:

[0008] Step (A): Blur and spatially downsample the hyperspectral dataset to obtain a low spatial resolution hyperspectral image (LR-HSI) and perform spectral downsampling to obtain a high spatial resolution multispectral image (HR-MSI), and use them as a pair of training samples;

[0009] Step (B): Select a certain proportion of training sample pairs as the training set, and generate corresponding tensors from these paired sample pairs; use these paired tensor pairs as input to the proposed convolutional network model to train the proposed cross-scale octave convolutional neural network, and obtain model parameters suitable for this hyperspectral dataset.

[0010] Step (C): Test the trained model on the test set and obtain the corresponding test results.

[0011] Preferably, in step (A), the hyperspectral dataset is blurred and spatially downsampled to obtain a low spatial resolution hyperspectral image (LR-HSI) and spectrally downsampled to obtain a high spatial resolution multispectral image (HR-MSI) as a pair of training samples, including the following specific steps:

[0012] Step (A1): Select the hyperspectral image (HR-HSI) dataset as the label;

[0013] Step (A2): Select a certain proportion of the hyperspectral images (HR-HSI) as the training set;

[0014] Step (A3): Spatial downsampling and spectral downsampling are performed on the training set to obtain paired training pairs.

[0015] Preferably, in step (A2), in order to reduce the experimental burden, the hyperspectral image (HR-HSI) of the training set is divided into a series of training blocks of size W*H.

[0016] Preferably, step (A3) includes the following specific steps:

[0017] Step (A31): The training block is sequentially subjected to Gaussian blur and downsampling to obtain a low spatial resolution hyperspectral image (LR-HSI) corresponding to the training block after spatial downsampling;

[0018] Step (A32): Apply the spectral response function to the training block for spectral downsampling to obtain a high spatial resolution multispectral image (HR-MSI) corresponding to the training block after spectral downsampling;

[0019] Step (A33): Use the low spatial resolution hyperspectral image (LR-HSI) and the high spatial resolution multispectral image (HR-MSI) as a training sample pair.

[0020] Preferably, in step (B), a certain proportion of training sample pairs are selected as the training set, and these paired sample pairs are used to generate corresponding tensors; these paired tensor pairs are then used as inputs to the convolutional network model to train the proposed cross-scale Octave convolutional neural network, thereby obtaining model parameters suitable for the hyperspectral dataset, including the following specific steps:

[0021] Step (B1): Use the hyperspectral dataset as the label for the training set, and perform spatial downsampling and spectral downsampling on the hyperspectral dataset to generate paired training sets;

[0022] Step (B2): Form a corresponding tensor for each pair of training samples in the training set;

[0023] Step (B3): Use the tensors and their labels corresponding to the training sample pairs as input to train the proposed cross-scale Octave convolutional neural network and obtain the model parameters suitable for this hyperspectral dataset.

[0024] Preferably, in step (B2), the low spatial resolution high image (LR-HSI) in the paired training pairs is extracted to form a tensor of w*h*L, and the high spatial resolution multispectral image (HR-MSI) is extracted to form a tensor of W*H*3, where W,H,l and w,h,L are the width, length and number of channels of the multispectral image and the hyperspectral image, respectively, and W / w=H / h=8.

[0025] Preferably, step (B3) includes the following specific steps:

[0026] Step (B31): Use discrete wavelet decomposition to extract low-frequency information from the high spatial resolution multispectral image (HR-MSI), and stitch it along the channel dimension with the upsampled low spatial resolution hyperspectral image (LR-HSI) by four times. Use a 3*3*64 convolution kernel to extract the spatial and spectral information of the two images simultaneously through a spatial spectral feature extraction network at four times the scale. Then, pass it through 3*3*128 and 3*3*L convolution kernels to form a (W / 2)*(H / 2)*L feature tensor.

[0027] Step (B32): Add the (W / 2)*(H / 2)*L feature tensor to the upsampled low spatial resolution hyperspectral image (LR-HSI) by four times to obtain a new (W / 2)*(H / 2)*L feature tensor, and use the feature tensor obtained after upsampling by two times as the input of the spatial-spectral feature extraction network at an eight-fold scale.

[0028] Step (B33): After using a 3*3*64 convolution kernel, the feature tensor is downsampled by two and four times respectively to obtain three feature tensors of different scales. These are used as inputs to a series of cross-scale octave convolutional networks to extract joint spatial and spectral features. Finally, the feature tensor obtained from the output of each cross-scale octave convolutional network is added to the low spatial resolution hyperspectral image (LR-HSI) upsampled by eight times, resulting in a total of 6 feature tensors of size W*H*L.

[0029] Step (B34): Name the first five feature tensors obtained in step (B33) respectively. Name the last feature tensor O. SR The reconstructed image is obtained by taking the multispectral image and hyperspectral image as input;

[0030] Step (B35): Input the 6 feature tensors of each pixel in step (B34) and their corresponding label information into the loss function to obtain the loss value, and perform backpropagation on the convolutional neural network based on the loss value to update the weight parameters;

[0031] The loss function of a convolutional network is:

[0032] l1(Z, O) SR )=||ZO SR ||1

[0033]

[0034]

[0035]

[0036] Among them, Z and O SR Z(i,j), O represent the target image and the reconstructed image, respectively; SR (i,j) represent Z and O respectively. SR The value of pixel (i,j); the network has n outputs, and the nth output is used as the reconstructed image; The k-th output of the network, k = 1, 2, ..., n-1; to avoid the denominator being 0, c = 1e-10; β and These are the trade-off factors, with the networks set to 1, 1, and 0.1 respectively;

[0037] Step (B36): When iteration = 20000, training ends, and the model parameters corresponding to the most recent decrease in loss value are saved as the optimal weight parameters.

[0038] Preferably, in step (C), the trained model is used to test the low spatial resolution hyperspectral image (LR-HSI) and high spatial resolution multispectral image (HR-MSI) captured by the sensor at the same time and in the same area to obtain a hyperspectral image with both high spatial resolution and high spectral resolution. Specifically, the tensor pairs of the tested multispectral and hyperspectral images are input into the trained model to obtain the result as the fusion result.

[0039] The beneficial effects of this invention are: the multispectral and hyperspectral image fusion method based on cross-scale Octave convolutional networks fully extracts spatial-spectral features from two images by using cross-scale Octave convolutional networks, and uses a spatial-spectral attention mechanism to make the network focus on those spatial-spectral information that are useful for image recovery, and reduce the focus on useless spatial-spectral information. This maximizes the reconstruction of spatial and spectral information while reducing information loss and confusion, thus obtaining satisfactory experimental results. Attached Figure Description

[0040] Figure 1This is a flowchart of the overall process of the multispectral and hyperspectral image fusion method based on cross-scale Octave convolutional networks of the present invention.

[0041] Figure 2 This is a flowchart of the spatial-spectral information reconstruction network at four times scale in the image fusion method of the present invention;

[0042] Figure 3 This is a flowchart of the spatial-spectral information reconstruction network at an eight-fold scale in the image fusion method of the present invention;

[0043] Figure 4 It is a hyperspectral image with low spatial resolution;

[0044] Figure 5 It is a high spatial resolution multispectral image;

[0045] Figure 6 This is a diagram showing the result of multispectral and hyperspectral image fusion according to the present invention;

[0046] Figure 7 It is a true hyperspectral image of the ground;

[0047] Figure 8 These are hyperspectral images of the WDCM dataset after spatial information degradation.

[0048] Figure 9 These are multispectral images obtained by degrading the spectral information of the WDCM dataset;

[0049] Figure 10 This is an image showing the multispectral and hyperspectral image fusion results of this invention on the WDCM dataset;

[0050] Figure 11 These are ground truth images from the WDCM dataset;

[0051] Figure 12 These are hyperspectral images of the IP dataset after spatial information degradation.

[0052] Figure 13 It is a multispectral image after spectral information degradation of the IP dataset;

[0053] Figure 14 This is a diagram showing the multispectral and hyperspectral image fusion results of this invention on the IP dataset. Detailed Implementation

[0054] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The advantages and features of the present invention will become clearer from the following description and claims. It should be noted that the drawings are all in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of the present invention.

[0055] like Figure 1 As shown, the multispectral and hyperspectral image fusion method based on cross-scale Octave convolutional networks of the present invention includes the following steps:

[0056] Step (A): Blur and spatially downsample the hyperspectral dataset to obtain a low spatial resolution hyperspectral image (LR-HSI) (e.g. Figure 4 The high spatial resolution multispectral image (HR-MSI) obtained by spectral downsampling (as shown in Figure 5) is used as a pair of training samples;

[0057] Step (B): Select a certain proportion of training sample pairs as the training set, generate corresponding tensors from these paired sample pairs, and use these paired tensor pairs as input to the convolutional network model to train the proposed cross-scale Octave convolutional neural network, thereby obtaining model parameters suitable for this hyperspectral dataset.

[0058] Step (C): Test the trained model on the test set and obtain the corresponding test results.

[0059] Specifically, in step (A), the hyperspectral dataset is blurred and spatially downsampled to obtain a low spatial resolution hyperspectral image (LR-HSI) and spectrally downsampled to obtain a high spatial resolution multispectral image (HR-MSI) as a pair of training samples, including the following specific steps:

[0060] Step (A1): Select the hyperspectral image (HR-HSI) dataset as the label;

[0061] Step (A2): Select a certain proportion of the hyperspectral images (HR-HSI) as the training set;

[0062] Step (A3): Spatial downsampling and spectral downsampling are performed on the training set to obtain paired training pairs;

[0063] Furthermore, in step (A2), in order to reduce the experimental burden, the hyperspectral images (HR-HSI) of the training set are divided into training blocks of size W*H.

[0064] Specifically, step (A3) includes the following steps:

[0065] Step (A31): The training block is sequentially subjected to Gaussian blur and downsampling to obtain a low spatial resolution hyperspectral image (LR-HSI) corresponding to the training block after spatial downsampling;

[0066] Step (A32): Apply the spectral response function to the training block for spectral downsampling to obtain a high spatial resolution multispectral image (HR-MSI) corresponding to the training block after spectral downsampling;

[0067] Step (A33): Use the low spatial resolution hyperspectral image (LR-HSI) and the high spatial resolution multispectral image (HR-MSI) as a training sample pair.

[0068] Specifically, in step (B), a certain proportion of training sample pairs are selected as the training set, and these paired sample pairs are used to generate corresponding tensors; these paired tensor pairs are then used as inputs to the convolutional network model to train the proposed cross-scale Octave convolutional neural network, obtaining model parameters suitable for the hyperspectral dataset, including the following specific steps:

[0069] Step (B1): Select a certain number of training sample pairs as the training set;

[0070] Step (B2): Form a corresponding tensor for each pair of training samples in the training set;

[0071] Step (B3): Use the tensors and their labels corresponding to the training sample pairs as input to train the proposed cross-scale Octave convolutional neural network and obtain the model parameters suitable for this hyperspectral dataset.

[0072] Specifically, in step (B2), the low spatial resolution high image (LR-HSI) in the paired training pairs is extracted to form a tensor of w*h*L, and the high spatial resolution multispectral image (HR-MSI) is extracted to form a tensor of W*H*l, where W,H,l and w,h,L are the width, length and number of channels of the multispectral image and the hyperspectral image, respectively, and W / w=H / h=8.

[0073] Specifically, step (B3) uses the tensors and their labels corresponding to the training sample pairs as input to train the proposed cross-scale Octave convolutional neural network, obtaining model parameters suitable for this hyperspectral dataset, including the following specific steps:

[0074] Step (B31): As Figure 2 and Figure 3 As shown, discrete wavelet decomposition is used to extract low-frequency information from high spatial resolution multispectral images (HR-MSI), and this information is then combined with the upsampled image along the channel dimension. Using a 3*3*64 convolution kernel, the input tensor of (W / 2)*(H / 2)*(L+l) is processed by a spatial spectral feature extraction network at a four-fold scale to simultaneously extract the spatial and spectral information of the two images. This information is then processed sequentially through 3*3*128 and 3*3*L convolution kernels to form a single (W / 2)*(H / 2)*L feature tensor.

[0075] Step (B32): Add the (W / 2)*(H / 2)*L feature tensor to the upsampled high image (LR-HSI) at four times the spatial resolution to obtain a new (W / 2)*(H / 2)*L feature tensor, and use the upsampled beam back as the input to the spatial-spectral feature extraction network at eight times the scale;

[0076] Step (B33): After using a 3*3*64 convolution kernel, the feature tensor is downsampled by two and four times respectively to obtain three feature tensors of different scales. These are used as inputs to a series of cross-scale octave convolutional networks to extract joint spatial and spectral features. Finally, the feature tensor obtained from the output of each cross-scale octave convolutional network is added to the low spatial resolution high image (LR-HSI) upsampled by eight times, resulting in a total of 6 feature tensors of size W*H*L.

[0077] Step (B34): Name the first five feature tensors obtained in step (B33) respectively. Name the last feature tensor O. SR The reconstructed image is obtained by taking the multispectral image and hyperspectral image as input;

[0078] Step (B35): Input the 6 feature tensors of each pixel in step (B34) and their corresponding label information into the loss function to obtain the loss value, and perform backpropagation on the convolutional neural network based on the loss value to update the weight parameters;

[0079] The loss function of a convolutional network is:

[0080] l1(Z, O) SR )=||ZO SR ||1

[0081]

[0082]

[0083]

[0084] Among them, Z and O SR Z(i,j), O represent the target image and the reconstructed image, respectively; SR (i,j) represent Z and O respectively. SR The value of pixel (i,j); the network has n outputs, and the nth output is used as the reconstructed image; The k-th output of the network, k = 1, 2, ..., n-1; to avoid the denominator being 0, c = 1e-10; β and These are the trade-off factors, with the networks set to 1, 1, and 0.1 respectively.

[0085] Specifically, in step (C), the trained model is used to train a pair of LR-HSI and HR-MSI images captured by the sensor at the same time and in the same region, resulting in a hyperspectral image with both high spatial and spectral resolution. Specifically, the tensors of the tested multispectral and hyperspectral images are input into the trained model, and the result is used as the fusion result. Figure 6 As shown.

[0086] Will Figures 4-6 respectively Figure 7 In comparison, the beneficial effects of this invention are as follows: The multispectral and hyperspectral image fusion method based on cross-scale Octave convolutional networks fully extracts spatial-spectral features from two images by using cross-scale Octave convolutional networks, and uses a spatial-spectral attention mechanism to make the network focus on those spatial-spectral information that are useful for image recovery, and reduce the focus on useless spatial-spectral information. This maximizes the reconstruction of spatial and spectral information while reducing information loss and confusion, thus obtaining satisfactory experimental results.

[0087] Example 1

[0088] To demonstrate the effectiveness of the multispectral and hyperspectral image fusion proposed in this invention, the Washington DCMall dataset was selected for experiments. The WDCMall dataset is an aerial hyperspectral image acquired by a Hydice sensor, containing 210 bands covering wavelengths from 0.4 to 2.5 μm. After removing blurred spectral bands such as those containing water vapor, this invention uses the remaining 191 bands. Each band has an image size of 1380×307 pixels. The middle 128×128 pixels were selected as the test set, and the remaining portion was used for training. The hyperspectral and multispectral images used as input for the test are as follows: Figures 8-9 As shown, the fused image and the real ground image are as follows: Figures 10-11 As shown.

[0089] Will Figures 8-9 respectively with Figures 10-11 By comparison, it can be clearly seen that the present invention fully extracts and integrates the rich spatial and spectral information in multispectral and hyperspectral images, thereby obtaining hyperspectral images with high spatial resolution.

[0090] Example 2

[0091] The Indian Pines dataset, imaged in 1992 by the Airborne Visible Infrared Imaging Spectroradiometer (AVIRIS) on an Indian pine tree in Indiana, USA, was used to demonstrate the effectiveness of the fusion method proposed in this invention. The IP dataset underwent spatial and spectral degradation. The degraded low-spatial-resolution hyperspectral image and high-spatial-resolution multispectral image are shown below. Figures 12-13 As shown, the degraded image is input into the network of this invention, and the resulting fused hyperspectral image is as follows. Figure 14 As shown, from Figures 12-14 As can be seen from this, the fusion method proposed in this invention effectively extracts the rich spectral information contained in the low spatial resolution hyperspectral image and the fine texture information in the high spatial resolution multispectral image, while minimizing the loss of spatial and spectral information.

[0092] The above are merely preferred embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A multispectral and hyperspectral image fusion method based on cross-scale Octave convolutional networks, characterized in that, Includes the following steps: Step (A): Spatially downsample and spectrally downsample the hyperspectral dataset to obtain a pair of multispectral images with high spatial resolution and a pair of hyperspectral images with low spatial resolution to simulate multispectral and hyperspectral image pairs captured by the sensor at the same time and in the same region; Step (B): Divide the image pairs into training and test sets according to a certain ratio, and generate tensor pairs corresponding to each training sample pair in the training set. Use the proposed fusion method to extract the spatial-spectral features of the tensor pairs, train the convolutional network model, and obtain the model parameters suitable for this hyperspectral dataset; the specific steps are as follows: Step (B1): Use the hyperspectral dataset as the label for the training set, and perform spatial downsampling and spectral downsampling on the hyperspectral dataset to generate paired training sets; Step (B2): Extract each training sample to form the corresponding tensor; Step (B3): Take the tensors corresponding to the training samples and their corresponding labels as input, extract spatial-spectral features for each pair of tensors through a convolutional neural network, and obtain the corresponding neural network model parameters; The specific steps of step (B3) are as follows: Step (B31): Use discrete wavelet decomposition to extract low-frequency information from the high spatial resolution multispectral image, and stitch it along the channel dimension with the upsampled low spatial resolution high spectral image. Use a 3*3*64 convolution kernel to extract the spatial and spectral information of the two images simultaneously through a spatial-spectral feature extraction network at a scale of four times, and then pass it through 3*3*128 and 3*3*L convolution kernels to form a (W / 2)*(H / 2)*L feature tensor. Step (B32): Add the (W / 2)*(H / 2)*L feature tensor to the upsampled hyperspectral image with a lower spatial resolution by four times to obtain a new (W / 2)*(H / 2)*L feature tensor, upsample it by two times and use it as the input of the spatial-spectral feature extraction network at an eight-fold scale; Step (B33): After using a 3*3*64 convolution kernel, the obtained feature tensors are downsampled by two times and four times respectively to obtain three feature tensors of different scales. These are used as inputs to a series of cross-scale Octave convolutional network modules to extract joint spatial and spectral features. Finally, the feature tensors obtained from the output of each cross-scale Octave convolutional network module are added to the LR-HSI upsampled by eight times, resulting in a total of 6 feature tensors of size W*H*L. Step (B34): Name the first five feature tensors obtained in step (B32) as follows: Name the last feature tensor O. SR The reconstructed image is obtained by taking the multispectral image and hyperspectral image as input; Step (B35): Input the 6 feature tensors obtained in step (B34) and their corresponding label information into the loss function to obtain the loss value, and perform backpropagation on the convolutional neural network based on the loss value to update the weight parameters; The loss function of a convolutional network is: l1(Z,O SR )=||ZO SR ||1 Among them, Z and O SR Z(i,j), O represent the target image and the reconstructed image, respectively; SR (i,j) represent Z and O respectively. SR The value of pixel (i,j); the network has n outputs, and the nth output is used as the reconstructed image; The k-th output of the network, k = 1, 2, ..., n-1; to avoid the denominator being 0, c = 1e-10; β and These are the trade-off factors, with the networks set to 1, 1, and 0.1 respectively; Step (B36): When iteration = 20000, training ends, and the model parameters corresponding to the most recent decrease in loss value are saved as the optimal weight parameters; Step (C): The trained model is fused with the sample pairs in the test set to obtain the final fusion result image.

2. The multispectral and hyperspectral image fusion method based on cross-scale Octave convolutional networks according to claim 1, characterized in that: In step (A), the hyperspectral dataset is blurred and spatially downsampled to obtain a low spatial resolution hyperspectral image (LR-HSI), and spectrally downsampled to obtain a high spatial resolution multispectral image, which are used as a pair of training samples. The specific steps are as follows: Step (A1): Select the hyperspectral image dataset as the label; Step (A2): Select a certain proportion of the hyperspectral images as the training set; Step (A3): Spatial downsampling and spectral downsampling are performed on the training set to obtain paired training pairs.

3. The multispectral and hyperspectral image fusion method based on cross-scale Octave convolutional networks according to claim 2, characterized in that: In step (A2), in order to reduce the experimental burden, the hyperspectral images of the training set are divided into a series of training blocks of size W*H, where W and H represent the width and length of the training block, respectively.

4. The multispectral and hyperspectral image fusion method based on cross-scale Octave convolutional networks according to claim 2, characterized in that: Step (A3) includes the following specific steps: Step (A31): The training block is sequentially subjected to Gaussian blur and downsampling to obtain a low spatial resolution hyperspectral image corresponding to the training block after spatial downsampling; Step (A32): Apply the spectral response function to the training block for spectral downsampling to obtain a high spatial resolution multispectral image corresponding to the training block after spectral downsampling; Step (A33): Use the low spatial resolution hyperspectral image and the high spatial resolution multispectral image as a pair of training samples.

5. The multispectral and hyperspectral image fusion method based on cross-scale Octave convolutional networks according to claim 1, characterized in that, In step (B2), tensors w*h*L and W*H*3 are formed for each pair of training samples, where W,H,3 and w,h,L are the width, length and number of channels of the multispectral image patch and the hyperspectral image patch, respectively, and W / w=H / h=8.

6. The multispectral and hyperspectral image fusion method based on cross-scale Octave convolutional networks according to claim 1, characterized in that, In step (C), the trained model is used to test the low spatial resolution hyperspectral image and the high spatial resolution multispectral image captured by the sensor at the same time and in the same area to obtain a hyperspectral image with both high spatial resolution and high spectral resolution. Specifically, the tensor pair of the tested multispectral and hyperspectral images is input into the trained model to obtain the result as the fusion result.

Images