A method and system for identifying minerals on the surface of Mars using hyperspectral images

Abstract

The present invention provides a method and system for identifying minerals on the surface of Mars using hyperspectral images, including: constructing a CRISM hyperspectral image data set, dividing the CRISM hyperspectral image data set into a training set and a test set; constructing a multi-branch segmentation network; selecting a loss function, iteratively training the multi-branch segmentation network through the loss function and the training set, and obtaining a trained deep learning model; and identifying minerals on the surface of Mars in CRISM hyperspectral images using the trained deep learning model. The multi-branch segmentation network proposed in the present invention extracts multi-level features of hyperspectral images in a multi-resolution feature map branch parallel manner, and finally aggregates the multi-scale spatial features of four branches, namely, a high spatial sparse spectral feature enhancement branch, a higher spatial near-sparse spectral feature enhancement branch, a lower spatial near-dense spectral feature enhancement branch, and a low spatial dense spectral feature enhancement branch, and classifies the multi-scale spectral features to achieve a better recognition effect.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a method and system for identifying minerals on the surface of Mars using hyperspectral images. Background Art

[0002] Based on the data collected by the CRISM Mars-specific small reconnaissance craft, the distribution of minerals such as carbonates, silicates, and sulfates on the surface of Mars can be analyzed, which will help us understand the geological activity process and climate evolution process of Mars. The detection and identification of water-containing minerals can be used to explore traces of water on Mars and provide a basis for the exploration of extraterrestrial life.

[0003] Traditional Martian surface mineral identification mainly includes mineral identification methods based on spectral matching and mineral identification methods based on spectral characteristic parameters. Among them: the spectral matching method has a simple principle, but it is more sensitive to noise and requires a high signal-to-noise ratio of the analyzed image spectrum; the method based on spectral characteristic parameters has achieved some results in the research on Martian surface mineral identification, but its degree of automation is not high and the identification accuracy is limited.

[0004] According to the feature information extracted by the deep learning network, the existing deep learning-based methods for identifying Martian surface minerals mainly include spectral feature classification networks and spectral-spatial feature classification networks. The spectral feature classification network uses the spectral information of hyperspectral data for classification, but does not make enough use of spatial information; some spatial-spectral feature classification networks have a shallow application of multi-scale information fusion and attention mechanisms. Moreover, most of these methods are pixel-by-pixel and based on the classification method of pixel blocks in the neighborhood of image pixels, which requires pixel-by-pixel overlapping sampling for mineral classification and identification, resulting in low recognition efficiency. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a method for identifying minerals on the surface of Mars using hyperspectral images, comprising the steps of:

[0006] S1: Construct the CRISM hyperspectral image dataset and divide the CRISM hyperspectral image dataset into a training set and a test set;

[0007] S2: Construct a multi-branch segmentation network for Mars surface mineral identification in hyperspectral images;

[0008] S3: Select a loss function, iteratively train the multi-branch segmentation network using the loss function and the training set to obtain a trained deep learning model;

[0009] S4: Identify Martian surface minerals in CRISM hyperspectral images using a trained deep learning model.

[0010] Preferably, the multi-branch segmentation network includes: a dimension reduction convolution unit, three branch convolution units including an original scale branch convolution unit, a 1 / 4 scale branch convolution unit, and a 1 / 16 scale branch convolution unit, four feature acquisition units including a high spatial sparse spectral feature acquisition unit, a higher spatial nearly sparse spectral feature acquisition unit, a lower spatial nearly dense spectral feature acquisition unit, and a low spatial dense spectral feature acquisition unit, and a segmentation head;

[0011] The dimension reduction convolution unit is connected to the high spatial sparse spectral feature acquisition unit;

[0012] The original scale branch convolution unit is connected to a higher spatial near-sparse spectral feature acquisition unit;

[0013] The 1 / 4 scale branch convolution unit is connected to the lower spatial near-dense spectral feature acquisition unit;

[0014] The 1 / 16 scale branch convolution unit is connected to the low spatial density spectral feature acquisition unit;

[0015] The four feature acquisition units are all connected to the segmentation head.

[0016] Preferably, the feature acquisition unit includes: a residual convolution structure RC, a channel attention structure CA, and a bilateral spatial-spectral fusion structure BSSF;

[0017] The residual convolution structure, channel attention structure and bilateral spatial-spectral fusion structure are connected in sequence;

[0018] The bilateral spatial-spectral fusion structure includes: a multi-scale spectral fusion sub-module and a multi-scale spatial enhancement sub-module.

[0019] Preferably, the workflow of the multi-branch segmentation network is:

[0020] S21: resize the input image to H×W×C, where H, W, and C represent the height, width, and number of spectral channels of the image, respectively;

[0021] S22: Perform spectral channel dimension reduction on the image through a dimension reduction convolution unit to obtain a feature map F1;

[0022] S23: downsampling the feature map F1 through the original scale branch convolution unit, the 1 / 4 scale branch convolution unit and the 1 / 16 scale branch convolution unit to obtain feature maps F2, F3 and F4;

[0023] S24: Input feature maps F1, F2, F3 and F4 into corresponding feature acquisition units respectively, and extract features, highlight important channels and enhance spatial-spectral features through residual convolution structure, channel attention structure and bilateral spatial-spectral fusion structure in sequence to obtain spatial-spectral enhanced feature map FM. BSSF1, Spatial spectrum enhancement feature map FM BSSF2 , Spatial spectrum enhancement feature map FM BSSF3 and spatial spectrum enhanced feature map FM BSSF4 ;

[0024] S25: The spatial spectrum is enhanced through the segmentation head to feature map FM BSSF1 , Spatial spectrum enhancement feature map FM BSSF2 , Spatial spectrum enhancement feature map FM BSSF3 and spatial spectrum enhanced feature map FM BSSF4 Perform splicing and recognition, and output the recognition results.

[0025] Preferably, step S24 is specifically as follows:

[0026] S241: Input the feature map into the residual convolution structure, highlight the mineral category information in the feature map through the residual convolution structure, and output the first stage feature map FM RC ;

[0027] S242: The first stage feature map FM RC Input channel attention structure, highlight the channel information in the feature map through the channel attention structure, and output the second stage attention feature map FM CA ;

[0028] S243: The second stage attention feature map FM CA Input bilateral spatial-spectral fusion structure, and strengthen FM through bilateral spatial-spectral fusion structure CA Channel information and spatial information of the output space spectrum enhancement feature map FM BSSF .

[0029] Preferably, step S25 is specifically as follows:

[0030] S251: Enhance the feature map FM with the empty spectrum BSSF2 , Spatial spectrum enhancement feature map FM BSSF3 and spatial spectrum enhanced feature map FM BSSF4 The input segmentation head is upsampled, and these feature maps are upsampled to the same spatial resolution as the input image of the multi-branch segmentation network, and the sampled spatial spectrum enhancement feature map FM is obtained UP2 , the spatial spectrum enhancement feature map FM after sampling UP3 And the spatial spectrum enhancement feature map FM after sampling UP4 ;

[0031] S252: Air spectrum enhancement feature map FM BSSF1 , the spatial spectrum enhancement feature map FM after sampling UP2 , the spatial spectrum enhancement feature map FM after sampling UP3 And the spatial spectrum enhancement feature map FM after sampling UP4Perform stitching in the channel dimension to obtain a stitched image;

[0032] S253: Input the stitched image into several 1×1 convolutions and output the mineral identification results. The number of convolution kernels is the number of Martian surface mineral categories that need to be identified.

[0033] Preferably, step S3 specifically comprises:

[0034] S31: Select the cross entropy loss function as the optimization direction of the multi-branch segmentation network and adjust the parameters of the multi-branch segmentation network;

[0035] S32: training the multi-branch segmentation network using the training set;

[0036] S33: Repeat steps S31-S32 until the value of the loss function reaches the minimum.

[0037] A hyperspectral image Mars surface mineral identification system, comprising:

[0038] The dataset acquisition module is used to construct the CRISM hyperspectral image dataset and divide the CRISM hyperspectral image dataset into a training set and a test set;

[0039] A network building module for constructing a multi-branch segmentation network for mineral identification on the surface of Mars in hyperspectral images;

[0040] The training module is used to select the loss function, iteratively train the multi-branch segmentation network through the loss function and the training set, and obtain a trained deep learning model;

[0041] The recognition module is used to identify Martian surface minerals in CRISM hyperspectral images using a trained deep learning model.

[0042] A storage device, wherein the storage device stores instructions and data for implementing the above-mentioned method for identifying minerals on the surface of Mars using hyperspectral images.

[0043] A hyperspectral image Martian surface mineral identification device comprises: a processor and a storage device; the processor loads and executes instructions and data in the storage device to implement the above-mentioned hyperspectral image Martian surface mineral identification method.

[0044] The present invention has the following beneficial effects:

[0045] 1. The multi-branch segmentation network proposed in the present invention extracts multi-level features of hyperspectral images in a parallel manner of multi-resolution feature map branches, and finally aggregates the multi-scale spatial features and multi-scale spectral features of four branches, namely, the high spatial sparse spectral feature enhancement branch, the higher spatial nearly sparse spectral feature enhancement branch, the lower spatial nearly dense spectral feature enhancement branch and the low spatial dense spectral feature enhancement branch, for classification to achieve better recognition effect;

[0046] 2. The multi-branch segmentation network proposed in the present invention introduces a residual convolution structure to extract deep features, and adds a channel attention structure to highlight more important channels; compared with traditional methods and common hyperspectral image classification and semantic segmentation networks, the generalization ability, accuracy and running speed of the multi-branch segmentation network on CRISM images can be improved to a certain extent;

[0047] 3. In order to better extract and fuse the spectral and spatial features of the image, a bilateral spatial-spectral fusion structure is designed to enhance the multi-scale spectral and texture features of the feature map; the multi-scale spectral fusion submodule and multi-scale spatial enhancement submodule that constitute the bilateral spatial-spectral fusion structure perform multi-scale spectral and spatial feature enhancement respectively, which can effectively distinguish similar minerals and further improve the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flow chart of a method according to an embodiment of the present invention;

[0049] Figure 2 A structural diagram of a multi-branch segmentation network provided by an embodiment of the present invention is shown;

[0050] Figure 3 The mineral spectrum curves of magnesium montmorillonite and prehnite provided by the embodiments of the present invention are shown;

[0051] Figure 4 The mineral spectrum curves of calcium carbonate / iron and aluminum montmorillonite provided by the embodiments of the present invention are shown;

[0052] Figure 5 The mineral spectrum curves of magnesium carbonate and chlorite provided by the embodiments of the present invention are shown;

[0053] Figure 6 The mineral spectrum curves of low-calcium pyroxene and fayalite provided by the embodiments of the present invention are shown;

[0054] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0055] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0056] First, some nouns or terms that appear in the process of describing the embodiments of the present invention are subject to the following explanations:

[0057] CRISM: Compact Reconnaissance Imaging Spectrometer for Mars (CRISM) is a spectrometer that uses optical and infrared wavelengths and is installed on the Mars Reconnaissance Orbiter (MRO) to search for minerals associated with past liquid water on Mars. The CRISM team is composed of scientists from more than ten universities in the United States, and the host is Scott Murchie. CRISM was designed and tested by the Johns Hopkins University Applied Physics Laboratory.

[0058] AdamW: An optimizer commonly used in deep learning. AdamW is an improved algorithm based on the Adam optimizer and L2 regularization.

[0059] Softmax: A common activation function in deep learning that can be used for multi-classification tasks. The Softmax function can convert the output values ​​of multiple categories into a probability distribution in the range of [0,1] and with a sum of 1.

[0060] Sigmoid: A common activation function in deep learning that can be used for binary classification tasks. The Sigmoid function can be used to convert the output value of the binary classification into a value in the range of [0,1].

[0061] Global average pooling: A common operation in deep learning. Specifically, all pixel values ​​in the feature map are added together to obtain the average value, and the average value is used to represent the corresponding feature map.

[0062] Atrous convolution: Atrous convolution is to inject holes on the basis of standard convolution, which can expand the receptive field of the convolution kernel without increasing the amount of calculation. Atrous convolution adds a hyperparameter - dilation rate on the basis of standard convolution. This hyperparameter refers to the interval of the convolution kernel. When the dilation rate is 1, the atrous convolution is equivalent to the ordinary convolution.

[0063] Dropout: Random deactivation is an operation to alleviate the overfitting problem in deep learning. Specifically, during the training of a neural network, for a certain layer of the neural network in an iteration, some neurons are randomly selected and temporarily hidden / discarded before model training and optimization.

[0064] Reference Figure 1 The present invention provides a method for identifying minerals on the surface of Mars using hyperspectral images, comprising the steps of:

[0065] S1: Construct the CRISM hyperspectral image dataset and divide the CRISM hyperspectral image dataset into a training set and a test set;

[0066] S2: Construct a multi-branch segmentation network for Mars surface mineral identification in hyperspectral images;

[0067] S3: Select a loss function, iteratively train the multi-branch segmentation network using the loss function and the training set to obtain a trained deep learning model;

[0068] S4: Identify Martian surface minerals in CRISM hyperspectral images using a trained deep learning model.

[0069] Furthermore, step S1 is specifically as follows: a certain number of CRISM hyperspectral images with mineral label information are selected for image preprocessing operations, wherein the mineral category label information is mainly collected by professionals using the spectral curve method, specifically: according to the degree of consistency between the spectral curve of the mineral label pixel in the image and the trend of the laboratory mineral spectral curve (especially the absorption peak), it can be judged whether a certain type of mineral exists in the pixel, such as magnesium montmorillonite generally has absorption peaks near wavelengths of 1.4, 1.9 and 2.3 μm. Figure 3-Figure 6 The spectral curves of 8 types of minerals in the CRISM image used in the present invention and their laboratory mineral standard spectral curves are shown. The horizontal axis in the figure is the wavelength, and the vertical axis is the pixel spectral reflectance value I / F.

[0070] This embodiment selects 6 CRISM images: FRT00003E12 (03E12), FRT00003FB9 (03FB9), FRT000050F2 (050F2), FRT000064D9 (064D9), FRT0000AA03 (0AA03), FRT0000B438 (0B438), all of which are from the Mars Image Viewer (http: / / viewer.mars.asu.edu / viewer), and the band range of all images is 1.0-2.6 μm, containing a total of 240 bands. Except for image 050F2, which is 450 pixels × 640 pixels in size, the sizes of the other 5 images are all 480 pixels × 640 pixels, and the image information and minerals are shown in Table 1.

[0071] Table 1 CRISM images and mineral information

[0072]

[0073] The hyperspectral data set is divided into a training set and a test set at a ratio of 1:9. Finally, the number of labeled training set and test set sample pixels is 11795 and 106066, respectively. The detailed number of each type of mineral in the CRISM data set after the training set and the test set are divided is shown in Table 2. In the CRISM used in the embodiment of the present invention, the number of magnesium montmorillonite pixels is the largest, reaching 50581; the number of aluminum montmorillonite pixels is the smallest, only 254.

[0074] Table 2 Number of various minerals in the CRISM dataset (unit: pixel)

[0075]

[0076]

[0077] Furthermore, the structure of the multi-branch segmentation network is as follows Figure 2 As shown in (a), unless otherwise specified, the convolution step size mentioned below defaults to 1;

[0078] The multi-branch segmentation network includes: the multi-branch segmentation network includes: a dimension reduction convolution unit, three branch convolution units including an original scale branch convolution unit, a 1 / 4 scale branch convolution unit, and a 1 / 16 scale branch convolution unit, four feature acquisition units including a high spatial sparse spectral feature acquisition unit, a higher spatial nearly sparse spectral feature acquisition unit, a lower spatial nearly dense spectral feature acquisition unit, and a low spatial dense spectral feature acquisition unit, and a segmentation head;

[0079] The dimension reduction convolution unit is connected to the high spatial sparse spectral feature acquisition unit;

[0080] The original scale branch convolution unit is connected to a higher spatial near-sparse spectral feature acquisition unit;

[0081] The 1 / 4 scale branch convolution unit is connected to the lower spatial near-dense spectral feature acquisition unit;

[0082] The 1 / 16 scale branch convolution unit is connected to the low spatial density spectral feature acquisition unit;

[0083] The four feature acquisition units are all connected to the segmentation head.

[0084] Specifically, a high-space sparse spectral feature acquisition unit (the sizes of the input feature map and the output feature map are 480×640×32 and 480×640×16 respectively; high-space means that the height and width of the input feature map are 480 and 640 respectively, and sparse spectrum means that the number of spectral channels of the input feature map is 32);

[0085] Higher spatial near-sparse spectral feature acquisition unit (the size of the input feature map and output feature map are 240×320×64 and 240×320×32 respectively; higher spatial refers to the height and width of the input feature map are 240 and 320 respectively, and near-sparse spectrum refers to the number of spectral channels of the input feature map is 64);

[0086] Lower spatial near-dense spectral feature acquisition unit (the sizes of the input feature map and the output feature map are 120×160×128 and 120×160×64 respectively; lower spatial refers to the height and width of the input feature map are 120 and 160 respectively, and near-dense spectrum refers to the number of spectral channels of the input feature map is 128);

[0087] Low-space dense spectral feature acquisition unit (the sizes of the input feature map and output feature map are 60×80×256 and 60×80×128 respectively; low space refers to the height and width of the input feature map are 60 and 80 respectively, and dense spectrum refers to the number of spectral channels of the input feature map is 256).

[0088] Furthermore, the feature acquisition unit includes: residual convolution structure RC, channel attention structure CA, bilateral spatial spectrum fusion structure BSSF;

[0089] The residual convolution structure, channel attention structure and bilateral spatial-spectral fusion structure are connected in sequence;

[0090] The bilateral spatial-spectral fusion structure includes: a multi-scale spectral fusion sub-module and a multi-scale spatial enhancement sub-module.

[0091] Furthermore, the workflow of the multi-branch segmentation network is:

[0092] S21: resize the input image to H×W×C, where H, W, and C represent the height, width, and number of spectral channels of the image, respectively;

[0093] Specifically, to facilitate network training and reasoning, before images are input into the network, images with a size less than 480 pixels × 640 pixels are uniformly padded with zero values ​​to ensure that the size of images input into the network is H × W × C (i.e., 480 × 640 × 240);

[0094] S22: Perform spectral channel dimension reduction on the image through a dimension reduction convolution unit to obtain a feature map F1;

[0095] Specifically, for the image input to the network, the spectral channel dimension of the image is first reduced by a dimension reduction convolution unit composed of 32 1×1 convolutions. At this time, the size of the feature map F1 obtained is 480×640×32;

[0096] S23: downsampling the feature map F1 through the original scale branch convolution unit, the 1 / 4 scale branch convolution unit and the 1 / 16 scale branch convolution unit to obtain feature maps F2, F3 and F4;

[0097] Specifically, the feature map F1 is input into the GN and GELU activation functions and then down-sampled in sequence using 2×2 convolutions with a step size of 2. The number of convolution kernels of the convolution layers of the original scale branch convolution unit, the 1 / 4 scale branch convolution unit, and the 1 / 16 scale branch convolution unit are 64, 128, and 156, respectively, and the feature maps F2, F3, and F4 with sizes of 240×320×64, 120×160×128, and 60×80×256 can be obtained (as shown in Fig. Figure 2 (a));

[0098] S24: Input feature maps F1, F2, F3 and F4 into corresponding feature acquisition units respectively, and extract features, highlight important channels and enhance spatial-spectral features through residual convolution structure, channel attention structure and bilateral spatial-spectral fusion structure in sequence to obtain spatial-spectral enhanced feature map FM. BSSF1 , Spatial spectrum enhancement feature map FM BSSF2 , Spatial spectrum enhancement feature map FM BSSF3 and spatial spectrum enhanced feature map FM BSSF4 ;

[0099] S25: The spatial spectrum is enhanced through the segmentation head to feature map FM BSSF1 , Spatial spectrum enhancement feature map FM BSSF2 , Spatial spectrum enhancement feature map FM BSSF3 and spatial spectrum enhanced feature map FM BSSF4 Perform splicing and recognition, and output the recognition results.

[0100] Further, step S24 is specifically as follows:

[0101] S241: Input the feature map into the residual convolution structure, highlight the mineral category information in the feature map through the residual convolution structure, and output the first stage feature map FM RC ;

[0102] S242: The first stage feature map FM RC Input channel attention structure, highlight the channel information in the feature map through the channel attention structure, and output the second stage attention feature map FM CA ;

[0103] S243: The second stage attention feature map FM CA Input bilateral spatial-spectral fusion structure, and strengthen FM through bilateral spatial-spectral fusion structure CAChannel information and spatial information of the output space spectrum enhancement feature map FM BSSF .

[0104] Specifically, each feature acquisition unit corresponds to a feature enhancement branch, that is, there are 4 feature enhancement branches in total. Taking the high spatial sparse spectral feature enhancement branch as an example, (1) the feature map F1 is input as follows Figure 2 (b) shows the residual convolution structure RC, which extracts shallower features of the hyperspectral image through two layers of 3×3 convolution with a step size of 1. The number of convolution kernels in the two convolution layers is 16 and 32 respectively, and the two convolution layers are followed by GN-GELU (GN-GELU means that the data is first input into the GN layer and then into the GELU activation function; the same below) and GN layer respectively; the result of the feature map F1 after passing through the second GN layer in RC will be element-wise added to the original feature map F1 to realize the residual connection. This residual connection avoids the gradient disappearance during deep neural network training; finally, RC outputs the first stage feature map FM RC ;

[0105] (2) FM RC Input such as Figure 2 (c) The channel attention structure CA shown in Figure 5 reweights the features through global average pooling and the information obtained by the multilayer perceptron (MLP) and highlights the more important features to improve the accuracy of the model; FM RC After global average pooling, a feature map of size 1×1×32 is obtained, which is then input into the MLP consisting of a fully connected layer-GELU-fully connected layer-Sigmoid activation function for feature enhancement, and feature maps FM of size 1×1×128 and 1×1×32 are obtained in turn. MLP1 and FM MLP2 , the feature map FM MLP2 With the feature map FM RC Perform element-wise multiplication to obtain the FM-based RC The second stage attention feature map FM with channel attention CA , its size is still 480×640×32;

[0106] (3) The second stage attention feature map FM CA enter Figure 2 In the bilateral space-spectrum fusion structure BSSF shown in (d), the bilateral space-spectrum fusion structure includes: a multi-scale spectral fusion sub-module and a multi-scale spatial enhancement sub-module;

[0107] Among them, 1) the multi-scale spectral fusion submodule MSF transforms the feature map FM CAThe number of spectral channels is compressed / expanded from 32 to 8, 32, and 128 by 1×1 convolution with a step size of 1 and the number of convolution kernels is 8, 32, and 128, respectively, to obtain the feature map FM MSF1 ,FM MSF2 ,FM MSF3 The above feature maps are passed through the GN layer and the GELU activation function in turn, and the three feature maps are concatenated in the spectral dimension to obtain an integrated feature map FM with a size of 480×640×168 MSF4 Finally, FM is transformed into MSF4 The number of spectral channels is reduced to 16, and the final output feature FM is 480×640×16 MSF ; By compressing / expanding the spectral dimension of the feature map, the information of a feature map changes in the spectral dimension, and the spectral dimension feature information becomes richer. Then, by fusing and reducing the dimension of these feature maps, it is possible to achieve multi-scale spectral dimension information fusion and enhancement without adding additional information while slightly increasing the number of parameters;

[0108] 2) The multi-scale spatial enhancement submodule MSA obtains receptive fields of different scales by changing the expansion rate and replacing the activation function and normalization method in the ASPP module, extracts multi-scale information, and achieves the effect of spatial feature enhancement and fusion; specifically, the feature map FM CA The 1×1 convolution-GN-GELU layer, the dilated convolution-GN-GELU layer with a dilation rate of 1, the 3×3 dilated convolution-GN-GELU layer with a dilation rate of 3, the 3×3 dilated convolution-GN-GELU layer with a dilation rate of 5, and the image pooling layer (including global average pooling, 1×1 convolution, GN, GELU and upsampling operations) are input respectively, and 5 feature maps of size 480×640×16 are output. The above 5 feature maps are tensor spliced ​​in the spectral dimension, and then the spectral channel dimension reduction is achieved through 16 1×1 convolutions, and GN-GELU-Dropout is used for normalization, nonlinear activation and random inactivation operations. Finally, the feature map FM of size 480×640×16 is output. MSA ; FM MSF With FM MSA The spectral dimension tensors are spliced, and the dimensionality reduction in the spectral channel direction is achieved through a 1×1 convolution-GN-GELU-Dropout layer to obtain a spatial spectrum enhancement feature map FM with a size of 240×320×16 BSSF1 .

[0109] Further, step S25 is specifically as follows:

[0110] S251: Enhance the feature map FM with the empty spectrum BSSF2, Spatial spectrum enhancement feature map FM BSSF3 and spatial spectrum enhanced feature map FM BSSF4 The input segmentation head is upsampled, and these feature maps are upsampled to the same spatial resolution as the input image of the multi-branch segmentation network, and the sampled spatial spectrum enhancement feature map FM is obtained UP2 , the spatial spectrum enhancement feature map FM after sampling UP3 And the spatial spectrum enhancement feature map FM after sampling UP4 ;

[0111] S252: Air spectrum enhancement feature map FM BSSF1 , the spatial spectrum enhancement feature map FM after sampling UP2 , the spatial spectrum enhancement feature map FM after sampling UP3 And the spatial spectrum enhancement feature map FM after sampling UP4 Perform stitching in the channel dimension to obtain a stitched image;

[0112] S253: Input the stitched image into several 1×1 convolutions and output the mineral identification results. The number of convolution kernels is the number of mineral categories on the Martian surface that need to be identified.

[0113] Furthermore, step S3 is specifically as follows:

[0114] S31: Select the cross entropy loss function as the optimization direction of the multi-branch segmentation network and adjust the parameters of the multi-branch segmentation network;

[0115] S32: training the multi-branch segmentation network using the training set;

[0116] S33: Repeat steps S31-S32 until the value of the loss function reaches the minimum.

[0117] Specifically, the multi-branch segmentation network optimizes the parameter values ​​of the model through iterative training so that the loss function value continuously approaches zero, thereby obtaining a model with good mineral identification effect; the present invention selects the cross entropy loss function as the direction of network optimization, and the calculation formula of the loss function is:

[0118]

[0119] Where N is the number of samples; p i is the probability distribution of the prediction result, where the value of each element is between [0,1]; i is the one-hot representation of the sample label;

[0120] First, the model parameters are initialized, and the multi-branch segmentation network is iteratively trained using the training data set. The AdamW optimizer is used to adjust the learning rate, and the initial learning rate of the training is 0.0001. The loss function value is calculated once each forward propagation is completed and the gradient is backpropagated; the gradient descent method is used to continuously optimize the loss, and finally a deep learning model that fits the distribution of the training set data is obtained; the output result will obtain the predicted probability value of each mineral category through the Softmax activation function, and the category with the largest predicted probability will be taken as the final category.

[0121] Furthermore, step S4 is specifically as follows:

[0122] The test set image data is input into the trained deep learning model, and the deep learning model proposed in the present invention is called to perform inference to obtain the mineral identification result image, so as to realize the efficient and accurate identification of minerals in the Mars CRISM hyperspectral image. For images with a size less than 480×640, the result image is automatically cropped from the upper left corner according to the size of the input image, and the cropped result image is the final classification result image.

[0123] In order to verify the effectiveness of the method for identifying minerals on the surface of Mars using hyperspectral images based on a multi-branch segmentation network proposed in an embodiment of the present invention, the accuracy of each category, overall accuracy (OA), average accuracy (AA), Kappa coefficient (k) and other indicators are selected to quantitatively analyze the test results. The experimental results are shown in Table 3, where MBSN is a multi-branch segmentation network.

[0124] Table 3 Comparison of mineral identification accuracy of different methods

[0125]

[0126]

[0127] It can be seen from the experimental results in Table 3 that the MBSN in the embodiment of the present invention can significantly improve the performance of identifying minerals on the surface of Mars. Even compared with the latest deep learning hyperspectral image classification network (such as SSFTT), MBSN still achieved a comprehensive improvement in indicators such as OA, AA, and k×100, among which OA increased by 1.3%, AA increased by 0.4%, and k×100 increased by 2.1. Except for the two minerals of prehnite and aluminum montmorillonite, the method proposed in the present invention has achieved the best accuracy in identifying other minerals.

[0128] In order to verify the effectiveness of the method for identifying minerals on the surface of Mars using hyperspectral images based on a multi-branch segmentation network proposed in the embodiment of the present invention, the recognition efficiencies of various mineral recognition methods are compared, and the comparison results can be seen in Table 4. The recognition efficiency of the MBSN proposed in the present invention is greatly improved compared to SVM, MLP, RNN, CDCNN and other methods, and is comparable to the recognition efficiency of the semantic segmentation network U-Net.

[0129] Table 4 Comparison of mineral identification efficiency of different methods

[0130] SVM MLP U-Net RNN CDCNN M3D-DCNN SSFTT MBSN Test(min) 10.24 1.01 0.44 1.50 10.27 5.19 1.99 0.44

[0131] In order to verify the effectiveness of the method for identifying minerals on the surface of Mars using hyperspectral images based on a multi-branch segmentation network proposed in the embodiment of the present invention, the mineral identification results of different methods are compared using image 0AA03 as an example. Figure 4 . Since the mineral identification results are too fragmented, in order to better compare the performance of different methods, a small area containing magnesium montmorillonite is enlarged in the figure. Obviously, the classification effects of the four methods of SVM, MLP, CDCNN, and M3D-DCNN are poor, and they are not able to effectively identify magnesium montmorillonite and other minerals in the area. The missed detection rate of U-Net recognition is low, but its accuracy is also low. It will mistakenly identify other minerals as magnesium montmorillonite, and magnesium montmorillonite as grape stone. In this area, SSFTT and the MBSN method proposed in this paper are more effective, and almost all magnesium montmorillonite is correctly identified. At the same time, neither of these two methods confuses magnesium montmorillonite with other minerals.

[0132] The present invention provides a hyperspectral image Martian surface mineral identification system, comprising:

[0133] The dataset acquisition module is used to construct the CRISM hyperspectral image dataset and divide the CRISM hyperspectral image dataset into a training set and a test set;

[0134] A network building module for constructing a multi-branch segmentation network for mineral identification on the surface of Mars in hyperspectral images;

[0135] The training module is used to select the loss function, iteratively train the multi-branch segmentation network through the loss function and the training set, and obtain a trained deep learning model;

[0136] The recognition module is used to identify Martian surface minerals in CRISM hyperspectral images using a trained deep learning model.

[0137] The present invention provides a storage device, wherein the storage device stores instructions and data for implementing the above-mentioned method for identifying minerals on the surface of Mars using hyperspectral images.

[0138] The present invention provides a hyperspectral image Martian surface mineral identification device, comprising: a processor and a storage device; the processor loads and executes instructions and data in the storage device to implement the above-mentioned hyperspectral image Martian surface mineral identification method.

[0139] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0140] The serial numbers of the embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In a unit claim that lists several means, several of these means may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order and these words may be interpreted as identifiers.

[0141] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for identifying minerals on the surface of Mars using hyperspectral images, characterized in that: Includes steps: S1: Construct the CRISM hyperspectral image dataset and divide the CRISM hyperspectral image dataset into a training set and a test set; S2: Construct a multi-branch segmentation network for Mars surface mineral identification in hyperspectral images; S3: Select a loss function, iteratively train the multi-branch segmentation network using the loss function and the training set to obtain a trained deep learning model; S4: Identification of Martian surface minerals in CRISM hyperspectral images using a trained deep learning model; The multi-branch segmentation network includes: a dimension reduction convolution unit, three branch convolution units including an original scale branch convolution unit, a 1 / 4 scale branch convolution unit, and a 1 / 16 scale branch convolution unit, four feature acquisition units including a high spatial sparse spectral feature acquisition unit, a higher spatial nearly sparse spectral feature acquisition unit, a lower spatial nearly dense spectral feature acquisition unit, and a low spatial dense spectral feature acquisition unit, and a segmentation head; The dimension reduction convolution unit is connected to the high spatial sparse spectral feature acquisition unit; The original scale branch convolution unit is connected to a higher spatial near-sparse spectral feature acquisition unit; The 1 / 4 scale branch convolution unit is connected to the lower spatial near-dense spectral feature acquisition unit; The 1 / 16 scale branch convolution unit is connected to the low spatial density spectral feature acquisition unit; All four feature acquisition units are connected to the segmentation head; The feature acquisition unit includes: residual convolution structure RC, channel attention structure CA, bilateral spatial spectrum fusion structure BSSF; The residual convolution structure, channel attention structure and bilateral spatial-spectral fusion structure are connected in sequence; The bilateral spatial-spectral fusion structure includes: a multi-scale spectral fusion sub-module and a multi-scale spatial enhancement sub-module.

2. The method for identifying minerals on the surface of Mars using hyperspectral images according to claim 1, characterized in that: The workflow of the multi-branch segmentation network is: S21: resize the input image to H×W×C, where H, W, and C represent the height, width, and number of spectral channels of the image, respectively; S22: Perform spectral channel dimension reduction on the image through a dimension reduction convolution unit to obtain a feature map F1; S23: downsampling the feature map F1 through the original scale branch convolution unit, the 1 / 4 scale branch convolution unit and the 1 / 16 scale branch convolution unit to obtain feature maps F2, F3 and F4; S24: Input feature maps F1, F2, F3 and F4 into corresponding feature acquisition units respectively, and extract features, highlight important channels and enhance spatial-spectral features through residual convolution structure, channel attention structure and bilateral spatial-spectral fusion structure in sequence to obtain spatial-spectral enhanced feature map FM. BSSF1 , Spatial spectrum enhancement feature map FM BSSF2 , Spatial spectrum enhancement feature map FM BSSF3 and spatial spectrum enhanced feature map FM BSSF4 ; S25: The spatial spectrum is enhanced through the segmentation head to feature map FM BSSF1 , Spatial spectrum enhancement feature map FM BSSF2 , Spatial spectrum enhancement feature map FM BSSF3 and spatial spectrum enhanced feature map FM BSSF4 Perform splicing and recognition, and output the recognition results.

3. The method for identifying minerals on the surface of Mars using hyperspectral images according to claim 2, characterized in that: Step S24 is specifically as follows: S241: Input the feature map into the residual convolution structure, highlight the mineral category information in the feature map through the residual convolution structure, and output the first stage feature map FM RC ; S242: The first stage feature map FM RC Input channel attention structure, highlight the channel information in the feature map through the channel attention structure, and output the second stage attention feature map FM CA ; S243: The second stage attention feature map FM CA Input bilateral spatial-spectral fusion structure, and strengthen FM through bilateral spatial-spectral fusion structure CA Channel information and spatial information of the output space spectrum enhancement feature map FM BSSF .

4. The method for identifying minerals on the surface of Mars using hyperspectral images according to claim 2, characterized in that: Step S25 is specifically as follows: S251: Enhance the feature map FM with the empty spectrum BSSF2 , Spatial spectrum enhancement feature map FM BSSF3 and spatial spectrum enhanced feature map FM BSSF4 The input segmentation head is upsampled, and these feature maps are upsampled to the same spatial resolution as the input image of the multi-branch segmentation network, and the sampled spatial spectrum enhancement feature map FM is obtained UP2 , the spatial spectrum enhancement feature map FM after sampling UP3 And the spatial spectrum enhancement feature map FM after sampling UP4 ; S252: Air spectrum enhancement feature map FM BSSF1 , the spatial spectrum enhancement feature map FM after sampling UP2 , the spatial spectrum enhancement feature map FM after sampling UP3 And the spatial spectrum enhancement feature map FM after sampling UP4 Perform stitching in the channel dimension to obtain a stitched image; S253: Input the stitched image into several 1×1 convolutions and output the mineral identification results. The number of convolution kernels is the number of mineral categories on the Martian surface that need to be identified.

5. The method for identifying minerals on the surface of Mars using hyperspectral images according to claim 1, characterized in that: Step S3 is specifically as follows: S31: Select the cross entropy loss function as the optimization direction of the multi-branch segmentation network and adjust the parameters of the multi-branch segmentation network; S32: training the multi-branch segmentation network using the training set; S33: Repeat steps S31-S32 until the value of the loss function reaches the minimum.

6. A hyperspectral imagery system for identifying minerals on the surface of Mars, characterized in that: include: The dataset acquisition module is used to construct the CRISM hyperspectral image dataset and divide the CRISM hyperspectral image dataset into a training set and a test set; A network building module for constructing a multi-branch segmentation network for mineral identification on the surface of Mars in hyperspectral images; The training module is used to select the loss function, iteratively train the multi-branch segmentation network through the loss function and the training set, and obtain a trained deep learning model; The recognition module is used to identify Martian surface minerals in CRISM hyperspectral images using a trained deep learning model; The multi-branch segmentation network includes: a dimension reduction convolution unit, three branch convolution units including an original scale branch convolution unit, a 1 / 4 scale branch convolution unit, and a 1 / 16 scale branch convolution unit, four feature acquisition units including a high spatial sparse spectral feature acquisition unit, a higher spatial nearly sparse spectral feature acquisition unit, a lower spatial nearly dense spectral feature acquisition unit, and a low spatial dense spectral feature acquisition unit, and a segmentation head; The dimension reduction convolution unit is connected to the high spatial sparse spectral feature acquisition unit; The original scale branch convolution unit is connected to a higher spatial near-sparse spectral feature acquisition unit; The 1 / 4 scale branch convolution unit is connected to the lower spatial near-dense spectral feature acquisition unit; The 1 / 16 scale branch convolution unit is connected to the low spatial density spectral feature acquisition unit; All four feature acquisition units are connected to the segmentation head; The feature acquisition unit includes: residual convolution structure RC, channel attention structure CA, bilateral spatial spectrum fusion structure BSSF; The residual convolution structure, channel attention structure and bilateral spatial-spectral fusion structure are connected in sequence; The bilateral spatial-spectral fusion structure includes: a multi-scale spectral fusion sub-module and a multi-scale spatial enhancement sub-module.

7. A storage device, characterized in that: The storage device stores instructions and data for implementing any one of the hyperspectral imagery Martian surface mineral identification methods described in claims 1 to 6.

8. A hyperspectral imaging device for identifying minerals on the surface of Mars, characterized in that: include: A processor and a storage device; the processor loads and executes instructions and data in the storage device to implement any one of the hyperspectral imagery Martian surface mineral identification methods described in claims 1 to 6.

Images