Hyperspectral image mineral collaborative recognition method and system based on deep neural network

Abstract

The present invention discloses a method and system for collaborative identification of minerals in hyperspectral images based on deep neural networks, and relates to the fields of mineral exploration and geoscience research. By collecting multi-source heterogeneous remote sensing data of main mineral categories and mineral elements in the coverage area of ​​the to-be-identified area, a spectral database is established, and a multi-scale feature extraction network of spatial and spectral dimensions is constructed using a hierarchical residual structure and different convolution kernels, multi-scale spatial-spectral features are extracted from multi-source remote sensing images, the extracted features are fused, and the information in the fusion process is adaptively adjusted using an attention mechanism, and a deep neural network model of a heterogeneous feature learning model is constructed. The neural network model is used to identify the characteristic minerals of the main mineral categories and mineral elements in the multi-source heterogeneous remote sensing images of the coverage area of ​​the to-be-identified area, and the deep neural network model is evaluated and optimized according to the identification results to obtain an identification model that is more suitable for characteristic minerals in the coverage area of ​​the to-be-identified area.

Description

Technical Field

[0001] The present invention relates to the field of mineral exploration and geological research, and in particular to a method and system for collaboratively identifying minerals using hyperspectral images based on a deep neural network. Background Art

[0002] Mineral identification is the basic work of geological prospecting and petrological research. Traditional mineral identification mainly relies on manual identification with optical microscopes, which is time-consuming, inefficient, and limited by the work experience of the identifier. With the rapid development of artificial intelligence technology, deep learning methods enable computers to automatically extract more accurate semantic information from images, thus providing an effective way for intelligent analysis of rock and mineral images. Mineral identification is one of the advantages of hyperspectral remote sensing technology. In particular, the innovation of sensors, communications, computers and other technologies has promoted the development of hyperspectral remote sensing mineral identification from mineral types to mineral subtypes, mineral components and other fine information identification.

[0003] At present, artificial intelligence has shown rapid growth in mineral identification research, realizing intelligent identification of minerals and lithology identification. The accuracy of some experiments exceeds 90%, but it is identified as a single mineral or a single rock, lacking comprehensive sorting, which has caused resource waste and repeated research to a certain extent. At the same time, the acquisition cost of geological mineral spectral data is often high, so the acquired data is mostly fragmented, and the data volume cannot meet the requirements of deep learning; due to the extremely strong professionalism of geological data, general artificial intelligence algorithms cannot be used directly, and the particularity and uncertainty of geological survey applications cannot find verified pre-trained models and prior knowledge in existing resource libraries. With the continuous deepening of application practice, the demand for fine mineral information is also increasing. Under the data-driven analysis mode, the data annotation amount of mineral elements and the fusion processing ability of multi-source information are the key to affecting the intelligent identification of mineral elements. Therefore, how to comprehensively utilize multi-source heterogeneous remote sensing data for effective feature extraction and collaborative classification to further improve the performance of classification and identification of typical mineral elements has urgent practical needs and important application value. However, due to the great differences in data structure and representation content of multi-source heterogeneous remote sensing data, and the fact that they are also very different from natural images, directly using network models in the field of computer vision cannot effectively perform collaborative classification of multi-source heterogeneous remote sensing data. The current mineral classification method can only be based on a single feature, resulting in poor classification accuracy and low generalization ability. To this end, we propose a hyperspectral image mineral collaborative recognition method and system based on deep neural networks. Summary of the invention

[0004] The main purpose of the present invention is to provide a method and system for collaborative identification of minerals using hyperspectral images based on deep neural networks. Aiming at the key problems of low precision and weak generalization ability of deep learning models in the current field of mineral identification, this project will adopt a series of cutting-edge deep learning technologies and data mining methods to build a set of efficient and accurate collaborative identification systems for mineral types using hyperspectral images, aiming to fully tap the potential value of massive multi-source heterogeneous geological data, break through the limitations of traditional mineral identification technology, and achieve high-precision and intelligent classification of mineral types, which can effectively solve the problems in the background technology.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] The hyperspectral image mineral collaborative identification method based on deep neural network includes:

[0007] Step 1: Collect multi-source heterogeneous remote sensing data of the main mineral categories and mineral elements in the coverage area of ​​the area to be identified, and establish a spectral database of the main mineral categories and mineral elements in the coverage area of ​​the area to be identified;

[0008] The specific process includes:

[0009] Step S11: collecting multimodal data covering different bands and different resolutions within the coverage area of ​​the area to be identified, and simultaneously collecting measured spectral data for verifying and calibrating the accuracy of the hyperspectral image data, wherein the multimodal data includes hyperspectral image data, infrared remote sensing data, radar data, and high spatial resolution data;

[0010] Step S12: Integrate the collected multimodal data and measured spectral data to construct a multi-source heterogeneous data set;

[0011] Step S13: preprocessing the acquired data, including data cleaning, noise removal, radiation correction, geometric correction, and normalization and standardization of the data;

[0012] Step S14: storing the processed data and establishing a spectral database of the main mineral categories and mineral elements in the coverage area of ​​the area to be identified;

[0013] Step 2: Construct a hierarchical residual network model, use the hierarchical residual structure and different convolution kernels to construct a multi-scale feature extraction network in spatial and spectral dimensions, and extract multi-scale spatial-spectral features from multi-source remote sensing images;

[0014] Step 3: Fuse the extracted multi-scale spatial-spectral features, and use the attention mechanism to adaptively adjust the information in the fusion process;

[0015] The specific process includes:

[0016] Step S31: using a hierarchical residual network to extract spatial-spectral features of different receptive fields from multi-source heterogeneous remote sensing data, and using a channel attention mechanism and a spatial attention mechanism to enhance the features;

[0017] Step S32: using the attention mechanism model to perform content-based nonlinear information fusion on heterogeneous features, optimizing the attention mechanism feature enhancement and fusion model under a unified network framework, and obtaining the optimal attention weight coefficient;

[0018] Step S33: using the acquired attention weight coefficient to effectively enhance the features and fuse the information of the heterogeneous features in the multi-source remote sensing data;

[0019] Step 4: construct deep neural network models including a deep hierarchical visual Transformer model and a heterogeneous feature learning model combining CNN and Transformer, and use the constructed neural network models to perform characteristic mineral recognition of the main mineral categories and mineral elements on the multi-source heterogeneous remote sensing images covering the area to be identified;

[0020] The deep hierarchical visual Transformer model is a structural form that includes three network branches for feature extraction and two layers of cross-attention mechanism feature fusion layers for fusing any two pairs of heterogeneous features, and the number of Transformer structures in each network branch is the same.

[0021] The network branches include:

[0022] Spectral branch for extracting spectral sequence information from hyperspectral images;

[0023] The first level Transformer branch used to extract hierarchical features from hyperspectral images;

[0024] The second-level Transformer branch used to extract spatial features from hyperspectral images.

[0025] The deep hierarchical visual Transformer model uses spectral Transformer and hierarchical Transformer structures to extract sequential spectral features and hierarchical spatial features from multi-source remote sensing data, and adopts a cross-attention mechanism model to adaptively fuse heterogeneous features. The specific process includes the following steps:

[0026] Step S31: in the spectral branch, multiple spectral Transformer structures are used to extract spectral sequence information in the hyperspectral image;

[0027] Step S32: For the spatial features of the hyperspectral image, PCA transformation is performed on it, and hierarchical features are extracted using a hierarchical Transformer;

[0028] Step S33: For the DSM and PolSAR data in the data set, since they mainly contain information of spatial dimensions, a hierarchical Transformer model is used to extract spatial features;

[0029] Step S34: the feature fusion stage includes two layers of cross-attention mechanism feature fusion layers, each of which fuses any two pairs of heterogeneous features;

[0030] Step S36: nonlinearly fuse the multi-scale features of the hyperspectral image in both spectral and spatial dimensions, and then perform information fusion between different remote sensing data.

[0031] The heterogeneous feature learning model combining CNN and Transformer adopts a heterogeneous neural network structure including two network branches, including a CNN network branch for extracting local spatial features and a Transformer structure network branch for extracting sequence spectral features. By performing structural transformation on the local spatial features represented by the feature graph and the spectral information contained in the embedded features, a multi-stage approach is adopted to extract heterogeneous features and fuse information, and the number of feature graph channels and the number of self-attention mechanism heads are gradually increased in each stage to improve the representation capabilities of the two heterogeneous features.

[0032] Step 5: Evaluate and optimize the constructed deep neural network model. Evaluate the classification performance of the model through at least one evaluation index including cross-validation, confusion matrix, and ROC curve. Iterate and optimize the model including adjusting the network structure, optimizing parameter settings, and introducing new features to further improve the accuracy and stability of the model and obtain an identification model that is more suitable for the characteristic minerals in the coverage area of ​​the area to be identified.

[0033] The hyperspectral image mineral collaborative identification system based on deep neural network includes multi-source heterogeneous remote sensing data acquisition module, data preprocessing module, spectral database, hierarchical residual network model module, deep neural network model module, and model evaluation and optimization module;

[0034] The multi-source heterogeneous remote sensing data acquisition module is used to collect multi-source heterogeneous remote sensing data of the main mineral categories and mineral elements in the coverage area of ​​the area to be identified, and the multi-source heterogeneous remote sensing data at least includes hyperspectral image data, infrared remote sensing data, radar data, high spatial resolution data and measured spectral data for verifying and calibrating the accuracy of hyperspectral image data;

[0035] The data preprocessing module is used to perform preprocessing including data cleaning, noise removal, radiation correction, geometric correction on the acquired multi-source heterogeneous remote sensing data, and normalize and standardize the data;

[0036] The spectral database is used to store processed multi-source heterogeneous remote sensing data;

[0037] The hierarchical residual network model module is used to construct a hierarchical residual network model, and uses a hierarchical residual structure and different convolution kernels to construct a multi-scale feature extraction network in spatial dimension and spectral dimension to extract multi-scale spatial-spectral features from multi-source remote sensing images;

[0038] The deep neural network model module is used to construct a deep neural network model including a deep hierarchical visual Transformer model and a heterogeneous feature learning model combining CNN and Transformer, respectively, and use the constructed neural network model to perform characteristic mineral identification of main mineral categories and mineral elements on multi-source heterogeneous remote sensing images of the coverage area of ​​the identification area;

[0039] The model evaluation and optimization module is used to evaluate and optimize the constructed deep neural network model to obtain an identification model that is more suitable for characteristic minerals in the coverage area of ​​the area to be identified.

[0040] The system also includes a memory, a processor, and a computer program stored on the memory and executable on the processor.

[0041] The present invention has the following beneficial effects:

[0042] Compared with existing technologies, this method combines advanced deep learning technologies such as convolutional neural networks and Transformer to comprehensively utilize the advantages of different network structures. CNN is good at capturing local features and spatial relationships, while Transformer is good at processing sequence data and global information. Through the combination of these two networks, automatic and intelligent collaborative classification of multi-source remote sensing data can be achieved;

[0043] Compared with the existing technology, in order to effectively extract multi-scale features, a hierarchical residual network model is used to extract features from pre-processed hyperspectral image data. This model can learn the feature information in the image layer by layer, thereby ensuring the effectiveness and stability of the features;

[0044] Compared with existing technologies, this method uses the spatial characteristics of hyperspectral image data, high spatial resolution images, infrared and radar data, and other multi-modal data complementarity to fuse the features of different modalities through a specific fusion strategy. This can fully utilize the advantages of each modality data and improve the comprehensiveness and accuracy of feature expression;

[0045] Compared with the existing technology, the attention mechanism is used to adaptively adjust the information in the fusion process to highlight important features and suppress noise information, thereby improving the expressiveness of features and the generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a technical flow chart of the hyperspectral image mineral collaborative identification method based on deep neural network of the present invention;

[0047] Figure 2 This is a structural block diagram of the hyperspectral image mineral collaborative identification system based on deep neural network of the present invention. DETAILED DESCRIPTION

[0048] The present invention will be further described below in conjunction with specific implementation methods, wherein the accompanying drawings are only used for exemplary descriptions and represent only schematic diagrams rather than actual drawings, and should not be understood as limiting the present invention. In order to better illustrate the specific implementation methods of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product.

[0049] In this embodiment, the present solution is explained by taking the western Henan Province of my country as an example. Among them, Henan Province is located in the central and eastern part of my country, the middle and lower reaches of the Yellow River, and is located in the southwest of the Huanghuaihai Plain. The topography is complex and diverse. The basic topography can be divided into five major areas: the eastern Henan Plain, the Nanyang Basin, the northern Henan Mountains, the western Henan Mountains, and the southern Henan Mountains. The terrain is basically high in the west and low in the east. The main surface coverage in the western Henan mountainous area is vegetation coverage and soil coverage, of which the vegetation coverage rate is about 60%. Due to the low degree of rock exposure, the coverage of the 1:50,000 geological survey in the province is relatively low.

[0050] The specific implementation process of the technical solution of the present invention includes the following steps:

[0051] (1) Establishing a multi-source heterogeneous image database: data acquisition and data preprocessing

[0052] Data collection: For the characteristic minerals in the coverage area, including quartz and plagioclase, multi-source heterogeneous data such as measured spectral data, hyperspectral data, high spatial resolution data, infrared data and radar data are collected.

[0053] ① Measured spectral data: Detailed spectral characteristics of quartz and plagioclase are obtained through field sampling and laboratory spectral measurement. These data are an important basis for building a refined expression system of mineral spectral characteristics. A ground feature spectrometer with a band width covering the mid-infrared band (wavelength>2500nm) is used. It is planned to use the Aopu Tiancheng ATP9110-110 ground feature spectrometer with an imaging band width of 300 to 11000nm.

[0054] ② Hyperspectral data: Hyperspectral data provides rich spectral band information, which can reflect the reflection or emission characteristics of objects at different wavelengths. For mineral identification, hyperspectral data is one of the indispensable data sources. It is planned to use drone hyperspectral imaging, equipped with Specim hyperspectral imager, with an imaging band width of 400-12000nm.

[0055] ③ High spatial resolution data: High spatial resolution data can provide detailed information on the surface morphology, which helps to consider factors such as topography and landforms when identifying mineral types. It is planned to use sub-meter satellite images or drone aerial images.

[0056] ④ Radar data: Radar data has a strong ability to penetrate surface materials and can obtain information below the surface, which is of great significance for identifying underground mineral deposits. In order to improve the spatial resolution, drone radar images are planned to be used.

[0057] ⑤ Infrared data: Infrared data can reflect the surface temperature distribution and the thermal radiation characteristics of materials, and plays an important role in identifying specific types of minerals such as hydrothermal deposits. It is planned to use drone infrared data.

[0058] Data preprocessing: First, the massive multi-source heterogeneous geological data of the coverage area are comprehensively analyzed and preprocessed, including denoising, atmospheric correction, normalization, orthorectification and other steps to ensure data quality.

[0059] ①Noise removal

[0060] Hyperspectral images usually contain a large number of spectral bands due to their high spectral resolution, but they may also be interfered by various noises, such as sensor noise, atmospheric scattering, terrain undulations, etc. There are many ways to remove noise. According to the characteristics of hyperspectral images, the following strategies can be adopted:

[0061] A: Filtering method:

[0062] Spatial filtering: such as mean filtering, median filtering, Gaussian filtering, etc. These methods reduce noise by smoothing the neighborhood around the pixel. However, it should be noted that excessive smoothing may blur the edge information and affect the accuracy of mineral identification.

[0063] Spectral filtering: Filtering using the similarity of spectral dimensions, such as Savitzky-Golay filtering, can remove noise while maintaining the spectral shape.

[0064] B: Transformation-based methods:

[0065] Principal component analysis (PCA): Through PCA transformation, hyperspectral data can be converted from the original high-dimensional space to a low-dimensional feature space, where the main noise components usually correspond to smaller principal components, and the noise can be reduced by removing these principal components.

[0066] Wavelet transform: Wavelet transform can decompose the signal into components of different scales, reduce the noise by removing the high-frequency part (usually noise) and reconstructing the signal.

[0067] C: Deep Learning Methods:

[0068] Convolutional Neural Network (CNN): CNN can automatically learn features in images, including noise features, through structures such as convolutional layers and pooling layers. Through training, CNN can learn how to distinguish between noise and useful information, thereby reducing the impact of noise in the feature extraction stage.

[0069] Denoising Autoencoder: This is a special type of autoencoder that learns a denoised representation of the data by adding noise to the input data and training the network to recover the original data from the noisy data.

[0070] ②Atmospheric correction

[0071] The influence of the atmosphere refers to the absorption and scattering of sunlight and radiation from the target by the atmosphere. The correction process to eliminate the influence of the atmosphere is usually called atmospheric correction, which is also the process of inverting the true reflectivity of the ground object. The important parameter for atmospheric correction is the band response function.

[0072] Atmospheric correction based on the radiation transfer equation is a method of atmospheric correction of remote sensing images using a model established based on the radiation transfer principle of electromagnetic waves in the atmosphere. This method converts the DN value of remote sensing images into surface reflectivity, surface emissivity, and surface temperature. Common correction methods include: MORTRAN model, 6S model, LOWTRAN, ultraviolet and visible radiation model UVRAD, and spatially distributed rapid atmospheric correction model ATCOR.

[0073] Adopting FLAASH method, based on MODTRAN 4+ radiation transfer model, it is a pixel-level correction, which corrects the collateral effect caused by diffuse reflection, including the classification map of cirrus and opaque clouds, and can adjust the spectral smoothing caused by artificial suppression. Atmospheric correction analysis of hyperspectral images can effectively eliminate the influence of factors such as atmosphere and light on the reflection of ground objects, and obtain more accurate reflectivity and emissivity, surface temperature and other real physical model parameters of ground objects.

[0074] ③ Normalization

[0075] The main purpose of image normalization is to adjust the range or distribution of hyperspectral image data to make it more suitable for subsequent data processing and analysis, especially in deep learning models, which can improve the training efficiency and prediction performance of deep learning models, thereby improving the accuracy and reliability of mineral species collaborative identification. The following are some common image normalization methods:

[0076] A: Linear Normalization

[0077] The data range of each band (or feature) of the hyperspectral image is linearly scaled to a specific interval, usually [0,1] or [-1,1].

[0078] The calculation formula is: Where X is the original data, X min and X max are the minimum and maximum values ​​of the band respectively. This method is simple and easy to implement, but it is sensitive to extreme values, and the new data may exceed the original range.

[0079] B: Z-score standardization

[0080] The data of each band is standardized to have a mean of 0 and a variance of 1.

[0081] The calculation formula is: Where μ is the mean of the band and σ is the standard deviation of the band. This method is suitable for most statistical analysis and deep learning models because it removes the impact of dimension and takes into account the variance of the data distribution.

[0082] C: Logarithmic transformation

[0083] For some hyperspectral imagery, especially areas of high reflectance, a logarithmic transformation can compress the data range and help reduce noise in the data.

[0084] The calculation formula is usually based on natural logarithms or base 10 logarithms, such as: X norm =log(X+1).

[0085] D: Histogram equalization

[0086] This method improves the contrast of the image by adjusting the histogram distribution of the image. In hyperspectral images, histogram equalization can be performed on each band separately, but it should be noted that this may change the relative relationship between the bands.

[0087] E: Normalization based on statistical distribution

[0088] Normalize the data according to its statistical distribution characteristics (such as Gaussian distribution) to make the data better conform to the model assumptions. This approach may require more complex statistical analysis and model fitting.

[0089] In the process of image normalization, it is necessary to select the appropriate method according to the specific project requirements and data characteristics. For example, in the mineral identification task, if there are extreme values ​​or a lot of noise in the data, linear normalization or logarithmic transformation may be needed to reduce the impact of these factors; if the data distribution is closer to the normal distribution, Z-score normalization can be used to improve the stability and generalization ability of the model.

[0090] ④Orthorectification

[0091] Orthorectification is a process of correcting geometric distortion of images. By selecting some ground control points on the image and using the digital elevation model (DEM) data, the image is tilted and projected at the same time to obtain a true plane image and eliminate the parallax caused by ground elevation. Common orthorectification algorithms include collinear equation model, strict geometric model based on affine transformation, improved polynomial model, rational function model, etc. This experiment uses the RPC parameters of the satellite sensor to convert pixel coordinates into geographic coordinates.

[0092] (2) Feature extraction

[0093] Using advanced feature extraction technology, especially high-precision feature extraction methods for hyperspectral images, we build a refined expression system of spectral features of major mineral categories and mineral elements. This step will deeply explore the intrinsic characteristics of geological data and lay a solid foundation for subsequent classification and identification. Hierarchical residual network data collaboration and adaptive feature fusion technology based on attention mechanism are two key technical innovations.

[0094] ① Hierarchical residual network data collaboration

[0095] Hierarchical residual network is a deep neural network structure that optimizes the information transfer of the network by introducing residual connections, effectively alleviating the gradient vanishing or gradient exploding problems in deep networks. In the task of mineral type recognition, HRN can extract features from images layer by layer, from low-level edge and texture features to high-level semantic features, forming a hierarchical feature expression.

[0096] Data collaboration refers to the use of multi-source, heterogeneous data sources (such as multispectral images, hyperspectral images, radar images, etc.) for joint analysis to obtain more comprehensive and accurate information. Through the hierarchical structure of HRN, the characteristics of different data sources can be processed separately, and feature fusion can be performed at different levels of the network to achieve the collaborative use of multi-source data. This collaborative approach can fully utilize the advantages of various data sources and improve the accuracy and robustness of mineral identification.

[0097] ② Adaptive feature fusion technology based on attention mechanism

[0098] The attention mechanism is a deep learning technology that simulates human visual attention behavior. It can automatically select key information and ignore irrelevant information when processing a large amount of information. In mineral type identification, the adaptive feature fusion technology based on the attention mechanism can dynamically adjust the importance of different features and achieve more accurate feature fusion.

[0099] The collaborative classification method is mainly divided into three stages, namely, multi-scale spatial-spectral feature extraction, self-attention mechanism feature enhancement, and feature fusion based on attention mechanism. The model uses a multi-branch network structure to extract different types of features from multi-source remote sensing data, in which the feature extraction module uses a hierarchical residual network. Channel and spatial attention mechanisms are used to enhance the spatial and spectral features of different receptive fields, respectively, thereby improving the discriminative ability of features.

[0100] The attention mechanism fusion model first nonlinearly fuses the multi-scale features of the two dimensions of the hyperspectral image, and then fuses the information between different remote sensing data, thereby effectively utilizing the complementarity of the content of multi-source remote sensing data for more accurate land object classification.

[0101] (3) Deep neural network model construction, evaluation and optimization

[0102] A deep neural network model was built to automatically identify quartz and plagioclase in the study area. A deep hierarchical visual Transformer model was compared with a heterogeneous feature learning model combining CNN and Transformer to obtain an identification model that is more suitable for the characteristic minerals in the study area.

[0103] ① Deep Hierarchical Visual Transformer Model

[0104] The model aims to gradually extract and fuse complex feature representations from input data by constructing multi-level Transformer blocks, thereby achieving more efficient and accurate image or video recognition tasks. The model uses a structure of multiple network branches for feature extraction. In the spectral branch, multiple spectral Transformer structures are used to extract spectral sequence information in hyperspectral images; for the spatial features of hyperspectral images, PCA transformation is first performed on them, and then hierarchical features are extracted using hierarchical Transformers. For DSM and PolSAR data in the dataset, since they mainly contain information in the spatial dimension, a hierarchical Transformer model is used to extract spatial features. In order to maintain the balance between different data, the same number of Transformer structures are used in the three network branches. The feature fusion stage includes two layers of cross-attention mechanism feature fusion layers, and each feature fusion layer fuses any two pairs of heterogeneous features. The attention mechanism fusion model first nonlinearly fuses the multi-scale features of the spectral and spatial dimensions of hyperspectral images, and then fuses the information between different remote sensing data, thereby effectively utilizing the complementarity of multi-source remote sensing data in content for more accurate land object classification.

[0105] ②Heterogeneous feature learning model combining CNN and Transformer

[0106] Combining the advantages of CNN and Transformer, a deep neural network model based on multi-source remote sensing data is constructed. CNN is good at automatic extraction and recognition of image features, while Transformer is good at processing long-distance dependencies in sequence data. Through the organic combination of the two, the automatic and intelligent collaborative classification of mineral elements is realized. In the process of model training, advanced optimization algorithms and regularization techniques are used to improve the convergence speed and generalization ability of the model.

[0107] ③Model evaluation and optimization

[0108] The constructed deep neural network model is rigorously evaluated and optimized. The classification performance of the model is comprehensively measured through various evaluation indicators such as cross-validation, confusion matrix, ROC curve, etc. In response to the problems exposed in the evaluation results, the model is iteratively optimized, including adjusting the network structure, optimizing parameter settings, introducing new features, etc., to further improve the accuracy and stability of the model.

[0109] ④Application and verification

[0110] The optimized deep neural network model is applied to actual geological exploration and mineral development work, and the granite pegmatite dominated by quartz and plagioclase outside the study area is intelligently interpreted and classified. The effectiveness and reliability of the model in practical application are evaluated by comparison and verification with actual geological data. At the same time, the model is further adjusted and improved based on application feedback to better meet the needs of actual work.

[0111] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for collaborative mineral recognition based on hyperspectral images using deep neural networks, characterized in that: include: Step 1: Collect multi-source heterogeneous remote sensing data of the main mineral categories and mineral elements in the coverage area of ​​the area to be identified, and establish a spectral database of the main mineral categories and mineral elements in the coverage area of ​​the area to be identified; Step 2: Construct a hierarchical residual network model, use the hierarchical residual structure and different convolution kernels to construct a multi-scale feature extraction network in spatial and spectral dimensions, and extract multi-scale spatial-spectral features from multi-source remote sensing images; Step 3: Fuse the extracted multi-scale spatial-spectral features, and use the attention mechanism to adaptively adjust the information in the fusion process; Step 4: construct deep neural network models including a deep hierarchical visual Transformer model and a heterogeneous feature learning model combining CNN and Transformer, and use the constructed neural network models to perform characteristic mineral recognition of the main mineral categories and mineral elements on the multi-source heterogeneous remote sensing images covering the area to be identified; Step 5: Evaluate and optimize the constructed deep neural network model. Evaluate the classification performance of the model through at least one evaluation index including cross-validation, confusion matrix, and ROC curve. Iterate and optimize the model including adjusting the network structure, optimizing parameter settings, and introducing new features to further improve the accuracy and stability of the model and obtain an identification model that is more suitable for the characteristic minerals in the coverage area of ​​the area to be identified.

2. The method for collaborative mineral identification based on hyperspectral images using deep neural networks according to claim 1 is characterized in that: The specific process of step one includes: Step S11: collecting multimodal data covering different bands and different resolutions within the coverage area of ​​the area to be identified, and simultaneously collecting measured spectral data for verifying and calibrating the accuracy of the hyperspectral image data, wherein the multimodal data includes hyperspectral image data, infrared remote sensing data, radar data, and high spatial resolution data; Step S12: Integrate the collected multimodal data and measured spectral data to construct a multi-source heterogeneous data set; Step S13: preprocessing the acquired data, including data cleaning, noise removal, radiation correction, geometric correction, and normalization and standardization of the data; Step S14: storing the processed data and establishing a spectral database of the main mineral categories and mineral elements in the coverage area of ​​the area to be identified.

3. The method for collaborative mineral identification based on hyperspectral images using deep neural networks according to claim 1 is characterized in that: The specific process of step three includes: Step S31: using a hierarchical residual network to extract spatial-spectral features of different receptive fields from multi-source heterogeneous remote sensing data, and using a channel attention mechanism and a spatial attention mechanism to enhance the features; Step S32: using the attention mechanism model to perform content-based nonlinear information fusion on heterogeneous features, optimizing the attention mechanism feature enhancement and fusion model under a unified network framework, and obtaining the optimal attention weight coefficient; Step S33: Use the acquired attention weight coefficient to effectively enhance the features and fuse the information of the heterogeneous features in the multi-source remote sensing data.

4. The method for collaborative mineral identification based on hyperspectral images using deep neural networks according to claim 1 is characterized in that: The deep hierarchical visual Transformer model is a structural form that includes three network branches for feature extraction and two layers of cross-attention mechanism feature fusion layers for fusing any two pairs of heterogeneous features, and the number of Transformer structures in each network branch is the same.

5. The method for collaborative mineral identification based on hyperspectral images using deep neural networks according to claim 4 is characterized in that: The network branches include: Spectral branch for extracting spectral sequence information from hyperspectral images; The first level Transformer branch used to extract hierarchical features from hyperspectral images; The second-level Transformer branch used to extract spatial features from hyperspectral images.

6. The method for collaborative mineral identification based on hyperspectral images using deep neural networks according to claim 1 is characterized in that: The deep hierarchical visual Transformer model uses spectral Transformer and hierarchical Transformer structures to extract sequential spectral features and hierarchical spatial features from multi-source remote sensing data, and adopts a cross-attention mechanism model to adaptively fuse heterogeneous features. The specific process includes the following steps: Step S31: in the spectral branch, multiple spectral Transformer structures are used to extract spectral sequence information in the hyperspectral image; Step S32: For the spatial features of the hyperspectral image, PCA transformation is performed on it, and hierarchical features are extracted using a hierarchical Transformer; Step S33: For the DSM and PolSAR data in the data set, since they mainly contain information of spatial dimensions, a hierarchical Transformer model is used to extract spatial features; Step S34: the feature fusion stage includes two layers of cross-attention mechanism feature fusion layers, each of which fuses any two pairs of heterogeneous features; Step S36: nonlinearly fuse the multi-scale features of the hyperspectral image in both spectral and spatial dimensions, and then perform information fusion between different remote sensing data.

7. The method for collaborative mineral identification based on hyperspectral images using deep neural networks according to claim 1 is characterized in that: The heterogeneous feature learning model combining CNN and Transformer adopts a heterogeneous neural network structure including two network branches, including a CNN network branch for extracting local spatial features and a Transformer structure network branch for extracting sequence spectral features. By performing structural transformation on the local spatial features represented by the feature graph and the spectral information contained in the embedded features, a multi-stage approach is adopted to extract heterogeneous features and fuse information, and the number of feature graph channels and the number of self-attention mechanism heads are gradually increased in each stage to improve the representation capabilities of the two heterogeneous features.

8. A hyperspectral image mineral collaborative identification system based on deep neural network, characterized by: The system is used to implement the steps of the method for collaboratively identifying minerals using hyperspectral images based on deep neural networks as described in any one of claims 1 to 7, comprising a multi-source heterogeneous remote sensing data acquisition module, a data preprocessing module, a spectral database, a hierarchical residual network model module, a deep neural network model module, and a model evaluation and optimization module; The multi-source heterogeneous remote sensing data acquisition module is used to collect multi-source heterogeneous remote sensing data of the main mineral categories and mineral elements in the coverage area of ​​the area to be identified, and the multi-source heterogeneous remote sensing data at least includes hyperspectral image data, infrared remote sensing data, radar data, high spatial resolution data and measured spectral data for verifying and calibrating the accuracy of hyperspectral image data; The data preprocessing module is used to perform preprocessing including data cleaning, noise removal, radiation correction, geometric correction on the acquired multi-source heterogeneous remote sensing data, and perform normalization and standardization on the data; The spectral database is used to store processed multi-source heterogeneous remote sensing data; The hierarchical residual network model module is used to construct a hierarchical residual network model, and uses a hierarchical residual structure and different convolution kernels to construct a multi-scale feature extraction network in spatial dimension and spectral dimension to extract multi-scale spatial-spectral features from multi-source remote sensing images; The deep neural network model module is used to construct a deep neural network model including a deep hierarchical visual Transformer model and a heterogeneous feature learning model combining CNN and Transformer, respectively, and use the constructed neural network model to perform characteristic mineral identification of main mineral categories and mineral elements on multi-source heterogeneous remote sensing images of the coverage area of ​​the identification area; The model evaluation and optimization module is used to evaluate and optimize the constructed deep neural network model, evaluate the classification performance of the model through at least one evaluation index including cross-validation, confusion matrix, and ROC curve, and iteratively optimize the model including adjusting the network structure, optimizing parameter settings, and introducing new features to further improve the accuracy and stability of the model and obtain an identification model that is more suitable for characteristic minerals in the coverage area of ​​the area to be identified.

9. The hyperspectral image mineral collaborative identification system based on deep neural network according to claim 8 is characterized in that: The system also includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, is capable of implementing the steps of the method for collaborative mineral identification based on hyperspectral images and deep neural networks as described in any one of claims 1 to 7.

Images