An object-oriented method for surface lithology identification and classification using hyperspectral remote sensing images
The spectrum and spatial characteristics of hyperspectral images are extracted through a dual-branch multi-scale dual-attention mechanism network, and combined with the leakage-free information data set division strategy, the problem of low recognition accuracy in hyperspectral remote sensing lithologic recognition is solved, significantly improving the recognition accuracy.
Patent Information
- Application Number
- CN202311811056.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-26
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2043-12-26
AI Technical Summary
The existing hyperspectral remote sensing lithologic recognition method has low recognition accuracy when there is no information leakage and the number of samples is small, so it is impossible to effectively extract high-dimensional spectral and spatial features.
The dual-branch multi-scale dual-attention mechanism network is adopted to extract the spectral characteristics and spatial characteristics of high-spectral images through spectral branches and spatial branches, and build the network using the leak-free information dataset division strategy to ensure that there is no overlap between training and testing data and generate The fusion characteristics are identified for surface lithology.
The identification accuracy in the absence of information leakage and small sample size was improved, the classification accuracy OA increased from 83.83% to 85.2%, AA increased from 78.89% to 84.09%, and Kappa coefficient increased from 0.7286 to 0.7523.
Smart Images

Figure CN117809179B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of hyperspectral remote sensing image processing, and in particular to an object-oriented hyperspectral remote sensing image surface lithology recognition and classification method. Background Art
[0002] Surface lithology identification is a very important and basic research direction in the field of geology, but it is currently still at the stage of manual field exploration and inspection, which greatly increases the time and material costs of this work. The emergence of hyperspectral remote sensing technology has not only promoted the development of other fields, but also shown great potential in the field of geological lithology identification.
[0003] At present, the earliest method used for lithology identification is the spectral angle matching method (SAM), but the SAM algorithm requires a lot of manual operation and has low accuracy. With the development of computer computing, its disadvantages have been revealed. The emergence of machine learning has greatly reduced manual work and improved efficiency. Common methods include principal component analysis (PCA), minimum noise fraction (MNF), independent component analysis (ICA) and greedy algorithms. However, with the increasing amount of data available in modern times, deep learning has begun to replace machine learning and has been widely used in other fields. However, the development in the field of hyperspectral remote sensing identification is still relatively slow. Most of them are simple convolutional networks, which cannot well extract the complex and high-dimensional spectral and spatial features in hyperspectral data. Therefore, the recognition accuracy of various types of objects is low when there is no information leakage and the number of samples is small. Summary of the invention
[0004] The purpose of the present invention is to provide an object-oriented hyperspectral remote sensing image surface lithology identification and classification method to solve the problem of low recognition accuracy of various types of land objects in existing lithology identification and classification methods without information leakage and with a small number of samples.
[0005] To achieve the above object, the present invention provides the following solutions:
[0006] An object-oriented hyperspectral remote sensing image surface lithology identification and classification method, comprising:
[0007] Determine the hyperspectral remote sensing images in the study area and the lithology type labels corresponding to each hyperspectral remote sensing image, and create a hyperspectral remote sensing dataset;
[0008] The pixels of the hyperspectral remote sensing image in the hyperspectral remote sensing dataset are divided into a training set and a test set by using a no-leakage information dataset division strategy; the no-leakage information dataset division strategy can ensure that there is no overlap between the training data in the training set and the test data in the test set;
[0009] Based on the deep learning method, a dual-branch multi-scale dual-attention mechanism network is used to extract the spectral features and spatial features of the hyperspectral remote sensing image to be tested and fuse them to generate fused features; the dual-branch multi-scale dual-attention mechanism network is trained and constructed through the training set and the test set; the dual-branch multi-scale dual-attention mechanism network includes a spectral branch, a spatial branch and a classification head; the spectral branch includes a MSeRA dense connection module and a spectral self-attention mechanism module, and the spectral branch is used to extract diagnostic spectral features from the spectral features; the spatial branch includes a spatial dense connection module and a spatial self-attention mechanism module, and the spatial branch is used to extract diagnostic spatial features from the spatial features; the classification head is used to fuse the diagnostic spectral features extracted by the spectral branch and the diagnostic spatial features extracted by the spatial branch to generate fused features; the fused features are used to characterize the surface lithology type of the hyperspectral remote sensing image to be tested.
[0010] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: relative to the technical solution of using a simple convolutional network for recognition and classification, the embodiment of the present invention uses a training set and a test set with no leakage information divided by a no leakage information data set division strategy to construct a dual-branch multi-scale dual attention mechanism network, which can ensure that there is no overlap between the training data in the training set and the test data in the test set. Therefore, the recognition and classification accuracy of the trained and constructed dual-branch multi-scale dual attention mechanism network will also be greatly improved, and the recognition accuracy of each category of objects can also be improved in the case of no information leakage and a small number of samples. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0012] Figure 1 A flow chart of an object-oriented hyperspectral remote sensing image surface lithology identification and classification method provided in the first embodiment of the present invention;
[0013] Figure 2 This is a diagram of the network structure of a dual-branch multi-scale dual-attention mechanism provided in the first embodiment of the present invention;
[0014] Figure 3 This is a structural diagram of a multi-scale spectral residual attention module provided in the first embodiment of the present invention;
[0015] Figure 4This is a flow chart of the object-oriented hyperspectral remote sensing image surface lithology identification and classification method provided in the second embodiment of the present invention. DETAILED DESCRIPTION
[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0017] The purpose of the present invention is to provide an object-oriented hyperspectral remote sensing image surface lithology identification and classification method, which can improve the recognition accuracy of various types of objects without information leakage and with a small number of samples.
[0018] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0019] Embodiment 1
[0020] like Figure 1 As shown, the present invention provides an object-oriented hyperspectral remote sensing image surface lithology identification and classification method, comprising:
[0021] Step 101: Determine the hyperspectral remote sensing images in the study area and the lithology type labels corresponding to each hyperspectral remote sensing image, and create a hyperspectral remote sensing dataset.
[0022] Step 102: Divide the pixels of the hyperspectral remote sensing image in the hyperspectral remote sensing dataset into a training set and a test set using a no-leakage information dataset partitioning strategy; the no-leakage information dataset partitioning strategy can ensure that there is no overlap between the training data in the training set and the test data in the test set.
[0023] Step 103: Based on the deep learning method, a dual-branch multi-scale dual-attention mechanism network is used to extract the spectral features and spatial features of the hyperspectral remote sensing image to be tested and fuse them to generate fused features; the dual-branch multi-scale dual-attention mechanism network is trained and constructed through the training set and the test set; the dual-branch multi-scale dual-attention mechanism network includes a spectral branch, a spatial branch and a classification head; the spectral branch includes a MSeRA dense connection module and a spectral self-attention mechanism module, and the spectral branch is used to extract diagnostic spectral features from the spectral features; the spatial branch includes a spatial dense connection module and a spatial self-attention mechanism module, and the spatial branch is used to extract diagnostic spatial features from the spatial features; the classification head is used to fuse the diagnostic spectral features extracted by the spectral branch and the diagnostic spatial features extracted by the spatial branch to generate fused features; the fused features are used to characterize the surface lithology type of the hyperspectral remote sensing image to be tested; the diagnostic spectral features are spectral features that play an important role in the final classification.
[0024] like Figure 2 As shown in Figure 1, the double-branch multi-scale dual-attention mechanism (DBMSDA) network mainly consists of three parts: spectral branch, spatial branch and classification head. Figure 2 The top part is the spectral branch, and the bottom part is the spatial branch. Finally, the features of the two are fused for classification. The spectral branch is mainly composed of the MSeRA dense connection module and the spectral attention module, and the spatial branch is mainly composed of the spatial dense connection module and the spatial attention module. Among them, p0 is the height and width of the hyperspectral data patch after division, and the two are generally set equal. b0 is the spectral dimension of the hyperspectral data patch after division, b1 is the spectral dimension obtained after a 3D convolution, and softmax is the normalized exponential function.
[0025] 3D-CNN is used as the basic convolutional structure of DBMSDA network to extract spatial and spectral information of hyperspectral images. Densely connected networks exist in both spectral and spatial branches. The spectral branch uses the multi-scale spectral residual attention (MSeRA) module as the connection unit of the densely connected network, such as Figure 3As shown in the figure, the multi-scale spectral residual attention (MSeRA) module includes the MSeRA dense connection module and the spectral self-attention module, which can avoid data explosion and gradient disappearance on the one hand, and fully extract the spectral features in the hyperspectral data on the other hand, making the extracted spectral features more refined and diagnostic. Among them, Conv is the convolution layer, ReLU is the activation function, CAM is the class activation map, Concatenate is the connection function, b1', b2 and b3 are the convolution kernel sizes of different scales in the multi-scale extraction module, which are set to b1=3, b2=5, and b3=7 respectively.
[0026] The spatial branch is composed of a simple 3D-CNN to avoid distortion of the learned spectral features due to excessive spatial information. Since HSI classification mainly relies on spectral information, spatial information only plays an auxiliary role. Excessive spatial information should be avoided to increase the difficulty of extracting spectral information.
[0027] The spectral dimension multiscale extraction module is applied in the multiscale spectral residual attention (MSeRA) module in order to extract the spectral information in the hyperspectral data. With the help of multiscale spectral dense blocks, the DBMSDA network can capture the spectral features of different levels corresponding to multiple receptive fields in each channel, thereby obtaining richer spectral information and improving the classification performance.
[0028] There are also two self-attention mechanisms, one in the spectral branch and the other in the spatial branch, called the spectral self-attention mechanism and the spatial self-attention mechanism respectively. Both of them are used to emphasize useful feature information and discard useless feature information. In addition, the self-attention mechanism also has the function of extracting global context information.
[0029] The present invention firstly uses 3D-CNN to extract the spectral information and spatial information of the entire hyperspectral data respectively, and then uses a densely connected network and a spectral self-attention mechanism in the spectral branch to extract diagnostic spectral features, and uses a densely connected network and a spatial self-attention mechanism in the spatial branch to extract diagnostic spatial features, and finally fuses the two for classification.
[0030] In practical applications, the present invention adopts 3D-CNN as the convolution structure of the DBMSDA network to extract hyperspectral information features. In order to avoid data explosion and gradient disappearance, the batch normalization (BatchNormalization, BN) + activation function Mish is used to standardize the results after each convolution in dense connection; then, the cube feature maps obtained by the two branches are converted into one-dimensional vector features using a random dropout layer and a global average pooling layer. Finally, the two one-dimensional vector features are connected into a whole, and then classified using a linear layer.
[0031] Step 104: Identify and classify the surface lithology of the hyperspectral remote sensing image according to the fused features.
[0032] The average accuracy OA of the final classification result of the present invention is improved from 83.83% to 85.2%, an increase of 1.37%, the overall accuracy AA is improved from 78.89% to 84.09%, an increase of 5.2%, and the Kappa coefficient is improved from 0.7286 to 0.7523.
[0033] Embodiment 2
[0034] Take the hyperspectral remote sensing image of Ziyuan-1 satellite as an example. Figure 4 As shown, the present invention includes the following steps.
[0035] S1. Obtain the free band hyperspectral remote sensing image of the Resource-1 remote sensing satellite, and preprocess the obtained Resource-1 hyperspectral remote sensing image; the data preprocessing process includes radiation correction, atmospheric correction and geometric correction.
[0036] S2. Based on the obtained Resource No. 1 hyperspectral remote sensing image, the study area is divided and corresponding labels are produced to obtain the hyperspectral remote sensing images and corresponding labels of the study area and produce a hyperspectral remote sensing dataset.
[0037] S3. Based on the obtained hyperspectral remote sensing images of the study area, the hyperspectral image pixels are divided into training set and test set using the no-leakage information dataset partitioning strategy.
[0038] S4. Based on the divided data set, a dual-branch multi-scale dual-attention mechanism network is used to extract and fuse its spectral features and spatial features respectively, and then it is trained and used for surface lithology identification and classification.
[0039] Step S2 includes dividing the study area and making labels, specifically including:
[0040] S2.1 The study area was selected mainly in areas with good bedrock exposure and no vegetation cover.
[0041] S2.2 Labeling mainly uses stratigraphic boundary data to automatically label each different type of lithology.
[0042] The research area here is mainly located in Kangyu Township, Bomi County, Nyingchi City, Tibet Autonomous Region. The characteristic of this area is that the bedrock is exposed and there is basically no vegetation cover.
[0043] Previously, each corresponding label was drawn manually when making relevant labels. As an improvement, the present invention uses ENVI and ArcGis software to automatically label each different type of lithology through stratigraphic boundary data to obtain a lithology label map of the area.
[0044] The size of the selected study area is 300 pixels×300 pixels, which ensures that all bedrock in the selected study area is exposed and also ensures that more lithology types are included. In this embodiment, the number of lithology types is 6.
[0045] Step S2 finally obtains a hyperspectral remote sensing image of size 300×300×166, where the two 300s are the height and width of the image, respectively, in pixels, and 166 is the number of bands; a lithology label map of size 300×300 is obtained, where 300 is the height and width, in pixels, with a total of 6 types of lithology, mainly metamorphic rocks such as schist and phyllite, siltstone, sedimentary rocks such as mudstone and sandstone, fine-grained lithic quartz sandstone, slate and Quaternary sediments.
[0046] Further, step S3 includes:
[0047] S3.1 divides the hyperspectral data into training set and test set according to its characteristics; the main feature of the hyperspectral data is that it has rich image information and spectral information, which can be regarded as a cube. Each pixel is a small cube in it. The surface contains spatial information and the depth direction has continuous spectral information.
[0048] S3.2 ensures that there is no information leakage between the divided dataset and the test set.
[0049] At present, the patch-based classification method used by most network models and its corresponding data partitioning strategy may have potential information leakage. Because the purpose of the patch-based classification method is to predict the label of the center pixel, but its partitioning strategy may cause overlapping patches in the training set and the test set. In the classification of hyperspectral images (HSI), the division of the training set and the test set will greatly affect the accuracy of the results.
[0050] Furthermore, after the feasibility analysis and judgment in S3.1, a data set partitioning strategy is needed to ensure that there is no information leakage between the training set and the test set. The steps of the data set partitioning strategy without information leakage provided in this embodiment specifically include:
[0051] 1) First, determine two important parameters: the ratio of pixels involved in training to all pixels in the original image, and the number of labeled pixels in each training block T. From this, we can know that the number of training blocks in each category is:
[0052]
[0053] Where i represents the pixel category, N i Indicates the number of training blocks in the corresponding category (N i≥
[0054] 1), n i It represents the total number of pixels of each category in the original image.
[0055] 2) According to the ratio λ of the pixels involved in the training, the training image to be divided is separated from the original image, and the remaining part is the verification-test image to be divided, and there is no overlap between the two images.
[0056] 3) According to the number of labeled pixels T in each training block, randomly select some labeled pixels in the training image to be divided and retain them, and define them as training images. Similarly, the verification-test images to be divided are also divided into verification images and test images according to the proposed verification / test sample ratio.
[0057] 4) The training image, leakage image, verification image, and test image obtained in the third step are divided into blocks of W×H×B, where W and H represent the width and height of the block, and B represents the number of spectral bands. After discarding those blocks that do not contain annotated pixels, all training blocks and test blocks are obtained.
[0058] 5) Use a sliding window strategy to obtain more training and testing patches from limited blocks.
[0059] As an improvement, S4 includes the use of 3D-CNN, densely connected network, multi-scale extraction module and self-attention mechanism in the dual-branch multi-scale dual attention mechanism network (DBMSDA) to extract spectral features and spatial features of the original hyperspectral data respectively, and then fuse the spectral features and spatial features for classification.
[0060] Features are extracted on the divided training set, and a dual-branch structure is selected to extract spectral features and spatial features respectively; the dual-branch structure includes a spectral branch and a spatial branch.
[0061] As an improvement, a multi-scale spectral residual attention (MSeRA) module is designed on the spectral branch as a densely connected network combined with a self-attention mechanism to more fully extract spectral information. The MSeRA module mainly combines the spectral dimension multi-scale extraction module and residual connection to extract spectral features of different scales; on the spatial branch, convolutional dense connections are used in combination with a self-attention mechanism to extract spatial information. Finally, the spectral features and spatial features are fused for classification.
[0062] The present invention divides the study area and produces corresponding labels based on the hyperspectral remote sensing image of the Resource-1 satellite. In simple terms, a hyperspectral remote sensing data set is produced, and the data set division strategy without leakage information is used to ensure that the training set and the test set after division have no information leakage; finally, the spectral features and spatial features of the hyperspectral data are extracted using a deep learning algorithm and then the two are integrated to achieve surface lithology identification and classification. The proposed dual-branch multi-scale dual attention mechanism network (DBMSDA) has better effects than the simple convolutional network.
[0063] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0064] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of the present invention. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.
Claims
1. An object-oriented hyperspectral remote sensing image surface lithology identification and classification method, characterized in that: include: Determine the hyperspectral remote sensing images in the study area and the lithology type labels corresponding to each hyperspectral remote sensing image, and create a hyperspectral remote sensing dataset; The pixels of the hyperspectral remote sensing image in the hyperspectral remote sensing dataset are divided into a training set and a test set by using a non-leakage information dataset division strategy; The non-leakage information data set partitioning strategy can ensure that there is no overlap between the training data in the training set and the test data in the test set; The steps of the no-leakage-information dataset partitioning strategy specifically include: 1) First, determine two important parameters: the ratio of pixels participating in training to all pixels in the original image, λ, and the number of labeled pixels in each training block, T; thus, the number of training blocks in each category is: in represents the pixel category, Indicates the number of training blocks in the corresponding category, It represents the total number of pixels of each category in the original image; 2) According to the ratio λ of the pixels involved in training, the training image to be separated is separated from the original image, and the remaining part is the verification-test image to be separated, and there is no overlap between the two images; 3) According to the number of labeled pixels T in each training block, randomly select some labeled pixels in the training image to be divided and retain them, and define them as training images; similarly, the verification-test image to be divided is also divided into verification images and test images according to the proposed verification / test sample ratio; 4) The training image, leakage image, verification image, and test image obtained in the third step are divided into blocks of W×H×B, where W and H represent the width and height of the block, and B represents the number of spectral bands; after discarding those blocks that do not contain annotated pixels, all training blocks and test blocks are obtained; 5) Using a sliding window strategy, more training and testing patches are obtained from a limited number of blocks; Based on the deep learning method, a dual-branch multi-scale dual-attention mechanism network is used to extract the spectral features and spatial features of the hyperspectral remote sensing image to be tested and fuse them to generate the fused features; the dual-branch multi-scale dual-attention mechanism network is trained and constructed through the training set and the test set; the dual-branch multi-scale dual-attention mechanism network includes a spectral branch, a spatial branch and a classification head; the spectral branch includes a MSeRA dense connection module and a spectral self-attention mechanism module, and the spectral branch is used to extract the diagnostic spectral features from the spectral features; the MSeRA dense connection module includes a spectral dimension multi-scale extraction module and a residual connection; the MSeRA dense connection module is used to extract Spectral features of the same scale; the spectral dimension multi-scale extraction module is applied in the MSeRA dense connection module, and with the help of the multi-scale spectral dense block, the dual-branch multi-scale dual-attention mechanism network can capture spectral features of different levels corresponding to multiple receptive fields in each channel; the spatial branch includes a spatial dense connection module and a spatial self-attention mechanism module, and the spatial branch is used to extract diagnostic spatial features from the spatial features; the classification head is used to fuse the diagnostic spectral features extracted by the spectral branch and the diagnostic spatial features extracted by the spatial branch to generate fused features; the fused features are used to characterize the surface lithology type of the hyperspectral remote sensing image to be measured.
2. The object-oriented hyperspectral remote sensing image surface lithology identification and classification method according to claim 1 is characterized in that: Determine the hyperspectral remote sensing images in the study area and the lithology type labels corresponding to each hyperspectral remote sensing image, and produce a hyperspectral remote sensing dataset, including: The areas with bedrock exposure higher than the set exposure area and no vegetation cover in the hyperspectral remote sensing images were selected as the hyperspectral remote sensing images of the study area; Using the stratigraphic boundary data, the pixels representing each type of lithology in the hyperspectral remote sensing image in the study area are automatically labeled to determine a lithology label map; A hyperspectral remote sensing dataset is produced according to the lithology label map and the corresponding hyperspectral remote sensing image.
3. The object-oriented hyperspectral remote sensing image surface lithology identification and classification method according to claim 2 is characterized in that: The pixels of the hyperspectral remote sensing image in the hyperspectral remote sensing dataset are divided into a training set and a test set using a no-leakage information dataset division strategy, specifically including: Determine training parameters, test parameters and verification parameters; the training parameters include the ratio of pixels participating in training to all pixels in the original image and the number of annotated pixels in each training block; the original image is a hyperspectral remote sensing image of the study area; the test parameters include the ratio of pixels participating in testing to all pixels in the original image and the number of annotated pixels in each test block; the verification parameters include the ratio of pixels participating in verification to all pixels in the original image and the number of annotated pixels in each verification block; According to the ratio of pixels participating in training to all pixels in the original image, a training image to be divided is divided from the original image; According to the number of annotated pixels in each training block, randomly select images with annotated pixels from the training images to be divided as training images, construct a training set, and use the remaining images with annotated pixels in the training images to be divided as leakage images; Determine a test image according to the test parameters and construct a test set; A verification image is determined according to the verification parameters, and a verification set is constructed.
4. The object-oriented hyperspectral remote sensing image surface lithology identification and classification method according to claim 3 is characterized in that: Determine the verification image according to the verification parameters, build the verification set, and then further include: Determine all training blocks and test blocks based on the training image, the leakage image, the test image, and the verification image; A sliding window strategy is adopted to obtain training patches and test patches from the training blocks and the test blocks.
5. The object-oriented hyperspectral remote sensing image surface lithology identification and classification method according to claim 1 is characterized in that: Determine the hyperspectral remote sensing images in the study area and the lithology type labels corresponding to each hyperspectral remote sensing image, and create a hyperspectral remote sensing dataset, which also includes: Preprocess the hyperspectral remote sensing images; the preprocessing process includes radiation correction, atmospheric correction and geometric correction.
