A method and system for lithological identification of hyperspectral images based on an object-oriented spatial spectrum enhancement convolutional neural network model

Abstract

The present invention discloses a hyperspectral image lithology recognition method based on an object-oriented spatial-spectral enhanced convolutional neural network model, which belongs to the field of remote sensing hyperspectral lithology mapping; the method includes: acquiring hyperspectral satellite images; preprocessing hyperspectral satellite images; obtaining samples and labels; model training; model prediction. The present invention also discloses a hyperspectral image lithology recognition system based on an object-oriented spatial-spectral enhanced convolutional neural network model. The present invention proposes a three-dimensional convolutional neural network that increases spatial information and fuses spectral feature information. On the basis of the traditional three-dimensional neural network for lithology classification, object-oriented sampling, three-dimensional space pyramid pooling and spectral cavity expansion methods are introduced to fully obtain spectral feature information of different scales, improve data utilization efficiency, and enhance lithology classification accuracy.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing hyperspectral lithology mapping, and in particular to a hyperspectral image lithology recognition method and system based on an object-oriented spatial-spectral enhanced convolutional neural network model. Background Art

[0002] The patent of this invention is aimed at the field of intelligent remote sensing geological mapping, and its purpose is to improve the digitalization and intelligence level of basic geological survey work. Lithology mapping is one of the key tasks of basic geological survey. Pre-mapping and investigation through remote sensing means can reduce the workload for field surveys. Satellite hyperspectral data contains hundreds of bands, and the information provided by a large number of spectral bands can provide qualitative and quantitative assistance for lithology classification.

[0003] With the development of computers, many scholars have introduced machine learning algorithms into the field of remote sensing lithology mapping, such as random forest (RF), support vector machine (SVM) supervised learning methods, and unsupervised learning frameworks combined with dimensionality reduction methods such as PCA and MNF. However, the above methods not only fail to fully exploit the spectral characteristics of hyperspectral data, but also fail to exploit the spatial information of hyperspectral images. In addition, although the object-oriented method combined with supervised classification can avoid the "salt and pepper effect" compared to the pixel-based calculation method, its accuracy is limited by the accuracy of the initial superpixel segmentation, and its generalization application ability is insufficient, and its ability to recognize small targets is poor.

[0004] Rocks present different spectral forms due to the influence of mineral composition, structure, weathering conditions, tectonic environment, alteration characteristics, etc. The same rock type may also reflect different spectral characteristics. Therefore, it is often difficult to find a suitable method for large-scale business applications in actual work.

[0005] In recent years, with the widespread application of deep learning convolutional neural networks in computer vision and remote sensing industries, some invention patents have been published on the application of lithology mapping using hyperspectral data combined with convolutional neural networks. Among them, the invention publication - 202011441451.0 - A method for lithology classification of hyperspectral SASI data, field rock identification in Liuyuan, Gansu, combined with hyperspectral remote sensing images and geological maps, selected typical rock pixel spectra in hyperspectral SASI images as sample data, and divided them into 4 types of rocks: alluvial, metamorphic, sedimentary and granite for identification, based on autoencoder + 2D convolution classification, the result accuracy is higher than traditional autoencoder classification. Invention Publication - 202111041101.X - A lithology identification method, device and medium based on unsupervised feature extraction. Considering that supervised classification greatly relies on a large number of high-quality and representative lithology samples, an unsupervised lithology classification method is proposed. A deep learning model is used to perform unsupervised lithology feature extraction on reflectance data to obtain predicted identification result data with the same resolution and spatial range as the original hyperspectral data. By constructing a lithology feature extraction network model, unsupervised extraction of spatial spectral features related to lithology identification is performed, which enhances the ability to obtain lithology spatial correlation information and improves the ability to screen effective features in the spectral dimension, thereby improving the extraction accuracy of lithology features.

[0006] In summary, for deep learning extraction of hyperspectral data of lithology, whether it is supervised classification or unsupervised classification, the core is to obtain the strong connection between spectral feature information and rock type, enhance spectral spatial information, and better reflect the rock type. Summary of the invention

[0007] The purpose of the present invention is to provide an automatic identification method for lithology extracted from hyperspectral images, build a three-dimensional convolutional neural network feature model through spatial spectrum enhancement, use the model for prediction, and achieve the goal of large-scale, big data lithology prediction. The technical problem to be solved by the present invention is to provide a lithology identification method and system based on the deficiencies of the prior art.

[0008] In order to solve the above technical problems, the present invention provides a hyperspectral image lithology recognition method based on an object-oriented spatial spectrum enhanced convolutional neural network model, comprising the following steps:

[0009] Acquire hyperspectral satellite imagery;

[0010] Preprocessing the hyperspectral satellite image to obtain a preprocessed hyperspectral image;

[0011] Obtain samples and labels based on the preprocessed hyperspectral images;

[0012] Input the samples and labels into the object-oriented space-spectrum enhanced convolutional neural network model for training, and obtain the trained object-oriented space-spectrum enhanced convolutional neural network model;

[0013] The hyperspectral satellite image of the data to be predicted is input into the trained object-oriented spatial-spectral enhanced convolutional neural network model for prediction to obtain the prediction result.

[0014] Preferably, the hyperspectral satellite image is preprocessed, specifically comprising the following steps:

[0015] Perform radiometric calibration, atmospheric correction, orthorectification and principal component analysis on hyperspectral satellite images.

[0016] Preferably, the radiation calibration is: using ENVI software, inputting the image to be calibrated and the image description file, and selecting Radiometric Correction>Radiometric Calibration in the Toolbox; performing radiation calibration to obtain the surface reflectivity after radiation correction;

[0017] The atmospheric correction is as follows: in the Toolbox software, select Radiometric Correction>AtmosphericCorrection Module>FLAASH Atmospheric Correction, perform atmospheric correction, and obtain the surface reflectance after atmospheric correction;

[0018] The orthorectification is as follows: using ENVI software, inputting hyperspectral satellite images, reference images and DEM data, outputting GCPs and performing orthorectification, eliminating the influence of terrain or deformation caused by camera orientation, and generating a planar orthorectified image;

[0019] The principal component analysis is:

[0020] Find the average of the x- and y-coordinates of the samples and

[0021] Find the characteristic covariance matrix; the covariance matrix is:

[0022]

[0023] in:

[0024]

[0025] Find the eigenvalues ​​and eigenvectors of the covariance;

[0026] Sort the eigenvalues ​​in descending order, select the largest c eigenvalues, and then use the corresponding c eigenvectors as column vectors to form an eigenvector matrix;

[0027] Project the sample points onto the selected eigenvectors. Assuming the number of samples is a and the number of features is b, the sample matrix after subtracting the mean is DataAdjust(a*b), the covariance matrix is ​​b*b, and the matrix composed of the selected c eigenvectors is EigenVectors(b*c); the projected data FinalData is:

[0028] FinalData(a*c)=DataAdjust(a*b)*EigenVectors(b*c)

[0029] The n-dimensional features of the sample are converted into k-dimensional, which is the projection of the original features on the k-dimensional.

[0030] Preferably, obtaining samples and labels according to the preprocessed hyperspectral image specifically includes the following steps:

[0031] Segment the preprocessed hyperspectral image to obtain object patches;

[0032] Create a sample window based on the object patch;

[0033] According to the sample window, the preprocessed hyperspectral image is cropped, and the corresponding lithology type labels are cropped to obtain samples and labels with different window sizes.

[0034] Preferably, segmenting the preprocessed hyperspectral image to obtain object spots specifically includes the following steps:

[0035] According to the set number of superpixels, seed points are evenly distributed in the preprocessed hyperspectral image;

[0036] Calculate the gradient values ​​of all pixels in the neighborhood of the seed point, and move the seed point to the pixel with the smallest gradient in the neighborhood;

[0037] Assign a class label to each pixel in the neighborhood around each seed point;

[0038] The distance between the pixel point and the seed point is measured; the formula of the distance measurement is:

[0039]

[0040]

[0041]

[0042] Among them, d cRepresents color distance; d s Represents the spatial distance; N s is the maximum spatial distance within the class; Nc is the maximum color distance; D is the distance metric;

[0043] Continue to iterate until the error converges and the optimal seed point is obtained;

[0044] According to the optimal seed point, the object patch is obtained.

[0045] Preferably, creating a sample window according to the object spot specifically includes the following steps:

[0046] Classify and sort the object spots according to their area sizes, and select some object spots in each category as reference object spots;

[0047] Obtain the centroid point of the reference object patch;

[0048] According to the centroid point, the bounding rectangle of the reference object spot is obtained as the sample window.

[0049] Preferably, the object-oriented spatial spectrum enhanced convolutional neural network model includes:

[0050] Multi-scale spatial feature fusion: N different pyramid scales are used to pool the feature maps to the target size, and then the pooled results are convolved. Each feature map is upsampled using bilinear interpolation to obtain the same size as the original feature map. The original feature map and the upsampled feature map are concatenated according to the channel dimension to obtain the output after feature fusion.

[0051] Three-dimensional spectral feature information expansion: The input data is convolved using a three-dimensional dilated convolution model; the length and width of the three-dimensional convolution kernel remain unchanged, and the three-dimensional spectral dilated convolution depth d is obtained by changing the dilated convolution rate.

[0052] d=d i (2r-1)-(r-1)

[0053] Where: r is the void convolution rate, the initial value is 1, and it increases successively, d i is the initial depth of the three-dimensional spectral hole convolution, and its initial value is 3;

[0054] Residual network connection: The residual network is connected between the module input and output to quickly transmit the input information to the deep layer. The output expression of the Ith block in the residual network is:

[0055] x I =F(x I-1 )H(x I-1 )

[0056] Among them, xI 、x I-1 They represent the outputs of the Ith block and the I-1th block respectively, F(x) represents the calculation result of the convolutional neural network, and H(x) represents the calculation result of the residual mapping;

[0057] Fully connected: The fully connected layer maps convolutional features to sample labels;

[0058] Activation function: The convolutional network uses the Relu function, and the function expression is:

[0059]

[0060] Among them, the max function means taking the maximum value between 0 and x;

[0061] Output: Output the trained object-oriented spatial spectrum enhanced convolutional neural network model.

[0062] Preferably, the method further comprises the following steps:

[0063] The prediction results were compared with the geological map to evaluate the accuracy.

[0064] Preferably, the prediction results are compared with the geological map to evaluate the accuracy, which specifically includes the following steps:

[0065] Perform accuracy evaluation on the prediction results; the accuracy evaluation includes overall accuracy OA, average accuracy AA, Kappa coefficient, precision P, recall R and F1 score;

[0066] The formula for the accuracy evaluation is:

[0067]

[0068]

[0069]

[0070]

[0071]

[0072]

[0073] Where: TP is the positive sample predicted by the model as positive; TN is the negative sample predicted by the model as negative; FP is the negative sample predicted by the model as positive; FN is the positive sample predicted by the model as negative;

[0074] The calculation formula of average precision AA is the average of the recall rate R of each category, and the calculation formula of P0 in the Kappa coefficient is equivalent to the precision rate P, P eIt represents the expected consistency and is calculated as:

[0075]

[0076] Where: R i is the total number of positive or negative examples of the true value; C i is the total number of positive or negative examples of the predicted value; N is the total number of true values ​​and predicted values.

[0077] The present invention also provides a hyperspectral image lithology recognition system based on an object-oriented spatial spectrum enhanced convolutional neural network model, comprising:

[0078] Acquisition module, used to acquire hyperspectral satellite images;

[0079] A preprocessing module is used to preprocess the hyperspectral satellite image to obtain a preprocessed hyperspectral image;

[0080] The sample and label making module is used to obtain samples and labels based on the pre-processed hyperspectral images;

[0081] A training module, used for inputting samples and labels into the object-oriented space-spectrum enhanced convolutional neural network model for training, so as to obtain a trained object-oriented space-spectrum enhanced convolutional neural network model;

[0082] The prediction module is used to input the hyperspectral satellite image of the data to be predicted into the trained object-oriented spatial-spectral enhanced convolutional neural network model for prediction to obtain the prediction result.

[0083] Compared with the prior art, the present invention has the following beneficial effects:

[0084] The present invention proposes a three-dimensional convolutional neural network that adds spatial information and fuses spectral feature information. On the basis of the traditional three-dimensional neural network for lithology classification, object-oriented sampling, three-dimensional spatial pyramid pooling and spectral cavity expansion methods are introduced to fully obtain spectral feature information of different scales, improve data utilization efficiency, and enhance lithology classification accuracy.

[0085] The present invention constructs a three-dimensional convolutional neural network feature model through spatial spectrum enhancement, and uses the model to make predictions, thereby achieving the goal of large-scale, big data lithology prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings.

[0087] Figure 1 Sampling graph for object-oriented;

[0088] Figure 2It is a schematic diagram of the process of the hyperspectral image lithology recognition method based on the object-oriented three-dimensional convolutional neural network model of the present invention;

[0089] Figure 3 Implementing process diagram for object-oriented 3D convolutional neural network model;

[0090] Figure 4 This is a framework diagram of the object-oriented three-dimensional convolutional neural network model;

[0091] Figure 5 This is the simulation result diagram of the object-oriented three-dimensional convolutional neural network model. DETAILED DESCRIPTION

[0092] Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention, so the present invention is not limited to the specific implementation disclosed below.

[0093] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0094] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0095] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0096] A hyperspectral image lithology recognition method based on an object-oriented three-dimensional convolutional neural network model includes the following specific steps:

[0097] Step 1, obtain hyperspectral satellite images;

[0098] Step 2, preprocessing the hyperspectral satellite image to obtain a preprocessed hyperspectral image;

[0099] Step 3, make samples and labels;

[0100] Step 4, inputting the samples and labels into the object-oriented spatial spectrum enhanced convolutional neural network model for training, and obtaining a trained object-oriented spatial spectrum enhanced convolutional neural network model;

[0101] Step 5: Input the hyperspectral satellite image of the data to be predicted into the trained object-oriented spatial-spectral enhanced convolutional neural network model for prediction to obtain the prediction result.

[0102] In the method, wherein the step 1 first selects a hyperspectral satellite image, the selection includes the following requirements:

[0103] 1.1. The spectral resolution of hyperspectral satellite images shall not be less than 10 nm;

[0104] 1.2, Hyperspectral satellite image spectral range 400-2500nm;

[0105] 1.3, the spatial resolution of hyperspectral satellite images shall not be less than 30m;

[0106] 1.4, for hyperspectral satellite images, you can choose Gaofen 5 or Resource 02E.

[0107] In the method, wherein the step 2 remote sensing image preprocessing comprises the following steps:

[0108] 2.1, Radiometric calibration: Use ENVI software to input the image to be calibrated and the image description file, and in the Toolbox, select Radiometric Correction>Radiometric Calibration. Perform radiometric calibration to obtain the surface reflectance after radiometric correction.

[0109] 2.2, Atmospheric Correction:

[0110] In the Toolbox software, select Radiometric Correction>Atmospheric CorrectionModule>FLAASH Atmospheric Correction, perform atmospheric correction, and obtain the surface reflectance after atmospheric correction.

[0111] 2.3. Orthorectification: Use ENVI software to input the image to be orthorectified, the reference image and DEM data, output GCPs and perform orthorectification to eliminate the influence of terrain or deformation caused by camera orientation, and generate a planar orthophoto.

[0112] 2.4, Principal Component Analysis (PCA): It is a commonly used dimensionality reduction method. It can realize hundreds of hyperspectral bands. The most important band information is obtained through dimensionality reduction operation and the output of model training is input. The specific steps are as follows:

[0113] 2.4.1, calculate the average value of x-coordinate and y-coordinate in the sample respectively and Then for all samples, subtract the corresponding mean.

[0114] 2.4.2, find the feature covariance matrix. If the feature is three-dimensional, the covariance matrix is:

[0115]

[0116] in:

[0117]

[0118] 2.4.3, find the eigenvalues ​​and eigenvectors of the covariance.

[0119] 2.4.4, sort the eigenvalues ​​in descending order, select the largest c of them, and then use the corresponding c eigenvectors as column vectors to form an eigenvector matrix.

[0120] 2.4.5, project the sample points onto the selected eigenvectors. Assume that the number of samples is a, the number of features is b, the sample matrix after subtracting the mean is DataAdjust(a*b), the covariance matrix is ​​b*b, and the matrix composed of the selected c eigenvectors is EigenVectors(b*c). Then the projected data FinalData is:

[0121] FinalData(a*c)=DataAdjust(a*b)*EigenVectors(b*c) (3)

[0122] In this way, the n-dimensional features of the original sample are transformed into k-dimensional, and this k-dimensional is the projection of the original features on the k-dimensional.

[0123] In the method, wherein the step 3 of preparing samples and labels comprises the following steps:

[0124] 3.1, Object-oriented segmentation: The pre-processed hyperspectral images and lithology categories are sampled in an object-oriented manner. They are cut into squares of a fixed size, the size of which is related to the GPU memory of the hardware configuration. Then, training and test data sets are randomly generated. The training and validation data sets are allocated in a certain proportion, and the training and test sets are required to be divided into two types of data: samples and labels. The training and test set data are normalized so that the band information corresponds to the lithology category.

[0125] 3.1.1 Segment the preprocessed hyperspectral image to obtain several object patches (s). The segmentation method is sampled from SLIC (simple linear iterative clustering), linear iterative clustering. It is an algorithm with simple ideas and convenient implementation. It converts the color image into a 5-dimensional feature vector in CIELAB color space and XY coordinates, and then constructs a distance metric for the 5-dimensional feature vector to perform local clustering of image pixels. Specific steps:

[0126] 1. Initialize seed points (cluster centers): Evenly distribute seed points in the image according to the set number of superpixels. Assuming that the image has a total of N pixels, which are pre-divided into K superpixels of the same size, then the size of each superpixel is N / K, and the distance (step length) between adjacent seed points is approximately S = sqrt(N / K).

[0127] 2. Reselect the seed point in the n*n neighborhood of the seed point (generally n=3). The specific method is: calculate the gradient values ​​of all pixels in the neighborhood and move the seed point to the place with the smallest gradient in the neighborhood. The purpose of this is to avoid the seed point falling on the contour boundary with a large gradient, so as not to affect the subsequent clustering effect.

[0128] 3. Assign a class label to each pixel in the neighborhood around each seed point (i.e., which cluster center it belongs to). Unlike the standard k-means search in the entire image, SLIC limits the search range to 2S*2S, which can accelerate the convergence of the algorithm.

[0129] 4. Distance measurement. Including color distance and spatial distance. For each searched pixel, calculate the distance between it and the seed point. The distance calculation method is as follows:

[0130]

[0131]

[0132] Where dc represents the color distance, ds represents the spatial distance, and Ns is the maximum spatial distance within the class, defined as Ns = S = sqrt(N / K), which is applicable to each cluster. The maximum color distance Nc varies from image to image and from cluster to cluster, so we use a fixed constant m instead. The final distance metric D is as follows:

[0133]

[0134] 5. Iterative optimization. In theory, the above steps are iterated continuously until the error converges (which can be understood as the cluster center of each pixel point no longer changes).

[0135] 3.2 Create sample window: Based on the object patch obtained by object-oriented segmentation in 3.1, create a sample window and extract samples. The sample window creation rules are as follows:

[0136] 1. Taking a single-scene hyperspectral image and the corresponding lithology sample as an example, the objects after the single-scene segmentation are sorted according to the area size level, and the same number of object patches are selected for different area levels, and the total number of selected object patches is not less than 60% of the total object patches.

[0137] 2. Take the centroid of the object spot, and take the circumscribed rectangle of the object spot according to the centroid. Set the circumscribed rectangle as the sample window.

[0138] 3.3 Make samples and labels: According to the sample window created in 3.2, crop the preprocessed hyperspectral image and the corresponding lithology type label, and then obtain samples and labels of different window sizes. In the subsequent model training, the size of each sample and label will be unified.

[0139] In the method, the model training in step 4 comprises the following steps:

[0140] 4.1 Input: Make three-dimensional data slices: Slice and standardize the samples and labels made in step 3 according to the window size specified by the model training, and then input them into the model.

[0141] 4.2 Multi-scale spatial feature fusion: In semantic segmentation, fusing features of different scales can increase the accuracy of segmentation. In order to make full use of spatial information and image features of different scales, multi-scale spatial feature fusion is required. The pyramid pooling model is a commonly used multi-scale spatial feature fusion model, which can effectively increase the receptive field and increase the utilization efficiency of global information. Using N different pyramid scales, the number of layers of the pyramid pooling module and the size of each layer can be modified. In the paper, the pyramid pooling module has N layers, the number of layers depends on the channel size, and the size of each layer is different, depending on the convolution kernel window size. First, the feature maps are pooled to the target size, and then the pooled results are convolved. Then, each feature map in the previous step is upsampled using bilinear interpolation to obtain the same size as the original feature map. Finally, the original feature map and the upsampled feature map are concatenated according to the channel dimension to obtain the output after feature fusion.

[0142] 4.3 Three-dimensional spectral feature information expansion: Using the three-dimensional hole convolution model, the input data is convolved to maximize the use of hyperspectral spectral information. r is the hole convolution rate, the initial value is 1, and it increases in sequence. iis the initial depth of the three-dimensional spectral hole convolution, and its initial value is 3. The length and width of the three-dimensional convolution kernel remain unchanged, and the three-dimensional spectral hole convolution depth d is obtained by changing the hole convolution rate.

[0143] d=d i (2r-1)-(r-1) (7)

[0144] 4.4 Residual network connection: In deep learning, the deepening of the model may cause the problem of gradient vanishing and gradient exploding. The residual network connection builds a "fast channel" between the module input and output, quickly transmitting the input information to the deep layer as an independent mapping to avoid the complex transformation of the hidden layer. The output expression of the Ith block in the residual network is:

[0145] x I =F(x I-1 )H(x I-1 )

[0146] Among them, x I 、x I-1 They represent the outputs of the I-th block and the I-1-th block respectively, F(x) represents the calculation result of the convolutional neural network, and H(x) represents the calculation result of the residual mapping.

[0147] 4.5 Fully connected: The fully connected layer maps the convolutional features to sample labels, which is equivalent to a classifier.

[0148] 4.6 Activation function: Convolutional neural network is a linear iterative process, which requires the introduction of nonlinear factors to isolate different modules and avoid model degradation. Activation functions are mostly common nonlinear functions. Convolutional networks usually use the Relu function, and the function expression is:

[0149]

[0150] The max function represents the maximum value between 0 and x.

[0151] 4.7 Output: Output the trained model.

[0152] In the method, wherein the model prediction in step 5 comprises the following steps:

[0153] 5.1 Prediction: Extract the algorithm model output in step 4.7, select other areas with the same lithology category as the training sample for prediction, and the prediction result is the lithology classification map of the area. Compare it with the geological map to obtain the classification accuracy result.

[0154] 5.2 Accuracy evaluation indicators: The classification accuracy uses overall accuracy OA, average accuracy AA, Kappa coefficient, precision P, recall rate R, and F1 score as result evaluation indicators. The calculation of these indicators depends on the basic concepts derived from four classification results, namely TP, FP, TN, and FN. Their corresponding concepts are:

[0155] TP: positive samples predicted by the model as positive;

[0156] TN: negative samples predicted by the model as negative classes;

[0157] FP: negative samples predicted as positive by the model;

[0158] FN: positive samples predicted by the model as negative classes;

[0159] The confusion matrix corresponding to the four is shown in the following table:

[0160] Table 1: Confusion matrix table corresponding to the four

[0161]

[0162] The definitions of each indicator are as follows:

[0163]

[0164]

[0165]

[0166]

[0167]

[0168]

[0169] The calculation formula of average precision AA is the average of the recall rate R of each category, and the calculation formula of P0 in the Kappa coefficient is equivalent to the precision rate P, P e It represents the expected consistency and is calculated as:

[0170]

[0171] Where: R i is the total number of positive or negative examples of the true value; C i is the total number of positive or negative examples of the predicted value; N is the total number of true values ​​and predicted values.

[0172] In order to better illustrate the technical effect of the present invention, the present invention provides the following specific embodiments to illustrate the above technical process:

[0173] Embodiment 1: A hyperspectral image lithology recognition method based on an object-oriented three-dimensional convolutional neural network model comprises the following steps:

[0174] The GF5B satellite data of the Saudi shield area is obtained. The satellite images meet the requirements of spatial and spectral resolution. The data is preprocessed by radiometric calibration, atmospheric correction, orthorectification, principal component analysis, and object segmentation. Samples and labels are made based on satellite images and vector geological maps. The seven category labels are: Quaternary alluvial layer, Quaternary gravel, granite and granodiorite, alkaline granite, schist and gneiss, gneiss and mixed rock, mica schist and schist. The samples and labels are put into the model training, and some areas in the samples are selected for model prediction, which includes the following steps:

[0175] 1) Obtain GF5B satellite hyperspectral data of the Saudi Arabian shield region, 290 bands, 30-meter spatial resolution, and a band range of 400-2500nm.

[0176] 2) Preprocess the remote sensing images, including radiation calibration, atmospheric correction, orthorectification, principal component analysis and object segmentation steps.

[0177] Step 2.1, radiometric calibration, use ENVI software, input the image to be radiometrically calibrated and the image description file, and in the Toolbox, select Radiometric Correction>Radiometric Calibration. Perform radiometric calibration to obtain the surface reflectance after radiometric correction.

[0178] Step 2.2, atmospheric correction, use ENVI software, in the Toolbox, select Radiometric Correction>Atmospheric Correction Module>FLAASH Atmospheric Correction, fill in the atmospheric model parameters according to the image header file information, perform atmospheric correction, and obtain the surface reflectance after atmospheric correction.

[0179] Step 2.3, orthorectification, use ENVI software, in the Toolbox, select Geometric Correction>Orthorectification>RPC Orthorectification, input the image to be orthorectified, the reference image and DEM data, output GCPs and perform orthorectification to generate a planar orthoimage.

[0180] Step 2.4, principal component analysis, use ENVI software, in the Toolbox, select Transform>PCARotation>Forward PCA Rotation Existing Statistics, input the image to be processed, set the number of output features k, and obtain the image after principal component transformation.

[0181] Step 2.5, object segmentation, use SLIC (simple linear iterative clustering), linear iterative clustering algorithm, to segment the pre-processed hyperspectral image to obtain several object patches (s). It is a simple and easy-to-implement algorithm that converts the color image into a 5-dimensional feature vector in CIELAB color space and XY coordinates, and then constructs a distance metric for the 5-dimensional feature vector to perform local clustering of image pixels.

[0182] Step 2.5.1, initialize seed points (cluster centers): distribute seed points evenly in the image according to the set number of superpixels. The image has a total of 110484 pixels, which are pre-divided into 200 superpixels of the same size. The size of each superpixel is 552, and the distance (step size) between adjacent seed points is 25.

[0183] Step 2.5.2, reselect the seed point in the 3*3 neighborhood of the seed point. The specific method is: calculate the gradient values ​​of all pixels in the neighborhood, and move the seed point to the place with the smallest gradient in the neighborhood. The purpose of this is to avoid the seed point falling on the contour boundary with a large gradient, so as not to affect the subsequent clustering effect.

[0184] Step 2.5.3, assign a class label to each pixel in the neighborhood around each seed point (i.e., which cluster center it belongs to). Unlike the standard k-means search in the entire image, the search range of SLIC is limited to 50*50, which can accelerate the convergence of the algorithm.

[0185] Step 2.5.4, distance measurement. Including color distance and spatial distance. For each searched pixel point, calculate the distance between it and the seed point. The distance calculation method is as follows:

[0186]

[0187]

[0188] Among them, dc represents the color distance, ds represents the spatial distance, and Ns is the maximum spatial distance within the class, which is defined as Ns = S = sqrt (N / K) = 25, which is applicable to each cluster. The maximum color distance Nc varies with different images and also with different clusters, so we use a fixed constant m instead. According to actual experience, m takes 9 for the best effect. The final distance metric D is as follows:

[0189]

[0190] Step 2.5.5, iterative optimization. Theoretically, the above steps are iterated continuously until the error converges (the cluster center of each pixel no longer changes).

[0191] 3) Create sample and lithology labels.

[0192] Step 3.1, based on the 200 object patches generated in step 2.5, create a sample window to extract samples. The sample window creation rules are as follows: (1) Taking a single-view hyperspectral image and the corresponding lithology sample as an example, sort the segmented objects according to their area size, select the same number of object patches for different area sizes, and the total number of selected object patches should not be less than 60% of the total object patches. (2) Take the centroid point of the object patch, and take the circumscribed rectangle of the object patch based on the centroid point. The circumscribed rectangle is defined as the sample window.

[0193] Table 2 records the number of samples for each lithology

[0194]

[0195]

[0196] Step 3.2: Based on the sample window created in step 3.1, crop the preprocessed hyperspectral image and the corresponding lithology type label, and then obtain samples and labels of different window sizes. In subsequent model training, the size of each sample and label will be unified.

[0197] 4) Model training

[0198] Step 4.1: Slice and standardize the samples and labels obtained in step 3.2 into a size of 5*5 and input them into the model.

[0199] In step 4.2, multi-scale spatial fusion includes three pooling layers, 2D convolution layers and upsampling layers. Therefore, the pyramid pooling module has three layers, and the pooling kernel sizes are 1*1, 3*3 and 5*5, respectively, which are used to capture spatial features of different scales. The convolution kernel window size is consistent with the pooling kernel. The upsampling layer uses bilinear interpolation to restore the original size of the feature map. Finally, the different feature maps are merged along the channel axis through the concatenate function.

[0200] Step 4.3: The three-dimensional spectral feature information is expanded using a three-dimensional dilated convolution model. While keeping the convolution kernel size unchanged, a dilated interval is introduced to expand the receptive field. The three convolutions with expansion rates of 1, 2, and 3 are designed, and the corresponding receptive fields are 3*3, 7*7, and 11*11.

[0201] Step 4.4, merging the convolution outputs with different expansion rates and putting them into the Resnet50 network for training. In the embodiment, different expansion rates are designed for the 1st, 2nd and 3rd layers of Resnet according to the receptive field of the expansion of the three-dimensional spectral feature information, and the learning rate is set to 0.00001.

[0202] Step 4.5, there are 3 fully connected layers, and the output parameters are 256, 128 and 7 respectively. A dropout layer is set in the middle of the fully connected layer with a parameter of 0.1.

[0203] In step 4.6, all activation functions in the model use the Relu activation function except the last fully connected layer which uses Softmax.

[0204] In step 4.7, the output model results are stored in hdf5 format.

[0205] 5) Prediction results.

[0206] Step 5.1, use the hdf5 file of the output model to predict the results and obtain the classification result graph.

[0207] Step 5.2, the classification results are compared with the geological map and the accuracy index is calculated.

[0208] Tables 3 and 4 record the algorithm accuracy.

[0209] Table 3 Accuracy evaluation table 1

[0210] Accuracy index Mix-3DCNN OA 0.8312 AA 0.7536 Kappa 0.7894

[0211] Table 4 Accuracy evaluation table 2

[0212]

[0213] The Kappa coefficient of Mix-3DCNN is greater than 0.75, indicating that the experimental repeatability is high. While fully acquiring spectral information, combined with spatial information reference, better classification accuracy and effect are obtained, and the classification accuracy maintains good consistency, which can meet the requirements of lithology classification in geological mapping.

[0214] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules, modules or units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units, modules or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0215] The units may or may not be physically separated, and the components displayed as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0216] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0217] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the central processing unit (CPU), the above-mentioned functions defined in the method of the present invention are executed. It should be noted that the above-mentioned computer-readable medium of the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of an electrical, magnetic, optical, electromagnetic, infrared segment, or semiconductor, or any combination of the above.

[0218] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0219] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A hyperspectral image lithology recognition method based on object-oriented spatial spectrum enhancement convolutional neural network model, It is characterized in that The following steps are involved: Acquire hyperspectral satellite imagery; Perform radiometric calibration, atmospheric correction, orthorectification and principal component analysis on hyperspectral satellite images to obtain pre-processed hyperspectral images; Obtain samples and labels based on the preprocessed hyperspectral images; Input samples and labels into the object-oriented spatial spectrum enhanced convolutional neural network model for training, and obtain the trained object-oriented spatial spectrum enhanced convolutional neural network model; the object-oriented spatial spectrum enhanced convolutional neural network model is: based on the lithology classification three-dimensional neural network, object-oriented sampling, three-dimensional space pyramid pooling and spectral cavity expansion method are introduced to fully obtain spectral feature information of different scales; The hyperspectral satellite image of the data to be predicted is input into the trained object-oriented spatial-spectral enhanced convolutional neural network model for prediction to obtain the prediction result; The object-oriented spatial spectrum enhanced convolutional neural network model includes: Multi-scale spatial feature fusion: N different pyramid scales are used to pool the feature maps to the target size, and then the pooled results are convolved. Each feature map is upsampled using bilinear interpolation to obtain the same size as the original feature map. The original feature map and the upsampled feature map are concatenated according to the channel dimension to obtain the output after feature fusion. Three-dimensional spectral feature information expansion: The input data is convolved using a three-dimensional dilated convolution model; the length and width of the three-dimensional convolution kernel remain unchanged, and the three-dimensional spectral dilated convolution depth d is obtained by changing the dilated convolution rate; d=d i (2r-1)-(r-1) Where: r is the dilated convolution rate, the initial value is 1, and it increases successively; d i is the initial depth of the three-dimensional spectral hole convolution, and its initial value is 3; Residual network connection: The residual network is connected between the module input and output to quickly transmit the input information to the deep layer. The output expression of the Ith block in the residual network is: x I =F(x I-1 )H(x I-1 ) Among them, x I 、x I-1 They represent the outputs of the Ith block and the I-1th block respectively, F(x) represents the calculation result of the convolutional neural network, and H(x) represents the calculation result of the residual mapping; Fully connected: The fully connected layer maps convolutional features to sample labels; Activation function: The convolutional network uses the Relu function, and the function expression is: Among them, the max function means taking the maximum value between 0 and x; Output: Output the trained object-oriented spatial spectrum enhanced convolutional neural network model.

2. The hyperspectral image lithology recognition method based on the object-oriented spatial spectrum enhanced convolutional neural network model according to claim 1, It is characterized in that According to the preprocessed hyperspectral image, samples and labels are obtained, which specifically includes the following steps: Segment the preprocessed hyperspectral image to obtain object patches; Create a sample window based on the object patch; According to the sample window, the preprocessed hyperspectral image is cropped, and the corresponding lithology type labels are cropped to obtain samples and labels with different window sizes.

3. The hyperspectral image lithology recognition method based on the object-oriented spatial spectrum enhanced convolutional neural network model according to claim 2, It is characterized in that According to the object spot, a sample window is created, which specifically includes the following steps: Classify and sort the object spots according to their area sizes, and select some object spots in each category as reference object spots; Obtain the centroid point of the reference object patch; According to the centroid point, the bounding rectangle of the reference object spot is obtained as the sample window.

4. The hyperspectral image lithology recognition method based on the object-oriented spatial spectrum enhanced convolutional neural network model according to claim 1, It is characterized in that The following steps are also included: The prediction results were compared with the geological map to evaluate the accuracy.

5. The hyperspectral image lithology recognition method based on the object-oriented spatial spectrum enhanced convolutional neural network model according to claim 4, It is characterized in that Compare the prediction results with the geological map and evaluate the accuracy, which includes the following steps: Perform accuracy evaluation on the prediction results; the accuracy evaluation includes overall accuracy OA, average accuracy AA, Kappa coefficient, precision P, recall R and F1 score; The formula for the accuracy evaluation is: Where: TP is the positive sample predicted by the model as positive; TN is the negative sample predicted by the model as negative; FP is the negative sample predicted by the model as positive; FN is the positive sample predicted by the model as negative; The calculation formula of average precision AA is the average of the recall rate R of each category, n represents the total number of samples, and P in the Kappa coefficient is 0 The calculation formula is equivalent to the accuracy P, P e It represents the expected consistency and is calculated as: Where: R i is the total number of positive or negative examples of the true value; C i is the total number of positive or negative examples of the predicted value; N is the total number of true values ​​and predicted values.

6. A hyperspectral image lithology recognition system based on an object-oriented spatial-spectral enhanced convolutional neural network model, used to implement the hyperspectral image lithology recognition method based on an object-oriented spatial-spectral enhanced convolutional neural network model as described in any one of claims 1 to 5, It is characterized in that include: Acquisition module, used to acquire hyperspectral satellite images; A preprocessing module is used to preprocess the hyperspectral satellite image to obtain a preprocessed hyperspectral image; The sample and label making module is used to obtain samples and labels based on the pre-processed hyperspectral images; A training module, used for inputting samples and labels into the object-oriented space-spectrum enhanced convolutional neural network model for training, so as to obtain a trained object-oriented space-spectrum enhanced convolutional neural network model; The prediction module is used to input the hyperspectral satellite image of the data to be predicted into the trained object-oriented spatial-spectral enhanced convolutional neural network model for prediction to obtain the prediction result.

Images