Sea fog hyperspectral remote sensing image recognition method and system based on DenseNet

Abstract

The invention relates to a sea fog hyperspectral remote sensing image recognition method and system based on DenseNet, and the method comprises the following steps: S1, obtaining hyperspectral image data and corresponding label data, carrying out the data analysis and processing, and constructing a hyperspectral data set for training; s2, constructing a DenseNet model, and training and optimizing the DenseNet model in combination with the hyperspectral data set to obtain a final DenseNet model; and S3, obtaining a target sea fog image to be recognized, and recognizing and classifying the target sea fog image through the final DenseNet model to obtain a recognition and classification result of the target sea fog image. According to the sea fog hyperspectral remote sensing image recognition method, the attention mechanism is introduced to enable the model to pay more attention to important areas in the image, areas related to sea fog recognition are highlighted, and compared with the original feature extraction capability only depending on DenseNet, the rich information can be more effectively mined and utilized to improve the classification precision.

Description

Technical Field

[0001] The present disclosure relates to the technical field of image processing, and in particular to a method and system for recognizing sea fog hyperspectral remote sensing images based on DenseNet. Background Art

[0002] In the analysis of hyperspectral remote sensing images in sea fog areas, the extraction of spatial texture features is crucial for image classification and recognition. Many scholars have tried to extract the spatial texture features of these hyperspectral remote sensing images through various methods and have achieved certain research results. However, different spatial texture features show significant differences in auxiliary classification, which leads to inconsistency in classification performance.

[0003] Although the existing DenseNet model has achieved certain results in extracting hyperspectral image features, it still has some shortcomings. When processing hyperspectral data, the DenseNet model fails to fully capture the feature information in the image, especially the global features. Since hyperspectral images usually contain rich spectral information and complex spatial structures, it is difficult to fully mine and utilize this information by relying solely on the existing feature extraction capabilities of the DenseNet model. Therefore, it has limited effect on improving the classification accuracy of hyperspectral remote sensing images. Summary of the invention

[0004] In order to solve the problems existing in the above-mentioned prior art, the purpose of the present disclosure is to provide a method and system for sea fog hyperspectral remote sensing image recognition based on DenseNet. The sea fog hyperspectral remote sensing image recognition method based on DenseNet introduces an attention mechanism to make the model pay more attention to important areas in the image, and highlights the areas related to sea fog recognition by learning the weight of each spatial position in the feature map, which to a certain extent makes up for the defect that the DenseNet model does not grasp the global spatial features; compared with relying solely on the original feature extraction capability of DenseNet, it can more effectively mine and utilize these rich information to improve classification accuracy.

[0005] The present invention discloses a method for recognizing sea fog hyperspectral remote sensing images based on DenseNet, comprising the following steps:

[0006] S1, obtain hyperspectral image data and corresponding label data, perform data analysis and processing, and construct a hyperspectral dataset for training;

[0007] S2, constructing a DenseNet model, and training and optimizing the DenseNet model in combination with the hyperspectral dataset to obtain a final DenseNet model;

[0008] S3. Obtain a target sea fog image to be identified, identify and classify the target sea fog image through the final DenseNet model, and obtain an identification and classification result of the target sea fog image.

[0009] Preferably, step S1 specifically includes:

[0010] Acquire hyperspectral image data and corresponding label data, wherein the hyperspectral image data includes image space dimensions and spectral band dimensions, and labels of the label data include sea fog areas, building areas, vegetation areas, and other areas;

[0011] Traversing the multidimensional array of the hyperspectral image data to obtain pixel reflectance, using a visualization library to draw a spectral curve according to the band number and reflectance, and obtaining spectral feature information data of different regions according to the spectral curve;

[0012] According to the hyperspectral image data, the texture features and edge features of different regions are analyzed by spatial dimension to obtain spatial feature information data of different regions;

[0013] Converting the hyperspectral image data into a digital matrix format and converting the label data into a digital label format;

[0014] StandardScaler is used to normalize the hyperspectral image data, and the normalized hyperspectral image data is divided into a training set and a test set in the ratio of A1:A2. The image data of the training set and the test set are paired and integrated with the corresponding label data to construct a hyperspectral dataset for training.

[0015] Preferably, step S2 specifically includes the following steps:

[0016] S201, construct a convolutional neural network basic feature extractor;

[0017] S202. Build a DenseNet model and introduce the attention mechanism;

[0018] S203, training the DenseNet model in combination with the training set, extracting features from the test set according to the trained DenseNet model and classifying the test set using a support vector machine to obtain a classification result;

[0019] S204, evaluating and analyzing the classification results to obtain evaluation results, and adjusting and optimizing the DenseNet model based on the evaluation results to obtain a final DenseNet model.

[0020] Preferably, step S201 specifically includes:

[0021] The hyperspectral data set is passed into the first convolution layer, and a convolution operation is performed by sliding a convolution kernel on the spectral band dimension of the hyperspectral image data to extract the correlation between the bands, and the specific band reflectance characteristics and the spectral curve shape characteristic spectral information related to the sea fog area are captured, and the specific band reflectance characteristics and the spectral curve shape characteristics are integrated into the desired output feature map, and the feature map is output;

[0022] After the first convolutional layer, add a 2D convolutional layer or a 3D convolutional layer.

[0023] If a two-dimensional convolutional layer is used, the convolution kernel is slid in the local spatial area of ​​the feature map to capture the spatial texture and edge information in the local area of ​​the image, and the feature map processed by the two-dimensional convolutional layer is output;

[0024] If a three-dimensional convolutional layer is used, the convolution kernel slides in the three-dimensional region including the spatial and spectral dimensions, and the complex local feature information is mined by integrating the spatial and spectral features, and the feature map processed by the three-dimensional convolutional layer is output;

[0025] A batch normalization layer is connected after the convolution layer to normalize the data of each channel of the feature map output by the convolution;

[0026] The ReLU activation function is used to perform nonlinear activation on the normalized feature map, and a maximum pooling layer is added to downsample the activated feature map to halve the spatial size of the feature map. The feature map is obtained after collaborative processing by each convolutional layer and related layers, completing preliminary feature extraction and data dimensionality reduction.

[0027] Preferably, in step S202, constructing a DenseNet model specifically includes:

[0028] According to the feature map obtained by the convolutional neural network basic feature extractor through feature extraction and the number of classifications required to be distinguished, a DenseNet model class is defined, wherein the DenseNet model class includes an input dimension, a growth rate, a number of layers, and a number of classifications;

[0029] Add multiple Dense blocks in the DenseNet model initialization function, each of which constructs a convolutional layer, a batch normalization layer, a ReLu activation layer, and a maximum pooling layer in sequence;

[0030] Construct a classifier linear layer, flatten the feature map output by the last Dense block and convert it into a one-dimensional vector form, use the classifier linear layer class to create a linear layer instance according to the dimension of the flattened vector and the number of categories, set the input dimension to correspond to the dimension of the flattened vector, and the output dimension to the number of categories, and pass the flattened vector into the classifier linear layer, and output the predicted probability value corresponding to each category.

[0031] Preferably, the channel attention mechanism and the spatial attention mechanism are sequentially inserted after the output of each convolutional layer of the DenseNet model to form a complete processing chain;

[0032] The channel attention mechanism is inserted into the DenseNet model including:

[0033] For the feature maps output by each convolutional layer in the DenseNet model, global average pooling and global maximum pooling are used to obtain the feature vectors representing the average eigenvalue and maximum eigenvalue of each channel;

[0034] The feature vectors representing the average eigenvalue and the maximum eigenvalue of each channel are concatenated and passed into a feedforward neural network consisting of two fully connected layers to generate a weight vector corresponding to each channel;

[0035] Multiply the generated weight vector by the original convolutional layer output feature map channel by channel in the channel dimension, adjust the characteristics of each channel according to the weight value, and output the feature map weighted by the channel attention mechanism;

[0036] The spatial attention mechanism is inserted into the DenseNet model including:

[0037] The feature maps processed by the channel attention mechanism are fused by splicing the channel dimensions to obtain a fused feature map containing all channel information;

[0038] The convolution operation is applied to the fused feature map, and a convolution layer instance is created through the convolution layer class. The weights of each spatial position of the feature map are learned through the convolution operation to obtain a spatial weight map.

[0039] The spatial weight map is multiplied pixel by pixel with the original feature map in the spatial dimension, and the feature map weighted by the spatial attention mechanism is output.

[0040] Preferably, step S203 specifically includes:

[0041] In combination with the training set, the model is trained using the cross entropy loss function and the Adam optimizer, and iterative training is performed according to the set number of training times K;

[0042] In each iteration, the prediction result is obtained by forward propagation, the loss value is calculated according to the prediction result and the true label, the gradient of each parameter is calculated according to the loss value by using the back propagation algorithm, and the parameters of the model are updated according to the gradient by the Adam optimizer;

[0043] The trained DenseNet model is used to extract features from the test set, and the extracted features are input into a support vector machine. By adjusting the kernel function type and penalty parameter of the support vector machine and selecting the optimal classification hyperplane according to the principle of the support vector machine, the classification of the sea fog area, the building area, the vegetation area and the other areas is obtained.

[0044] Preferably, step S204 specifically includes:

[0045] According to the classification results, the classification accuracy is judged by the confusion matrix CM, the overall classification accuracy OA, the average classification accuracy AA and the statistical coefficient Kappa;

[0046] The confusion matrix CM is:

[0047]

[0048] The overall classification accuracy OA is:

[0049]

[0050] The average classification accuracy AA is:

[0051]

[0052] The statistical coefficient Kappa is:

[0053]

[0054] Among them, m i，j Indicates the number of objects in the i-th category that are misclassified into the j-th category; m i，i represents the number of objects in the i-th category correctly classified into the i-th category; n represents the number of categories of objects; N represents the total number of objects in all categories; N i represents the total number of objects in the i-th category; i = 1, 2, ..., n; j = 1, 2, ..., n;

[0055] The accuracy of classification is judged by analyzing the overall classification ability and the classification balance between categories;

[0056] If the overall classification ability and the classification balance between categories are qualified, it is judged that the classification accuracy meets the requirements;

[0057] If the overall classification capability and / or the classification balance between categories are not up to standard, the classification accuracy is judged to be unsatisfactory and the DenseNet model is improved;

[0058] The overall classification capabilities include:

[0059] If OA ≥ 0.8 and Kappa ≥ 0.7, the overall classification ability is qualified;

[0060] If OA < 0.8 and / or Kappa < 0.7, the overall classification ability is unsatisfactory;

[0061] Classification balance between categories includes:

[0062] Calculate the difference Q between the maximum and minimum number of correctly classified samples corresponding to each category on the diagonal of the confusion matrix CM, and set the difference range threshold to: [Q min , Q max ];

[0063] If AA ≥ 0.8 and Q = [Q min , Q max ], then the classification balance between categories is qualified;

[0064] If AA<0.8 and / or Q≠[Q min , Q max ], then the classification balance between categories is unqualified;

[0065] After the classification accuracy meets the requirements, draw the classification result map and compare it with the heat map of the actual label;

[0066] After the comparison is completed, the final recognition result is output, and the final recognition result includes the classification accuracy P F , the distribution rate of error pixels P E and the recognition accuracy rate P of the sea fog area S ;

[0067] Set the classification accuracy threshold to P 1 , the error pixel distribution rate threshold is P 2 , the recognition accuracy threshold of the sea fog area is P 3 ;

[0068] If P F ≥P 1 , P E ≤P 2 and P S ≥P 3If both meet the requirements, the DenseNet model is determined to meet the requirements and the DenseNet model is used as the final DenseNet model. If not, the DenseNet model is determined to meet the requirements and the DenseNet model is adjusted and optimized until P F ≥P 1 , P E ≤P 2 and P S ≥P 3 All satisfied.

[0069] Preferably, step S3 specifically includes:

[0070] A target sea fog image to be identified is obtained, and the target sea fog image is identified and classified by the final DenseNet model to obtain identification and classification results of the sea fog area, the building area, the vegetation area, and the other areas, and the identification and classification results are visualized.

[0071] The present disclosure also provides a sea fog hyperspectral remote sensing image recognition system based on DenseNet, comprising:

[0072] A hyperspectral dataset construction module is used to obtain hyperspectral image data and corresponding label data, perform data analysis and processing, and construct a hyperspectral dataset for training;

[0073] A model building module, used for building a DenseNet model, and training and optimizing the DenseNet model in combination with the hyperspectral dataset;

[0074] The sea fog image recognition module is used to obtain a target sea fog image to be recognized, and recognize and classify the target sea fog image according to the DenseNet model.

[0075] The advantages of the sea fog hyperspectral remote sensing image recognition method and system based on DenseNet described in the present disclosure are:

[0076] 1. A sea fog hyperspectral remote sensing image recognition method based on DenseNet disclosed in the present invention introduces an attention mechanism to make the model pay more attention to important areas in the image, and highlights the areas related to sea fog recognition by learning the weight of each spatial position in the feature map. This makes up for the defect of the DenseNet model in not grasping the global spatial features to a certain extent, so that the model can focus on the macro-level feature performance such as the distribution of the sea fog area in the entire image; in the feature extraction process, firstly, one-dimensional convolution is used to extract spectral features to obtain the correlation between each band, and then two-dimensional or three-dimensional convolution is used to extract spatial features. Integrating feature information from different dimensions is also helpful to capture some cross-regional global feature connections related to sea fog; in terms of spectral feature extraction, the use of One-dimensional convolution extracts the correlation between bands, which can mine the characteristic information hidden in many spectral bands, find the reflectivity changes in the sea fog area in specific bands and the key distinguishing basis such as the shape characteristics of the spectral curve. In terms of spatial feature extraction, two-dimensional or three-dimensional convolution layers are used to capture the spatial features such as texture and edge of the local area, and the attention mechanism is used to highlight the performance of these spatial features in the sea fog area, suppress irrelevant background information, and make the model more fully utilize the complex spatial structure. In addition, the spectral features and spatial features are integrated through the attention mechanism and the subsequent fully connected layer to achieve the deep fusion and coordinated utilization of the two. Compared with relying solely on the original feature extraction capabilities of DenseNet, this can more effectively mine and utilize these rich information to improve the classification accuracy. Therefore, the sea fog hyperspectral remote sensing image recognition method based on DenseNet has improved the problem of insufficient feature information captured by DenseNet to a certain extent, and better mine and utilize the spectral and spatial features of hyperspectral images to improve classification accuracy.

[0077] 2. A sea fog hyperspectral remote sensing image recognition system based on DenseNet disclosed in the present invention includes a hyperspectral data set construction module, which is used to obtain hyperspectral image data and corresponding label data, perform data analysis and processing, and construct a hyperspectral data set for training; a model construction module, which is used to construct a DenseNet model, and train and optimize the DenseNet model in combination with a hyperspectral data set; a sea fog image recognition module, which is used to obtain a target sea fog image to be identified, and identify and classify the target sea fog image according to the DenseNet model. The hyperspectral image data and the corresponding label data are obtained through the hyperspectral data set construction module, and data analysis and processing are performed, so that the quality of the hyperspectral data set used for training can be ensured; the model construction module focuses on the construction and training optimization of the DenseNet model, and can make targeted adjustments to the structure and parameters of the model according to the characteristics of the hyperspectral data set and the requirements of the sea fog recognition task; the sea fog image recognition module is specifically used to obtain the target sea fog image to be identified, and identify and classify it according to the trained DenseNet model, so that the system can well realize the specific application scenario of sea fog image recognition, and can seamlessly connect the model with the actual sea fog image recognition task, and is used to process target sea fog images from different data sources. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 It is a flow chart of a method for recognizing sea fog hyperspectral remote sensing images based on DenseNet described in the present disclosure;

[0079] Figure 2 It is a flowchart of the steps of a method for recognizing sea fog hyperspectral remote sensing images based on DenseNet described in the present disclosure;

[0080] Figure 3 It is an architecture diagram of the DenseNet model in the sea fog hyperspectral remote sensing image recognition method based on DenseNet described in the present invention. DETAILED DESCRIPTION

[0081] like Figure 1 - Figure 3 As shown, the sea fog hyperspectral remote sensing image recognition method based on DenseNet described in the present disclosure includes the following steps:

[0082] S1, obtain hyperspectral image data and corresponding label data, perform data analysis and processing, and construct a hyperspectral dataset for training;

[0083] S2, build a DenseNet model, and train and optimize the DenseNet model in combination with the hyperspectral dataset to obtain the final DenseNet model;

[0084] S3. Obtain the target sea fog image to be identified, identify and classify the target sea fog image through the final DenseNet model, and obtain the recognition and classification results of the target sea fog image.

[0085] Furthermore, in this embodiment, step S1 specifically includes:

[0086] Obtaining hyperspectral image data and corresponding label data, the hyperspectral image data includes image space dimensions and spectral band dimensions, and the labels of the label data include sea fog areas, building areas, vegetation areas and other areas;

[0087] Traverse the multidimensional array of hyperspectral image data to obtain pixel reflectance, use the visualization library to draw spectral curves according to the band number and reflectance, and obtain spectral feature information data of different regions based on the spectral curves;

[0088] According to the hyperspectral image data, the texture features and edge features of different regions are analyzed through the spatial dimension to obtain the spatial feature information data of different regions;

[0089] Convert the hyperspectral image data into digital matrix form and the label data into digital label format;

[0090] StandardScaler was used to normalize the hyperspectral image data, and the normalized hyperspectral image data was divided into a training set and a test set according to the ratio of A1:A2. The image data of the training set and the test set were paired and integrated with the corresponding label data to construct a hyperspectral dataset for training. The optimal value of A1 was 8, and the optimal value of A2 was 2.

[0091] Here is an example:

[0092] The dimensions of the acquired hyperspectral image data are (100, 100, 200), where the first two numbers 100 represent the number of rows and columns of the image in the spatial dimension (i.e., the image spatial dimension), and the last number 200 represents the spectral band dimension, that is, each pixel has reflectance information of 200 different bands;

[0093] The corresponding label data is a two-dimensional array of the same size (100,100), where each element corresponds to the region category to which the pixel at the corresponding position belongs. The region categories include sea fog region, building region, vegetation region, and other regions.

[0094] Through programming (such as Python), loop through the multidimensional array of hyperspectral image data to extract the reflectance value of each pixel in each spectral band. For example, for a pixel with coordinate position (30,40), its reflectance value in 200 bands can be obtained, such as [0.1, 0.2, 0.3, ..., 0.5];

[0095] Using a visualization library (such as the Matplotlib library), the spectral curve of each pixel is drawn according to the band number (from 1 to 200) as the horizontal axis and the reflectance value as the vertical axis. For example, the spectral curves drawn for multiple pixels in the sea fog area may show that their reflectance is generally low in certain band ranges (such as bands 50-70), and the curve shape shows a characteristic of a relatively gentle decline and then a slow rise; while the pixel spectral curve in the building area has an obvious peak between bands 80-100. Through such analysis and comparison, the spectral characteristic information data of different areas are obtained;

[0096] According to the hyperspectral image data, the texture features and edge features of different regions are analyzed through spatial dimensions. For example, the gray-level co-occurrence matrix (GLCM) method is used to calculate the texture features of different regions. For the sea fog area, the gray-level co-occurrence matrix element values ​​in different directions (horizontal, vertical, diagonal, etc.) are calculated, and it is found that the contrast is low and the entropy value is small, which means that the texture of the sea fog area is relatively uniform and smooth, while the contrast of the building area is high and the entropy value is large, reflecting that the building has more complex and changeable texture characteristics.

[0097] The edge features are obtained by edge detection operators (such as Sobel operator). At the boundary between the sea fog area and the surrounding area, the detected edge strength is weak and the coherence is poor, indicating that the boundary is fuzzy. However, at the boundary between the building area and other areas, the edge strength is high and relatively coherent, showing a clear geometric shape boundary, and the spatial feature information data of different areas are obtained.

[0098] The hyperspectral image data with the original dimension of (100, 100, 200) is converted into a digital matrix form. For example, the 200 band reflectance values ​​of each pixel are arranged in a certain order (such as row by row and column by column), and finally a two-dimensional matrix is ​​obtained, and the shape becomes (10000, 200), where 10000 = 100 * 100, which represents the total number of pixels;

[0099] Convert the label data into a digital label format, for example, "1" represents the sea fog area, "2" represents the building area, "3" represents the vegetation area, and "4" represents other areas;

[0100] StandardScaler is used to normalize the converted hyperspectral image data, mapping the reflectance value on each band to a specific interval (e.g., a standard normal distribution interval with a mean of 0 and a variance of 1), so that data in different bands have similar scales.

[0101] The normalized hyperspectral image data is divided into a training set and a test set according to the ratio of 8:2 (i.e., A1=8, A2=2). Then the training set data contains 8000 samples (spectral data corresponding to pixels), and the test set contains 2000 samples.

[0102] The image data of the training set and test set are paired and integrated with the corresponding label data. For example, each spectral data sample in the training set corresponds to the category label marked at its original position in the image, thus constructing a hyperspectral dataset for training.

[0103] Furthermore, in this embodiment, step S2 specifically includes the following steps:

[0104] S201, construct a convolutional neural network basic feature extractor;

[0105] S202. Build a DenseNet model and introduce the attention mechanism;

[0106] S203, training the DenseNet model by combining the training set, extracting features from the test set according to the trained DenseNet model and classifying the test set by a support vector machine to obtain a classification result;

[0107] S204, evaluating and analyzing the classification results to obtain evaluation results, and adjusting and optimizing the DenseNet model based on the evaluation results to obtain the final DenseNet model.

[0108] Furthermore, in this embodiment, step S201 specifically includes:

[0109] The hyperspectral data set is passed into the first convolution layer, and the convolution operation is performed by sliding the convolution kernel on the spectral band dimension of the hyperspectral image data to extract the correlation between the bands, capture the specific band reflectance characteristics and spectral curve shape characteristics related to the sea fog area, integrate the specific band reflectance characteristics and spectral curve shape characteristics into the required output feature map, and output the feature map;

[0110] After the first convolutional layer, add a 2D convolutional layer or a 3D convolutional layer.

[0111] If a two-dimensional convolutional layer is used, the convolution kernel slides in the local spatial area of ​​the image to capture the spatial texture and edge information in the local area of ​​the image, and outputs the feature map processed by the two-dimensional convolutional layer;

[0112] If a three-dimensional convolutional layer is used, the convolution kernel slides in the three-dimensional region including the spatial and spectral dimensions, and the complex local feature information is mined by integrating the spatial and spectral features, and the feature map processed by the three-dimensional convolutional layer is output;

[0113] A batch normalization layer is connected after the convolution layer to normalize the data of each channel of the feature map output by the convolution;

[0114] The ReLU activation function is used to perform nonlinear activation on the normalized feature map, and a maximum pooling layer is added to downsample the activated feature map, halving the spatial size of the feature map. The feature map is obtained after the collaborative processing of each convolutional layer and related layers, completing the preliminary feature extraction and data dimensionality reduction.

[0115] Here is an example:

[0116] The convolution kernel size of the first convolution layer is (1,3), where the first dimension 1 means that no convolution is performed on the spatial dimension, and only the spectral band dimension is operated. The second dimension 3 represents the number of spectral bands covered by the convolution kernel. The number of convolution kernels is 16, the step size is 1, and the padding is 0.

[0117] When the hyperspectral image data samples in the training set are passed into the first convolution layer, the convolution kernel slides on the spectral band dimension to perform convolution operations. For example, for a training sample with a shape of (100, 100, 200), the convolution kernel starts from the first band and moves one band position each time, performing convolution operations such as weighted summation with the corresponding three band data to extract the correlation between the bands.

[0118] After training and convolution operations on all samples in the test set, the reflectance characteristics of specific bands and the characteristic spectral information of the spectral curve shape related to the sea fog area are captured. For example, it is found that when the convolution kernel is convolved at the band 30-32 position, the output feature values ​​corresponding to the pixels in the sea fog area have obvious characteristic patterns (such as a specific numerical range or a numerical change trend that is significantly different from other areas), and after multiple such convolution kernel operations, these spectral features related to sea fog are integrated into the output feature map. At this time, the shape of the output feature map becomes (100, 100, 16), that is, the spatial dimension remains unchanged, and the number of channels becomes 16, which means that 16 different spectral feature maps are extracted, and then this feature map output is passed to the next layer;

[0119] Two-dimensional convolutional layer:

[0120] Choose to add a 2D convolution layer with a kernel size of (3,3), which means that convolution operations are performed in both the row and column dimensions of the space. The number of convolution kernels is 32, the stride is 1, and the padding is 1.

[0121] The convolution kernel of the two-dimensional convolution layer slides in the local spatial area of ​​the image, that is, the spatial dimension of (100,100);

[0122] For the feature map (100, 100, 16) output by the first convolution layer, the convolution kernel starts from the upper left corner of the image and moves one pixel at a time to perform convolution operations on the data of the corresponding channel in the 3×3 local spatial area. In this way, the spatial texture and edge information in the local area of ​​the image are captured. For example, in the sea fog area, after being processed by the two-dimensional convolution layer, the feature maps of some channels show relatively smooth texture features with slow grayscale changes (corresponding to texture information), as well as edge features with non-dramatic grayscale changes at the boundaries (corresponding to edge information). Finally, the feature map processed by the two-dimensional convolution layer is output (the shape becomes (100, 100, 32), and the number of channels increases to 32, indicating that richer spatial features are extracted);

[0123] Three-dimensional convolutional layer:

[0124] A three-dimensional convolutional layer is used, and its convolution kernel size is set to (3,3,3), which means that convolution operations are performed on the row, column and spectral dimensions of the space at the same time. The number of convolution kernels is 24, the step size is (1,1,1), and the padding is (1,1,1);

[0125] For the feature map (100, 100, 16) output by the first convolution layer, the 3D convolution kernel will slide in the 3D region containing the spatial and spectral dimensions. For example, a 3×3 local region in space is considered, and the convolution operation is performed together with the features of the three adjacent bands in the spectral dimension. In this way, the spatial and spectral features are integrated to mine complex local feature information. Taking the sea fog area as an example, it is found that at certain spatial positions, the features of several specific bands are combined to present a unique feature pattern that is different from other regions. Finally, the feature map processed by the 3D convolution layer is output (the shape becomes (100, 100, 24), and the number of channels becomes 24, reflecting the feature extraction result after integrating the spatial and spectral dimensions);

[0126] A batch normalization layer is connected after the convolution layer. Take the feature map of the two-dimensional convolution layer output (100, 100, 32) as an example: the batch normalization layer will normalize the data of each channel of the feature map output by the convolution. For each channel (32 channels), the mean and variance of the data of all samples in the channel (the feature map corresponding to the 8000 samples in the training set at this stage) are calculated, and the data of each sample in the channel is normalized according to the standard normal distribution with a mean of 0 and a variance of 1, making the data distribution more stable, which helps to speed up the model training and improve the generalization ability of the model.

[0127] After batch normalization, the shape of the feature map is still (100, 100, 32), but the distribution characteristics of each channel data have changed;

[0128] The ReLU activation function is used for nonlinear activation. For the feature map after batch normalization, take (100,100,32) as an example: the ReLU activation function will set the values ​​less than 0 on each pixel position and each channel in the feature map to 0, and retain the values ​​greater than or equal to 0. This introduces nonlinear factors and allows the model to learn more complex feature representations. For example, the original feature values ​​on a certain channel are [-0.2, 0.3, 0.5,...], which become [0, 0.3, 0.5,...] after ReLU activation;

[0129] Add a maximum pooling layer, assuming that the pooling window size is (2,2) and the step size is 2. Downsample the activated feature map, and select the maximum value in the window as the new feature value in the spatial dimension in units of 2×2 windows. Reduce the spatial size of the feature map by half (for example, from (100,100) to (50,50)). Finally, obtain the feature map after the progressive layer-by-layer convolution and the coordinated processing of related layers (the shape becomes (50,50,32)), completing the preliminary feature extraction and data dimensionality reduction.

[0130] Furthermore, in this embodiment, in step S202, constructing a DenseNet model specifically includes:

[0131] According to the feature map obtained by the convolutional neural network basic feature extractor through feature extraction and the number of categories to be distinguished, the DenseNet model class is defined. The DenseNet model class includes input dimension, growth rate, number of layers, and number of categories;

[0132] Add multiple Dense blocks to the DenseNet model initialization function. Each Dense block constructs a convolution layer, a batch normalization layer, a ReLu activation layer, and a maximum pooling layer in sequence.

[0133] Construct a classifier linear layer, flatten the feature map output by the last Dense block into a one-dimensional vector form, use the classifier linear layer class to create a linear layer instance according to the flattened vector dimension and the number of categories, set the input dimension to correspond to the flattened vector dimension, and the output dimension to the number of categories, and pass the flattened vector into the classifier linear layer, and output the predicted probability value corresponding to each category;

[0134] Here is an example:

[0135] After being processed by the previous convolutional neural network basic feature extractor, the resulting feature map has a shape of (50, 50, 32) (spatial dimension is 50×50, number of channels is 32), and the types of objects to be distinguished, that is, the number of categories, include sea fog areas, building areas, vegetation areas and other areas.

[0136] Define the DenseNet model class, set the input dimension to (50,50,32), the growth rate to 12, where the growth rate represents the number of channels added to the output feature map of each Dense block compared to the input feature map, the number of layers to 4 (i.e., it contains 4 Dense blocks), and the number of categories to 4, so as to construct the DenseNet model structure framework for the current hyperspectral image classification task;

[0137] Add the Dense block to the DenseNet model initialization function;

[0138] The first Dense block:

[0139] Convolutional layer: Construct a convolutional layer (nn.Conv2d) with the number of input channels equal to the number of channels of the current input feature map, 32, the number of output channels set to 12 according to the growth rate, the convolution kernel size set to (3,3), and the padding set to 1;

[0140] The convolution layer performs a convolution operation on the input feature map ((50,50,32)). For example, for each pixel position in the feature map, the convolution kernel performs weighted summation and other calculations on the data of the corresponding channel in the surrounding 3×3 local area according to the set weight parameters, and outputs a feature map with 12 channels and a shape of (50,50,12).

[0141] Batch normalization layer: add a batch normalization layer (nn.BatchNorm2d) to normalize the data of the 12 channels output by the convolution layer, calculate the mean and variance of each channel data respectively, and standardize the data to a distribution with a mean of 0 and a variance of 1, so that the data distribution is more conducive to subsequent training and calculation. After this layer, the shape of the feature map is still (50,50,12);

[0142] ReLU activation layer: Add a ReLU activation layer (nn.ReLU), use in-place activation, set the element values ​​less than 0 in the feature map after batch normalization to 0, introduce nonlinear factors, enhance the expressiveness of the model, and keep the shape of the feature map after processing (50,50,12) unchanged;

[0143] Max pooling layer: Add a max pooling layer (nn.MaxPool2d), set the pooling window size to (2,2), the step size to 2, and take the maximum value in the window as the new feature value in the spatial dimension of 2×2. Downsample the feature map so that the spatial size is halved (to (25,25)). The shape of the output feature map becomes (25,25,12). This completes the construction and processing of the first Dense block.

[0144] Subsequent Dense blocks, taking the second Dense block as an example:

[0145] Input feature changes: Due to the dense connection characteristics of DenseNet, the input feature map of the second Dense block contains the (25, 25, 12) feature map output by the previous Dense block, and connects the feature maps output by all layers of the previous Dense block, and concatenates them along the channel dimension. For example, the number of channels of the feature map output by the first Dense block is 12. After concatenation, the number of channels of the feature map input to the second Dense block becomes 12+12=24 (the number of channels is accumulated), and the shape becomes (25, 25, 24);

[0146] Convolutional layer: Construct a convolutional layer (nn.Conv2d) with 24 input channels (the number of channels after concatenation). The number of output channels increases by 12 to 24+12=36 (the number of input channels plus the growth rate). The kernel size is still (3,3), and the padding is set to 1. This convolutional layer performs a convolution operation on the concatenated feature map, and the output channel number becomes a feature map with 36 channels and a shape of (25,25,36).

[0147] Batch normalization layer, ReLU activation layer, and maximum pooling layer: Similar to the first Dense block, a batch normalization layer (nn.BatchNorm2d) is added in sequence for data normalization, a ReLU activation layer (nn.ReLU) introduces nonlinearity, and a maximum pooling layer (nn.MaxPool2d) performs downsampling. Finally, after being processed by the second Dense block, the shape of the feature map becomes (12, 12, 36), that is, the spatial dimension becomes 12×12 after pooling;

[0148] According to the above-mentioned method of adding the second Dense block, the remaining Dense blocks (a total of 4 Dense blocks in this embodiment) are constructed and processed in sequence. As the Dense blocks are continuously stacked, the number of channels of the feature map will continue to increase, and the spatial dimension will gradually decrease. At the same time, the features learned in the previous layers are integrated to achieve feature reuse and deep fusion;

[0149] After being processed by the last Dense block (for example, the shape of the final output feature map is (6,6,48)), a classifier linear layer is constructed to convert it into predicted probability values ​​for each category;

[0150] Flatten the feature map output by the last Dense block, that is, convert the three-dimensional feature map (6,6,48) into a one-dimensional vector form. By calculation, the vector length is 6×6×48=1728;

[0151] According to the flattened vector dimension of 1728 and the number of categories of 4 (corresponding to the 4 types of objects to be distinguished), the classifier linear layer class (nn.Linear) is used to create a linear layer instance;

[0152] Set the input dimension of the linear layer to 1728 (corresponding to the dimension of the flattened vector), the output dimension to 4 (the number of categories), and pass the flattened vector into the classifier linear layer;

[0153] For example, after the flattened vector is passed into the linear layer as input, the linear layer will perform matrix multiplication and other operations with the input vector based on its internally learned weight parameters (continuously adjusted and optimized through the training process), and finally output a vector of length 4, which corresponds to the predicted probability values ​​of the four categories of sea fog area, building area, vegetation area and other areas, indicating that the model believes that the current input hyperspectral image data belongs to each category. The higher the probability value, the more likely it is to belong to the category, thus completing the entire process from feature extraction to the final classification probability output based on the DenseNet model.

[0154] Furthermore, in this embodiment, in step S202, introducing the attention mechanism specifically includes:

[0155] Insert the channel attention mechanism and spatial attention mechanism into the output of each convolutional layer of the DenseNet model in turn to form a complete processing chain;

[0156] The channel attention mechanism is inserted into the DenseNet model including:

[0157] For the feature maps output by each convolutional layer in the DenseNet model, global average pooling and global maximum pooling are used to obtain the feature vectors representing the average eigenvalue and maximum eigenvalue of each channel;

[0158] The feature vectors representing the average eigenvalue and the maximum eigenvalue of each channel are concatenated and passed into a feedforward neural network consisting of two fully connected layers to generate a weight vector corresponding to each channel;

[0159] Multiply the generated weight vector by the original convolutional layer output feature map channel by channel in the channel dimension, adjust the characteristics of each channel according to the weight value, and output the feature map weighted by the channel attention mechanism;

[0160] The spatial attention mechanism is inserted into the DenseNet model including:

[0161] The feature maps processed by the channel attention mechanism are fused by splicing the channel dimension to obtain a fused feature map containing all channel information;

[0162] The convolution operation is applied to the fused feature map, and a convolution layer instance is created through the convolution layer class. The weights of each spatial position of the feature map are learned through the convolution operation to obtain a spatial weight map.

[0163] Multiply the spatial weight map with the original feature map pixel by pixel in the spatial dimension, and output the feature map weighted by the spatial attention mechanism;

[0164] By inserting the channel attention mechanism and the spatial attention mechanism into the output of each convolutional layer of the DenseNet model, the entire model can more intelligently focus on the feature information in the hyperspectral image that is critical to the classification task. Whether in the channel dimension or the spatial dimension, it can better mine and utilize features to accurately distinguish different types of objects such as sea fog areas, building areas, vegetation areas, and other areas.

[0165] Here is an example:

[0166] The channel attention mechanism is inserted into the DenseNet model as follows:

[0167] Take the feature map output by a convolutional layer in the DenseNet model as an example to explain the operation process of the channel attention mechanism in detail;

[0168] According to the previously constructed DenseNet model, the convolutional layer in the second Dense block outputs a feature map with a shape of (25, 25, 36), where 25×25 is the spatial dimension and 36 is the number of channels;

[0169] Global average pooling and global maximum pooling:

[0170] Perform a global average pooling operation on the feature map, perform an average calculation in the spatial dimension (i.e., 25×25), and compress the entire feature map into a value in each channel, which is equivalent to calculating the average value of all pixel values ​​in each channel. In this way, a feature vector of length 36 is obtained, and each element represents the average feature value of the corresponding channel. For example, after calculation, this feature vector is [0.2, 0.3, 0.15, ..., 0.25];

[0171] Perform a global maximum pooling operation on the feature map, also in the spatial dimension, taking the maximum value of all pixels on each channel to obtain a feature vector of length 36, representing the maximum eigenvalue of each channel. For example, the vector obtained is [0.5, 0.6, 0.4, ..., 0.7];

[0172] Feature vector concatenation and feedforward neural network processing:

[0173] The two eigenvectors representing the average eigenvalue and the maximum eigenvalue of each channel are concatenated along the dimension to obtain a new eigenvector with a length of 72 (because it is the concatenation of two vectors with a length of 36);

[0174] The concatenated feature vector is passed into a feedforward neural network containing two fully connected layers. The number of neurons in the first fully connected layer is set to 32 (the number of neurons here can be adjusted according to actual conditions), and the activation function uses the ReLU activation function to perform a nonlinear transformation on the input feature vector; then the number of neurons in the second fully connected layer is set to 36 (consistent with the number of channels in the original feature map), and the Sigmoid function is used as the activation function to limit the output value between [0,1], and generate a weight vector corresponding to each channel. For example, after being processed by this feedforward neural network, the obtained weight vector is [0.3, 0.6, 0.4, ..., 0.8], and each value corresponds to the weight of the corresponding channel in the original feature map;

[0175] Multiply channel by channel and output weighted feature map:

[0176] Multiply the generated weight vector by the original convolutional layer output feature map (25,25,36) channel by channel in the channel dimension. That is, for each channel of the feature map, the value of each pixel position is multiplied by the corresponding weight value. For example, if a pixel value of a channel in the original feature map is 0.5, the weight of the corresponding channel is 0.6, and after multiplication, it becomes 0.5×0.6=0.3;

[0177] The features of each channel are adjusted according to the weight value, and the feature map weighted by the channel attention mechanism is output. Its shape is still (25, 25, 36), but the features of each channel have been enhanced or suppressed according to the weight, so that the model can pay more attention to the channel features that are more helpful for the classification task. For example, those channel features that are more important for distinguishing sea fog areas will be amplified, while relatively unimportant channel features will be weakened;

[0178] After each convolutional layer output of the DenseNet model, the channel attention mechanism is inserted according to the above steps to form a complete processing chain, so that the feature map output by each layer undergoes such channel feature weight adjustment, continuously optimizes feature representation, and improves the model's ability to distinguish different types of objects (sea fog area, building area, vegetation area and other areas);

[0179] The spatial attention mechanism is inserted into the DenseNet model as follows:

[0180] Feature map fusion:

[0181] After being processed by the channel attention mechanism, each convolutional layer will output the corresponding weighted feature map (taking the convolutional layer in the second Dense block above as an example, it is a weighted feature map with a shape of (25, 25, 36));

[0182] The feature maps processed by the channel attention mechanism are fused by splicing in the channel dimension, and the information of all channels is integrated together to obtain a fused feature map containing all channel information. For example, the weighted feature maps output by the three convolutional layers are (25, 25, 36), (25, 25, 48), and (25, 25, 60). By splicing in the channel dimension, the shape of the final fused feature map becomes (25, 25, 36 + 48 + 60 = 144);

[0183] Convolution operations learn spatial weights:

[0184] The convolution operation is used on the fused feature map to learn the weights of each spatial position of the feature map. A convolution layer instance is created through the convolution layer class. For example, the convolution kernel size is (3,3), the number of convolution kernels is 1, the step size is 1, and the padding is 1. The convolution layer slides on the spatial dimension (25×25) of the fused feature map to perform convolution operations. By learning information such as the feature combination relationship at different spatial positions in the feature map, a spatial weight map is obtained. For example, the shape of the spatial weight map obtained after the convolution operation is also (25,25). The value of each pixel position represents the weight of the corresponding spatial position, and the value range is between [0,1]. For example, a position in the spatial weight map has a value of 0.8, which means that the position will be the focus of subsequent feature weighting, while the position with a value of 0.2 is relatively unimportant.

[0185] Pixel-by-pixel multiplication and output weighted feature map:

[0186] The obtained spatial weight map is multiplied pixel by pixel with the feature map (taking the feature map of (25,25,36) as an example) after the channel attention mechanism is processed and before the fusion operation is performed. That is, for each pixel position of the feature map, the values ​​on each channel are multiplied by the corresponding spatial weight value. For example, the channel values ​​of a pixel position in the feature map are [0.3,0.4,0.5], and the weight value of the corresponding spatial position is 0.6. After multiplication, it becomes [0.3×0.6,0.4×0.6,0.5×0.6]=[0.18,0.24,0.3]. After pixel-by-pixel multiplication, the feature map weighted by the spatial attention mechanism is output, and its shape is still (25,25,36), but the spatial features in the feature map have been adjusted according to the weights, so that the model focuses more on the important spatial areas in the image related to classification tasks such as sea fog area recognition, suppresses interference information such as less important background areas, and further improves the classification performance of the model;

[0187] The spatial attention mechanism is inserted in sequence according to the above process after the output of each convolutional layer of the DenseNet model, and cooperates with the channel attention mechanism to process the feature map output by each layer. The feature map after such layer-by-layer processing will carry information that is more conducive to classification, and will be subsequently passed to the next layer of the DenseNet model or the final classifier linear layer for further feature fusion, classification and other operations, thereby improving the classification accuracy of the entire model for different areas in hyperspectral remote sensing images.

[0188] Furthermore, in this embodiment, step S203 specifically includes:

[0189] Combined with the training set, the cross entropy loss function and Adam optimizer are used for model training, and iterative training is performed according to the set number of training times K;

[0190] In each iteration, the prediction result is obtained by forward propagation, the loss value is calculated according to the prediction result and the true label, the gradient of each parameter is calculated according to the loss value by back propagation algorithm, and the parameters of the model are updated according to the gradient by Adam optimizer;

[0191] The trained DenseNet model is used to extract features from the test set, and the extracted features are input into the support vector machine. By adjusting the kernel function type and penalty parameter of the support vector machine and selecting the optimal classification hyperplane according to the principle of the support vector machine, the classification of sea fog area, building area, vegetation area and other areas is obtained.

[0192] Here is an example:

[0193] Model training includes:

[0194] The training times K is set to 100 times. The training set contains 8000 hyperspectral image data samples and corresponding category labels. The category labels are "1" for sea fog areas, "2" for building areas, "3" for vegetation areas, and "4" for other areas.

[0195] Forward propagation and loss calculation, taking the first iteration as an example:

[0196] Select a sample in the training set. For example, the original hyperspectral image data dimension of the sample is (100, 100, 200). After the above series of preprocessing and feature extraction steps, it is converted into a feature map of (50, 50, 32) to meet the input dimension requirements of the constructed DenseNet model. It is input into the DenseNet model to start forward propagation.

[0197] The sample data passes through each Dense block in the DenseNet model in turn (each Dense block contains convolution layer, batch normalization layer, ReLU activation layer and maximum pooling layer operations), and after each convolution layer output, it will also be inserted into the channel attention mechanism and spatial attention mechanism to optimize the features, continuously integrate and refine the feature information, and finally output the prediction probability value corresponding to each category through the classifier linear layer. For example, for this sample, the output prediction probability value is [0.2, 0.3, 0.4, 0.1], and each value corresponds to the prediction probability of category "1" sea fog area, "2" building area, "3" vegetation area and "4" other areas;

[0198] The loss value is calculated based on the prediction result of the sample and the corresponding true label. For example, the true label of the sample is "3", which is the vegetation area. The cross entropy loss function is used here. The calculation formula of the cross entropy loss function is:

[0199]

[0200] Where C represents the number of categories, C = 4); y i represents the one-hot encoding form corresponding to the true label (for the true label "3", its one-hot encoding is [0,0,1,0]); P i represents the predicted probability value, i.e., [0.2, 0.3, 0.4, 0.1] obtained above;

[0201] The loss value of this sample is calculated according to the formula: -[0*log(0.2)+0*log(0.3)+1*log(0.4)+0*log(0.1)]=0.398;

[0202] For all 8000 samples in the training set, forward propagation and loss calculation are performed according to this process, and the average loss is calculated to obtain the average loss value corresponding to the entire training set in this iteration, so as to measure the performance of the current model on this batch of training data;

[0203] Back propagation and parameter update:

[0204] The back propagation algorithm is used to calculate the gradients of various parameters (including the weights and biases of all convolutional layers in the DenseNet model, the weights and biases of the fully connected layers, and the various parameters involved in the attention mechanism) based on the average loss value just calculated;

[0205] Back propagation starts from the last position where the loss is calculated. According to the chain rule, the influence of each parameter on the loss value is calculated layer by layer along the network structure in reverse, that is, the gradient. For example, for a weight parameter in a convolutional layer in the DenseNet model, after back propagation calculation, its corresponding gradient value is 0.03.

[0206] The Adam optimizer is used to update the model parameters according to the calculated gradient. The Adam optimizer comprehensively considers the first-order moment estimate (similar to the average value of the gradient) and the second-order moment estimate (similar to the average value of the square of the gradient) of the gradient, as well as some other hyperparameters (for example, the learning rate is set to 0.001), and then adjusts the update step size of each parameter to promote the parameter to be updated in the direction that can reduce the loss value. For example, the convolution layer weight parameter with a gradient value of 0.03 mentioned above will be updated to a new value after the Adam optimizer follows its internal calculation rules and adjustment logic. The above completes the parameter update process;

[0207] In each iteration, the above forward propagation, loss calculation, back propagation and parameter update operations are repeated. As the number of iterations increases, the parameters of the model are continuously adjusted and optimized. Under normal circumstances, the classification accuracy of the model for the training set data will gradually improve.

[0208] After 100 iterations of training, the training process of the DenseNet model based on the training set is completed;

[0209] Using the trained model to extract features and classify using support vector machines includes:

[0210] The test set (including the corresponding number of hyperspectral image data samples) is input into the trained DenseNet model, and the forward propagation operation is also performed. However, the focus is not on the final classification probability output, but on the features extracted in the middle layer of the model. For example, after the DenseNet model has been processed by several specific Dense blocks and the attention mechanism, a feature map with a shape of (10,10,64) is output at a certain middle layer position. This feature map is flattened to convert it into a one-dimensional vector (the calculated vector length is 10×10×64=6400);

[0211] In this way, the corresponding feature vector is extracted for each test set sample, and the feature vector set corresponding to all samples in the test set is obtained. For example, if the number of test set samples is 2000, then 2000 feature vectors with a dimension of 6400 will be obtained. These feature vectors contain the representative feature information learned and extracted by the DenseNet model from the hyperspectral image data, and then they are input into the support vector machine for classification operation;

[0212] Select an appropriate kernel function type. Common kernel functions include linear kernel function, polynomial kernel function and Gaussian radial basis kernel function (RBF). For example, select a linear kernel function, provide the extracted 2000 feature vectors and their corresponding true category labels ("1", "2", "3" and "4" represent different area categories) as input data to the support vector machine model, and set an initial penalty parameter C (for example, the penalty parameter C is set to 1.0. The penalty parameter is mainly used to control the tolerance for classification errors. Its value will affect the complexity and generalization ability of the support vector machine model);

[0213] The support vector machine starts training based on these data and set parameters to obtain an optimal classification hyperplane, so that samples of different categories (sea fog area, building area, vegetation area and other areas) can be accurately distinguished in the feature space;

[0214] Evaluation indicators (such as accuracy and recall) are used to evaluate the classification effect of the current support vector machine model based on the linear kernel function and with a penalty parameter of C = 1.0 on the test set;

[0215] If the classification effect is not ideal, adjust the kernel function type (for example, change to RBF kernel function), and adjust the value of penalty parameter C (for example, change the value of penalty parameter C to 0.5 or 2.0, etc.), then carry out support vector machine training and classification effect evaluation again, and try different kernel function and penalty parameter combinations repeatedly until the optimal parameter combination and corresponding classification hyperplane that enable the support vector machine to achieve good classification accuracy on the test set are found. For example, after many attempts, it is found that when the RBF kernel function is used and the penalty parameter C is set to 1.5, the support vector machine has an accuracy of 85% in classifying the test set samples. At this time, the optimal classification hyperplane of the support vector machine and the corresponding classification results based on the current DenseNet model feature extraction are determined, thereby achieving the classification tasks of sea fog areas, building areas, vegetation areas and other areas.

[0216] Furthermore, in this embodiment, step S204 specifically includes:

[0217] According to the classification results, the classification accuracy is judged by the confusion matrix CM, the overall classification accuracy OA, the average classification accuracy AA and the statistical coefficient Kappa;

[0218] The confusion matrix CM is:

[0219]

[0220] The overall classification accuracy OA is:

[0221]

[0222] The average classification accuracy AA is:

[0223]

[0224] The statistical coefficient Kappa is:

[0225]

[0226] Among them, m i，j Indicates the number of objects in the i-th category that are misclassified into the j-th category; m i，i represents the number of objects in the i-th category correctly classified into the i-th category; n represents the number of categories of objects; N represents the total number of objects in all categories; N i represents the total number of objects in the i-th category; i = 1, 2, ..., n; j = 1, 2, ..., n;

[0227] The accuracy of classification is judged by analyzing the overall classification ability and the classification balance between categories;

[0228] If the overall classification ability and the classification balance between categories are qualified, it is judged that the classification accuracy meets the requirements;

[0229] If the overall classification capability and / or the classification balance between categories are not up to standard, the classification accuracy is judged to be unsatisfactory and the DenseNet model is improved;

[0230] The overall classification capabilities include:

[0231] If OA ≥ 0.8 and Kappa ≥ 0.7, the overall classification ability is qualified;

[0232] If OA < 0.8 and / or Kappa < 0.7, the overall classification ability is unsatisfactory;

[0233] Classification balance between categories includes:

[0234] Calculate the difference Q between the maximum and minimum number of correctly classified samples corresponding to each category on the diagonal of the confusion matrix CM, and set the difference range threshold to: [Q min , Q max ];

[0235] If AA ≥ 0.8 and Q = [Q min , Q max ], then the classification balance between categories is qualified;

[0236] If AA<0.8 and / or Q≠[Q min , Q max ], then the classification balance between categories is unqualified;

[0237] After the comparison is completed, the final recognition result is output, which includes the classification accuracy P F , the distribution rate of error pixels P E And the recognition accuracy of sea fog area P S ;

[0238] Set the classification accuracy threshold to P 1 , the error pixel distribution rate threshold is P 2 , the recognition accuracy threshold of the sea fog area is P 3 ;

[0239] If P F ≥P 1 , P E ≤P 2 and P S ≥P 3 If all meet the requirements, the DenseNet model is judged to meet the requirements and the DenseNet model is used as the final DenseNet model. If not, the DenseNet model is judged to meet the requirements and the DenseNet model is adjusted and optimized until P F ≥P 1 , P E ≤P 2 and P S ≥P 3 All are satisfied;

[0240] Here is an example:

[0241] n=4,N=2000,N 1 =500, N 2 =500, N 3 =500, N 4 =500; where N=2000 is the total number of samples in the test set;

[0242] i = 1 indicates category 1 (sea fog area), i = 2 indicates category 2 (building area), i = 3 indicates category 3 (vegetation area), and i = 4 indicates category 4 (other areas);

[0243] j=1 means classification into the first category, j=2 means classification into the second category, j=3 means classification into the third category, and j=4 means classification into the fourth category;

[0244]

[0245] In this matrix, m 1，1 =350 indicates the number of areas that actually belong to the sea fog area (Class 1) and are correctly classified as the sea fog area; m 1，2=30 means the number of samples that were originally in the sea fog area but were misclassified as the building area (Class 2), and so on;

[0246]

[0247] Q min =50, Q max =150;

[0248] The difference between the maximum and minimum number of correctly classified samples corresponding to each category on the diagonal of the confusion matrix CM is Q = 480-350 = 130;

[0249] Therefore, OA>0.8 and Kappa>0.7, the overall classification ability is qualified;

[0250] AA>0.8 and Q=[50,150], the classification balance between categories is qualified;

[0251] Therefore, the overall classification ability and the classification balance between categories are qualified, and the classification accuracy meets the requirements;

[0252] If the classification accuracy meets the requirements, the classification result graph is drawn and compared with the heat map of the actual label. After the comparison is completed, the final recognition result is output to obtain the classification accuracy P F =90%, distribution rate of error pixels P E =5% and the recognition accuracy of the sea fog area P S =85%;

[0253] Set the classification accuracy threshold to P 1 =80%, the error pixel distribution rate threshold is P 2 =10%, the recognition accuracy threshold of the sea fog area is P 3 =80%;

[0254] Then P F >80%、P E <10% and P S >80%, so P F ≥P 1 , P E ≤P 2 and P S ≥P 3 If all of them are satisfied, the DenseNet model is judged to meet the requirements and the DenseNet model is used as the final DenseNet model.

[0255] Furthermore, in this embodiment, step S3 also includes:

[0256] Obtain the target sea fog image to be identified, identify and classify the target sea fog image through the final DenseNet model, obtain the identification and classification results of the sea fog area, building area, vegetation area and other areas, and visualize the identification and classification results;

[0257] Visual display can be presented in the form of charts or statistical graphs, so that the identification results of sea fog areas, building areas, vegetation areas and other areas can be clearly determined;

[0258] Here is an example:

[0259] Take an image of a coastal area acquired by a hyperspectral remote sensing device as an example of a target sea fog image;

[0260] Obtain the target sea fog image to be identified, the spatial dimension of the image is 500 pixels long and 600 pixels wide, and has 200 spectral bands;

[0261] Preprocess the target sea fog image, normalize the 200 spectral band reflectance data of each pixel, and adjust the dimensional information of the image;

[0262] The processed target sea fog image is input into the final DenseNet model. The target sea fog image data starts forward propagation inside the model, and passes through the convolution layer, batch normalization layer, ReLU activation layer and maximum pooling layer in each Dense block for feature extraction and feature optimization. During this period, the channel attention mechanism will highlight the feature channels that are important for classification, and the spatial attention mechanism will focus on the key spatial positions, allowing the model to capture and use features more accurately.

[0263] After a series of model processing, the prediction probability values ​​corresponding to the four categories of sea fog area, building area, vegetation area and other areas are output for each pixel in the image. For example, the pixel with coordinates (200,300) in the image has the prediction probabilities of [0.4, 0.25, 0.2, 0.15] after model operation. According to the maximum probability principle, the pixel is judged to belong to the sea fog area. In this way, every pixel of the entire image is judged, and finally the recognition and classification results of the category to which each pixel of the entire image belongs are obtained;

[0264] In order to intuitively present the classification results, different colors are used to represent different area categories for visualization. Light blue represents sea fog areas, dark gray represents building areas, bright green represents vegetation areas, and light yellow represents other areas.

[0265] Through visualization, we can clearly see the distribution of regions divided by different colors in the image. For example, the large area near the sea is light blue, indicating that the model recognizes that this part is covered by sea fog. Dark gray blocks can be seen along the coastline, corresponding to the distribution of buildings along the coast. There are many bright green patches scattered on the land, representing the location of vegetation. There are also some light yellow areas mixed in, which are other landforms.

[0266] Through the above visualization, we can intuitively understand the classification and recognition effect of the DenseNet model on this target sea fog image.

[0267] The disclosed embodiment also provides a sea fog hyperspectral remote sensing image recognition system based on DenseNet, including:

[0268] A hyperspectral dataset construction module is used to obtain hyperspectral image data and corresponding label data, perform data analysis and processing, and construct a hyperspectral dataset for training;

[0269] Model building module, used to build DenseNet model and train and optimize the DenseNet model in combination with hyperspectral dataset;

[0270] The sea fog image recognition module is used to obtain the target sea fog image to be identified, and identify and classify the target sea fog image according to the DenseNet model.

[0271] In the description of the present disclosure, it is necessary to understand that the directions or positional relationships indicated by directional words such as "front, back, up, down, left, right", "lateral, vertical, horizontal" and "top, bottom" are usually based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present disclosure and simplifying the description. Unless otherwise specified, these directional words do not indicate or imply that the device or element referred to must have a specific orientation or be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the scope of protection of the present disclosure.

[0272] For those skilled in the art, various other corresponding changes and deformations can be made according to the technical solutions and concepts described above, and all of these changes and deformations should fall within the protection scope of the claims of the present disclosure.

Claims

1. A sea fog hyperspectral remote sensing image recognition method based on DenseNet, characterized in that: The following steps are involved: S1, obtain hyperspectral image data and corresponding label data, perform data analysis and processing, and construct a hyperspectral dataset for training; S2, constructing a DenseNet model, and training and optimizing the DenseNet model in combination with the hyperspectral dataset to obtain a final DenseNet model; S3. Obtain a target sea fog image to be identified, identify and classify the target sea fog image through the final DenseNet model, and obtain an identification and classification result of the target sea fog image.

2. The sea fog hyperspectral remote sensing image recognition method based on DenseNet according to claim 1, characterized in that: Step S1 specifically includes: Acquire hyperspectral image data and corresponding label data, wherein the hyperspectral image data includes image space dimensions and spectral band dimensions, and labels of the label data include sea fog areas, building areas, vegetation areas, and other areas; Traversing the multidimensional array of the hyperspectral image data to obtain pixel reflectance, using a visualization library to draw a spectral curve according to the band number and reflectance, and obtaining spectral feature information data of different regions according to the spectral curve; According to the hyperspectral image data, the texture features and edge features of different regions are analyzed by spatial dimension to obtain spatial feature information data of different regions; Converting the hyperspectral image data into a digital matrix format and converting the label data into a digital label format; StandardScaler is used to normalize the hyperspectral image data, and the normalized hyperspectral image data is divided into a training set and a test set according to the ratio of A1:A2. The image data of the training set and the test set are paired and integrated with the corresponding label data to construct a hyperspectral dataset for training.

3. The sea fog hyperspectral remote sensing image recognition method based on DenseNet according to claim 2 is characterized in that: Step S2 specifically includes the following steps: S201, construct a convolutional neural network basic feature extractor; S202. Build a DenseNet model and introduce the attention mechanism; S203, training the DenseNet model in combination with the training set, extracting features from the test set according to the trained DenseNet model and classifying the test set using a support vector machine to obtain a classification result; S204, evaluating and analyzing the classification results to obtain evaluation results, and adjusting and optimizing the DenseNet model based on the evaluation results to obtain a final DenseNet model.

4. The sea fog hyperspectral remote sensing image recognition method based on DenseNet according to claim 3 is characterized in that: Step S201 specifically includes: The hyperspectral data set is passed into the first convolution layer, and a convolution operation is performed by sliding a convolution kernel on the spectral band dimension of the hyperspectral image data to extract the correlation between the bands, and the specific band reflectance characteristics and the spectral curve shape characteristic spectral information related to the sea fog area are captured, and the specific band reflectance characteristics and the spectral curve shape characteristics are integrated into the desired output feature map, and the feature map is output; After the first convolutional layer, add a 2D convolutional layer or a 3D convolutional layer. If a two-dimensional convolutional layer is used, the convolution kernel is slid in the local spatial area of ​​the feature map to capture the spatial texture and edge information in the local area of ​​the image, and the feature map processed by the two-dimensional convolutional layer is output; If a three-dimensional convolutional layer is used, the convolution kernel slides in the three-dimensional region including the spatial and spectral dimensions, and the complex local feature information is mined by integrating the spatial and spectral features, and the feature map processed by the three-dimensional convolutional layer is output; A batch normalization layer is connected after the convolution layer to normalize the data of each channel of the feature map output by the convolution; The ReLU activation function is used to perform nonlinear activation on the normalized feature map, and a maximum pooling layer is added to downsample the activated feature map to halve the spatial size of the feature map. The feature map is obtained after collaborative processing by each convolutional layer and related layers, completing preliminary feature extraction and data dimensionality reduction.

5. The sea fog hyperspectral remote sensing image recognition method based on DenseNet according to claim 4 is characterized in that: In step S202, constructing a DenseNet model specifically includes: According to the feature map obtained by the convolutional neural network basic feature extractor through feature extraction and the number of classifications required to be distinguished, a DenseNet model class is defined, wherein the DenseNet model class includes an input dimension, a growth rate, a number of layers, and a number of classifications; Add multiple Dense blocks in the DenseNet model initialization function, each of which constructs a convolutional layer, a batch normalization layer, a ReLu activation layer, and a maximum pooling layer in sequence; Construct a classifier linear layer, flatten the feature map output by the last Dense block and convert it into a one-dimensional vector form, use the classifier linear layer class to create a linear layer instance according to the dimension of the flattened vector and the number of categories, set the input dimension to correspond to the dimension of the flattened vector, and the output dimension to the number of categories, and pass the flattened vector into the classifier linear layer, and output the predicted probability value corresponding to each category.

6. The sea fog hyperspectral remote sensing image recognition method based on DenseNet according to claim 5, characterized in that: In step S202, introducing the attention mechanism specifically includes: Insert the channel attention mechanism and spatial attention mechanism into the output of each convolutional layer of the DenseNet model in turn to form a complete processing chain; The channel attention mechanism inserted into the DenseNet model includes: For the feature maps output by each convolutional layer in the DenseNet model, global average pooling and global maximum pooling are used to obtain the feature vectors representing the average eigenvalue and maximum eigenvalue of each channel; The feature vectors representing the average eigenvalue and the maximum eigenvalue of each channel are concatenated and passed into a feedforward neural network consisting of two fully connected layers to generate a weight vector corresponding to each channel; Multiply the generated weight vector by the original convolutional layer output feature map channel by channel in the channel dimension, adjust the characteristics of each channel according to the weight value, and output the feature map weighted by the channel attention mechanism; The spatial attention mechanism is inserted into the DenseNet model including: The feature maps processed by the channel attention mechanism are fused by splicing the channel dimensions to obtain a fused feature map containing all channel information; The convolution operation is applied to the fused feature map, and a convolution layer instance is created through the convolution layer class. The weights of each spatial position of the feature map are learned through the convolution operation to obtain a spatial weight map. The spatial weight map is multiplied pixel by pixel with the original feature map in the spatial dimension, and the feature map weighted by the spatial attention mechanism is output.

7. The sea fog hyperspectral remote sensing image recognition method based on DenseNet according to claim 6, characterized in that: Step S203 specifically includes: In combination with the training set, the model is trained using the cross entropy loss function and the Adam optimizer, and iterative training is performed according to the set number of training times K; In each iteration, the prediction result is obtained by forward propagation, the loss value is calculated according to the prediction result and the true label, the gradient of each parameter is calculated according to the loss value by using the back propagation algorithm, and the parameters of the model are updated according to the gradient by the Adam optimizer; The trained DenseNet model is used to extract features from the test set, and the extracted features are input into a support vector machine. By adjusting the kernel function type and penalty parameter of the support vector machine and selecting the optimal classification hyperplane according to the principle of the support vector machine, the classification of the sea fog area, the building area, the vegetation area and the other areas is obtained.

8. The sea fog hyperspectral remote sensing image recognition method based on DenseNet according to claim 7, characterized in that: Step S204 specifically includes: According to the classification results, the classification accuracy is judged by the confusion matrix CM, the overall classification accuracy OA, the average classification accuracy AA and the statistical coefficient Kappa; The confusion matrix CM is: The overall classification accuracy OA is: The average classification accuracy AA is: The statistical coefficient Kappa is: Among them, m i，j Indicates the number of objects in the i-th category that are misclassified into the j-th category; m i，i represents the number of objects in the i-th category correctly classified into the i-th category; n represents the number of categories of objects; N represents the total number of objects in all categories; N i represents the total number of objects in the i-th category; i = 1, 2, ..., n; j = 1, 2, ..., n; The accuracy of classification is judged by analyzing the overall classification ability and the classification balance between categories; If the overall classification ability and the classification balance between categories are qualified, it is judged that the classification accuracy meets the requirements; If the overall classification capability and / or the classification balance between categories are not up to standard, it is judged that the classification accuracy does not meet the requirements, and the DenseNet model is improved; The overall classification capabilities include: If OA ≥ 0.8 and Kappa ≥ 0.7, the overall classification ability is qualified; If OA < 0.8 and / or Kappa < 0.7, the overall classification ability is unsatisfactory; Classification balance between categories includes: Calculate the difference Q between the maximum and minimum number of correctly classified samples corresponding to each category on the diagonal of the confusion matrix CM, and set the difference range threshold to: [Q min , Q max ]; If AA ≥ 0.8 and Q = [Q min , Q max ], then the classification balance between categories is qualified; If AA<0.8 and / or Q≠[Q min , Q max ], then the classification balance between categories is unqualified; After the classification accuracy meets the requirements, draw the classification result map and compare it with the heat map of the actual label; After the comparison is completed, the final recognition result is output, and the final recognition result includes the classification accuracy P F , the distribution rate of error pixels P E and the recognition accuracy rate P of the sea fog area S ; The classification accuracy threshold is set to P1, the error pixel distribution threshold is set to P2, and the recognition accuracy threshold of the sea fog area is set to P3; If P F ≥P1, P E ≤P2 and P S ≥P3 are satisfied, the DenseNet model is determined to meet the requirements and the DenseNet model is used as the final DenseNet model. If not, the DenseNet model is determined to meet the requirements and the DenseNet model is adjusted and optimized until P F ≥P1, P E ≤P2 and P S ≥P3 are all satisfied.

9. The sea fog hyperspectral remote sensing image recognition method based on DenseNet according to claim 8, characterized in that: Step S3 specifically includes: A target sea fog image to be identified is obtained, and the target sea fog image is identified and classified by the final DenseNet model to obtain identification and classification results of the sea fog area, the building area, the vegetation area, and the other areas, and the identification and classification results are visualized.

10. A sea fog hyperspectral remote sensing image recognition system based on DenseNet, characterized in that: include: A hyperspectral dataset construction module is used to obtain hyperspectral image data and corresponding label data, perform data analysis and processing, and construct a hyperspectral dataset for training; A model building module, used for building a DenseNet model, and training and optimizing the DenseNet model in combination with the hyperspectral dataset; The sea fog image recognition module is used to obtain a target sea fog image to be recognized, and recognize and classify the target sea fog image according to the DenseNet model.

Images