Comprehensive evaluation method for land classification, electronic equipment and storage medium

Abstract

A comprehensive land classification evaluation method, electronic device, and storage medium, belonging to the technical field of remote sensing image processing and evaluation. To address the accuracy evaluation problem after accurate classification of similar categories and category boundaries in land use classification methods, the present invention includes constructing a multi-scale normalized spatial attention mechanism layer; constructing a multi-scale normalized directional position attention mechanism layer; constructing a multi-scale normalized channel attention mechanism layer; and constructing a land use classification network model, including a normalized multi-attention feature extraction network and a multi-deep and shallow feature fusion void convolution network. Data from an urban-rural domain adaptive land cover dataset is collected and preprocessed to obtain a land dataset, which is then input into the constructed land use classification network model for land classification training. A land classification model based on deep and shallow features and multi-attention mechanism fusion is obtained. The predicted classification categories of the prediction set and the actual sample label classification categories are used to construct a confusion matrix for accuracy evaluation.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing image processing and evaluation, and in particular relates to a comprehensive evaluation method for land classification, electronic equipment and storage medium. Background Art

[0002] Land use can be understood as a series of products created by the interaction of human activities with land. It can also be understood as the exchange and conversion of matter, energy, value, and information generated by the interaction between the Earth's surface and the development of social civilization. Currently, land use is widely used in urban management, ecological protection, agricultural supervision, and other fields, such as urban regional planning, illegal construction monitoring, ecological pattern assessment, environmentally vulnerable area assessment, and non-agricultural and non-grain production supervision. To this day, how to efficiently, accurately, frequently, and cost-effectively utilize remote sensing imagery to obtain land use classification results remains a hotly debated research issue.

[0003] Satellite remote sensing imagery, with its wide coverage, short replay cycles, and historical traceability, provides effective data support for land use and change monitoring. Machine learning methods based on supervised classification, object-oriented learning, artificial neural networks, and support vector machines have been widely and early applied to land use. Key techniques include image semantic segmentation based on thresholds, edges, regions, and specific theories. However, these methods often utilize relatively shallow semantic information for image segmentation, failing to extract deeper contextual information about similar categories. This makes it difficult to account for all land types in multi-class classification. Segmentation for multiple categories, similar categories, and category boundaries remains a significant challenge, and there is no comprehensive evaluation method for post-classification results. Summary of the Invention

[0004] The problem to be solved by the present invention is to solve the problem of accuracy evaluation after accurate classification of similar categories and category boundaries in the land utilization classification method, and propose a comprehensive evaluation method for land classification, electronic equipment and storage medium.

[0005] To achieve the above object, the present invention is implemented through the following technical solutions:

[0006] A comprehensive evaluation method for land classification includes the following steps:

[0007] S1. Construct a multi-scale normalized spatial attention mechanism layer MANSAM;

[0008] S2. Construct a multi-scale normalized direction position attention mechanism layer MANDPAM;

[0009] S3, construct a multi-scale normalized channel attention mechanism layer MANCAM;

[0010] S4. Construct a land use classification network model, including a normalized multi-attention feature extraction network NSCPANet and a multi-deep and shallow feature fusion dilated convolutional network MNSDDNet. The normalized multi-attention feature extraction network NSCPANet includes a shallow feature extraction network and a deep feature extraction network. The multi-deep and shallow feature fusion dilated convolutional network MNSDDNet uses a deep and shallow feature fusion method SDFF to fuse the features of the shallow feature extraction network and the deep feature extraction network.

[0011] S5. Collect data from the urban-rural domain adaptive land cover dataset, perform data preprocessing, obtain a land dataset, and divide it into a land dataset prediction set and a land dataset training set;

[0012] S6. Input the land dataset training set obtained in step S5 into the land use classification network model constructed in step S4, perform land classification training based on the fusion of deep and shallow features and multi-attention mechanism, obtain a land classification model based on the fusion of deep and shallow features and multi-attention mechanism, and then perform sample prediction on the land dataset prediction set to obtain the predicted classification category of the land dataset prediction set;

[0013] S7. Use the predicted classification categories of the land dataset prediction set obtained in step S6 and the actual sample label classification categories of the dataset to construct a confusion matrix, and then use the average intersection-over-union ratio, category average pixel accuracy, overall classification accuracy, accuracy, recall rate, and F1 score based on the confusion matrix to evaluate the accuracy of the land classification model based on the fusion of deep and shallow features and multi-attention mechanism.

[0014] Furthermore, the specific implementation method of step S1 includes the following steps:

[0015] S1.1. Perform channel-wise average pooling and maximum pooling on the input feature data. The calculation expression of the average pooling feature A(C) is:

[0016]

[0017] Where C represents the total number of input channels, i represents any one of C, and x(C, i) represents the input feature data of the i-th channel;

[0018] The calculation expression of the maximum pooling feature M(C) is:

[0019] M(C)=Max(x(C,i))

[0020] Among them, Max represents the maximum pooling operation;

[0021] S1.2. Connect A(C) and M(C) in the channel direction to obtain the connection feature Y(C). The calculation expression is:

[0022] Y(C)=Cat(A(C),M(C))

[0023] Among them, Cat represents the connection operation;

[0024] S1.3. Perform a convolution operation on Y(C) with a 7×7 convolution kernel to obtain the convolution feature Y(F). The calculation expression is:

[0025] Y(F)=∫ 7×7 (Y(C))

[0026] Among them, ∫ 7×7 Represents the convolution operation of the 7×7 convolution kernel;

[0027] S1.4. Perform the scaling factor weight operation in batch normalization BN on Y(F) to obtain the scaling factor weight feature Y(BN) in the normalized BN. The calculation expression is:

[0028] Y(BN)=ω λ (BN(Y(F)))

[0029]

[0030]

[0031] Where λ represents the scaling factor of each channel, ω λ Represents the obtained weight, μ B and σ B Represent the mean and standard deviation of the mini-batch, r and β represent the trainable finite transformation scale and offset, BN(s) represents the scale factor, Y represents the penalty factor, Bin out Represents the output features of the previous layer;

[0032] S1.5. Apply the sigmoid activation function to Y(BN) to obtain the attention vector, and then assign the attention vector to the input feature data to complete the multi-scale normalized spatial attention mechanism layer MANSAM assignment to obtain the multi-scale normalized spatial attention mechanism layer output Y. The calculation expression is:

[0033] Y = sigmoid(Y(BN))×input x

[0034] Among them, sigmoid represents the activation function operation, input x Represents input feature data.

[0035] Furthermore, the specific implementation method of step S2 includes the following steps:

[0036] S2.1, perform average pooling of the input feature data in the direction of feature length H, and the average pooling feature in the direction of feature length H The calculation expression is:

[0037]

[0038] Among them, W represents the feature width, H represents the feature length, i represents the number of width features, x C (H, i) represents the i-th width feature data;

[0039] Perform maximum pooling of the input feature data in the direction of feature length H, and the maximum pooling feature in the direction of feature length H The calculation expression is:

[0040]

[0041] The input feature data is average pooled in the feature width W direction, and the average pooled feature in the feature width W direction is The calculation expression is:

[0042]

[0043] Where j represents any one of W;

[0044] Perform maximum pooling of feature width W direction on the input feature data, and the maximum pooling feature of feature width W direction The calculation expression is:

[0045]

[0046] S2.2, yes Perform the scaling factor weight operation in batch normalization BN, and the calculation expression is:

[0047]

[0048]

[0049]

[0050]

[0051] in, is the scaling factor weight feature in the normalized BN of the average pooling in the feature length H direction, is the scaling factor weight feature in the normalized BN of the maximum pooling in the feature length H direction, is the scaling factor weight feature in the normalized BN of the average pooling in the feature width W direction, The scaling factor weight feature in the normalized BN of the maximum pooling in the feature width W direction;

[0052] S2.3, yes Perform matrix dot multiplication to obtain the matrix dot product feature Y(H) in the feature length H direction. The calculation expression is:

[0053]

[0054] right Perform matrix dot multiplication to obtain the matrix dot product feature Y(W) in the feature width W direction. The calculation expression is:

[0055]

[0056] S2.4. Transpose Y(W) and connect it with Y(H) along the H direction to obtain the connection feature Y(H, W) with the characteristic length in the H direction and the characteristic width in the W direction. The calculation expression is:

[0057] Y(H,W)=Cat(Y(H),Y(W).transpsition)

[0058] Among them, transpsition represents matrix transposition;

[0059] S2.5. Perform Conv2d, 1×1 convolution, batch normalization (BN), and activation function h_swish on Y(H, W) to obtain the multi-step operation feature F(H, W) in the feature length direction H and feature width direction W. The calculation expression is:

[0060] F(H, W) = h_swish(BN(∫ 1×1 (Y(H,W))))

[0061] Among them, h_swish represents the activation function operation, ∫ 1×1 Represents the convolution operation of 1×1 convolution kernel;

[0062] S2.6. Split F(H, W) and calculate the expression:

[0063] F(H) (C，H，1) ,F(W) (C，1，W) =split(F(H,W))

[0064] Where split represents the feature splitting operation, (C, H, 1) and (C, 1, W) represent the feature size, and F(H) (C，H，1) Indicates the segmentation feature in the feature length H direction, F(W) (C，1，W) Represents the segmentation feature in the feature width W direction;

[0065] S2.7, respectively for F(H) (C，H，1) 、F(W) (C，1，W) Perform Conv2d, 1×1 convolution operations and activation function h_swish operations to obtain the attention vector Y(Ht) of the segmentation feature in the feature length H direction and the attention vector Y(Wt) of the segmentation feature in the feature width W direction. The calculation expression is:

[0066] Y(Ht)=sigmoid(∫ 1×1 (F(H) (C，H，1) ))

[0067] Y(Wt)=sigmoid(∫ 1×1 (F(W) (C，1，W) ));

[0068] S2.8. Assign the attention vector Y(Ht) of the segmentation feature in the direction of feature length H and the attention vector Y(Wt) of the segmentation feature in the direction of feature width W to the input feature data, complete the multi-scale normalized direction position attention mechanism layer MANDPAM allocation, and obtain the multi-scale normalized direction position attention mechanism layer feature output M(y). The calculation expression is:

[0069] M(y)=Y(Ht)×Y(Wt)×input x .

[0070] Furthermore, the specific implementation method of step S3 includes the following steps:

[0071] S3.1. Perform global average pooling on the input feature data to obtain the global average pooling feature A C (H, W), the calculation expression is:

[0072]

[0073] Perform global maximum pooling on the input feature data to obtain the global maximum pooling feature M C (H, W), the calculation expression is:

[0074] M C (H, W) = Max(x C (H, W, i, j));

[0075] S3.2, respectively for A C (H, W), M C(H, W) performs batch normalization BN scaling factor weight analysis and activation function Relu6 operation, and finally performs batch normalization BN scaling factor weight analysis to obtain the scaling factor weight feature BN(A) in the normalized BN of the global average pooling feature and the scaling factor weight feature BN(M) in the normalized BN of the global maximum pooling feature. The calculation expression is:

[0076] BN(A)=ω λ (BN(Relu6(ω λ (BN(A C (H,W))))))

[0077] BN(M)=ω λ (BN(Relu6(ω λ (BN(M C (H, W))))));

[0078] S3.3. Perform a sum operation on BN(A) and BN(M) to obtain the sum operation feature M(x). The calculation expression is:

[0079] M(x)=BN(A)+BN(M);

[0080] S3.4. Perform the activation function h_swish on M(x) to obtain the attention vector of the summation feature.

[0081] Assign the summation feature attention vector to the input feature data, complete the multi-scale normalized channel attention mechanism layer MANCAM allocation, and obtain the multi-scale normalized channel attention mechanism layer feature C(y). The calculation expression is:

[0082] C(y)=sigmoid(M(x))×input x .

[0083] Furthermore, the specific implementation method of step S4 includes the following steps:

[0084] S4.1. Construct a normalized multi-attention feature extraction network NSCPANet, wherein the normalized multi-attention feature extraction network NSCPANet includes a shallow feature extraction network and a deep feature extraction network;

[0085] S4.1.1. Construct a shallow feature extraction network Shallow_bneck, which is built on the basis of the MobileNetV3 model bneck. First, the input features are convolved with a 1×1 convolution kernel size to increase the dimension of the feature channel. Then, a depth-wise separable convolution with a 3×3 convolution kernel size is used to extract the residual edge and feature information. Then, the multi-scale normalized spatial attention mechanism layer MANSAM constructed in step S1 is used to extract the feature space weight. Then, the multi-scale normalized directional position attention mechanism layer MANDPAM constructed in step S2 is used to extract the feature directional position weight. Then, a convolution operation with a 1×1 convolution kernel size is used to reduce the dimensionality of the feature map and restore the channel size of the input features to obtain the shallow feature extraction network Shallow_bneck.

[0086] S4.1.2. Construct a deep feature extraction network Deep_bneck, which is built on the MobileNetV3 model bneck. First, the input features are convolved with a 1×1 convolution kernel size to increase the dimension of the feature channels. Then, a depthwise separable convolution with a 3×3 convolution kernel size is used to extract residual edges and feature information. Then, the multi-scale normalized channel attention mechanism layer MANCAM layer constructed in step S3 is used to extract feature channel weights. Finally, a convolution operation with a 1×1 convolution kernel size is used to reduce the dimension of the feature map and restore the channel size of the input features to obtain the deep feature extraction network Deep_bneck.

[0087] S4.1.3. Construct a normalized multi-attention feature extraction network NSCPANet, which is based on the NSCPANet model of the MobileNetV3 model. First, perform a convolution operation Covn2d on the input image data with a convolution kernel size of 3×3, a step size of 2, and a feature map of 16. Then perform batch normalization BatchNorm2d and activation function h_swish operation to perform 1 / 2 feature downsampling to obtain the first output feature out1; the first output feature out1 is input into Shallow_bneck for 1 / 4 and 1 / 8 feature downsampling. Sampling is performed to obtain the second output feature out2 and the third output feature out3, and then the third output feature out3 is input into Deep_bneck for 1 / 16 feature downsampling to obtain the fourth output feature out4, and then the fourth output feature out4 is input into Deep_bneck for 1 / 32 feature downsampling, and then the convolution operation with a 3×3 convolution kernel size, a step size of 2, and a feature map of 960 is used for feature expansion, and then batch normalization BatchNorm2d and activation function h_swish are performed to obtain the fifth output feature out5;

[0088] S4.2. Constructing a multi-deep and shallow feature fusion dilated convolutional network MNSDDNet;

[0089] S4.2.1. Construct a deep and shallow feature fusion method SDFF. The deep and shallow feature fusion method SDFF uses the normalized multi-attention feature extraction network NSCPANet constructed in step S4.1 to downsample the input data by 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 respectively. Then, the downsampled 1 / 2, 1 / 8, 1 / 16, and 1 / 32 features are respectively subjected to the convolution algorithm Covn2d, 1×1, and upsampling algorithm to restore the number of channels and size to the same as the 1 / 4 downsampling and connect them to complete the deep and shallow feature fusion. The calculation expression is:

[0090] FT=Cat(interpolate(∫ 1×1 (FT 1 / 2 , FT 1 / 8 , FT 1 / 16 , FT 1 / 32 )), FT 1 / 4 )

[0091] Among them, FT 1 / 2 Indicates the features of downsampling 1 / 2, FT 1 / 4 Indicates the feature of downsampling 1 / 4, FT 1 / 8 Indicates the feature of downsampling 1 / 8, FT 1 / 16 Indicates the feature of downsampling 1 / 16, FT 1 / 32 represents the feature of downsampling 1 / 32, ∫ 1×1 Represents the convolution operation of the 1×1 convolution kernel, and interpolate represents the sampling operation on the enhanced feature edge;

[0092] S4.2.2. Construct a multi-deep and shallow feature fusion void convolution network MNSDDNet, which is constructed based on the MNSDDNet model of the DeeplabV3+ model. First, the fifth output feature out5 obtained in step S4.1 is subjected to void spatial convolution pooling pyramid operation and feature connection operation to obtain feature x. Then, the deep and shallow feature fusion method SDFF constructed in S4.2.1 is used to fuse the features of x, out1, out2, out3, and out4 to obtain x1. The convolution of x1 is performed with a convolution kernel size of 1×1, a step size of 1, and a feature map of the number of classifications num_classes. Then, the interpolate upsampling method is used to restore the original feature size for land use classification prediction.

[0093] Furthermore, the specific implementation method of step S5 includes the following steps:

[0094] S5.1. Collect data from the urban-rural domain adaptive land cover dataset and modify the original dataset size of 1024×1024 to 512×512.

[0095] S5.2. Label the data in the modified dataset, where 0 represents no data area, 1 represents background, 2 represents urban and rural residential land, 3 represents road, 4 represents water area, 5 represents unused land, 6 represents forest land, and 7 represents cultivated land;

[0096] S5.3. Perform rotation, mirroring, blurring, and noise preprocessing on the labeled dataset to obtain the land dataset.

[0097] Furthermore, step S6 inputs the land dataset obtained in step S5 into the land use classification network model constructed in step S4, performs land classification training based on the fusion of deep and shallow features and multi-attention mechanisms, and completes the training using the dice_loss loss function, sgd optimization algorithm, and cos loss attenuation strategy to obtain a land classification model based on the fusion of deep and shallow features and multi-attention mechanisms.

[0098] Furthermore, the specific implementation method of step S7 includes the following steps:

[0099] S7.1. Construct a confusion matrix using the predicted classification categories of the land dataset prediction set obtained in step S6 and the actual sample label classification categories of the dataset to obtain a classification confusion matrix table, where the sum of each row in the classification confusion matrix table is the actual number of samples in the classification, and the sum of each column is the number of samples predicted to be classified;

[0100] S7.2. Perform mean intersection over union (mIoU) accuracy evaluation based on the confusion matrix. Calculate IoU0, IoU1, IoU2, IoU3, IoU4, IoU5, IoU6, and IoU7 for the seven categories using the confusion matrix.

[0101] Then calculate the average intersection over union (mIoU), and the calculation expression is:

[0102]

[0103] Then the calculation expression of mIoU is:

[0104]

[0105] Among them, k represents the number of categories, p ij Indicates that i is predicted to be j, which is a false negative FN; p ji Indicates that j is predicted to be i, which is a false positive FP; p ii It means that i is predicted to be i, which is the true TP;

[0106] S7.3. Perform category average pixel accuracy (mPA) evaluation based on the confusion matrix. Calculate the pixel accuracy of seven categories (CPA0, CPA1, CPA2, CPA3, CPA4, CPA5, CPA6, and CPA7) using the confusion matrix.

[0107] Then calculate the average CPA of each category to get mPA, the calculation expression is:

[0108]

[0109] Then the calculation expression of mPA is:

[0110]

[0111] Among them, k represents the number of categories, i represents the classification category, and CPA represents the category pixel accuracy;

[0112] S7.4. Calculate the overall classification accuracy OA, which is the sum of correctly classified pixels divided by the total number of pixels. The number of correctly classified pixels is distributed along the diagonal of the confusion matrix, and the total number of pixels is the total number of pixels with sample labels. The calculation expression is:

[0113]

[0114] Among them, k represents the number of categories, i represents the actual classification category, j represents the predicted classification category, and p ij Indicates that i is predicted to be j, which is a false negative; p ii It means that i is predicted to be i, which is the true TP;

[0115] S7.5. Calculate the accuracy, which is the ratio of the number of correctly classified positive samples to the total number of positive samples in the predicted classification. The calculation expression is:

[0116]

[0117] Among them, TP represents the positive samples that are correctly predicted and classified, i.e. true positives, and FP represents the negative samples that are incorrectly predicted and classified, i.e. false positives;

[0118] S7.6. Calculate the recall rate (Recall), which is the ratio of the number of correctly classified positive samples to the total number of positive samples. The calculation expression is:

[0119]

[0120] Among them, TP represents the positive samples that are correctly predicted and classified, i.e. true positive samples, and FN represents the positive samples that are incorrectly predicted and classified, i.e. false negative positive samples;

[0121] S7.7. Calculate the F1 score, which is the harmonic mean of precision and recall. The calculation expression is:

[0122]

[0123] Among them, Precision represents the accuracy rate and Recall represents the recall rate.

[0124] The electronic device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps of the comprehensive evaluation method for land classification are realized.

[0125] A computer-readable storage medium stores a computer program, which implements the comprehensive evaluation method for land classification when executed by a processor.

[0126] Beneficial effects of the present invention:

[0127] The comprehensive land classification evaluation method described in the present invention constructs a land use classification network model that can not only more accurately extract scenes with fewer categories, such as water bodies, lake ice, and individual buildings and roads, but can also generate higher-precision land use classification images in complex multi-category remote sensing images. The comprehensive land classification evaluation method described in the present invention performs a multi-dimensional accuracy evaluation on the land use classification network model, comprehensively utilizing overall classification accuracy, average pixel accuracy, average intersection-over-union ratio of categories, and single-category accuracy, recall rate, and F1 score accuracy evaluation methods to comprehensively evaluate the model classification accuracy, ensuring the accuracy and reliability of the land use classification accuracy evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0128] Figure 1 This is a flow chart of a comprehensive evaluation method for land classification according to the present invention;

[0129] Figure 2 This is a diagram of the NSCPANet_NSCPANet architecture of the land use classification network model in the comprehensive evaluation method for land classification according to the present invention;

[0130] Figure 3 This is a Shallow_backkeck architecture diagram of a land use classification network model in a comprehensive land classification evaluation method according to the present invention;

[0131] Figure 4 This is a Deep_bneck architecture diagram of the land use classification network model in the comprehensive evaluation method for land classification described in the present invention;

[0132] Figure 5 This is a MANSAM architecture diagram of a land use classification network model in a comprehensive land classification evaluation method according to the present invention;

[0133] Figure 6 This is a MANDPAM architecture diagram of a land use classification network model in a comprehensive land classification evaluation method according to the present invention;

[0134] Figure 7 This is a MANCAM architecture diagram of a land use classification network model in a comprehensive land classification evaluation method according to the present invention;

[0135] Figure 8 This is a comparison chart of the mIoU accuracy evaluation of the comprehensive evaluation method for land classification described in the present invention;

[0136] Figure 9 This is an mPA evaluation comparison chart of a comprehensive evaluation method for land classification according to the present invention;

[0137] Figure 10 This is an F1 evaluation comparison chart of a comprehensive evaluation method for land classification according to the present invention;

[0138] Figure 11 This is a comparison chart of the precision evaluation of the comprehensive evaluation method for land classification according to the present invention;

[0139] Figure 12 This is a Recall evaluation comparison chart of the comprehensive evaluation method for land classification described in the present invention. DETAILED DESCRIPTION

[0140] In order to make the objectives, technical solutions, and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present invention and are not intended to limit the present invention. That is, the specific embodiments described herein are only some embodiments of the present invention, not all embodiments. Generally, the components of the specific embodiments of the present invention described and illustrated in the drawings herein can be arranged and designed in various different configurations, and the present invention can also have other embodiments.

[0141] Therefore, the following detailed description of the specific embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but is merely representative of selected specific embodiments of the present invention. All other specific embodiments obtained by those skilled in the art based on the specific embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0142] In order to further understand the content, features and effects of the present invention, the following specific embodiments are given as examples, and the attached Figure 1 -Attached Figure 12 The detailed instructions are as follows: Specific implementation method one:

[0144] A comprehensive evaluation method for land classification includes the following steps:

[0145] S1. Construct a multi-scale normalized spatial attention mechanism layer MANSAM;

[0146] Furthermore, the specific implementation method of step S1 includes the following steps:

[0147] S1.1. Perform channel-wise average pooling and maximum pooling on the input feature data. The calculation expression of the average pooling feature A(C) is:

[0148]

[0149] Where C represents the total number of input channels, i represents any one of C, and x(C, i) represents the input feature data of the i-th channel;

[0150] The calculation expression of the maximum pooling feature M(C) is:

[0151] M(C)=Max(x(C,i))

[0152] Among them, Max represents the maximum pooling operation;

[0153] S1.2. Connect A(C) and M(C) in the channel direction to obtain the connection feature Y(C). The calculation expression is:

[0154] Y(C)=Cat(A(C),M(C))

[0155] Among them, Cat represents the connection operation;

[0156] S1.3. Perform a convolution operation on Y(C) with a 7×7 convolution kernel to obtain the convolution feature Y(F). The calculation expression is:

[0157] Y(F)=∫ 7×7 (Y(C))

[0158] Among them, ∫ 7×7 Represents the convolution operation of the 7×7 convolution kernel;

[0159] S1.4. Perform the scaling factor weight operation in batch normalization BN on Y(F) to obtain the scaling factor weight feature Y(BN) in the normalized BN. The calculation expression is:

[0160] Y(BN)=ω λ (BN(Y(F)))

[0161]

[0162]

[0163] Where λ represents the scaling factor of each channel, ω λ Represents the obtained weight, μ B and σ B Represent the mean and standard deviation of the mini-batch, r and β represent the trainable finite transformation scale and offset, BN(s) represents the scale factor, ∈ represents the penalty factor, Bin out Represents the output features of the previous layer;

[0164] S1.5. Apply the sigmoid activation function to Y(BN) to obtain the attention vector, and then assign the attention vector to the input feature data to complete the multi-scale normalized spatial attention mechanism layer MANSAM assignment to obtain the multi-scale normalized spatial attention mechanism layer output Y. The calculation expression is:

[0165] Y = sigmoid(Y(BN))×input x

[0166] Among them, sigmoid represents the activation function operation, input x Represents input feature data;

[0167] S2. Construct a multi-scale normalized direction position attention mechanism layer MANDPAM;

[0168] Furthermore, the specific implementation method of step S2 includes the following steps:

[0169] S2.1, perform average pooling of the input feature data in the direction of feature length H, and the average pooling feature in the direction of feature length H The calculation expression is:

[0170]

[0171] Among them, W represents the feature width, H represents the feature length, i represents the number of width features, x C (H, i) represents the i-th width feature data;

[0172] Perform maximum pooling of the input feature data in the direction of feature length H, and the maximum pooling feature in the direction of feature length H The calculation expression is:

[0173]

[0174] The input feature data is average pooled in the feature width W direction, and the average pooled feature in the feature width W direction is The calculation expression is:

[0175]

[0176] Where j represents any one of W;

[0177] Perform maximum pooling of feature width W direction on the input feature data, and the maximum pooling feature of feature width W direction The calculation expression is:

[0178]

[0179] S2.2, yes Perform the scaling factor weight operation in batch normalization BN, and the calculation expression is:

[0180]

[0181]

[0182]

[0183]

[0184] in, is the scaling factor weight feature in the normalized BN of the average pooling in the feature length H direction, is the scaling factor weight feature in the normalized BN of the maximum pooling in the feature length H direction, is the scaling factor weight feature in the normalized BN of the average pooling in the feature width W direction, The scaling factor weight feature in the normalized BN of the maximum pooling in the feature width W direction;

[0185] S2.3, yes Perform matrix dot multiplication to obtain the matrix dot product feature Y(H) in the feature length H direction. The calculation expression is:

[0186]

[0187] right Perform matrix dot multiplication to obtain the matrix dot product feature Y(W) in the feature width W direction. The calculation expression is:

[0188]

[0189] S2.4. Transpose Y(W) and connect it with Y(H) along the H direction to obtain the connection feature Y(H, W) with the characteristic length in the H direction and the characteristic width in the W direction. The calculation expression is:

[0190] Y(H,W)=Cat(Y(H),Y(W).transpsition)

[0191] Among them, transpsition represents matrix transposition;

[0192] S2.5. Perform Conv2d, 1×1 convolution, batch normalization (BN), and activation function h_swish on Y(H, W) to obtain the multi-step operation feature F(H, W) in the feature length direction H and feature width direction W. The calculation expression is:

[0193] F(H, W) = h_swish(BN(∫ 1×1 (Y(H,W))))

[0194] Among them, h_swish represents the activation function operation, ∫ 1×1 Represents the convolution operation of 1×1 convolution kernel;

[0195] S2.6. Split F(H, W) and calculate the expression:

[0196] F(H) (C，H，1) ,F(W) (C，1，W) =split(F(H,W))

[0197] Where split represents the feature splitting operation, (C, H, 1) and (C, 1, W) represent the feature size, and F(H) (C，H，1) Indicates the segmentation feature in the feature length H direction, F(W) (C，1，W) Represents the segmentation feature in the feature width W direction;

[0198] S2.7, respectively for F(H) (C，H，1) 、F(W) (C，1，W) Perform Conv2d, 1×1 convolution operations and activation function h_swish operations to obtain the attention vector Y(Ht) of the segmentation feature in the feature length H direction and the attention vector Y(Wt) of the segmentation feature in the feature width W direction. The calculation expression is:

[0199] Y(Ht)=sigmoid(∫ 1×1 (F(H) (C，H，1) ))

[0200] Y(Wt)=sigmoid(∫ 1×1 (F(W) (C，1，W) ));

[0201] S2.8. Assign the attention vector Y(Ht) of the segmentation feature in the direction of feature length H and the attention vector Y(Wt) of the segmentation feature in the direction of feature width W to the input feature data, complete the multi-scale normalized direction position attention mechanism layer MANDPAM allocation, and obtain the multi-scale normalized direction position attention mechanism layer feature output M(y). The calculation expression is:

[0202] M(y)=Y(Ht)×Y(Wt)×input x ;

[0203] S3, construct a multi-scale normalized channel attention mechanism layer MANCAM;

[0204] Furthermore, the specific implementation method of step S3 includes the following steps:

[0205] S3.1. Perform global average pooling on the input feature data to obtain the global average pooling feature A C (H, W), the calculation expression is:

[0206]

[0207] Perform global maximum pooling on the input feature data to obtain the global maximum pooling feature M C (H, W), the calculation expression is:

[0208] M C (H, W) = Max(x C (H, W, i, j));

[0209] S3.2, respectively for A C (H, W), M C (H, W) performs batch normalization BN scaling factor weight analysis and activation function Relu6 operation, and finally performs batch normalization BN scaling factor weight analysis to obtain the scaling factor weight feature BN(A) in the normalized BN of the global average pooling feature and the scaling factor weight feature BN(M) in the normalized BN of the global maximum pooling feature. The calculation expression is:

[0210] BN(A)=ω λ (BN(Relu6(ω λ (BN(A C (H,W))))))

[0211] BN(M)=ω λ (BN(Relu6(ω λ (BN(M C (H, W))))));

[0212] S3.3. Perform a sum operation on BN(A) and BN(M) to obtain the sum operation feature M(x). The calculation expression is:

[0213] M(x)=BN(A)+BN(M);

[0214] S3.4. Perform the activation function h_swish on M(x) to obtain the attention vector of the summation feature.

[0215] Assign the summation feature attention vector to the input feature data, complete the multi-scale normalized channel attention mechanism layer MANCAM allocation, and obtain the multi-scale normalized channel attention mechanism layer feature C(y). The calculation expression is:

[0216] C(y)=sigmoid(M(x))×input x ;

[0217] S4. Construct a land use classification network model, including a normalized multi-attention feature extraction network NSCPANet and a multi-deep and shallow feature fusion dilated convolutional network MNSDDNet. The normalized multi-attention feature extraction network NSCPANet includes a shallow feature extraction network and a deep feature extraction network. The multi-deep and shallow feature fusion dilated convolutional network MNSDDNet uses a deep and shallow feature fusion method SDFF to fuse the features of the shallow feature extraction network and the deep feature extraction network.

[0218] Furthermore, the specific implementation method of step S4 includes the following steps:

[0219] S4.1. Construct a normalized multi-attention feature extraction network NSCPANet, wherein the normalized multi-attention feature extraction network NSCPANet includes a shallow feature extraction network and a deep feature extraction network;

[0220] S4.1.1. Construct a shallow feature extraction network Shallow_bneck, which is built on the basis of the MobileNetV3 model bneck. First, the input features are convolved with a 1×1 convolution kernel size to increase the dimension of the feature channel. Then, a depth-wise separable convolution with a 3×3 convolution kernel size is used to extract the residual edge and feature information. Then, the multi-scale normalized spatial attention mechanism layer MANSAM constructed in step S1 is used to extract the feature space weight. Then, the multi-scale normalized directional position attention mechanism layer MANDPAM constructed in step S2 is used to extract the feature directional position weight. Then, a convolution operation with a 1×1 convolution kernel size is used to reduce the dimensionality of the feature map and restore the channel size of the input features to obtain the shallow feature extraction network Shallow_bneck.

[0221] S4.1.2. Construct a deep feature extraction network Deep_bneck, which is built on the MobileNetV3 model bneck. First, the input features are convolved with a 1×1 convolution kernel size to increase the dimension of the feature channels. Then, a depthwise separable convolution with a 3×3 convolution kernel size is used to extract residual edges and feature information. Then, the multi-scale normalized channel attention mechanism layer MANCAM layer constructed in step S3 is used to extract feature channel weights. Finally, a convolution operation with a 1×1 convolution kernel size is used to reduce the dimension of the feature map and restore the channel size of the input features to obtain the deep feature extraction network Deep_bneck.

[0222] S4.1.3. Construct a normalized multi-attention feature extraction network NSCPANet, which is based on the NSCPANet model of the MobileNetV3 model. First, perform a convolution operation Covn2d on the input image data with a convolution kernel size of 3×3, a step size of 2, and a feature map of 16. Then perform batch normalization BatchNorm2d and activation function h_swish operation to perform 1 / 2 feature downsampling to obtain the first output feature out1; the first output feature out1 is input into Shallow_bneck for 1 / 4 and 1 / 8 feature downsampling. Sampling is performed to obtain the second output feature out2 and the third output feature out3, and then the third output feature out3 is input into Deep_bneck for 1 / 16 feature downsampling to obtain the fourth output feature out4, and then the fourth output feature out4 is input into Deep_bneck for 1 / 32 feature downsampling, and then the convolution operation with a 3×3 convolution kernel size, a step size of 2, and a feature map of 960 is used for feature expansion, and then batch normalization BatchNorm2d and activation function h_swish are performed to obtain the fifth output feature out5;

[0223] S4.2. Constructing a multi-deep and shallow feature fusion dilated convolutional network MNSDDNet;

[0224] S4.2.1. Construct a deep and shallow feature fusion method SDFF. The deep and shallow feature fusion method SDFF uses the normalized multi-attention feature extraction network NSCPANet constructed in step S4.1 to downsample the input data by 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 respectively. Then, the downsampled 1 / 2, 1 / 8, 1 / 16, and 1 / 32 features are respectively subjected to the convolution algorithm Covn2d, 1×1, and upsampling algorithm to restore the number of channels and size to the same as the 1 / 4 downsampling and connect them to complete the deep and shallow feature fusion. The calculation expression is:

[0225] FT=Cat(interpolate(∫ 1×1 (FT1 / 2 , FT 1 / 8 , FT 1 / 16 , FT 1 / 32 )), FT 1 / 4 )

[0226] Among them, FT 1 / 2 Indicates the features of downsampling 1 / 2, FT 1 / 4 Indicates the feature of downsampling 1 / 4, FT 1 / 8 Indicates the feature of downsampling 1 / 8, FT 1 / 16 Indicates the feature of downsampling 1 / 16, FT 1 / 32 represents the feature of downsampling 1 / 32, ∫ 1×1 Represents the convolution operation of the 1×1 convolution kernel, and interpolate represents the sampling operation on the enhanced feature edge;

[0227] S4.2.2. Construct a multi-deep and shallow feature fusion atrous convolutional network (MNSDDNet). This is based on the MNSDDNet model of the DeeplabV3+ model. First, perform atrous spatial convolution pooling pyramid operations and feature concatenation operations on the fifth output feature out5 obtained in step S4.1 to obtain feature x. Then, use the deep and shallow feature fusion method SDFF constructed in S4.2.1 to fuse the features of x, out1, out2, out3, and out4 to obtain x1. Convolution is performed on x1 with a convolution kernel size of 1×1, a stride of 1, and a feature map with the number of classes num_classes. Then, use the interpolate upsampling method to restore the original feature size for land use classification prediction.

[0228] Table 1: NSCPANet model network structure

[0229]

[0230] S5. Collect data from the urban-rural domain adaptive land cover dataset, perform data preprocessing, obtain a land dataset, and divide it into a land dataset prediction set and a land dataset training set;

[0231] Furthermore, the specific implementation method of step S5 includes the following steps:

[0232] S5.1. Collect data from the urban-rural domain adaptive land cover dataset and modify the original dataset size of 1024×1024 to 512×512.

[0233] S5.2. Label the data in the modified dataset, where 0 represents no data area, 1 represents background, 2 represents urban and rural residential land, 3 represents road, 4 represents water area, 5 represents unused land, 6 represents forest land, and 7 represents cultivated land;

[0234] S5.3. Rotate, mirror, blur, and perform noise preprocessing on the labeled dataset to obtain a land dataset;

[0235] S6. Input the land dataset training set obtained in step S5 into the land use classification network model constructed in step S4, perform land classification training based on the fusion of deep and shallow features and multi-attention mechanism, obtain a land classification model based on the fusion of deep and shallow features and multi-attention mechanism, and then perform sample prediction on the land dataset prediction set to obtain the predicted classification category of the land dataset prediction set;

[0236] Furthermore, step S6 inputs the land dataset obtained in step S5 into the land use classification network model constructed in step S4, performs land classification training based on the fusion of deep and shallow features and multi-attention mechanisms, and completes the training using the dice_loss loss function, sgd optimization algorithm, and cos loss attenuation strategy to obtain a land classification model based on the fusion of deep and shallow features and multi-attention mechanisms.

[0237] Furthermore, the training parameter settings include: batch_size=8, optimizer=sgd(lr=0.001), lr_decay_type=cos, loss=dice_loss.

[0238] S7. Use the predicted classification categories of the land dataset prediction set obtained in step S6 and the actual sample label classification categories of the dataset to construct a confusion matrix, and then use the average intersection-over-union ratio, category average pixel accuracy, overall classification accuracy, accuracy, recall rate, and F1 score based on the confusion matrix to evaluate the accuracy of the land classification model based on the fusion of deep and shallow features and multi-attention mechanism;

[0239] Furthermore, the specific implementation method of step S7 includes the following steps:

[0240] S7.1. Construct a confusion matrix using the predicted classification categories of the land dataset prediction set obtained in step S6 and the actual sample label classification categories of the dataset to obtain a classification confusion matrix table, where the sum of each row in the classification confusion matrix table is the actual number of samples in the classification, and the sum of each column is the number of samples predicted to be classified;

[0241] The confusion matrix is ​​shown in Table 2:

[0242] Table 2 Confusion matrix

[0243]

[0244] From the confusion matrix, we can see that the sum of each row is the actual number of samples of that class, and the sum of each column is the number of samples predicted to be of that class. The letters in the table represent the number of samples predicted by the model for different categories. gt0, gt1, gt2, gt3, gt4, gt5, gt6, and gt7 represent sample data for the correctly predicted corresponding categories, and the others represent the number of samples incorrectly predicted to be of different categories. For example, for category 0, there are gt0 samples that are actually class 0 and are classified as class 0, b0 samples that are actually class 0 and are classified as class 1, c0 samples that are actually class 0 and are classified as class 2, d0 samples that are actually class 0 and are classified as class 3, e0 samples that are actually class 0 and are classified as class 4, f0 samples that are actually class 0 and are classified as class 5, g0 samples that are actually class 0 and are classified as class 6, and h0 samples that are actually class 0 and are classified as class 7. The other categories are distributed in the same regular pattern.

[0245] S7.2. Perform mean intersection over union (mIoU) accuracy evaluation based on the confusion matrix. Calculate IoU0, IoU1, IoU2, IoU3, IoU4, IoU5, IoU6, and IoU7 for the seven categories using the confusion matrix.

[0246] Taking the IoU calculation of category 0 as an example, the IoU calculation of other categories is similar. The calculation expression is:

[0247] IoU0=gt0 / ((gt0+b0+c0+d0+e0+f0+g0+h0)+(gt0+a1+a2+a3+a4+a5+a6+a7)-gt0)

[0248] Among them, gt0 represents the number of samples correctly classified as class 0, b0, c0, d0, e0, f0, g0, and h0 represent the number of samples that should be class 0 but are predicted to be class 1, class 2, class 3, class 4, class 5, class 6, and class 7; a1, a2, a3, a4, a5, a6, and a7 represent the number of samples predicted by the model to be class 1, class 2, class 3, class 4, class 5, class 6, and class 7;

[0249] Then calculate the average intersection over union (mIoU), and the calculation expression is:

[0250]

[0251] Then the calculation expression of mIoU is:

[0252]

[0253] Among them, k represents the number of categories, p ij Indicates that i is predicted to be j, which is a false negative FN; p ji Indicates that j is predicted to be i, which is a false positive FP; p iiIt means that i is predicted to be i, which is the true TP;

[0254] S7.3. Perform category average pixel accuracy (mPA) evaluation based on the confusion matrix. Calculate the pixel accuracy of seven categories (CPA0, CPA1, CPA2, CPA3, CPA4, CPA5, CPA6, and CPA7) using the confusion matrix.

[0255] Taking the CPA calculation of category 0 as an example, the CPA calculation of other categories is similar. The calculation expression is:

[0256] CPA0=gt0 / (gt0+b0+c0+d0+e0+f0+g0+h0)

[0257] Among them, gt0 represents the number of samples correctly classified as class 0, b0, c0, d0, e0, f0, g0, and h0 represent the number of samples that should be class 0 but are predicted to be class 1, class 2, class 3, class 4, class 5, class 6, and class 7;

[0258] Then calculate the average CPA of each category to get mPA, the calculation expression is:

[0259]

[0260] Then the calculation expression of mPA is:

[0261]

[0262] Among them, k represents the number of categories, i represents the classification category, and CPA represents the category pixel accuracy;

[0263] S7.4. Calculate the overall classification accuracy OA, which is the sum of correctly classified pixels divided by the total number of pixels. The number of correctly classified pixels is distributed along the diagonal of the confusion matrix, and the total number of pixels is the total number of pixels with sample labels. The calculation expression is:

[0264]

[0265] Among them, k represents the number of categories, i represents the actual classification category, j represents the predicted classification category, and p ij Indicates that i is predicted to be j, which is a false negative; p ii It means that i is predicted to be i, which is the true TP;

[0266] S7.5. Calculate the accuracy, which is the ratio of the number of correctly classified positive samples to the total number of positive samples in the predicted classification. The calculation expression is:

[0267]

[0268] Among them, TP represents the positive samples that are correctly predicted and classified, i.e. true positives, and FP represents the negative samples that are incorrectly predicted and classified, i.e. false positives;

[0269] S7.6. Calculate the recall rate (Recall), which is the ratio of the number of correctly classified positive samples to the total number of positive samples. The calculation expression is:

[0270]

[0271] Among them, TP represents the positive samples that are correctly predicted and classified, i.e. true positive samples, and FN represents the positive samples that are incorrectly predicted and classified, i.e. false negative positive samples;

[0272] S7.7. Calculate the F1 score, which is the harmonic mean of precision and recall. The calculation expression is:

[0273]

[0274] Among them, Precision represents the accuracy rate, and Recall represents the recall rate;

[0275] Furthermore, comparative analysis was conducted with U_net (ResNet50), pspnet (ResNet50), DeeplabV3+, and Segmenter models, using mIoU, mPA, OA, Precision, Recall, and F1 scores for evaluation. The land use classification network model was evaluated in multiple dimensions. The evaluation results are shown in Table 3. In addition, an ablation experiment was conducted to verify the impact of the added modules on the classification structure. The evaluation results are shown in Table 4.

[0276] Table 3 Comparison of classification evaluation of land types by different models

[0277]

[0278]

[0279] Table 4 Ablation experiment

[0280]

[0281] As can be seen from Table 4, the land use classification network model in this implementation has a better model accuracy, with OA reaching 84.20, F1 score reaching 85.90%, mAP reaching 85.95%, and mIoU reaching 73.81%. Specific implementation method two:

[0283] The electronic device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps of a comprehensive evaluation method for land classification described in the first specific implementation method are implemented.

[0284] The computer device of the present invention may include a processor and a memory, such as a single-chip microcomputer including a central processing unit. Furthermore, the processor is configured to execute a computer program stored in the memory to implement the steps of the above-mentioned comprehensive land classification evaluation method.

[0285] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0286] The memory may mainly include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area may store data created based on the use of the mobile phone (such as audio data, a phone book, etc.). In addition, the memory may include a high-speed random access memory and may also include a non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Specific implementation method three:

[0288] A computer-readable storage medium stores a computer program, which implements the comprehensive evaluation method for land classification when executed by a processor.

[0289] The computer-readable storage medium of the present invention can be any form of storage medium that can be read by the processor of a computer device, including but not limited to non-volatile memory, volatile memory, ferroelectric memory, etc. The computer-readable storage medium stores a computer program. When the processor of the computer device reads and executes the computer program stored in the memory, the steps of the above-mentioned comprehensive evaluation method for land classification can be implemented.

[0290] The computer program includes computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0291] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0292] Although the present application has been described above with reference to specific embodiments, various modifications may be made thereto and components may be substituted with equivalents without departing from the scope of the present application. In particular, as long as there are no structural conflicts, the various features of the embodiments disclosed herein may be combined with each other in any manner, and the omission of an exhaustive description of these combinations in this specification is solely for the sake of space and resource conservation. Therefore, the present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions within the scope of the claims.

Claims

1. A comprehensive evaluation method for land classification, characterized in that: The steps include: S1. Construct a multi-scale normalized spatial attention mechanism layer MANSAM; S2. Construct a multi-scale normalized direction position attention mechanism layer MANDPAM; S3, construct a multi-scale normalized channel attention mechanism layer MANCAM; S4. Construct a land use classification network model, including a normalized multi-attention feature extraction network NSCPANet and a multi-deep and shallow feature fusion dilated convolutional network MNSDDNet. The normalized multi-attention feature extraction network NSCPANet includes a shallow feature extraction network and a deep feature extraction network. The multi-deep and shallow feature fusion dilated convolutional network MNSDDNet uses a deep and shallow feature fusion method SDFF to fuse the features of the shallow feature extraction network and the deep feature extraction network. S5. Collect data from the urban-rural domain adaptive land cover dataset, perform data preprocessing, obtain a land dataset, and divide it into a land dataset prediction set and a land dataset training set; S6. Input the land dataset training set obtained in step S5 into the land use classification network model constructed in step S4, perform land classification training based on the fusion of deep and shallow features and multi-attention mechanism, obtain a land classification model based on the fusion of deep and shallow features and multi-attention mechanism, and then perform sample prediction on the land dataset prediction set to obtain the predicted classification category of the land dataset prediction set; S7. Use the predicted classification categories of the land dataset prediction set obtained in step S6 and the actual sample label classification categories of the dataset to construct a confusion matrix, and then use the average intersection-over-union ratio, category average pixel accuracy, overall classification accuracy, accuracy, recall rate, and F1 score based on the confusion matrix to evaluate the accuracy of the land classification model based on the fusion of deep and shallow features and multi-attention mechanism.

2. A comprehensive evaluation method for land classification according to claim 1, characterized in that: The specific implementation method of step S1 includes the following steps: S1.

1. Perform channel-wise average pooling and maximum pooling on the input feature data. The calculation expression of the average pooling feature a(C) is: Where C represents the total number of input channels, i represents any one of C, and x(C,i) represents the input feature data of the i-th channel; The calculation expression of the maximum pooling feature M(C) is: M(C)=Max(x(C,i)) Among them, Max represents the maximum pooling operation; S1.

2. Connect A(C) and M(C) in the channel direction to obtain the connection feature Y(C). The calculation expression is: Y(C)=Cat(A(C),M(C)) Among them, Cat represents the connection operation; S1.

3. Perform a convolution operation on Y(C) with a 7×7 convolution kernel to obtain the convolution feature Y(F). The calculation expression is: Y(F)=∫ 7×7 (Y(C)) Among them, ∫ 7×7 Represents the convolution operation of the 7×7 convolution kernel; S1.

4. Perform the scaling factor weight operation in batch normalization BN on Y(F) to obtain the scaling factor weight feature Y(BN) in the normalized BN. The calculation expression is: Y(BN)=ω λ (BN(Y(F))) Where λ represents the scaling factor of each channel, ω λ Represents the obtained weight, μ B and σ B Represent the mean and standard deviation of the mini-batch, r and β represent the trainable finite transformation scale and offset, BN(s) represents the scale factor, ∈ represents the penalty factor, Bin out Represents the output features of the previous layer; S1.

5. Apply the sigmoid activation function to Y(BN) to obtain the attention vector, and then assign the attention vector to the input feature data to complete the multi-scale normalized spatial attention mechanism layer MANSAM assignment to obtain the multi-scale normalized spatial attention mechanism layer output Y. The calculation expression is: Y=sigmoid(Y(BN))×input x Among them, sigmoid represents the activation function operation, input x Represents input feature data.

3. A comprehensive evaluation method for land classification according to claim 2, characterized in that: The specific implementation method of step S2 includes the following steps: S2.1, perform average pooling of the input feature data in the direction of feature length H, and the average pooling feature in the direction of feature length H The calculation expression is: Among them, W represents the feature width, H represents the feature length, i represents the number of width features, x C (H,i) represents the i-th width feature data; Perform maximum pooling of the input feature data in the direction of feature length H, and the maximum pooling feature in the direction of feature length H The calculation expression is: The input feature data is average pooled in the feature width W direction, and the average pooled feature in the feature width W direction is The calculation expression is: Where j represents any one of W; Perform maximum pooling of feature width W direction on the input feature data, and the maximum pooling feature of feature width W direction The calculation expression is: S2.2, yes Perform the scaling factor weight operation in batch normalization BN, and the calculation expression is: in, is the scaling factor weight feature in the normalized BN of the average pooling in the feature length H direction, is the scaling factor weight feature in the normalized BN of the maximum pooling in the feature length H direction, is the scaling factor weight feature in the normalized BN of the average pooling in the feature width W direction, The scaling factor weight feature in the normalized BN of the maximum pooling in the feature width W direction; S2.3, yes Perform matrix dot multiplication to obtain the matrix dot product feature Y(H) in the feature length H direction. The calculation expression is: right Perform matrix dot multiplication to obtain the matrix dot product feature Y(W) in the feature width W direction. The calculation expression is: S2.

4. Transpose Y(W) and connect it with Y(H) along the H direction to obtain the connection feature Y(H,W) with the characteristic length in the H direction and the characteristic width in the W direction. The calculation expression is: Y(H,W)=Cat(Y(H),Y(W).transpsition) Among them, transpsition represents matrix transposition; S2.

5. Perform Conv2d, 1×1 convolution, batch normalization (BN), and activation function h_swish on Y(H,W) to obtain the multi-step operation feature F(H,W) in the feature length direction H and feature width direction W. The calculation expression is: F(H,W)=h_swish(BN(∫ 1×1 (Y(H,W)))) Among them, h_swish represents the activation function operation, ∫ 1×1 Represents the convolution operation of 1×1 convolution kernel; S2.

6. Split F(H,W) and calculate the expression: F(H) (C,H,1) ,F(W) (C,1,W) =split(F(H,W)) Among them, split represents the feature splitting operation, (C,H,1) and (C,1,W) represent the feature size, F(H) (c,H,1) Indicates the segmentation feature in the feature length H direction, F(W) (C,1,W) Represents the segmentation feature in the feature width W direction; S2.7, respectively for F(H) (C,H,1) 、F(W) (C,1,W) Perform Conv2d, 1×1 convolution operations and activation function h_swish operations to obtain the attention vector Y(Ht) of the segmentation feature in the feature length H direction and the attention vector Y(Wt) of the segmentation feature in the feature width W direction. The calculation expression is: Y(Ht)=sigmoid(∫ 1×1 (F(H) (C,H,1) )) Y(Wt)=sigmoid(∫ 1×1 (F(W) (C,1,W) )); S2.

8. Assign the attention vector Y(Ht) of the segmentation feature in the direction of feature length H and the attention vector Y(Wt) of the segmentation feature in the direction of feature width W to the input feature data, complete the multi-scale normalized direction position attention mechanism layer MANDPAM allocation, and obtain the multi-scale normalized direction position attention mechanism layer feature output M(y). The calculation expression is: M(y)=Y(Ht)×Y(Wt)×input x 。 4. A comprehensive evaluation method for land classification according to claim 3, characterized in that: The specific implementation method of step S3 includes the following steps: S3.

1. Perform global average pooling on the input feature data to obtain the global average pooling feature A C (H,W), the calculation expression is: Perform global maximum pooling on the input feature data to obtain the global maximum pooling feature M C (H,W), the calculation expression is: M C (H,W)=Max(x C (H,W,i,j)); S3.2, respectively for A C (H,W),M C (H, W) performs batch normalization BN scaling factor weight analysis and activation function Relu6 operation, and finally performs batch normalization BN scaling factor weight analysis to obtain the scaling factor weight feature BN(A) in the normalized BN of the global average pooling feature and the scaling factor weight feature BN(M) in the normalized BN of the global maximum pooling feature. The calculation expression is: BN(A)=ω λ (BN(Relu6(ω λ (BN(A C (H,W)))))) BN(M)=ω λ (BN(Relu6(ω λ (BN(M C (H,W)))))); S3.

3. Perform a sum operation on BN(A) and BN(M) to obtain the sum operation feature M(x). The calculation expression is: M(x)=BN(A)+BN(M); S3.

4. Perform the activation function h_swish on M(x) to obtain the attention vector of the summation feature. Assign the summation feature attention vector to the input feature data, complete the multi-scale normalized channel attention mechanism layer MANCAM allocation, and obtain the multi-scale normalized channel attention mechanism layer feature C(y). The calculation expression is: C(y)=sigmoid(M(x))×input x 。 5. A comprehensive evaluation method for land classification according to claim 4, characterized in that: The specific implementation method of step S4 includes the following steps: S4.

1. Construct a normalized multi-attention feature extraction network NSCPANet, wherein the normalized multi-attention feature extraction network NSCPANet includes a shallow feature extraction network and a deep feature extraction network; S4.1.

1. Construct a shallow feature extraction network Shallow_bneck, which is built on the basis of the MobileNetV3 model bneck. First, the input features are convolved with a 1×1 convolution kernel size to increase the dimension of the feature channel. Then, a depth-wise separable convolution with a 3×3 convolution kernel size is used to extract the residual edge and feature information. Then, the multi-scale normalized spatial attention mechanism layer MANSAM constructed in step S1 is used to extract the feature space weight. Then, the multi-scale normalized directional position attention mechanism layer MANDPAM constructed in step S2 is used to extract the feature directional position weight. Then, a convolution operation with a 1×1 convolution kernel size is used to reduce the dimensionality of the feature map and restore the channel size of the input features to obtain the shallow feature extraction network Shallow_bneck. S4.1.

2. Construct a deep feature extraction network Deep_bneck, which is built on the MobileNetV3 model bneck. First, the input features are convolved with a 1×1 convolution kernel size to increase the dimension of the feature channels. Then, a depthwise separable convolution with a 3×3 convolution kernel size is used to extract residual edges and feature information. Then, the multi-scale normalized channel attention mechanism layer MANCAM layer constructed in step S3 is used to extract feature channel weights. Finally, a convolution operation with a 1×1 convolution kernel size is used to reduce the dimension of the feature map and restore the channel size of the input features to obtain the deep feature extraction network Deep_bneck. S4.1.

3. Construct a normalized multi-attention feature extraction network NSCPANet, which is based on the NSCPANet model of the MobileNetV3 model. First, perform a convolution operation Covn2d on the input image data with a convolution kernel size of 3×3, a step size of 2, and a feature map of 16. Then perform batch normalization BatchNorm2d and activation function h_swish operation to perform 1 / 2 feature downsampling to obtain the first output feature out1; the first output feature out1 is input into Shallow_bneck for 1 / 4 and 1 / 8 feature downsampling. Sampling is performed to obtain the second output feature out2 and the third output feature out3, and then the third output feature out3 is input into Deep_bneck for 1 / 16 feature downsampling to obtain the fourth output feature out4, and then the fourth output feature out4 is input into Deep_bneck for 1 / 32 feature downsampling, and then the convolution operation with a 3×3 convolution kernel size, a step size of 2, and a feature map of 960 is used for feature expansion, and then batch normalization BatchNorm2d and activation function h_swish are performed to obtain the fifth output feature out5; S4.

2. Construct a multi-deep and shallow feature fusion dilated convolutional network MNSDDNet; S4.2.

1. Construct a deep and shallow feature fusion method SDFF. The deep and shallow feature fusion method SDFF uses the normalized multi-attention feature extraction network NSCPANet constructed in step S4.1 to downsample the input data by 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 respectively. Then, the downsampled 1 / 2, 1 / 8, 1 / 16, and 1 / 32 features are respectively subjected to the convolution algorithm Covn2d, 1×1, and upsampling algorithm to restore the number of channels and size to the same as the 1 / 4 downsampling and connect them to complete the deep and shallow feature fusion. The calculation expression is: FT=Cat(interpolate(∫ 1×1 (FT 1 / 2 ,FT 1 / 8 ,FT 1 / 16 ,FT 1 / 32 )),FT 1 / 4 ) Among them, FT 1 / 2 Indicates the features of downsampling 1 / 2, FT 1 / 4 Indicates the feature of downsampling 1 / 4, FT 1 / 8 Indicates the feature of downsampling 1 / 8, FT 1 / 16 Indicates the feature of downsampling 1 / 16, FT 1 / 32 represents the feature of downsampling 1 / 32, ∫ 1×1 Represents the convolution operation of the 1×1 convolution kernel, and interpolate represents the sampling operation on the enhanced feature edge; S4.2.

2. Construct a multi-deep and shallow feature fusion void convolution network MNSDDNet, which is constructed based on the MNSDDNet model of the DeeplabV3+ model. First, the fifth output feature out5 obtained in step S4.1 is subjected to void spatial convolution pooling pyramid operation and feature connection operation to obtain feature x. Then, the deep and shallow feature fusion method SDFF constructed in S4.2.1 is used to fuse the features of x, out1, out2, out3, and out4 to obtain x1. The convolution of x1 is performed with a convolution kernel size of 1×1, a step size of 1, and a feature map of the number of classifications num_classes. Then, the interpolate upsampling method is used to restore the original feature size for land use classification prediction.

6. A comprehensive evaluation method for land classification according to claim 5, characterized in that: The specific implementation method of step S5 includes the following steps: S5.

1. Collect data from the urban-rural domain adaptive land cover dataset and modify the original dataset size of 1024×1024 to 512×512. S5.

2. Label the data in the modified dataset, where 0 represents no data area, 1 represents background, 2 represents urban and rural residential land, 3 represents road, 4 represents water area, 5 represents unused land, 6 represents forest land, and 7 represents cultivated land; S5.

3. Perform rotation, mirroring, blurring, and noise preprocessing on the labeled dataset to obtain the land dataset.

7. A comprehensive evaluation method for land classification according to claim 6, characterized in that: Step S6 inputs the land dataset obtained in step S5 into the land use classification network model constructed in step S4, performs land classification training based on the fusion of deep and shallow features and multi-attention mechanisms, and completes the training using the dice_loss loss function, sgd optimization algorithm, and cos loss attenuation strategy to obtain a land classification model based on the fusion of deep and shallow features and multi-attention mechanisms.

8. A comprehensive evaluation method for land classification according to claim 7, characterized in that: The specific implementation method of step S7 includes the following steps: S7.

1. Construct a confusion matrix using the predicted classification categories of the land dataset prediction set obtained in step S6 and the actual sample label classification categories of the dataset to obtain a classification confusion matrix table, where the sum of each row in the classification confusion matrix table is the actual number of samples in the classification, and the sum of each column is the number of samples predicted to be classified; S7.

2. Perform mean intersection over union (mIoU) accuracy evaluation based on the confusion matrix. Calculate IoU0, IoU1, IoU2, IoU3, IoU4, IoU5, IoU6, and IoU7 for the seven categories using the confusion matrix. Then calculate the average intersection over union (mIoU), and the calculation expression is: Then the calculation expression of mIoU is: Among them, k represents the number of categories, p ij Indicates that i is predicted to be j, which is a false negative FN; p ij Indicates that j is predicted to be i, which is a false positive FP; p ii It means that i is predicted to be i, which is the true TP; S7.

3. Perform category average pixel accuracy (mPA) evaluation based on the confusion matrix. Calculate the pixel accuracy of seven categories (CPA0, CPA1, CPA2, CPA3, CPA4, CPA5, CPA6, and CPA7) using the confusion matrix. Then calculate the average CPA of each category to get mPA, the calculation expression is: Then the calculation expression of mPA is: Among them, k represents the number of categories, i represents the classification category, and CPA represents the category pixel accuracy; S7.

4. Calculate the overall classification accuracy OA, which is the sum of correctly classified pixels divided by the total number of pixels. The number of correctly classified pixels is distributed along the diagonal of the confusion matrix, and the total number of pixels is the total number of pixels with sample labels. The calculation expression is: Among them, k represents the number of categories, i represents the actual classification category, j represents the predicted classification category, and p ij Indicates that i is predicted to be j, which is a false negative; p ii It means that i is predicted to be i, which is the true TP; S7.

5. Calculate the accuracy, which is the ratio of the number of correctly classified positive samples to the total number of positive samples in the predicted classification. The calculation expression is: Among them, TP represents the positive samples that are correctly predicted and classified, i.e. true positives, and FP represents the negative samples that are incorrectly predicted and classified, i.e. false positives; S7.

6. Calculate the recall rate (Recall), which is the ratio of the number of correctly classified positive samples to the total number of positive samples. The calculation expression is: Among them, TP represents the positive samples that are correctly predicted and classified, i.e. true positive samples, and FN represents the positive samples that are incorrectly predicted and classified, i.e. false negative positive samples; S7.

7. Calculate the F1 score, which is the harmonic mean of precision and recall. The calculation expression is: Among them, Precision represents the accuracy rate and Recall represents the recall rate.

9. An electronic device, characterized in that The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of a comprehensive evaluation method for land classification as described in any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the comprehensive evaluation method for land classification according to any one of claims 1 to 8 is implemented.

Images