Soil classification method based on prototype correction and feature distance enhancement

Abstract

The invention provides a soil classification method based on prototype correction and feature distance enhancement, and the method comprises the following steps: S1, collecting image information of sample soil, segmenting the image information of the sample soil, and removing a background image to obtain a sample soil image; s2, dividing the sample soil image into a support set S and a query set Q; s3, constructing an improved prototype network; s4, dividing a sample soil image into a support set S and a query set Q, and inputting the support set S and the query set Q into the improved prototype network to train a soil classification model; and S5, acquiring image information of to-be-detected soil, repeating the processing processes in the step S1 and the step S2 on the image information of the to-be-detected soil to obtain a support set S and a query set Q of an image of the to-be-detected soil, and inputting the support set S and the query set Q of the image of the to-be-detected soil into the trained soil type classification model to obtain a final soil type classification result.

Description

Technical Field

[0001] The invention relates to a soil type classification method, in particular to a soil type classification method based on prototype correction and feature distance enhancement. Background Art

[0002] Accurate soil classification is the basic work for agricultural resource survey and crop planting.

[0003] In the prior art, machine vision is generally used to identify and classify soil types. The neural network involved in the existing machine vision processing method requires a large number of sample images to enable the neural network's own parameter update to meet the accuracy requirements. When the number of samples is small, the existing neural network cannot achieve accurate identification of soil types.

[0004] Therefore, a prototype network is also proposed in the prior art. The prototype network divides the sample image into a query set and a support set, and extracts features through a feature extractor (such as a RESNET network, a CNN network, etc.) to form query set features and support set features. Then, the Euclidean distance between the support set features and the query set features is used as a metric to determine the final soil classification result. However, the accuracy of the classification result of this method is still low. This is because when a certain support set image feature deviates, this will lead to the accuracy of the classification result. On the other hand, when the distinguishing features between the query set features and the support set features are not obvious, the accuracy of the final result will also be reduced.

[0005] Therefore, in order to solve the above technical problems, it is urgent to propose a new technical means. Summary of the invention

[0006] In view of this, the purpose of the present invention is to provide a soil type classification method based on prototype correction and feature distance enhancement. An adaptive cosine similarity enhancement module and a separable feature distance enhancement module are designed in the existing prototype network. The prototype is reconstructed to form a class prototype based on the adaptive cosine similarity enhancement module, thereby avoiding the deviation of the support set image features. In addition, the separable feature distance enhancement module increases the expressiveness of the metric function on the difference of class features, thereby effectively improving the accuracy of the final soil type identification.

[0007] The present invention provides a soil type classification method based on prototype correction and feature distance enhancement, comprising the following steps:

[0008] S1. Collecting image information of sample soil, and segmenting the image information of the sample soil to remove the background image to obtain a sample soil image;

[0009] S2. Divide the sample soil image into a support set S and a query set Q;

[0010] S3. Constructing an improved prototype network, the improved prototype network includes a feature extractor, an adaptive cosine similarity enhancement module, a separable feature distance enhancement module, a feature reconstruction module and a metric module;

[0011] The feature extractor inputs a support set S and a query set Q and extracts support set features and query set features, the support set features and query set features are input into an adaptive cosine similarity enhancement module for processing to obtain an adaptive cosine similarity enhancement mask, the adaptive cosine similarity enhancement mask and the support set features are input into a feature reconstruction module to obtain a class prototype feature, the class prototype feature and the query set feature are input into a Euclidean distance module to obtain the Euclidean distance between the class prototype feature and the query set feature, the Euclidean distance is input into a separable feature distance enhancement module for processing to obtain a feature weight, the Euclidean distance between the class prototype feature and the query set feature and the feature weight are input into a metric module to reconstruct a metric function, and a final soil species classification probability is obtained from the metric function;

[0012] S4. Divide the sample soil image into a support set S and a query set Q and input them into the improved prototype network to train the soil type classification and recognition model. During the training, the obtained soil type classification probability is fed back to the feature extractor and the parameters of the feature extractor are updated;

[0013] S5. Obtain the image information of the soil to be tested, and repeat the processing in steps S1 and S2 for the image information of the soil to be tested to obtain the support set S and query set Q of the soil image to be tested, and input the support set S and query set Q of the soil image to be tested into the trained soil type classification and recognition model to obtain the final soil type classification result.

[0014] Further, the adaptive cosine similarity enhancement module includes a cosine similarity calculation unit, a K-MAX pooling unit and a multi-layer perceptron;

[0015] The cosine similarity calculation unit is used to calculate the cosine similarity between the query set features and the support set features;

[0016] The cosine similarity calculation unit inputs the calculation result into the K-MAX pooling unit to obtain the maximum u cosine similarities, and inputs them into the multi-layer perceptron unit, which outputs an adaptive cosine similarity enhancement mask.

[0017] Further, the cosine similarity calculation unit calculates the cosine similarity by the following method:

[0018]

[0019] Where: R q,s represents the cosine similarity, R q,s =[r1,r2,…,rWH ] T , represents the similarity between the local area of ​​the support set sample and each local area feature of the query image, W represents the width of the image, H represents the height of the image, and Z s represents the support set image features, ||.||2 represents the modulus length, Z q Represents the query set image features.

[0020] Further, the K-MAX pooling unit selects K maximum values ​​specifically including:

[0021]

[0022] in:

[0023] Furthermore, the multi-layer perceptron unit determines the adaptive cosine similarity enhancement mask by the following method:

[0024]

[0025] σ represents the activation function and τ is the temperature coefficient. represents the adaptive cosine similarity enhancement mask, whose set can be expressed as:

[0026]

[0027] Further, the feature reconstruction module processing process is as follows:

[0028] Reconstruct the support set image feature map:

[0029]

[0030] in: Represents the reconstructed support set image feature map;

[0031] Constructing prototypes with feature space differentiation:

[0032]

[0033] Where: C n Represents a prototype with feature space differences; y s represents the support set image category label, n represents the category, and K represents the number of support set samples in the corresponding category.

[0034] Furthermore, the processing process of the separable feature distance enhancement module is as follows:

[0035] Construct a feature distance matrix between the query set image features and the prototypes with feature space differences:

[0036]

[0037] in: N represents the number of categories, and C represents the number of feature channels;

[0038] Calculate feature distance variance:

[0039]

[0040] in: represents the distance variance of the l-th dimension feature, represents the mean distance of the l-th dimension feature, and D(.) represents the Euclidean distance;

[0041] Normalize the feature distance variance:

[0042]

[0043] in: and Represent the mean, maximum and minimum values ​​of feature distance variance respectively;

[0044] Determine feature weights:

[0045] in: is the weight value of the l-th dimension feature.

[0046] Furthermore, the measurement module reconstructs the measurement function by the following method:

[0047]

[0048] in: Represents the query set image and class prototype C n The Euclidean distance between the l-th dimension features.

[0049] Furthermore, the soil type classification probability is determined by the following method:

[0050]

[0051] The beneficial effects of the present invention are as follows: through the present invention, an adaptive cosine similarity enhancement module and a separable feature distance enhancement module are designed in the existing prototype network, and the prototype is reconstructed to form a class prototype based on the adaptive cosine similarity enhancement module, thereby avoiding the deviation of the support set image features, and the separable feature distance enhancement module increases the expressiveness of the metric function on the difference of class features, thereby effectively improving the accuracy of the final soil type identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0053] Figure 1It is a flow chart of the present invention.

[0054] Figure 2 This is a schematic diagram of the improved prototype network structure of the present invention.

[0055] Figure 3 This is a curve showing the accuracy change of the model test with different K values ​​in a specific example of the present invention.

[0056] Figure 4 This is the accuracy change curve of the verification of the present invention and the comparison network model.

[0057] Figure 5 It is a soil image in a specific example. DETAILED DESCRIPTION

[0058] The present invention is further described in detail below:

[0059] The present invention provides a soil type classification method based on prototype correction and feature distance enhancement, comprising the following steps:

[0060] S1. Collecting image information of sample soil, and segmenting the image information of the sample soil to remove the background image to obtain a sample soil image;

[0061] S2. Divide the sample soil image into a support set S and a query set Q;

[0062] S3. Constructing an improved prototype network, the improved prototype network includes a feature extractor, an adaptive cosine similarity enhancement module, a separable feature distance enhancement module, a feature reconstruction module and a metric module;

[0063] The feature extractor inputs a support set S and a query set Q and extracts support set features and query set features, the support set features and query set features are input into an adaptive cosine similarity enhancement module for processing to obtain an adaptive cosine similarity enhancement mask, the adaptive cosine similarity enhancement mask and the support set features are input into a feature reconstruction module to obtain a class prototype feature, the class prototype feature and the query set feature are input into a Euclidean distance module to obtain the Euclidean distance between the class prototype feature and the query set feature, the Euclidean distance is input into a separable feature distance enhancement module for processing to obtain a feature weight, the Euclidean distance between the class prototype feature and the query set feature and the feature weight are input into a metric module to reconstruct a metric function, and a final soil species classification probability is obtained from the metric function;

[0064] S4. Divide the sample soil image into a support set S and a query set Q and input them into the improved prototype network to train the soil type classification and recognition model. During the training, the obtained soil type classification probability is fed back to the feature extractor and the parameters of the feature extractor are updated;

[0065] S5. Obtain the image information of the soil to be tested, and repeat the processing in step S1 and step S2 for the image information of the soil to be tested to obtain the support set S and query set Q of the soil image to be tested, and input the support set S and query set Q of the soil image to be tested into the trained soil type classification and recognition model to obtain the final soil type classification result. Through the above, an adaptive cosine similarity enhancement module and a separable feature distance enhancement module are designed in the existing prototype network, and the prototype is reconstructed to form a class prototype based on the adaptive cosine similarity enhancement module, thereby avoiding the deviation of the support set image features, and the separable feature distance enhancement module increases the expressiveness of the metric function on the difference of the class features, thereby effectively improving the accuracy of the final soil type recognition.

[0066] For the existing prototype network, Snell et al. (2017) proposed a small sample learning prototype network. Assume that F θ is a feature extractor with trainable parameters θ, then it supports image features Z s =F θ (x s ), query image feature Z q =F θ (x q ),and Among them, C, H, and W represent the number of channels, height, and width of the feature map respectively. n Defined as:

[0067]

[0068] Among them, n represents the category, and K is the number of supporting samples in the corresponding category.

[0069] The prototype network uses the distance measurement to measure the difference between the prototype and the query image. That is, the measurement function D(·) is defined as:

[0070]

[0071] in Represents Z q The l-th dimension feature of .

[0072] Classification results of the model The model uses mean square error (MSE) as the loss function:

[0073]

[0074] in, Represents y q It is 1 if it is equal to n, otherwise it is 0.

[0075] Among them, when the feature extractor adopts the existing RESNET network (residual neural network) or CNN network (convolutional neural network), its loss function adopts the above-mentioned mean square error MSE as the loss function.

[0076] In this embodiment, the adaptive cosine similarity enhancement module includes a cosine similarity calculation unit, a K-MAX pooling unit and a multi-layer perceptron;

[0077] The cosine similarity calculation unit is used to calculate the cosine similarity between the query set features and the support set features;

[0078] The cosine similarity calculation unit inputs the calculation result into the K-MAX pooling unit to obtain the maximum u cosine similarities, and inputs them into the multi-layer perceptron unit, which outputs an adaptive cosine similarity enhancement mask.

[0079] Wherein: the cosine similarity calculation unit calculates the cosine similarity by the following method:

[0080]

[0081] Where: R q,s represents the cosine similarity, R q,s =[r1,r2,…,r WH ] T , represents the similarity between the local area of ​​the support set sample and each local area feature of the query image, W represents the width of the image, H represents the height of the image, and Z s represents the support set image features, ||.||2 represents the modulus length, Z q Represents the query set image features.

[0082] The K-MAX pooling unit selects K maximum values ​​specifically including:

[0083]

[0084] in:

[0085] The multi-layer perceptron unit determines the adaptive cosine similarity enhancement mask as follows:

[0086]

[0087] σ represents the activation function, τ is the temperature coefficient, and are the weights of each layer of the multilayer perceptron. represents the adaptive cosine similarity enhancement mask, whose set can be expressed as:

[0088]

[0089] Reconstruct the support set image feature map:

[0090]

[0091] in: Represents the reconstructed support set image feature map;

[0092] Constructing prototypes with feature space differentiation:

[0093]

[0094] Where: C n It represents a prototype with feature space differences, that is, prototype-like features. The construction process is the correction process of the prototype; y s Represents the support set image category label, n represents the category, and K represents the number of support set samples in the corresponding category. Through the above, the similarity between the query image and the support image is used to construct a local similarity enhancement mask, which masks the feature blocks with weak similarity in the support image. The reconstructed support image features enhance the spatial differences between support images of different categories, thereby increasing the distance between class prototypes, and reconstructing the prototype network to form a class prototype network, which can effectively improve the accuracy of the final soil species recognition.

[0095] In this embodiment, the processing process of the separable feature distance enhancement module is as follows:

[0096] Construct a feature distance matrix between the query set image features and the prototypes with feature space differences:

[0097]

[0098] in: N represents the number of categories, and C represents the number of feature channels;

[0099] Calculate feature distance variance:

[0100]

[0101] in: represents the distance variance of the l-th dimension feature, represents the mean distance of the l-th dimension feature, and D(.) represents the Euclidean distance;

[0102] Normalize the feature distance variance:

[0103]

[0104] in: and Represent the mean, maximum and minimum values ​​of feature distance variance respectively;

[0105] Determine feature weights:

[0106] in: is the weight value of the l-th dimension feature.

[0107] Among them: The measurement module reconstructs the measurement function through the following method:

[0108]

[0109] in: Represents the query set image and class prototype C n The Euclidean distance between the l-th dimension features.

[0110] The soil classification probability is determined by the following method:

[0111] This probability will be fed back to the feature extractor for parameter update when the prototype network training is improved. When the prototype network training is completed, the classification probability is the final soil classification result when the soil image to be tested is input. The above reconstruction of the image feature map by feature weight regression can effectively improve the final recognition accuracy.

[0112] The present invention is further described below with specific examples:

[0113] (1) Experiment 1: Experiment on the influence of K-MAX pooling with different K values ​​on the soil classification accuracy of the improved prototype network of the present invention

[0114] Different K values ​​are selected for K-MAX pooling of the adaptive cosine similarity enhancement mask module to obtain a better K value.

[0115] (2) Experiment 2: Comparative experiment on soil classification accuracy

[0116] The improved prototype network of the present invention is compared with small sample classification networks such as MatchingNet (Vinyals et al., 2016), ProtoNet (Snell et al., 2017), MAML (Finn et al., 2017), RelationNet (Sung et al., 2018), DSN (Simon et al., 2020), RENet (Kang et al., 2021) and ReNAP (Li et al., 2023) to verify the classification accuracy of the improved prototype network of the present invention.

[0117] (3) Experiment 3: Model ablation experiment

[0118] Taking ResNet-18 as the feature extraction module, the effectiveness of the adaptive cosine similarity enhancement mask and separability feature distance enhancement module in the improved prototype network of the present invention is verified.

[0119] The improved prototype network of the present invention is trained with different K values ​​(K is [1, 2, 3, 4, 5]), the model verification accuracy and test accuracy are calculated, and a better K value is obtained. The accuracy change curve of the improved prototype network of the present invention under different K values ​​is as follows Figure 3 As shown:

[0120] Figure 3 It shows that after the model converges, the model verification accuracy of K=3 is the highest. When K is less than 3, the test accuracy of the improved prototype network of the present invention increases with the increase of K value, which is mainly affected by the number of similar areas between the support image and the query image. When K is greater than 3, the test accuracy decreases with the increase of K value. The main reason is that as the K value increases, the adaptive cosine similarity enhancement mask introduces areas with less similarity, which increases the redundant features in the support image, resulting in a decrease in the similarity between the reconstructed support image and the query image, thereby reducing the classification accuracy. K=3 is the optimal parameter of the improved prototype network of the present invention for classifying soil species, and is used as the K value for subsequent experiments.

[0121] Due to the limitation of the number of categories (soil types), the number of support set categories is set to 4, the number of epochs is set to 200, and the improved prototype network of the present invention is trained to compare the network model with that of Experiment 2. After each epoch of training is completed, the model is repeatedly verified 100 times by the verification set, and the average value is calculated as the verification accuracy. The verification accuracy change curve during the training process is shown in Figure 4 As shown:

[0122] After the training is completed, the model is tested 600 times using the test set, and the average value is calculated as the test accuracy.

[0123] Table 1. Soil classification test accuracy of the improved prototype network and the comparative network model in Experiment 2

[0124]

[0125] * Experimental parameters, soil sub-map size: 224*224; Batchsize: 16; initial learning rate: 10 -3 ; Optimizer: Adam; Confidence: 95%.

[0126] Figure 4It shows that the improved prototype network of the present invention converges at 90 epochs and 130 epochs in 1-shot and 5-shot training respectively, and the verification accuracy after convergence is higher than that of other small sample classification models. At the same time, during the entire training process, the verification accuracy of the improved prototype network of the present invention is always higher than that of the original prototype network ProtoNet, indicating that the improved prototype network of the present invention can effectively extract the key features of soil images during the training process. Before the model converges, 1-shot training uses only one image as a guide, and the support set cannot provide sufficient reference information to guide the query image classification. The probability of misclassification is also relatively large, resulting in large fluctuations in the verification accuracy change curve.

[0127] The experimental data results in Table 2 show that the classification accuracy of the improved prototype network of the present invention after 1-shot and 5-shot training is greater than that of other comparison network models, indicating that it is superior to other small sample classification models in soil image classification tasks and can achieve higher accuracy soil classification. Compared with the original prototype network ProtoNet, the soil classification of the improved prototype network of the present invention after 1-shot training is improved by 14.93%, and the soil classification of the network after 5-shot training is improved by 16.97%, proving that the improved prototype network of the present invention constructed by improving the prototype network ProtoNet with adaptive cosine similarity enhancement mask and separability feature distance enhancement module is effective and can greatly improve the accuracy of soil classification.

[0128] In order to prove the effectiveness of the adaptive cosine similarity enhancement mask (ACSEM) and the separable feature distance enhancement module (DFDE), the components of the improved prototype network of the present invention are removed by stripping to conduct ablation experiments. The experiment includes four models: prototype network (ProtoNet), prototype network plus adaptive cosine similarity enhancement mask (ProtoNet+ACSEM), prototype network plus separable feature distance enhancement module (ProtoNet+DFDE) and improved prototype network of the present invention. The above four models are trained using the training set and the validation set. After the training is completed, the model is tested 600 times by the test set, and the average test accuracy is calculated. Ablation experiment results:

[0129] Table 2. Soil classification accuracy in ablation experiment 3

[0130]

[0131] Table 2 shows that compared with ProtoNet, the test accuracy of ProtoNet+ACSEM is improved by 9.64% (1-shot) and 14.04% (5-shot), respectively. ACSEM enhances the spatial difference between the support images of different categories, so that the distance between class prototypes increases, and the classification accuracy is improved. Compared with ProtoNet, the test accuracy of ProtoNet+DFDE is improved by 5.91% (1-shot) and 9.63% (5-shot), respectively. DFDE increases the expressiveness of the distance metric function on the difference of class features, thereby improving the classification accuracy of the model. Compared with ProtoNet, ProtoNet+ACSEM and ProtoNet+DFDE, the improved prototype network of the present invention is improved by 14.93%, 5.29% and 9.02% in 1-shot, and by 16.97%, 2.93% and 7.34% in 5-shot. For the classification of small sample soil species, increasing the distance between class prototypes and enhancing distinguishability features are both crucial. Therefore, the improved prototype network constructed by combining ACSEM and DFDE can further improve the accuracy of soil type classification.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A soil type classification method based on prototype correction and feature distance enhancement, characterized in that: The following steps are involved: S1. Collecting image information of sample soil, and segmenting the image information of the sample soil to remove the background image to obtain a sample soil image; S2. Divide the sample soil image into a support set S and a query set Q; S3. Constructing an improved prototype network, the improved prototype network includes a feature extractor, an adaptive cosine similarity enhancement module, a separable feature distance enhancement module, a feature reconstruction module and a metric module; The feature extractor inputs a support set S and a query set Q and extracts support set features and query set features, the support set features and query set features are input into an adaptive cosine similarity enhancement module for processing to obtain an adaptive cosine similarity enhancement mask, the adaptive cosine similarity enhancement mask and the support set features are input into a feature reconstruction module to obtain a class prototype feature, the class prototype feature and the query set feature are input into a Euclidean distance module to obtain the Euclidean distance between the class prototype feature and the query set feature, the Euclidean distance is input into a separable feature distance enhancement module for processing to obtain a feature weight, the Euclidean distance between the class prototype feature and the query set feature and the feature weight are input into a metric module to reconstruct a metric function, and a final soil species classification probability is obtained from the metric function; S4. Divide the sample soil image into a support set S and a query set Q and input them into the improved prototype network to train the soil type classification and recognition model. During the training, the obtained soil type classification probability is fed back to the feature extractor and the parameters of the feature extractor are updated; S5. Obtain the image information of the soil to be tested, and repeat the processing in steps S1 and S2 for the image information of the soil to be tested to obtain the support set S and query set Q of the soil image to be tested, and input the support set S and query set Q of the soil image to be tested into the trained soil type classification and recognition model to obtain the final soil type classification result.

2. The soil type classification method based on prototype correction and feature distance enhancement according to claim 1 is characterized in that: The adaptive cosine similarity enhancement module includes a cosine similarity calculation unit, a K-MAX pooling unit and a multi-layer perceptron; The cosine similarity calculation unit is used to calculate the cosine similarity between the query set features and the support set features; The cosine similarity calculation unit inputs the calculation result into the K-MAX pooling unit to obtain the maximum u cosine similarities, and inputs them into the multi-layer perceptron unit, which outputs an adaptive cosine similarity enhancement mask.

3. The soil type classification method based on prototype correction and feature distance enhancement according to claim 2 is characterized in that: The cosine similarity calculation unit calculates the cosine similarity by the following method: Where: R q,s represents the cosine similarity, R q,s =[r1,r2,…,r WH ] T , represents the similarity between the local area of ​​the support set sample and each local area feature of the query image, W represents the width of the image, H represents the height of the image, and Z s represents the support set image features, ||.||2 represents the modulus length, Z q Represents the query set image features.

4. The soil type classification method based on prototype correction and feature distance enhancement according to claim 3 is characterized in that: The K-MAX pooling unit selects K maximum values ​​specifically including: in:

5. The soil type classification method based on prototype correction and feature distance enhancement according to claim 4 is characterized in that: The multi-layer perceptron unit determines the adaptive cosine similarity enhancement mask as follows: σ represents the activation function and τ is the temperature coefficient. represents the adaptive cosine similarity enhancement mask, whose set can be expressed as:

6. The soil type classification method based on prototype correction and feature distance enhancement according to claim 5 is characterized in that: The feature reconstruction module processing process is as follows: Reconstruct the support set image feature map: in: Represents the reconstructed support set image feature map; Constructing prototypes with feature space differentiation: Where: C n Represents a prototype with feature space differences, y s represents the support set image category label, n represents the category, and K represents the number of support set samples in the corresponding category.

7. The soil type classification method based on prototype correction and feature distance enhancement according to claim 6 is characterized in that: The processing process of the separable feature distance enhancement module is as follows: Construct a feature distance matrix between the query set image features and the prototypes with feature space differences: in: N represents the number of categories, and C represents the number of feature channels; Calculate feature distance variance: in: represents the distance variance of the l-th dimension feature, represents the mean distance of the l-th dimension feature, and D(.) represents the Euclidean distance; Normalize the feature distance variance: in: and Represent the mean, maximum and minimum values ​​of feature distance variance respectively; Determine feature weights: in: is the weight value of the l-th dimension feature.

8. The soil type classification method based on prototype correction and feature distance enhancement according to claim 7 is characterized in that: The measurement module reconstructs the measurement function in the following way: in: Represents the query set image and class prototype C n The Euclidean distance between the l-th dimension features.

9. The soil type classification method based on prototype correction and feature distance enhancement according to claim 8 is characterized in that: The soil classification probability is determined by the following method:

Images