Soil image recognition method based on multi-color channel and PH channel attention mechanism

Abstract

The invention provides a soil image recognition method based on a multi-color channel and PH channel attention mechanism, and the method comprises the steps: S1, obtaining a sample soil image, and determining the PH value of the sample soil; s2, respectively mapping the sample soil image to an LAB color space, an HSL color space, an HSV color space and a YUV color space from an RGB color space, and removing brightness components in the LAB color space, the HSL color space, the HSV color space and the YUV color space to form first input information; s3, determining a correlation coefficient between each sub-graph and the soil PH value under the RGB three color channels; the PH value, the correlation coefficient and the subgraph of the sample soil image form second input information; s4, constructing a soil image classification prediction model; S5, inputting the first input information and the second input information into the soil image classification prediction model, and training the soil image classification prediction model; and S6, obtaining an image of to-be-detected soil, determining first input information and second input information of the to-be-detected soil image through the steps S1 to S3, and inputting the first input information and the second input information into the trained soil image classification prediction model to obtain a final soil classification result.

Description

Technical Field

[0001] The present invention relates to a soil classification and recognition method, and in particular to a soil image recognition method based on a multi-color channel and PH channel attention mechanism. Background Art

[0002] Soil classification is an important part of agricultural production. In the prior art, soil type identification is generally based on soil and uses neural networks to identify the soil, which can improve the efficiency of soil type identification. However, the recognition accuracy of the processing method in the prior art is relatively low.

[0003] This is because although soils of different soil categories have great differences in physical and chemical properties, the soil images collected are relatively similar, resulting in a lower accuracy rate in classification and recognition. On the other hand, the collection of soil images is affected by factors such as lighting conditions and the time span when the image is acquired, so that the brightness of the image has a great impact on the accuracy of soil type recognition, resulting in the inability to accurately identify the soil type.

[0004] Therefore, in order to solve the above technical problems, it is urgent to propose a new technical means. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a soil image recognition method based on multi-color channel and PH channel attention mechanism, which maps the soil image into different color spaces, eliminates the brightness dimension to form the first input information, and also considers the influence of PH value on soil color to establish the second input information, and establishes a soil recognition prediction model, and trains the soil recognition prediction model with the first input information and the second input information, thereby effectively avoiding the influence of brightness and PH on the recognition accuracy, effectively improving the accuracy of the final soil identification, and the whole process is relatively simple, adaptable and robust.

[0006] The present invention provides a soil image recognition method based on a multi-color channel and PH channel attention mechanism, comprising the following steps:

[0007] S1. Obtain a sample soil image and determine the pH value of the sample soil;

[0008] S2. Mapping the sample soil image from the RGB color space to the LAB color space, HSL color space, HSV color space and YUV color space, and removing the brightness component in the LAB color space, HSL color space, HSV color space and YUV color space to form the first input information;

[0009] S3. Divide each original image of the sample soil image into N sub-images, and determine the correlation coefficient between each sub-image and the soil pH value under the three RGB color channels; the pH value, the correlation coefficient and the sub-image of the sample soil image form the second input information;

[0010] S4. Construct a soil image classification prediction model, the prediction model includes a first Vision Transfomer neural network, a second Vision Transfomer neural network and a PH channel attention mechanism module; wherein the outputs of the first Vision Transfomer neural network and the second Vision Transfomer neural network are added to obtain a final classification result, the input end of the first Vision Transfomer neural network inputs the first input information, the PH channel attention mechanism module inputs the second input information, and the output end of the PH channel attention mechanism module is connected to the input end of the second Vision Transfomer neural network;

[0011] S5. Inputting the first input information and the second input information into the soil image classification prediction model and training the soil image classification prediction model until the training is completed;

[0012] S6. Obtain an image of the soil to be tested, and determine the first input information and the second input information of the soil image to be tested through steps S1 to S3 and input them into the trained soil image classification prediction model to obtain the final soil classification result.

[0013] Furthermore, the PH channel attention mechanism module includes a convolutional layer, a PH value processing module, and a sigmod module;

[0014] The pH value processing module multiplies the pH value of the soil by the correlation coefficient to obtain a pH characteristic;

[0015] The convolution layer includes a first convolution layer and a second convolution layer. The sub-image of the soil image is processed by the first convolution layer and then input into the second convolution layer. The output of the first convolution layer is multiplied by the PH feature and then added to the output result of the second convolution layer and then input into the sigmod module for processing. The output of the sigmod module is multiplied by the original image information of the sub-image of the soil image and then input into the second Vision Transfomer neural network for processing.

[0016] Furthermore, the loss function of the soil image classification prediction model is as follows:

[0017]

[0018] Where: q *Indicates the predicted probability value of the position corresponding to 1 in the label, q max Indicates that the predicted value is divided by q * The maximum predicted value outside the range, q(x i ) represents the predicted value, p(x i ) represents the label value of the sample image.

[0019] Beneficial effects of the present invention: Through the present invention, the soil image is mapped to different color spaces, and the brightness dimension is eliminated to form the first input information. The influence of the pH value on the soil color is also considered to establish the second input information, and a soil recognition prediction model is established. The first input information and the second input information are used to train the soil recognition prediction model, thereby effectively avoiding the influence of brightness and pH on the recognition accuracy, effectively improving the accuracy of the final soil identification, and the whole process is relatively simple, highly adaptable and robust. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0021] Figure 1 It is a flow chart of the present invention.

[0022] Figure 2 It is a schematic diagram of the soil image classification prediction model structure of the present invention.

[0023] Figure 3 Schematic diagram of the PH channel attention mechanism module structure of the present invention.

[0024] Figure 4 The histograms of the same soil image in different color spaces of the present invention.

[0025] Figure 5 This is an example sample soil image of the present invention.

[0026] Figure 6 This is a test accuracy curve diagram of the present invention and other existing algorithms.

[0027] Figure 7 is the confusion matrix. DETAILED DESCRIPTION

[0028] The present invention is further described in detail below:

[0029] The present invention provides a soil image recognition method based on a multi-color channel and PH channel attention mechanism, comprising the following steps:

[0030] S1. Obtain a sample soil image and determine the pH value of the sample soil;

[0031] S2. The sample soil image is mapped from the RGB color space to the LAB color space, the HSL color space, the HSV color space and the YUV color space, and the brightness component in the LAB color space, the HSL color space, the HSV color space and the YUV color space is removed to form the first input information; wherein the histograms of different color spaces are as follows Figure 4 As shown: From the figure, it can be seen that the brightness information in different color spaces is different. The main reason is that in the process of soil image processing, different color spaces have different structures and characteristics, and the performance and interpretation of brightness information are significantly different. Through the above processing, the interference of brightness is eliminated, so that the accuracy of the final result can be improved. Since each color space has a brightness component, after the brightness component is eliminated, each color space has two components left. Then, in the LAB color space, HSL color space, HSV color space and YUV color space, the brightness component is eliminated, and the remaining color components in the four spaces are left, a total of 8 components, thereby forming a feature vector and inputting it into the first Vision Transfomer neural network;

[0032] S3. Each original image of the sample soil image is divided into N sub-images, and the correlation coefficient between each sub-image and the soil pH value under the three RGB color channels is determined; the pH value, the correlation coefficient and the sub-image of the sample soil image form the second input information; different pH values ​​of the soil are correlated with each color channel in the RGB space of the soil image, taking purple soil as an example, its correlation coefficient is shown in Table 1:

[0033] Table 1 Correlation coefficient between pH value and RGB color channel

[0034] Color channels R G B Correlation coefficient -0.157 -0.421 -0.500

[0035] The correlation coefficients for different soils are different and can be determined using existing methods.

[0036] By analyzing the correlation coefficient table, we can see that the soil color channels R, G and B show different sensitivities when the pH value changes. The correlation coefficient between the red channel R and the pH value is -0.157, indicating that the R channel changes in the opposite direction to the pH value, that is, the pH value increases, and the R channel value decreases. In contrast, the sensitivity of the G channel is relatively high, that is, within the same pH change range, the G channel has a larger change amplitude. The change of the B channel has a higher sensitivity than the other two channels. The B channel is more sensitive to pH changes.

[0037] S4. Construct a soil image classification prediction model, the prediction model includes a first Vision Transfomer neural network, a second Vision Transfomer neural network and a PH channel attention mechanism module; wherein the outputs of the first Vision Transfomer neural network and the second Vision Transfomer neural network are added to obtain a final classification result, the input end of the first Vision Transfomer neural network inputs the first input information, the PH channel attention mechanism module inputs the second input information, and the output end of the PH channel attention mechanism module is connected to the input end of the second Vision Transfomer neural network;

[0038] S5. Inputting the first input information and the second input information into the soil image classification prediction model and training the soil image classification prediction model until the training is completed;

[0039] S6. Obtain an image of the soil to be tested, and determine the first input information and the second input information of the soil image to be tested through steps S1 to S3 and input them into the trained soil image classification prediction model to obtain the final soil classification result; through the above method, the soil image is mapped to different color spaces, and the brightness dimension is eliminated to form the first input information, and the influence of the pH value on the soil color is also considered to establish the second input information, and a soil recognition prediction model is established, and the first input information and the second input information are used to train the soil recognition prediction model, thereby effectively avoiding the influence of brightness and pH on the recognition accuracy, and effectively improving the accuracy of the final soil genus recognition, and the whole process is relatively simple, adaptable and robust.

[0040] It should be noted that the images mentioned above are all soil images after denoising and background removal.

[0041] The Vision Transfomer neural network can adopt the existing neural network structure, and its structure and principle will not be described in detail here.

[0042] In this embodiment, the PH channel attention mechanism module includes a convolutional layer, a PH value processing module and a sigmod module;

[0043] The pH value processing module multiplies the pH value of the soil by the correlation coefficient to obtain a pH characteristic;

[0044] The convolution layer includes a first convolution layer and a second convolution layer. The sub-image of the soil image is processed by the first convolution layer and then input into the second convolution layer. The output of the first convolution layer is multiplied by the pH feature and then added to the output result of the second convolution layer and then input into the sigmod module for processing. The output of the sigmod module is multiplied by the original image information of the sub-image of the soil image and then input into the second Vision Transfomer neural network for processing. Through the above processing, since the sensitivity of the soil color channels R, G and B to pH changes is inconsistent, when extracting soil image features, the reaction characteristics of the three channels are comprehensively considered, and different weights are assigned to each color channel by pH value and correlation coefficient, so as to obtain more comprehensive soil feature information, thereby ensuring the accuracy of the final result.

[0045] In this embodiment, the loss function of the soil image classification prediction model is as follows:

[0046]

[0047] Where: q * Indicates the predicted probability value of the position corresponding to 1 in the label, q max Indicates that the predicted value is divided by q * The maximum predicted value outside the range, q(x i ) represents the predicted value, p(x i ) represents the label value of the sample image. Loss is the loss function, which is the focused cross entropy loss function.

[0048] The following is a specific example:

[0049] In this example, data set 1 includes 122 images of dark purple clay (G23211), 226 images of gray-brown purple clay (G23212), and 98 images of red-brown purple clay (G23312), a total of 445 original images of soil with the background removed, and 3 soil genera.

[0050] Dataset 2: Using the subimage selection algorithm on the soil images in Dataset 1

[12] Cut sub-graphs, and obtain a total of 13666 sub-graphs, including 3701 dark purple clay (G23211), 7091 gray-brown purple clay (G23212), and 2874 red-brown purple clay (G23312). Take 80% of the three soil categories as the training set of the model and 20% as the test set of the model. The sample images of the above data sets are as follows: Figure 5 shown.

[0051] In order to verify the performance of the model, it is necessary to conduct experiments on the model. The various parts of the experiment are designed as follows.

[0052] Define variables: TP is the number of positive samples with correct prediction results. FP is the number of negative samples with incorrect prediction results. TN is the number of negative samples with correct prediction results. FN is the number of negative samples with incorrect prediction results.

[0053] Experimental evaluation criteria:

[0054] (1) Accuracy refers to the ratio of the number of correct predictions to the total number of predictions in the prediction results. The correct predictions are the total number of TPs and TNs. The higher the accuracy, the better the model effect. The calculation formula is:

[0055]

[0056] (2) Precision refers to the proportion of positive samples predicted as the result among all correctly predicted positive samples. The higher the precision, the higher the accuracy of the model in predicting positive samples. The calculation formula is:

[0057]

[0058] (3) Recall rate refers to the proportion of all positive samples whose prediction results are also positive samples. The higher the recall rate, the more positive samples the model can correctly identify and the better the recognition ability. The calculation formula is:

[0059]

[0060] (4) F1-Score takes into account both precision and recall, and is the harmonic mean of these two indicators. It has a more comprehensive model evaluation capability than either of these two indicators. Its calculation formula is:

[0061]

[0062] Design experiments to verify:

[0063] (1) Experiment 1: On Dataset 2, a comparative experiment was conducted with ResNet, MobileNetV2, MobileNetV3, MobileNetV4, EfficientNetV1, EfficientNetV2, VVT and Swin Transformer algorithms according to the experimental evaluation criteria to prove the advancedness of the algorithm proposed in this chapter in soil image recognition.

[0064] (2) Experiment 2: Use the optimal model in this chapter to verify the three indicators of Precision, Recall and F1-Score of the three categories of red-brown purple clay, dark purple clay and gray-brown purple clay on the test set, and characterize the confusion matrix.

[0065] Experimental parameter settings:

[0066] The Epoch number is 200, the initial learning rate is 0.0001, and the BatchSize is set to 8. GPU acceleration is used for all algorithms. Except for the algorithm in this paper, all other comparison algorithms use the cross entropy loss function.

[0067] Experiment 1 Comparative Experimental Results and Analysis,The comparative experiment conducted on Dataset 2 proves the advancedness of the model in this chapter. The final results of the four indicators of Accuracy, Precision, Recall and F1-score are shown in Table 2. The test set accuracy curve is shown in Figure 6 As shown:

[0068] Table 2 Comparative experimental results

[0069]

[0070]

[0071] Depend on Figure 6 It can be seen that the test accuracy of the method in the present invention (represented by ours in the figure) and the comparison model in the soil attribute recognition task of 0-20 cm surface soil images gradually improves with the increase of training times, and the test accuracy reaches a stable state after the 150th round of training.

[0072] In the experimental results, the algorithm proposed in this chapter has achieved a high level in Dataset 2. The specific reasons are as follows: ResNet improves the trainability of the network through residual connections, but in the soil and soil image recognition task, the data set is small, and ResNet's feature extraction ability is limited. When processing fine-grained images such as soil images, it is unable to fully extract its fine-grained features; MobileNet series networks are a lightweight neural network architecture that focuses on efficient computing on mobile and edge devices. MobileNetV2 and MobileNetV3 networks introduce deep separable convolutions to reduce the amount of computation and the number of parameters. Although the MobileNet series networks perform well in computational efficiency, they are relatively weak in feature expression capabilities. In the soil image recognition task, complex soil image features cannot be captured, so their performance is not as good as other more complex models. Based on MobileNetV3, MobileNetV4 uses a new activation function and a more flexible network structure, but its training process is not stable enough, especially on complex soil image datasets, where gradients are prone to vanishing or exploding, resulting in lower-than-expected performance in soil image recognition tasks; the EfficientNet series of networks is a convolutional neural network based on compound scaling, which improves efficiency and accuracy by systematically scaling the depth, width, and resolution of the network. Although EfficientNet's use of compound scaling can theoretically improve efficiency, it is very complicated to choose the right scaling ratio for different types of soil images. Although EfficientNet is a lightweight model, it takes longer and more resources to train than other lightweight networks. EfficientNet performs well on specific datasets, but due to the small size of the soil image dataset, the model's generalization ability is insufficient, resulting in poor results. Although EfficientNetV2 has optimized EfficientNet in many aspects, its complexity and high computational requirements make it a challenge for users to choose a suitable model to balance accuracy and efficiency in soil image recognition tasks. A local bias algorithm of two-dimensional Manhattan distance is introduced in transformer, including a feature reduction attention module and a feature preservation connection module to control computational overhead and improve feature extraction capabilities. However, the differences between classes in soil image datasets are usually small, and this model lacks the ability to dynamically adapt to features of different categories, resulting in poor performance in soil image recognition tasks. Swin Transformer is a visual Transformer model with a hierarchical structure. It uses a sliding window mechanism for image processing, has strong feature extraction capabilities, and has a relatively complex structure, requiring high computing resources and memory.In soil image classification tasks, especially when used on edge devices, its computational requirements may become a bottleneck. Compared with traditional convolutional neural networks, the training time of Swin Transformer is usually longer, which affects the work efficiency for soil image classification tasks that require fast iteration and optimization. This chapter proposes an algorithm that fully utilizes the color differences between soil species and uses the color channel features in different color spaces to increase the discrimination of soil image. And according to the correlation between soil pH and the three RGB channels of soil images combined with the ECA channel attention mechanism, weights are assigned to each channel, highlighting the characteristics of different color channels.

[0073] Experiment 2 Experimental results and analysis, the model trained in this chapter was verified on three soil types: dark purple clay, gray-brown purple clay and red-brown purple clay using the three indicators of Precision, Recall and F1-Score. The results are shown in Table 3. The model was also verified and analyzed by the corresponding confusion matrix. The confusion matrix verification result is shown in Figure 3. Figure 7 .

[0074] Table 3 Soil classification verification results

[0075]

[0076] The three indicators of Precision, Recall and F1-Score show that the present invention predicts positive samples. The present invention performs well in identifying dark purple clay samples, with fewer misclassified samples, indicating that the model has a strong recognition ability for this category. The recall rate is high, indicating that the model can effectively identify most of the dark purple clay samples, and there are very few missed samples. F1-Score comprehensively reflects the balance between precision and recall, indicating that the model performs well in the recognition of dark purple clay. The model has extremely high classification accuracy for gray-brown purple clay, and misclassification is almost non-existent. The recall rate is also close to 100%, indicating that the model has almost no missed samples and has strong recognition ability.

[0077] Figure 7 The confusion matrix shows that the number of correctly identified images in this chapter's model is high, the number of incorrectly identified images is small, and the number of images distributed in other classes is relatively even without clustering. This further shows the role of the focused cross entropy loss function.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A soil image soil attribute recognition method based on multi-color channel and PH channel attention mechanism, characterized by: The following steps are involved: S1. Obtain a sample soil image and determine the pH value of the sample soil; S2. Mapping the sample soil image from the RGB color space to the LAB color space, HSL color space, HSV color space and YUV color space, and removing the brightness component in the LAB color space, HSL color space, HSV color space and YUV color space to form the first input information; S3. Divide each original image of the sample soil image into N sub-images, and determine the correlation coefficient between each sub-image and the soil pH value under the three RGB color channels; the pH value, the correlation coefficient and the sub-image of the sample soil image form the second input information; S4. Construct a soil image classification prediction model, the prediction model includes a first Vision Transfomer neural network, a second Vision Transfomer neural network and a PH channel attention mechanism module; wherein the outputs of the first Vision Transfomer neural network and the second Vision Transfomer neural network are added to obtain a final classification result, the input end of the first Vision Transfomer neural network inputs the first input information, the PH channel attention mechanism module inputs the second input information, and the output end of the PH channel attention mechanism module is connected to the input end of the second Vision Transfomer neural network; S5. Inputting the first input information and the second input information into the soil image classification prediction model and training the soil image classification prediction model until the training is completed; S6. Obtain an image of the soil to be tested, and determine the first input information and the second input information of the soil image to be tested through steps S1 to S3 and input them into the trained soil image classification prediction model to obtain the final soil classification result.

2. The soil attribute recognition method of soil image based on multi-color channel and PH channel attention mechanism according to claim 1 is characterized by: The PH channel attention mechanism module includes convolutional layer, PH value processing module and sigmod module; The pH value processing module multiplies the pH value of the soil by the correlation coefficient to obtain a pH characteristic; The convolution layer includes a first convolution layer and a second convolution layer. The sub-image of the soil image is processed by the first convolution layer and then input into the second convolution layer. The output of the first convolution layer is multiplied by the PH feature and then added to the output result of the second convolution layer and then input into the sigmod module for processing. The output of the sigmod module is multiplied by the original image information of the sub-image of the soil image and then input into the second Vision Transfomer neural network for processing.

3. The soil attribute recognition method of soil image based on multi-color channel and PH channel attention mechanism according to claim 1 is characterized in that: The loss function of the soil image classification prediction model is as follows: Where: q * Indicates the predicted probability value of the position corresponding to 1 in the label, q max Indicates that the predicted value is divided by q * The maximum predicted value outside the range, q(x i ) represents the predicted value, p(x i ) represents the label value of the sample image.

Images