Soil erosion yellow mud image recognition method based on MobileNetV3

Abstract

The application provides a soil erosion yellow mud water image recognition method based on MobileNetV3, comprising the following steps: acquiring a multi-period heterogeneous data set of normal water quality and soil erosion yellow mud water images and performing pretreatment; constructing an enhanced MobileNetV3 small model fusing a multi-dimensional attention mechanism; training the small model by adopting a dynamic learning rate scheduling and a weight decay strategy; evaluating the small model after training based on a multi-dimensional evaluation framework, and combining Grad-CAM to output a multi-dimensional explainability analysis result. The enhanced MobileNetV3 small model of the application combines channel, spatial and spectral attention mechanisms, significantly improves the recognition ability of the model in a complex environment, makes the detection of the yellow mud water phenomenon more accurate and efficient, effectively improves the generalization ability and prediction accuracy of the model, and improves the model accuracy to 99.53%.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and environmental monitoring, and particularly relates to a soil erosion yellow mud water image recognition method based on MobileNetV3. BACKGROUND

[0002] With the rapid development of industrialization and urbanization, soil erosion problems are becoming increasingly serious, which has a significant impact on water quality. Traditionally, detecting whether water is affected by soil erosion mainly relies on physical and chemical analysis methods, which are not only time-consuming, labor-intensive and costly, but also require professional equipment and personnel to operate, making it difficult to achieve large-scale, low-cost and real-time monitoring.

[0003] In recent years, with the development of computer vision technology, especially the breakthrough of deep learning in image recognition, image-based water quality monitoring has become possible. However, existing image recognition methods still face challenges in dealing with water quality changes in complex backgrounds, such as changes in lighting conditions, water color similarity, etc., which directly affect the accuracy and reliability of the model. Moreover, although traditional convolutional neural networks (CNN) have achieved remarkable results in image recognition tasks, they often require high computational resources when dealing with high-resolution images or large-scale datasets, and are prone to overfitting. SUMMARY

[0004] To address the shortcomings of the prior art, the present application provides a soil erosion yellow mud water image recognition method based on MobileNetV3. By introducing a channel and spatial attention mechanism (CBAM) and a spectral attention module, the present application can effectively capture key features in images, thereby improving the model's recognition ability in complex environments. This solves the problem of high cost and low efficiency in large-scale, real-time monitoring of traditional water quality monitoring methods, and improves the recognition accuracy and frequency of river water quality changes, especially the yellow mud water phenomenon caused by soil erosion. It can timely discover and alarm the soil erosion yellow mud water.

[0005] The technical solution of the present application is a soil erosion yellow mud water image recognition method based on MobileNetV3, which includes the following steps:

[0006] S1), acquire a multi-period heterogeneous dataset of normal water quality and soil erosion yellow mud water images, and perform dynamic domain enhancement preprocessing on the multi-period heterogeneous dataset to divide it into a training set, a test set and a validation set;

[0007] S2), construct an enhanced MobileNetV3 small model that integrates multi-dimensional attention mechanisms;

[0008] S3), training the enhanced MobileNetV3 small model through the training set and adopting a dynamic learning rate schedule and a weight decay strategy, and dynamically saving the optimal weight through the validation set;

[0009] S4), evaluating the trained enhanced MobileNetV3 small model based on a multi-dimensional evaluation framework, and outputting a multi-dimensional explainability analysis result in combination with Grad-CAM.

[0010] Preferably, in step S2), the construction of the enhanced MobileNetV3 small model comprises introducing a hierarchical embedded channel-spatial attention module CBAM and a spectral attention module into the original MobileNetV3 architecture to strengthen the attention to important features.

[0011] Preferably, in step S2), the construction of the enhanced MobileNetV3 small model further comprises replacing the classifier of the last layer of the original MobileNetV3 architecture with a new classifier comprising two linear layers and a Hardswish activation function.

[0012] Preferably, in step S2), the hierarchical embedded channel-spatial attention module CBAM comprises a channel attention CA and a spatial attention SA, and the channel attention CA and the spatial attention SA are used to enhance the feature map.

[0013] Preferably, in step S2), the channel attention CA performs global average pooling and global maximum pooling operations on the feature map F output by the previous layer, which has a shape of [N, C, H, W], to generate two tensors with a shape of [N, C, 1, 1], and then flattens the two tensors into one-dimensional vectors with a shape of [N, C]; then uses the same two fully connected layers to process the two one-dimensional vectors, first reduces the dimension through a fully connected layer with a weight of W0, then applies a ReLU activation function to increase the nonlinear expression capability, and finally restores to the original dimension through a fully connected layer with a weight of W1 to generate the importance score of each channel; adds the scores corresponding to the two pooling results to form a comprehensive channel importance score; then applies a Sigmoid function to convert these scores into attention weights, ensuring that the value range is between [0, 1], expands the generated attention weights from the shape [N, C] back to the original spatial dimension [N, C, 1, 1], and applies them to the original feature map in an element-level multiplication manner to complete the application of the channel attention mechanism, where N is the batch size, C is the number of channels, H and W are the height and width of the feature map respectively, and the calculation expression is:

[0014]

[0015] wherein, W1, W0 are the weights of the fully connected layer; σ is the Sigmoid function; are the results of average pooling and max pooling in channel dimension, respectively; M c (F) represents the channel attention weight.

[0016] As preferred, in step S2), the spatial attention SA is applied to the feature map F of the previous layer, which has a shape of [N, C, H, W]; global average pooling and global max pooling operations are performed respectively to generate two tensors with a shape of [N, 1, H, W]; the two tensors are concatenated along the channel dimension to form a new feature map with a shape of [N, 2, H, W]; a convolution layer with a 7x7 convolution kernel is used to process the concatenated feature map to convert it into a single-channel spatial attention weight map with a shape of [N, 1, H, W]; the Sigmoid function σ is applied to convert the scores into attention weights, ensuring that the value range is between [0, 1]; the generated spatial attention weights are applied to the original feature map in an element-level multiplication manner to complete the application of the spatial attention mechanism, and the calculation expression is as follows:

[0017]

[0018] wherein, M s (F) represents the spatial attention weight; Conv 7×7 represents a 7x7 convolution layer; are the results of average pooling and max pooling in spatial dimension, respectively.

[0019] As preferred, in step S2), the spectral attention module is applied to the feature map F of the previous layer, which has a shape of [N, C, H, W]; a convolution layer with a 1x1 convolution kernel is applied to dynamically learn the importance weight of each channel, and the output shape remains [N, C, H, W]; the Sigmoid function σ is applied to convert the scores into attention weights, ensuring that the value range is between [0, 1]; the generated spectral attention weights are applied to the original feature map in an element-level multiplication manner to complete the application of the spatial attention mechanism, and the calculation expression is as follows:

[0020] M λ (F) = σ(Conv 1×1 (F)

[0021] wherein, M λ (F) represents the spectral attention weight; Conv 1×1 represents a 1x1 convolution layer; σ is the Sigmoid function.

[0022] As preferred, in step S3), the weights of the strong MobileNetV3 small model are initialized using the method of transfer learning, and the pre-trained weights in the large-scale image classification task are used as the starting point.

[0023] As preferred, in step S3), the model training is performed using the cross-entropy loss function and the AdamW optimizer (initial learning rate 1e-4, weight decay 1e-4), while combining the cosine annealing learning rate scheduler to dynamically adjust the learning rate, and at the end of each epoch, the model performance on the validation set is evaluated, and the best model state is recorded and saved.

[0024] As preferred, in step S3), the update of the learning rate is represented as:

[0025]

[0026] wherein, η t is the updated learning rate, η min , η max are the minimum and maximum values of the learning rate respectively; T max is the period length; T cur is the current iteration number.

[0027] As preferred, in step S3), the expression of the cross-entropy loss function is:

[0028]

[0029] wherein, Loss is the average loss of the batch, N is the number of samples in the batch, C is the number of classes; y i,c is the indicator variable of the true label; p i,c is the predicted probability of the model that the sample i belongs to class c.

[0030] As preferred, in step S4), the river images under different environmental conditions in the test set are input into the trained enhanced MobileNetV3 small model, and the corresponding prediction results are generated, and the output variable is a binary classification label, wherein the normal water quality and the yellow mud water correspond to “Normal” and “Muddy” respectively;

[0031] The confusion matrix diagram is generated using the prediction results of the enhanced MobileNetV3 small model.

[0032] The Grad-CAM is used to generate the explainable heat map, which highlights the image area that the enhanced MobileNetV3 small model focuses on when making decisions; and an explainable analysis report is generated, including the original image, the Grad-CAM heat map and the superimposed image.

[0033] As preferred, in step S4), the Grad-CAM is used to generate an interpretable heat map, specifically including the following steps:

[0034] S41), a class-related heat map is generated by calculating the gradient weighted activation map of the target layer;

[0035] The third layer from the bottom of the enhanced MobileNetV3 small model is selected as the target layer, the Grad-CAM object is initialized, and the input image tensor is passed to the model for forward propagation to generate an activation map

[0036] S42), the input image tensor is converted into a PIL image format, and the activation map is processed to generate a color heat map;

[0037] S43), the heat map is superimposed on the original image to generate a visualization result.

[0038] The beneficial effects of the present application are:

[0039] 1、The enhanced MobileNetV3 small model of the present application significantly improves the recognition ability of the model in complex environments by combining channel, spatial and spectral attention mechanisms, making the detection of yellow mud water phenomenon more accurate and efficient;

[0040] 2、The present application can effectively improve the generalization ability and prediction accuracy of the model, and the model accuracy is improved by 99.53%, which is 8.2% higher than the original MobileNetV3, and the accuracy remains at a high level in strong light, weak light and other interference environments;

[0041] 3、The present application comprehensively evaluates the model through Grad-CAM, confusion matrix, accuracy, precision, recall and F1 score and other evaluation indexes, and provides a visualization tool to help understand the working principle of the model, thereby enhancing the transparency and credibility of the model. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 The flowchart of the method of the present application is shown in the figure;

[0043] Figure 2 The structure framework diagram of the enhanced MobileNetV3 small model constructed by the present application is shown in the figure;

[0044] Figure 3 The confusion matrix result graph of the test set in the embodiment of the present application is shown in the figure;

[0045] Figure 4 The evaluation index value graph of the test set in the embodiment of the present application is shown in the figure;

[0046] Figure 5 The Grad-CAM visualization graph in the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0047] The specific embodiments of the present application are further described below with reference to the accompanying drawings:

[0048] As Figure 1 shown, the embodiment provides a soil erosion yellow mud water image recognition method based on MobileNetV3, comprising the following steps:

[0049] S1), obtain a multi-period heterogeneous data set of normal water quality and soil erosion yellow mud water images, and divide the multi-period heterogeneous data set into a training set, a test set and a validation set after dynamic domain enhancement preprocessing;

[0050] In the embodiment, the multi-period heterogeneous data set is derived from river monitoring sites at different geographical locations and different time periods, specifically 69 soil erosion monitoring sites; wherein the normal pictures and the yellow mud water pictures caused by soil erosion are all more than 3000.

[0051] In the embodiment, the multi-period heterogeneous data set is dynamically divided into enhanced preprocessing, comprising the following steps:

[0052] S11), adjust the size of the original image to 224x224 pixels, and crop or fill to adapt to the model input requirement;

[0053] S12), perform row normalization processing on the image processed in step S11) to make the pixel value in the [0, 1] interval;

[0054] S13), enhance the image after normalization, that is:

[0055] I A = ColorJitter(I O ,β Bright = 0.3,β contrast = 0.3,β saturation = 0.3)

[0056] In the formula, I A represents the enhanced image, I O is the image after normalization;β Bright = 0.3,β conrrast = 0.3,β saturation = 0.3 are the channel jitter intensity parameters respectively.

[0057] The training set, test set and validation set are divided according to the ratio of 7:2:1.

[0058] S2), construct an enhanced MobileNetV3 small model with a fusion multi-dimensional attention mechanism;

[0059] As Figure 2 shown, the strong MobileNet V3 small model is constructed as follows:

[0060] On the basis of the original MobileNet V3 architecture, the hierarchical embedded channel-spatial attention module CBAM and the spectral attention module are introduced to strengthen the attention to important features.

[0061] And a new classifier containing two linear layers and a Hardswish activation function is used to replace the classifier of the last layer of the original MobileNet V3 architecture.

[0062] In this embodiment, the hierarchical embedded channel-spatial attention module CBAM includes channel attention CA and spatial attention SA, which enhance the feature map through the channel attention CA and spatial attention SA.

[0063] Among them, the channel attention CA performs global average pooling and global maximum pooling operations on the feature map F output by the previous layer, which has a shape of [N, C, H, W], to generate two tensors with a shape of [N, C, 1, 1], and then flatten the two tensors into one-dimensional vectors with a shape of [N, C]; Then use the same two fully connected layers to process the two one-dimensional vectors, first reduce the dimension through the fully connected layer with weight W0, then apply the ReLU activation function to increase the nonlinear expression ability, and finally restore to the original dimension through the fully connected layer with weight W1, to generate the importance score of each channel; And add the scores corresponding to the two kinds of pooling results to form a comprehensive channel importance score; Then apply the Sigmoid function to convert these scores into attention weights, ensuring that the value range is between [0, 1], and the generated attention weights are expanded from the shape [N, C] back to the original spatial dimension [N, C, 1, 1], and applied to the original feature map in an element-level multiplication manner, completing the application of the channel attention mechanism, where N is the batch size, C is the number of channels, H and W are the height and width of the feature map respectively, and its calculation expression is:

[0064]

[0065] In the formula, W1 and W0 are the weights of the fully connected layer; σ is the Sigmoid function; respectively represent the average pooling and maximum pooling results of the channel dimension; M c (F) represents the channel attention weight.

[0066] The spatial attention SA is applied to the feature map F of the previous layer, which has a shape of [N, C, H, W]. Global average pooling and global maximum pooling operations are performed respectively to generate two tensors with a shape of [N, 1, H, W]. The two tensors are spliced along the channel dimension to form a new feature map with a shape of [N, 2, H, W]. A convolutional layer with a 7x7 convolutional kernel is used to process the spliced feature map to convert it into a single-channel spatial attention weight map with a shape of [N, 1, H, W]. The Sigmoid function σ is applied to convert the scores into attention weights, ensuring that the value range is between [0, 1]. The generated spatial attention weights are applied to the original feature map in an element-level multiplication manner to complete the application of the spatial attention mechanism, and the calculation expression is as follows:

[0067]

[0068] In the formula, M s (F) represents the spatial attention weight; Conv 7×7 represents a 7x7 convolutional layer; respectively represent the average pooling and maximum pooling results of the spatial dimension.

[0069] In this embodiment, the spectral attention module is applied to the feature map F of the previous layer, which has a shape of [N, C, H, W]. A convolutional layer with a 1x1 convolutional kernel is applied to dynamically learn the importance weight of each channel, and the output shape remains [N, C, H, W]. The Sigmoid function σ is applied to convert the scores into attention weights, ensuring that the value range is between [0, 1]. The generated spectral attention weights are applied to the original feature map in an element-level multiplication manner to complete the application of the spatial attention mechanism, and the calculation expression is as follows:

[0070] M λ (F) = σ(Conv 1×1 (F)

[0071] In the formula, M λ (F) represents the spatial attention weight; Conv 1×1 represents a 1x1 convolutional layer; and σ is the Sigmoid function.

[0072] S3), train the enhanced MobileNetV3 small model through the training set and using a dynamic learning rate scheduling and weight decay strategy, and dynamically save the optimal weights through the validation set;

[0073] In this embodiment, the method of transfer learning is used to initialize the weights of the strong MobileNetV3 small model, and the pre-trained weights in a large-scale image classification task are used as a starting point.

[0074] The model training is performed using the cross-entropy loss function and the AdamW optimizer (initial learning rate 1e-4, weight decay 1e-4), combined with the cosine annealing learning rate scheduler to dynamically adjust the learning rate, and the model performance on the validation set is evaluated at the end of each epoch, and the best model state is recorded and saved.

[0075] wherein the update of the learning rate is represented as:

[0076]

[0077] wherein η t is the updated learning rate, η min and η max are the minimum and maximum values of the learning rate, respectively; T max is the period length; and T cur is the current iteration number.

[0078] The expression of the cross-entropy loss function is:

[0079]

[0080] wherein Loss is the average loss of the batch, N is the number of samples in the batch, and C is the number of classes; y i,c is the indicator variable of the true label; and p i,c is the predicted probability of the model that the sample i belongs to the class c.

[0081] In the embodiment, the forward propagation process of the strong MobileNet V3 small model can be described as follows:

[0082] The feature map F with the shape of [N, C, H, W] is passed through the basic feature extraction layer of the pre-trained strong MobileNet V3 small model to obtain the feature map of the last layer; the CBAM module is applied to the feature map of the last layer, including channel attention and spatial attention, to generate an enhanced feature map; the spectral attention module is applied to further enhance the feature map by dynamically learning the importance weight of each channel using a 1x1 convolution kernel; the enhanced feature map is converted into a single vector through the global average pooling layer and flattened into a one-dimensional vector; then the two linear layers and the Hardswish activation function are sequentially passed through, and the Dropout layer is used to prevent overfitting, and finally the classification result with the shape of [N, num_classes] is output, wherein num_classes is the number of target classes of the classification task.

[0083] S4), based on the multi-dimensional evaluation framework, the enhanced MobileNetV3 small model after training is evaluated, and the multi-dimensional explainability analysis result is output combined with Grad-CAM.

[0084] In this embodiment, the river images under different environmental conditions in the test set are input into the trained enhanced MobileNetV3 small model, and the corresponding prediction results are generated, and the output variable is a binary classification label, in which the normal water quality and the muddy water correspond to "Normal" and "Muddy", respectively;

[0085] And the enhanced MobileNetV3 small model is evaluated by using multi-dimensional indexes such as accuracy, precision, recall and F1 score, etc. Figure 4 As shown in the figure.

[0086] Then, the confusion matrix diagram is generated by using the prediction results of the enhanced MobileNetV3 small model.

[0087] The Grad-CAM is used to generate an interpretable heat map to highlight the image area that the enhanced MobileNetV3 small model focuses on when making a decision; and an interpretable analysis report is generated, including the original image, the Grad-CAM heat map and the superimposed image.

[0088] The Grad-CAM is used to generate an interpretable heat map, which specifically includes the following steps:

[0089] S41), a heat map related to the class is generated by calculating the gradient weighted activation map of the target layer;

[0090] The third layer from the bottom of the enhanced MobileNetV3 small model is selected as the target layer, the Grad-CAM object is initialized, and the input image tensor is transmitted to the model for forward propagation to generate an activation map

[0091] S42), the input image tensor is converted into a PIL image format, and the activation map is processed to generate a color heat map;

[0092] S43), the heat map is superimposed with the original image to generate a visual result.

[0093] In this embodiment, the test set confusion matrix result diagram is shown in the figure. Figure 3 As shown in the figure, Figure 3 The prediction effect of the model on the test set is shown in detail, which is specifically that 342 samples are correctly identified as "Normal", 300 samples are correctly identified as "Muddy", 2 samples are misidentified as "Muddy" from "Normal", and 1 sample is misidentified as "Normal" from "Muddy". This indicates that the model has high accuracy and reliability in distinguishing normal and soil erosion muddy water images, only a small number of samples are misclassified, and the overall performance is excellent and stable.

[0094] In this embodiment, the Grad-CAM visualization result graph involved is as shown in Figure 5 The left image shows the actual situation of the river; the middle image is the heat map generated by Grad-CAM, highlighting the key areas that the model pays attention to; and the right image is the superimposed image of the heat map and the original image, which intuitively shows the attention area of the model. These visualization results show that the model can accurately identify the yellow mud water image caused by soil erosion, and Grad-CAM heat map provides clear decision basis, enhancing the model's explainability and credibility.

[0095] The above embodiments and descriptions described in the specification are only to illustrate the principles and best embodiments of the present application, and various changes and improvements can be made without departing from the spirit and scope of the present application, and these changes and improvements all fall within the scope of the claimed present application.

Claims

1. A soil erosion yellow mud image recognition method based on MobileNetV3, characterized in that, The method comprises the following steps: S1) acquiring a multi-period heterogeneous dataset of normal water quality and soil erosion yellow mud water images, and performing dynamic domain enhancement preprocessing on the multi-period heterogeneous dataset, and then dividing the multi-period heterogeneous dataset into a training set, a test set and a validation set; S2) constructing an enhanced MobileNetV3 small model with a multi-dimensional attention mechanism; the construction of the enhanced MobileNetV3 small model comprises introducing a hierarchical embedded channel-spatial attention module CBAM and a spectral attention module based on the original MobileNetV3 architecture; S3) training the enhanced MobileNetV3 small model by using the training set and adopting a dynamic learning rate scheduling and weight decay strategy, and dynamically saving the optimal weight by using the validation set; S4) evaluating the trained enhanced MobileNetV3 small model based on a multi-dimensional evaluation framework, and combining a Grad-CAM to output a multi-dimensional explainability analysis result.

2. The soil erosion yellow mud water image recognition method based on MobileNetV3 according to claim 1, characterized in that: In step S2), the construction of the enhanced MobileNetV3 small model further comprises replacing a classifier of a last layer of the original MobileNetV3 architecture with a new classifier comprising two linear layers and a Hardswish activation function.

3. The soil erosion yellow mud water image recognition method based on MobileNetV3 according to claim 1, characterized in that: In step S2), the hierarchical embedded channel-spatial attention module CBAM comprises a channel attention CA and a spatial attention SA, and the channel attention CA and the spatial attention SA are used to enhance a feature map.

4. The soil erosion yellow mud water image recognition method based on MobileNetV3 according to claim 3, characterized in that: In step S2), a calculation expression for processing the feature map F output by a previous layer by the channel attention CA is as follows: In the formula, W1, W0 are full connection layer weights; and σ is a Sigmoid function. are the results of average pooling and max pooling of the channel dimension, respectively; M c (F) represents the channel attention weight.

5. The soil erosion yellow mud water image recognition method based on MobileNetV3 according to claim 4, characterized in that: In step S2), a calculation expression for processing the feature map F of the previous layer by the spatial attention SA is as follows: In the formula, M s (F) represents the spatial attention weight; f 7×7 represents a 7x7 convolutional layer; respectively represent the average pooling and the maximum pooling results of the spatial dimension.

6. The MobileNetV3-based soil erosion yellow mud water image recognition method according to claim 1, characterized in that: In step S2), a calculation expression for processing the feature map F of the previous layer by the spectral attention module is as follows: M λ (F) = σ(Conv 1×1 (F) In the formula, M λ (F) denotes the spectral attention weight; Conv 1×1 denotes a 1x1 convolution layer; and σ is a Sigmoid function.

7. The MobileNetV3-based soil erosion yellow mud water image recognition method according to claim 1, characterized in that: In step S3), a cross-entropy loss function and an AdamW optimizer are used for model training, an initial learning rate is 1e-4, and a weight decay is 1e-4; meanwhile, a cosine annealing learning rate scheduler is used to dynamically adjust the learning rate, the model performance on the validation set is evaluated at the end of each epoch, the best model state is recorded and saved; The update of the learning rate is represented as follows: where η t is the updated learning rate, η min , η max are the minimum and maximum values of the learning rate, respectively; T max is the cycle length; and T cur is the current iteration number.

8. The MobileNetV3-based soil erosion yellow mud water image recognition method according to claim 1, characterized in that: In step S4), river images under different environmental conditions in the test set are input into the trained enhanced MobileNetV3 small model, and corresponding prediction results are generated, and the output variable is a binary classification label, wherein normal water quality and yellow mud water correspond to "Normal" and "Muddy", respectively; A confusion matrix diagram is generated by using the prediction results of the enhanced MobileNetV3 small model, An explainability heat map is generated by using the Grad-CAM, which highlights the image regions that the enhanced MobileNetV3 small model focuses on when making decisions; and an explainability analysis report is generated, which comprises an original image, a Grad-CAM heat map and a superimposed image.

9. The MobileNetV3-based soil erosion yellow mud water image recognition method according to claim 8, characterized in that: In step S4), the explainability heat map is generated by using the Grad-CAM, and the method comprises the following steps: S41), generate a heat map related to the category by calculating the gradient weighted activation map of the target layer; The third layer from the bottom of the enhanced MobileNetV3 small model is selected as the target layer, the GradCAM object is initialized, and the input image tensor is passed to the model for forward propagation to generate the activation map; S42), convert the input image tensor to the PIL image format, and process the activation map to generate a color heat map; S43), superimpose the heat map with the original image to generate a visualization result.

Images