Tomato leaf insect pest area segmentation method based on improved deeplabv3+ model
By improving the deeplabv3+ model, utilizing data augmentation and feature extraction network replacement, and combining it with the CBAM convolutional attention mechanism, we achieved accurate segmentation of insect-infested areas on tomato leaves, solving the problem of inaccurate segmentation in existing technologies and reducing pesticide use and planting workload.
Patent Information
- Application Number
- CN202310281424.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-03-22
- Publication Date
- 2026-01-13
- Estimated Expiration
- 2043-03-22
AI Technical Summary
Existing technologies cannot effectively monitor and accurately locate pest-infested areas on tomato leaves, resulting in excessive pesticide use and inaccurate segmentation, which affects tomato growth.
An improved deeplabv3+ model was used to construct a dataset by capturing tomato leaf images using industrial cameras and ordinary mobile phone cameras. Various data augmentations were performed, the backbone feature extraction network was replaced with MobileNetV3, the aspp module was improved to ddaspp, a cbam convolutional attention mechanism was added, and feature fusion of the decoder was increased to improve recognition accuracy.
It improved the segmentation accuracy and generalization ability of insect-infested areas on tomato leaves, reduced pesticide use, decreased planting workload, and enhanced the robustness and recognition accuracy of the model.
Smart Images

Figure CN116433901B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical field of insect pest region segmentation on tomato leaves, and in particular to a method for segmenting insect pest regions on tomato leaves based on an improved deeplabv3+ model. Background Technology
[0002] During cultivation, tomatoes are susceptible to various pests, such as leafminers, whose damage to tomatoes continues throughout their entire life cycle. The leafminer infests tomatoes from young leaves to older leaves. When the larvae burrow into the leaves, they primarily feed on the leaf tissue, leaving white areas of varying shapes and sizes that eventually cause the leaves to yellow and wither. This leads to the decomposition of chlorophyll, a decrease in sugar content, and in severe cases, yellowing and shedding of leaves, even resulting in plant death. Therefore, it is crucial to eliminate leafminer larvae as soon as they infest the tomato leaves to minimize damage and prevent yield decline. However, growers cannot constantly monitor the condition of tomato leaves, especially when cultivating large areas.
[0003] In recent years, with the development of smart agriculture, computer vision technology and artificial intelligence technology have been gradually applied to the agricultural field. Tomato cultivation can also introduce related technologies to achieve precise location of pest areas. Then, combined with technologies such as robots and robotic arms, it can achieve precise and timely spraying of pesticides, minimizing the consequences of pests, protecting the growth of tomatoes, and reducing the amount of pesticides used and the workload of planting.
[0004] Based on the above discussion, the invention of an improved deeplabv3+ model for segmenting insect-infested regions on tomato leaves has high practical application value. Summary of the Invention
[0005] The purpose of this invention is to overcome the shortcomings and deficiencies of existing technologies and propose a method for segmenting tomato leaf pest regions based on an improved deeplabv3+ model. This method primarily utilizes images of tomato leaves captured by industrial cameras and ordinary mobile phone cameras to construct an original dataset of tomato pest regions. Various data augmentation techniques are then employed to expand the original dataset, improving recognition accuracy. Training, validation, and test datasets are constructed. Furthermore, the backbone feature extraction network of the deeplabv3+ model is replaced and converted to MobileNetV3. Targeted improvements are then made to both the decoder and encoder, and an attention mechanism is incorporated to further enhance the recognition effect. Finally, through training, the optimal model is saved to achieve accurate segmentation of tomato leaf pest regions.
[0006] To achieve the above objectives, the technical solution provided by this invention is as follows: a method for segmenting insect-infested regions on tomato leaves based on an improved deeplabv3+ model. This improved deeplabv3+ model is an improvement on the original deeplabv3+ model in four parts. The first part is an improvement to the original backbone feature extraction network: the original backbone feature extraction network xception is replaced with the MobileNetV3 backbone feature extraction network, while the last three convolutional layers are removed and one downsampling is reduced to reduce information loss. The second part is an improvement to the aspp module: to compensate for the disadvantage of the original aspp structure discarding a lot of information due to dilated convolution, the idea of DenseNet is adopted, and the dilation rates of the three convolution operations are set to 3, 5 and 7 respectively. At the same time, a convolution operation with a dilation rate of 11 is added in parallel with the above three convolution operations. The output of the low dilation rate is added to the input of the high dilation rate. At the same time, to compensate for the information loss of the original max pooling operation, the original max pooling is replaced with soft pooling, which can reduce the information loss during pooling. In addition, to reduce the number of parameters and computation, all ordinary convolutions are replaced with depth-wise convolutions. The first part is convolution, hence the improved aspp structure is named ddaspp; the second part is to add a cbam convolutional attention mechanism module after the output of the ddaspp structure to enhance the extraction of local information; the third part is to add a low-level feature fusion in the decoder stage to improve the segmentation effect.
[0007] The specific implementation of this method for segmenting insect-infested areas on tomato leaves includes the following steps:
[0008] 1) Collect images of tomato leaves grown in cherry tomato fields and improve image quality using a super-resolution restoration algorithm;
[0009] 2) Use LabelMe software to segment the insect-infested areas on tomato leaves in the images and construct the original dataset;
[0010] 3) Based on the environment of the cherry tomato growing area and the growth characteristics of cherry tomatoes, data augmentation techniques were used to augment the original dataset to construct training dataset, validation dataset and test dataset;
[0011] 4) After converting the size of the training dataset, batch normalization is performed uniformly to accelerate the convergence speed of the model in subsequent training processes.
[0012] 5) Set training parameters for the improved deeplabv3+ model and train it using the training dataset. By obtaining the probability value of image segmentation, calculate the loss value by combining the binary cross-entropy loss with the manually labeled real value. Finally, update the weights based on the loss value calculated by the model. Adjust the model parameters and update the learning rate according to the number of iterations during training. After each specific number of model training iterations, use the validation dataset to verify the training effect of the model. Iterate until the total loss of the validation dataset reaches the minimum state. Finally, save the optimal model after training and validation.
[0013] 6) After performing size conversion and normalization on the cherry tomato leaf images to be identified in the test dataset, the images are input into the saved optimal model for inference. The saved optimal model can segment the pest-infested areas of the cherry tomato leaf images to be identified, thus completing the accurate segmentation of the pest-infested areas of the tomato leaves.
[0014] Furthermore, in step 3), the specific details of the data augmentation methods include:
[0015] Rotate: Rotate the image clockwise by 90°, 180°, or 270°;
[0016] Flip: Flips the image horizontally and vertically;
[0017] Brightness Variation: Randomly adjusts the brightness in the image to simulate images under different lighting conditions;
[0018] Double dropout: Because the model contains a dropout module, some neurons are randomly hidden. The outputs of the same input dropped twice are similar but inconsistent. This effect is used to achieve the purpose of data augmentation.
[0019] Furthermore, in step 5), the improved deeplabv3+ model is divided into an encoder and a decoder. The encoder consists of a backbone feature extraction network MobileNetV3, a ddaspp structure, and a cbam convolutional attention mechanism module. The decoder includes two multi-scale feature fusion modules.
[0020] Furthermore, the backbone feature extraction network MobileNetV3 comprises four interconnected parts, as detailed below:
[0021] The first part consists of two 3x3 bottleneck modules connected together, with output M1;
[0022] The second part consists of a 3*3 bottleneck module and a 5*5 bottleneck module, with an output of M2;
[0023] The third part consists of two 5x5 bottleneck modules and one 3x3 bottleneck module, with an output of M3;
[0024] The fourth part consists of five 3x3 bottleneck modules and three 5x5 bottleneck modules, with an output of M4.
[0025] Furthermore, the Ddaspp structure consists of five parts: a 3x3 depthwise separable convolution with a dilation rate of 3, a 3x3 depthwise separable convolution with a dilation rate of 5, a 3x3 depthwise separable convolution with a dilation rate of 7, a 3x3 depthwise separable convolution with a dilation rate of 11, and a softpooling module. The inputs of the 3x3 depthwise separable convolution with a dilation rate of 3 and the softpooling module are the outputs of the fourth part of the backbone feature extraction network MobileNetV3. The input of the 3x3 depthwise separable convolution with a dilation rate of 5 is the output of the 3x3 depthwise separable convolution with a dilation rate of 3 and the fourth part of the backbone feature extraction network MobileNetV3. The inputs of the 3x3 depthwise separable convolution with a dilation rate of 7 are the outputs of the 3x3 depthwise separable convolution with a dilation rate of 3, the 3x3 depthwise separable convolution with a dilation rate of 5, and the fourth part of the backbone feature extraction network MobileNetV3. The inputs of the 3x3 depthwise separable convolution with a dilation rate of 11 are the outputs of the 3x3 depthwise separable convolution with a dilation rate of 3, the 3x3 depthwise separable convolution with a dilation rate of 5, the 3x3 depthwise separable convolution with a dilation rate of 7, and the fourth part of the backbone feature extraction network MobileNetV3. The outputs of these five parts are concatenated and used as the output of the ddaspp part.
[0026] Furthermore, the cbam convolutional attention mechanism module includes a spatial attention mechanism module and a channel attention mechanism module, as detailed below:
[0027] The input to the CBAM convolutional attention mechanism module is the channel attention mechanism. The input feature map is subjected to global max pooling and global average pooling based on width and height, respectively, and then passed through a shared fully connected layer. The features output from the shared fully connected layer are summed element-wise, followed by a sigmoid activation operation to generate the final channel attention feature map. This final channel attention feature map is then multiplied element-wise with the input to the CBAM convolutional attention mechanism module. The resulting feature map is then used as the input to the spatial attention mechanism module. This input feature map is subjected to average pooling and max pooling in the channel dimension. The outputs of the two modules are merged, then activated by a sigmoid operation to generate the final spatial attention feature map. This final spatial attention feature map is then multiplied element-wise with the input to the spatial attention mechanism module, and the resulting feature map is used as the output of CBAM.
[0028] Furthermore, the decoder is described in detail below:
[0029] The details of the first multi-scale feature fusion module are as follows:
[0030] The output of the cbam convolutional attention mechanism module and the output of the fourth part of the backbone feature extraction network MobileNetV3 M3 are respectively processed by 1*1 depthwise separable convolution to change the number of channels. After concatenating the two to complete one feature fusion, they are upsampled by two times and then output to the second multi-scale feature fusion module.
[0031] The details of the second multi-scale feature fusion module are as follows:
[0032] The output of the first multi-scale feature fusion module and the output M3 of the fourth part of the backbone feature extraction network MobileNetV3 are respectively processed by 1*1 depthwise separable convolution to change the number of channels. After concatenating the two to complete one feature fusion, they are upsampled by two times and then output to 3*3 depthwise separable convolution, which is then upsampled by four times as the output of the improved deeplabv3+ model.
[0033] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0034] 1. Considering the complex environment of outdoor tomato growing areas and the growth characteristics of tomato leaves, various data augmentation techniques were used to effectively expand the original dataset of tomato leaf pest areas, improve the model's segmentation accuracy and generalization ability for tomato leaf pest areas, and avoid overfitting.
[0035] 2. The original deeplabv3+ model structure was improved by replacing the backbone feature extraction network and using separable deep convolutions, which reduced the time, computing power and memory consumption of the model during training and inference, and improved the model's segmentation accuracy and generalization ability for insect-infested areas on tomato leaves.
[0036] 3. Drawing inspiration from DenseNet, we constructed the ddaspp structure, which overcomes the shortcomings of the original aspp structure in losing too much information and improves segmentation accuracy.
[0037] 4. A cbam convolutional attention mechanism module has been added to enhance the extraction of local information.
[0038] 5. An additional fusion of low-level features was added, which solved the problem of holes in the segmentation results of the original deeplabv3+ model.
[0039] 6. Compared with traditional image processing methods, the improved deeplabv3+ model structure has higher robustness and recognition accuracy. Attached Figure Description
[0040] Figure 1 The structure diagram for improving the deeplabv3+ model.
[0041] Figure 2 This is a structural diagram of the cbam convolutional attention mechanism module. Detailed Implementation
[0042] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.
[0043] like Figure 1 and Figure 2As shown, this embodiment provides a method for segmenting insect-infested regions on tomato leaves based on an improved DeepLabv3+ model. This improved DeepLabv3+ model is an improvement on the original DeepLabv3+ model in four parts. The first part improves the original backbone feature extraction network: replacing the original Xception backbone feature extraction network with a MobileNetV3 backbone feature extraction network, while removing the last three convolutional layers and reducing downsampling by one, thus reducing information loss. The second part improves the ASPP module: to compensate for the disadvantage of the original ASPP structure discarding a large amount of information due to dilated convolution, borrowing the idea of DenseNet, the dilation rates of the three convolution operations are set to 3, 5, and 7 respectively, while adding a convolution operation with a dilation rate of 11 in parallel with the above three convolution operations. The output of the low dilation rate is added to the input of the high dilation rate. Simultaneously, to compensate for the information loss in the original max pooling operation, the original max pooling is replaced with soft pooling, which reduces the information loss during pooling. Furthermore, to reduce the number of parameters and computational load, all ordinary convolutions are replaced with depth-wise convolutions. The first part is convolution, hence the improved aspp structure is named ddaspp; the second part is to add a cbam convolutional attention mechanism module after the output of the ddaspp structure to enhance the extraction of local information; the third part is to add a low-level feature fusion in the decoder stage to improve the segmentation effect.
[0044] The specific implementation of this method for segmenting insect-infested areas on tomato leaves includes the following steps:
[0045] 1) Images of pest-infested areas on tomato leaves grown outdoors were collected, and the LabelMe software was used to segment these areas to construct the original dataset. The application of computer vision and artificial intelligence in agriculture is not yet widespread. Currently, there are relatively few researchers in the field of segmenting pest-infested areas on tomato leaves against complex backgrounds, and no publicly available datasets are available. Therefore, images of pest-infested areas on tomatoes were collected independently to create the dataset.
[0046] 2) Based on the complex environment of outdoor tomato growing areas and the growth characteristics of tomato leaves, different types of data augmentation methods were used to augment the original dataset, constructing training, validation, and test datasets to improve model performance. The data augmentation techniques used included:
[0047] Cropping: Cropping the original image;
[0048] Rotate: Rotate the cropped image clockwise by 90°, 180°, or 270°;
[0049] Flip: Flips the cropped image horizontally and vertically;
[0050] Brightness Variation: Randomly adjusts the brightness in the image to simulate images under different lighting conditions;
[0051] r-dropout: Because the model contains a dropout module, the outputs of the same input dropped twice are similar but inconsistent. This effect is used to achieve the purpose of data augmentation.
[0052] 3) Perform size conversion and data normalization on the training dataset to accelerate the convergence speed of the model during subsequent training. Specifically, the size conversion process converts the images input to the network into images of a size of 224×224×3 suitable for the network.
[0053] 4) Set training parameters for the improved deeplabv3+ model (set the experimental optimizer to Adam, batch size to 8, total number of iterations to 200 epochs, and learning rate to 0.0001), and train using the training dataset. By obtaining the probability value of image segmentation, calculate the loss value by combining the binary cross-entropy loss with the manually labeled true value. Finally, update the weights based on the loss value calculated by the model. Adjust the model parameters and update the learning rate according to the number of iterations during training. After each specific number of training iterations, use the validation dataset to verify the training effect of the model. Iterate until the total loss of the validation dataset reaches the minimum state. Finally, save the optimal model after training and validation.
[0054] The improved deeplabv3+ model is divided into an encoder and a decoder. The encoder consists of a backbone feature extraction network MobileNetV3, a ddaspp structure, and a cbam convolutional attention mechanism module. The decoder includes two multi-scale feature fusion modules.
[0055] The backbone feature extraction network MobileNetV3 consists of four interconnected parts, as detailed below:
[0056] The first part consists of two 3x3 bottleneck modules connected together, with output M1;
[0057] The second part consists of a 3*3 bottleneck module and a 5*5 bottleneck module, with an output of M2;
[0058] The third part consists of two 5x5 bottleneck modules and one 3x3 bottleneck module, with an output of M3;
[0059] The fourth part consists of five 3x3 bottleneck modules and three 5x5 bottleneck modules, with an output of M4.
[0060] The model's input size is 224*224*3; the image is input from the backbone feature extraction network MobileNetV3, and outputs to different parts in the second, third, and fourth parts.
[0061] The Ddaspp structure consists of five parts: a 3x3 depthwise separable convolution with a dilation rate of 3, a 3x3 depthwise separable convolution with a dilation rate of 5, a 3x3 depthwise separable convolution with a dilation rate of 7, a 3x3 depthwise separable convolution with a dilation rate of 11, and a softpooling module. The inputs to the 3x3 depthwise separable convolution with a dilation rate of 3 and the softpooling module are the outputs of the fourth part of the MobileNetV3 backbone feature extraction network. The inputs to the 3x3 depthwise separable convolution with a dilation rate of 5 are the outputs of the 3x3 depthwise separable convolution with a dilation rate of 3 and the fourth part of the MobileNetV3 backbone feature extraction network. The inputs to the 3x3 depthwise separable convolution with a dilation rate of 7 are the outputs of the 3x3 depthwise separable convolution with a dilation rate of 3, the 3x3 depthwise separable convolution with a dilation rate of 5, and the output of the fourth part of the backbone feature extraction network MobileNetV3. The inputs to the 3x3 depthwise separable convolution with a dilation rate of 11 are the outputs of the 3x3 depthwise separable convolution with a dilation rate of 3, the 3x3 depthwise separable convolution with a dilation rate of 5, the 3x3 depthwise separable convolution with a dilation rate of 7, and the output of the fourth part of the backbone feature extraction network MobileNetV3. The outputs of these five parts are concatenated and then used as the output of the ddaspp part.
[0062] The CBAM convolutional attention mechanism module includes a spatial attention mechanism module and a channel attention mechanism module, as detailed below:
[0063] The input to the CBAM convolutional attention mechanism module is the channel attention mechanism. The input feature map is subjected to global max pooling and global average pooling based on width and height, respectively, and then passed through a shared fully connected layer. The features output from the shared fully connected layer are summed element-wise, followed by a sigmoid activation operation to generate the final channel attention feature map. This final channel attention feature map is then multiplied element-wise with the input to the CBAM convolutional attention mechanism module. The resulting feature map is then used as the input to the spatial attention mechanism module. This input feature map is subjected to average pooling and max pooling in the channel dimension. The outputs of the two modules are merged, then activated by a sigmoid operation to generate the final spatial attention feature map. This final spatial attention feature map is then multiplied element-wise with the input to the spatial attention mechanism module, and the resulting feature map is used as the output of CBAM.
[0064] The decoder is described in the following details:
[0065] The details of the first multi-scale feature fusion module are as follows:
[0066] The output of the cbam convolutional attention mechanism module and the output of the fourth part of the backbone feature extraction network MobileNetV3 M3 are respectively processed by 1*1 depthwise separable convolution to change the number of channels. After concatenating the two to complete one feature fusion, they are upsampled by two times and then output to the second multi-scale feature fusion module.
[0067] The details of the second multi-scale feature fusion module are as follows:
[0068] The output of the first multi-scale feature fusion module and the output M3 of the fourth part of the backbone feature extraction network MobileNetV3 are respectively processed by 1*1 depthwise separable convolution to change the number of channels. After concatenating the two to complete one feature fusion, they are upsampled by two times and then output to 3*3 depthwise separable convolution, which is then upsampled by four times as the output of the improved deeplabv3+ model.
[0069] 5) After performing size conversion and normalization on the cherry tomato leaf images to be identified in the test dataset, the images are input into the saved optimal model for inference. The saved optimal model can segment the pest-infested areas of the cherry tomato leaf images to be identified, thus completing the accurate segmentation of the pest-infested areas of the tomato leaves.
[0070] In summary, this invention provides a new method for controlling tomato pests, which can be combined with mechanical structures to achieve precise and timely pesticide spraying, minimizing the consequences of pests, protecting tomato growth, and reducing pesticide usage and planting workload.
[0071] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.
Claims
1. A tomato leaf insect pest area segmentation method based on an improved deeplabv3+ model, characterized in that, The improved deeplabv3+ model is improved in four parts from the original deeplabv3+ model. The first part is to improve the original backbone feature extraction network: replace the original backbone feature extraction network xception with MobileNetV3 backbone feature extraction network, meanwhile, delete the last three convolution layers and reduce one down-sampling to reduce the loss of information; the second part is to improve the aspp module: in order to make up for the shortcomings of the original aspp structure that discards a large amount of information because of the dilated convolution, learn from the idea of densenet, set the dilated rate of three convolution operations to 3, 5 and 7 respectively, and increase a convolution operation with a dilated rate of 11 in parallel with the above three convolution operations, the output of low dilated rate will be added to the input of high dilated rate, at the same time, make up for the loss of information of the original maximum pooling operation, replace the original maximum pooling with soft pooling, which can reduce the loss of information in the pooling process, and in order to reduce the parameter quantity and operation quantity, replace all ordinary convolution with depth wise convolution, so the improved aspp structure is named as ddaspp; the third part is to add a cbam convolution attention mechanism module after the output of the ddaspp structure to enhance the extraction of local information; the last part is to add a bottom feature fusion in the decoder stage to make the segmentation effect better; The specific implementation of the tomato leaf pest area segmentation method includes the following steps: 1) Collect tomato leaf images planted in cherry tomato planting land, and use a super-resolution recovery algorithm to improve the quality of the images; 2) Use the labelme software to segment the tomato leaf pest area in the image to construct an original data set; 3) According to the environment of cherry tomato planting land and the growth characteristics of cherry tomato, use data enhancement means to enhance the original data set to construct a training data set, a verification data set and a test data set; 4) After converting the size of the training data set, uniformly perform batch normalization processing to speed up the convergence speed of the model in the subsequent training process; 5) Set the training parameters of the improved deeplabv3+ model and use the training data set for training, calculate the loss value by obtaining the probability value of the identified image segmentation, calculating the loss value with the real value annotated by artificial, and finally updating the weight based on the loss value calculated by the model, adjusting the model parameters according to the number of iterations in the training process, updating the learning rate, and verifying the training effect of the model with the verification data set after the model is trained for a certain number of times, iterating until the total loss of the verification data set reaches the minimum state, and finally saving the optimal model after training and verification; 6) After size conversion and normalization processing of the cherry tomato leaf image to be identified in the test data set, input it into the saved optimal model for inference, the saved optimal model can segment the pest area of the cherry tomato leaf image to be identified, and complete the accurate segmentation of the tomato leaf pest area.
2. The tomato leaf insect-pest area segmentation method based on the improved deeplabv3+ model according to claim 1, wherein, In step 3), the specific circumstances of the data enhancement means include: Rotation: rotate the image clockwise 90°, 180°, 270°; Flip: flip the image horizontally and vertically; Brightness transformation: randomly adjust the brightness of the picture to simulate the picture under different light intensity conditions; Two times dropout: because the model contains a dropout module, some neurons are randomly hidden, the output of the same input dropout twice is similar but not consistent, which is used to achieve the purpose of data enhancement.
3. The tomato leaf insect-pest area segmentation method based on the improved deeplabv3+ model according to claim 2, characterized in that, In step 5), the improved deeplabv3+ model is divided into an encoder and a decoder, wherein the encoder is divided into a backbone feature extraction network MobileNetV3, a ddaspp structure and a cbam convolution attention mechanism module, and the decoder includes two multi-scale feature fusion modules.
4. The tomato leaf insect-pest area segmentation method based on the improved deeplabv3+ model according to claim 3, characterized in that, The backbone feature extraction network MobileNetV3 includes four connected parts, as follows: The first part is two 3*3 bottleneck modules connected, and the output is M1; The second part is a 3*3 bottleneck module and a 5*5 bottleneck module, and the output is M2; The third part is two 5*5 bottleneck modules and a 3*3 bottleneck module, and the output is M3; The fourth part is five 3*3 bottleneck modules and three 5*5 bottleneck modules, and the output is M4.
5. The tomato leaf pest area segmentation method based on the improved deeplabv3+ model according to claim 4, characterized in that, The ddaspp structure is composed of five parts: a 3*3 depth separable convolution with an expansion rate of 3, a 3*3 depth separable convolution with an expansion rate of 5, a 3*3 depth separable convolution with an expansion rate of 7, a 3*3 depth separable convolution with an expansion rate of 11, and a softpool module. The input of the 3*3 depth separable convolution with an expansion rate of 3 and the softpool module is the output of the fourth part of the backbone feature extraction network MobileNetV3, the input of the 3*3 depth separable convolution with an expansion rate of 5 is the output of the 3*3 depth separable convolution with an expansion rate of 3 and the output of the fourth part of the backbone feature extraction network MobileNetV3, the input of the 3*3 depth separable convolution with an expansion rate of 7 is the output of the 3*3 depth separable convolution with an expansion rate of 3, the output of the 3*3 depth separable convolution with an expansion rate of 5 and the output of the fourth part of the backbone feature extraction network MobileNetV3, the input of the 3*3 depth separable convolution with an expansion rate of 11 is the output of the 3*3 depth separable convolution with an expansion rate of 3, the output of the 3*3 depth separable convolution with an expansion rate of 5, the output of the 3*3 depth separable convolution with an expansion rate of 7 and the output of the fourth part of the backbone feature extraction network MobileNetV3, and the outputs of the five parts are concatenated as the output of the ddaspp part.
6. The tomato leaf insect-pest area segmentation method based on the improved deeplabv3+ model according to claim 5, characterized in that, The cbam convolution attention mechanism module includes a spatial attention mechanism module and a channel attention mechanism module, as follows: The input to the CBAM convolutional attention mechanism module is the channel attention mechanism. The input feature map is subjected to global max pooling and global average pooling based on width and height, respectively, and then passed through a shared fully connected layer. The features output from the shared fully connected layer are summed element-wise, followed by a sigmoid activation operation to generate the final channel attention feature map. This final channel attention feature map is then multiplied element-wise with the input to the CBAM convolutional attention mechanism module. The resulting feature map is then used as the input to the spatial attention mechanism module. This input feature map is subjected to average pooling and max pooling in the channel dimension. The outputs of the two modules are merged, then activated by a sigmoid operation to generate the final spatial attention feature map. This final spatial attention feature map is then multiplied element-wise with the input to the spatial attention mechanism module, and the resulting feature map is used as the output of CBAM.
7. The tomato leaf insect-pest area segmentation method based on the improved deeplabv3+ model according to claim 6, characterized in that, The decoder is described in the following details: The details of the first multi-scale feature fusion module are as follows: The output of the cbam convolutional attention mechanism module and the output of the fourth part of the backbone feature extraction network MobileNetV3 M3 are respectively processed by 1*1 depthwise separable convolution to change the number of channels. After concatenating the two to complete one feature fusion, they are upsampled by two times and then output to the second multi-scale feature fusion module. The details of the second multi-scale feature fusion module are as follows: The output of the first multi-scale feature fusion module and the output M3 of the fourth part of the backbone feature extraction network MobileNetV3 are respectively processed by 1*1 depthwise separable convolution to change the number of channels. After concatenating the two to complete one feature fusion, they are upsampled by two times and then output to 3*3 depthwise separable convolution, which is then upsampled by four times as the output of the improved deeplabv3+ model.
Citation Information
Patent Citations
Image segmentation method and system based on multi-scale residual error coding and decoding network
CN114004811A
Straw mushroom growth state identification method based on convolutional neural network
CN114781483A