Rice disease identification method and system

Abstract

The application provides a rice disease identification method and system, the method comprises the following steps: S100, annotating the disease area label of the rice leaf image to obtain a segmentation label image; S101, extracting features from the rice leaf image; S102, enhancing part of the feature boundary information to obtain first enhanced features; S103, searching part of the feature target to obtain second enhanced features; S104, receiving part of the second enhanced features and the first enhanced features, and outputting decoding features; S105, receiving part of the decoding features, and outputting semantic prediction results; S106, extracting the boundary of the segmentation label image to obtain a boundary label image; S107, receiving the first enhanced features and part of the second enhanced features, and constructing a boundary supervision loss; S108, receiving the boundary prediction results and the semantic prediction results, and constructing a regularization term loss; S109, applying the constructed loss to a rice disease identification model for training, realizing the boundary information of the disease area of interest, obtaining accurate boundary segmentation effects, and improving the rice disease detection precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of rice disease detection, and in particular to a rice disease recognition method and system. BACKGROUND

[0002] In recent years, most of the researches related to rice disease segmentation mainly focus on designing more efficient semantic segmentation models. These methods do not achieve a good balance between the calculation of segmentation accuracy and the complexity of the model. In addition, these methods perform poorly in the face of small target diseases and chaotic target scenes. Object boundaries are key attributes of objects themselves, which are rich in discriminative information and help accurate pixel-level classification. Current methods do not fully utilize the information obtained by the feature extractor at each layer, resulting in poor segmentation results, which seriously affects researchers' accurate assessment and analysis of rice diseases.

[0003] The Chinese patent application with document number CN112465820A, entitled "Rice disease detection method based on semantic segmentation and fusion of global context information", discloses a rice disease detection method, which constructs a rice disease detection model based on semantic segmentation and fusion of global context information, thereby improving the accuracy of rice disease detection. The Chinese patent application with document number CN118334642A, entitled "Rice disease recognition method and system based on image processing", discloses a rice disease detection method, which combines three tasks of foreground segmentation, boundary detection and image classification, and constructs an image processing rice disease recognition method.

[0004] The above two rice disease detection methods are implemented by using traditional or deep learning or a combination of both to detect rice diseases. However, their segmentation models are too simple or lack robustness, which can lead to poor recognition accuracy when facing scenes not present in the data set. In particular, the hierarchical detection process used in the Chinese patent with document number CN118334642A, i.e., first using the OTSU method to obtain the foreground rice leaf, then using the HED edge detection method to detect the edge of the rice leaf based on the foreground rice leaf, and finally using the DenseNet classification network to classify the disease after morphological processing, the whole process is relatively cumbersome, the model is large and not conducive to practical deployment and application, and the HED boundary detection model has not been trained on the rice data set, so its performance may not be excellent. Therefore, the above method cannot ensure the accuracy of rice disease detection. SUMMARY

[0005] In view of the above, the present application aims to provide a rice disease recognition method and system to solve or at least partially solve the above-mentioned problems in the prior art.

[0006] To achieve the above-mentioned purpose, the first aspect of the present application provides a rice disease recognition method, which comprises:

[0007] S100, collect rice leaf image with disease, and label the disease area in the collected rice leaf image to obtain a segmentation label image;

[0008] S101, performing feature extraction on the collected rice leaf image to obtain multi-layer feature information of the rice leaf image;

[0009] S102, performing boundary information enhancement processing on part of the extracted multi-layer features to obtain first enhanced features containing boundary and texture information;

[0010] S103, performing target search on part of the extracted multi-layer features to find multi-scale target features, and using different size convolution kernel combinations in multiple parallel branches to adaptively extract target features of different scales to obtain second enhanced features containing target features of various scales;

[0011] S104, receiving the first enhanced features and the second enhanced features, performing deep feature decoding and fusion on the received features, and constructing auxiliary semantic loss through an auxiliary semantic prediction head, and finally outputting multiple decoding features;

[0012] S105, receiving part of the decoding features, performing Dropout operation, convolutional neural network operation and bilinear upsampling processing on the received features, and constructing semantic loss with the segmentation label image to output semantic prediction results;

[0013] S106, performing boundary extraction on the segmentation label image to obtain a boundary label image;

[0014] S107, receiving the first enhanced features and part of the second enhanced features, fusing the received features and outputting boundary prediction results of the rice disease area, and constructing boundary supervision loss by combining the boundary prediction results with the boundary label image;

[0015] S108, receiving the boundary prediction results and the semantic prediction results, constructing semantic-boundary loss and boundary-semantic loss by combining the segmentation label image and the boundary label image, and constructing regular term loss based on the semantic-boundary loss and the boundary-semantic loss;

[0016] S109, constructing a rice disease identification model, and applying the constructed semantic loss, auxiliary semantic loss, boundary supervision loss and regular term loss in the training of the rice disease identification model, and applying the rice disease identification model to disease identification of rice pictures.

[0017] Further, the step S102 specifically comprises the following steps:

[0018] S21, boundary information enhancement processing is performed, including a depth information extraction branch, a boundary information acquisition branch and a residual connection branch, part of the extracted multi-layer features is received by the boundary information acquisition branch, first through Sobel operator operation, the gradients in the horizontal and vertical directions are calculated, the absolute values are added to obtain first boundary features, then a convolution layer with a convolution kernel size of 1*1 is used to realize feature interaction and reorganization of the first boundary features, finally, the first boundary features are normalized by a Sigmoid activation function;

[0019] S22, part of the extracted multi-layer features is received by the depth information extraction branch, and the deep information in the shallow features is mined through two "convolution + batch normalization + ReLU" operation operations, the local features are extracted and the spatial information is enhanced, and the second boundary features are output;

[0020] S23, the first boundary features and the second boundary features output by the boundary information acquisition branch and the depth information extraction branch are multiplied to activate the boundary information contained therein;

[0021] S24, the residual connection structure in the residual connection branch is added to the part of the features extracted originally, and after one "convolution + batch normalization + ReLU" operation operation, the first enhanced features containing boundary and texture information are obtained.

[0022] Further, the step S103 specifically includes the following steps:

[0023] S31, part of the extracted multi-layer features is received by the first branch, first one convolution operation with a convolution kernel width and height of 1 and 3 is performed, then the local features in the horizontal direction are subjected to one convolution operation with a convolution kernel width and height of 3 and 1, and the features in the vertical direction are subjected to one dilated convolution operation with a convolution kernel size of 3*3 and an expansion rate of 3, so that the receptive field is expanded to 7*7;

[0024] S32, part of the extracted multi-layer features is received by the second branch, first one convolution operation with a convolution kernel width and height of 1 and 5 is performed, then the local features in the horizontal direction are subjected to one convolution operation with a convolution kernel width and height of 5 and 1, and the features in the vertical direction are subjected to one dilated convolution operation with a convolution kernel size of 5*5 and an expansion rate of 5, so that the receptive field is expanded to 21*21;

[0025] S33, receiving part of the multi-layer features extracted through the third branch, first performing a convolution operation with a kernel size of 1*7, then performing a convolution operation with a kernel size of 7*1 on the local features in the horizontal direction, and performing a dilated convolution operation with a kernel size of 7*7 and a dilation rate of 7 on the local features in the vertical direction to expand the receptive field to 43*43;

[0026] S34, receiving part of the multi-layer features extracted through the fourth branch, first performing a convolution operation with a kernel size of 1*9, that is, performing a convolution operation only in the width direction, then performing a convolution operation with a kernel size of 9*1 on the local features in the horizontal direction, that is, performing a convolution operation only in the height direction, and performing a dilated convolution operation with a kernel size of 9*9 and a dilation rate of 9 on the local features in the vertical direction to expand the receptive field to 73*73;

[0027] S35, receiving the features output by the four branches in steps S31-S34, splicing the features along the channel dimension to generate fusion features with multi-scale perception ability, then performing a convolution operation with a kernel size of 1*1 on the fusion features to compress the channel dimension of the features, and finally adding the features compressed in the channel dimension to part of the multi-layer features extracted originally to fuse local details and global context.

[0028] Further, the step S105 specifically comprises the following steps:

[0029] For the received part of the decoding features, first performing a Dropout operation with a probability of 0.1, then performing two groups of "convolution + batch normalization + ReLU" operation, and then performing a two times bilinear up-sampling processing to restore the feature tensor to 64*150*150 size, filtering redundant features, and then performing a group of "convolution + batch normalization + ReLU" operation on the 64*150*150 size feature tensor, and then performing a two times bilinear up-sampling processing to restore the feature tensor to 64*300*300 size, and finally performing semantic decoding by using a layer of convolution to obtain an output tensor of 2*300*300, and constructing a semantic loss with a segmentation label image to output a semantic prediction result.

[0030] Further, the step S106 specifically comprises the following steps:

[0031] S61, specifying the image path requiring boundary extraction and the path of the output boundary label image;

[0032] S62, read all segmentation label images under the image path needing to extract the boundary, traverse each segmentation label image one by one, and create a single-channel PNG image consistent with the size of the segmentation label image and the pixel, which is used for filling in the boundary information subsequently, to obtain a boundary label image;

[0033] S63, fill in the pixel boundary for the four sides of the segmentation label image respectively, and create a 5*5 size sliding window, and check the 5*5 neighborhood of each pixel;

[0034] S64, after the 5*5 size sliding window traverses the current image, the obtained boundary label image is saved in the path of the output boundary label image.

[0035] Further, the step S107 specifically comprises the following steps:

[0036] S71, a part of the first enhanced feature is subjected to a convolution operation, and then up-sampled to the same size as another part of the first enhanced feature, the up-sampled feature is spliced along the channel dimension with another part of the first enhanced feature, to obtain a fusion feature, and the fusion feature is subjected to a convolution + batch normalization + ReLU operation again, to obtain a primary fusion feature;

[0037] S72, a part of the second enhanced feature is subjected to a convolution and bilinear up-sampling operation, to obtain the same size as another part of the first enhanced feature, and the feature is spliced along the channel dimension with the primary fusion feature, and then subjected to a convolution + batch normalization + ReLU operation again, to compress the feature channel number and sufficiently fuse the feature information, to obtain an intermediate fusion feature;

[0038] S73, after global average pooling processing of the intermediate fusion feature, the intermediate fusion feature is subjected to a convolution operation first, to realize dimension reduction of the feature channel, and then subjected to a convolution operation again, to realize dimension increase of the feature channel, filter redundant information, and the filtered feature is subjected to Sigmoid function processing, to activate effective features, and the effective features are multiplied with the original intermediate fusion feature, to obtain a feature after boundary information enhancement;

[0039] S74, the enhanced feature is subjected to a convolution operation and then bilinear up-sampling, to obtain a boundary prediction result tensor.

[0040] The second aspect of the application provides a rice disease identification system, the system comprises:

[0041] An image acquisition module is configured to acquire a rice leaf image with a disease and transmit the rice leaf image with the disease to the feature extraction module;

[0042] Image annotation module: used to label the diseased areas in the collected rice leaf images to obtain an annotated rice leaf image, i.e., a segmented label image;

[0043] Feature extraction module: Used to receive rice leaf images transmitted by the image acquisition module, and to extract features from the rice leaf images to obtain multi-layer features of the rice leaf images;

[0044] Boundary generation module: It is used to receive some features from the feature extraction module and perform boundary information enhancement processing on the received partial features to obtain the first enhanced feature containing boundary and texture information;

[0045] Target search module: It is used to receive some features from the feature extraction module, and perform target search on each of the partial features to find multi-scale target features and obtain a second enhanced feature containing target features of various scales;

[0046] Feature aggregation module: It is used to receive the first enhanced feature and the second enhanced feature, randomly discard the received features, and then restore the feature resolution through convolution and upsampling operations. The upsampled features are fused with the features of the previous stage on the one hand, and an intermediate supervision signal is generated through the auxiliary semantic prediction head on the other hand. It is also used to construct an auxiliary semantic loss with the segmentation label image, and finally output the fused multi-layer decoding features.

[0047] Boundary label generation module: used to obtain the target boundary from the segmented label image to obtain the boundary label image;

[0048] Boundary supervision module: It is used to receive the first enhanced features and part of the second enhanced features, fuse the received features and output the boundary prediction results of the rice disease area. The output of the boundary supervision module is used to construct the boundary supervision loss together with the boundary label image.

[0049] The segmentation head output module is used to receive the output of the last layer feature aggregation module and output the semantic prediction result to construct the semantic loss. The segmentation head output module consists of DropOut operation, two sets of "convolution + batch normalization + ReLU" operation, double bilinear upsampling, one set of "convolution + batch normalization + ReLU" operation, and convolution operation.

[0050] Regularization term loss module: It receives the semantic prediction results from the boundary supervision module and the segmentation head output module. It constructs semantic-boundary loss and boundary-semantic loss by segmenting the label image and the boundary label image, and constructs regularization term loss based on semantic-boundary loss and boundary-semantic loss.

[0051] Furthermore, the boundary generation module includes a depth information extraction branch, a boundary information acquisition branch, and a residual connection branch;

[0052] The deep information extraction branch uses two "convolution + batch normalization + ReLU" operations to mine deep information in shallow features and output the second boundary feature.

[0053] The boundary information acquisition branch is used to obtain boundary discrimination information through Sobel operator operation, activate boundary features through convolution and Sigmoid operation to obtain the first boundary feature, and multiply the second boundary feature with the first boundary feature to obtain the boundary activated and enhanced feature. The feature is added to the received partial features through the residual connection branch, and after one "convolution + batch normalization + ReLU" operation, the first enhanced feature containing boundary and texture information is obtained.

[0054] Furthermore, the target search module includes a multi-level receptive field feature acquisition branch and a residual connection branch;

[0055] The multi-level receptive field feature acquisition branch includes four sets of convolution operations at different scales. Each set consists of horizontal and vertical one-dimensional convolution, standard two-dimensional convolution, and dilated convolution to capture local and global features at different scales. The local and global features at different scales are spliced ​​and fused using the channel dimension, then compressed by convolution, and feature enhancement is performed using residual connection branches.

[0056] Compared with the prior art, the beneficial effects of the present invention are:

[0057] This invention proposes a method and system for identifying rice diseases. By fusing features and outputting the boundary prediction results of rice disease areas, and constructing a boundary supervision loss, the computational complexity of the model during testing is reduced, providing a novel solution for improving the accuracy of rice disease detection. By performing boundary information enhancement processing and target search on the features, more boundary information of rice disease targets and contextual information of disease targets at various scales are captured respectively. By performing feature decoding and fusion, and constructing an auxiliary semantic loss, the previous layer and enhanced features are effectively combined to achieve efficient feature decoding. This invention can focus on the boundary information of disease areas and significantly improve the performance of disease boundary segmentation, obtaining more accurate boundary segmentation results, thereby improving the accuracy of rice disease detection. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only preferred embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1This is a schematic flowchart of a rice disease identification method provided in an embodiment of the present invention;

[0060] Figure 2 This is a comparison diagram of the results of a rice disease identification method provided in an embodiment of the present invention with other existing image segmentation methods;

[0061] Figure 3 This is a schematic diagram of rice leaf image acquisition provided in an embodiment of the present invention;

[0062] Figure 4 This is a schematic diagram of a rice disease identification system according to another embodiment of the present invention;

[0063] Figure 5 A schematic diagram of a boundary generation module provided in another embodiment of the present invention;

[0064] Figure 6 A schematic diagram of a target search module provided in another embodiment of the present invention;

[0065] Figure 7 This is a schematic diagram of a feature aggregation module provided in another embodiment of the present invention;

[0066] Figure 8 A schematic diagram of a segmentation head output module provided in another embodiment of the present invention;

[0067] Figure 9 A schematic diagram of a boundary supervision module provided in another embodiment of the present invention;

[0068] Figure 10 This is a detection interface for a rice disease identification system provided in another embodiment of the present invention. Detailed Implementation

[0069] The principles and features of the present invention are described below with reference to the accompanying drawings. The listed embodiments are only used to explain the present invention and are not intended to limit the scope of the present invention.

[0070] Reference Figure 1 This embodiment provides a method for identifying rice diseases, the method comprising the following steps:

[0071] S100. Collect images of rice leaves with diseases, and then label the diseased areas in the collected rice leaf images to obtain segmented labeled images.

[0072] S101. Extract features from the collected rice leaf images to obtain multi-layer feature information of the rice leaf images. The multi-layer feature information consists of 4 layers of feature information, which are labeled as R1, R2, R3 and R4 respectively.

[0073] S102. Perform boundary information enhancement processing on the R1 and R2 features extracted from the multi-layer features to obtain the first enhanced features E1 and E2 containing boundary and texture information. This specifically includes the following steps:

[0074] S21. Perform boundary information enhancement processing, including a depth information extraction branch, a boundary information acquisition branch, and a residual connection branch. The boundary information acquisition branch receives the R1 and R2 features extracted from the multi-layer features. First, the Sobel operator is used to calculate the gradients in the horizontal and vertical directions. The absolute values ​​are then added together to obtain the first boundary feature. Subsequently, a convolutional layer with a kernel size of 1×1 is used to perform feature interaction and recombination on the first boundary feature, reducing the amount of computation and optimizing the feature expression. Finally, the Sigmoid activation function is used to normalize the first boundary feature, enhancing the response of the boundary region while suppressing noise in the non-boundary region.

[0075] S22. The deep information extraction branch receives the R1 and R2 features extracted from the multi-layer features respectively. After two "3×3 convolution + batch normalization + ReLU" operations, the deep information in the shallow features is mined to extract local features and enhance spatial information. The second boundary feature is output. The output second boundary feature is used to supplement the boundary information and obtain the detailed information that the branch may have missed.

[0076] S23. Multiply the first boundary feature and the second boundary feature output by the boundary information acquisition branch and the depth information extraction branch to activate the boundary information contained therein.

[0077] S24. The residual connection structure in the residual connection branch is added to the original extracted R1 and R2 features to ensure effective gradient backpropagation and prevent deep network degradation, resulting in the first enhanced features E1 and E2 containing boundary and texture information.

[0078] S103. The extracted third and fourth layer features R3 and R4 from the multi-layer features are used for target search to find multi-scale target features. Multiple parallel branches are used with convolutional kernels of different sizes to adaptively extract target features at different scales, resulting in a second enhanced feature containing target features of various scales. This significantly improves the model's ability to perceive multi-scale targets. Specifically, this includes the following steps:

[0079] S31. The first branch receives the R3 and R4 features extracted from the multi-layer features respectively. First, a convolution operation with a kernel width and height of 1 and 3 is performed, that is, convolution calculation is performed only in the width direction to extract local features in the horizontal direction. Then, the local features in the horizontal direction are subjected to another convolution operation with a kernel width and height of 3 and 1, that is, convolution calculation is performed only in the height direction to extract local features in the vertical direction. The combination of the two operations is equivalent to a single convolution operation with a kernel size of 3×3, but the combination of the two operations reduces the number of parameters and has higher computational efficiency. The features in the vertical direction are then subjected to a dilated convolution operation with a kernel size of 3×3 and an expansion rate of 3, so as to expand the receptive field to 7×7 without increasing the number of parameters and capture a wider range of contextual information.

[0080] S32. The second branch receives the R3 and R4 features extracted from the multi-layer features respectively. First, a convolution operation with a kernel width and height of 1 and 5 is performed, that is, convolution calculation is performed only in the width direction to enhance the capture ability of wide-range horizontal features. Then, the local features in the horizontal direction are subjected to another convolution operation with a kernel width and height of 5 and 1 to enhance the capture ability of wide-range vertical features. The combination of the two operations is equivalent to a single convolution operation with a kernel size of 5×5, but the computational complexity is significantly reduced. The features in the vertical direction are subjected to a dilated convolution operation with a kernel size of 5×5 and an expansion rate of 5 to expand the receptive field to 21×21 and further fuse large-scale target features.

[0081] S33. The R3 and R4 features extracted from the multi-layer features are received through the third branch. First, a convolution operation with a kernel width and height of 1 and 7 is performed, that is, convolution calculation is performed only in the width direction to extract ultra-wide horizontal features. Then, the local features in the horizontal direction are subjected to another convolution operation with a kernel width and height of 7 and 1, that is, convolution calculation is performed only in the height direction to extract ultra-wide vertical features. The combination of the two operations is equivalent to a convolution operation with a kernel size of 7×7, but the number of parameters is greatly reduced. The features in the vertical direction are subjected to a dilated convolution operation with a kernel size of 7×7 and an expansion rate of 7 to expand the receptive field to 43×43 and cover a larger range of global context information.

[0082] S34. The fourth branch receives the R3 and R4 features extracted from the multi-layer features respectively. First, a convolution operation with a kernel width and height of 1 and 9 is performed, that is, convolution calculation is performed only in the width direction to capture extremely wide horizontal features. Then, the local features in the horizontal direction are subjected to another convolution operation with a kernel width and height of 9 and 1, that is, convolution calculation is performed only in the height direction to capture extremely wide vertical features. The combination of the two operations is equivalent to a single convolution operation with a kernel size of 9×9, but the computational cost is significantly optimized. The features in the vertical direction are then subjected to a dilated convolution operation with a kernel size of 9×9 and an expansion rate of 9 to expand the receptive field to 73×73, achieving ultra-large range feature perception.

[0083] S35. Receive the features output from the four branches in steps S31-S34 and concatenate them along the channel dimension to generate fused features with multi-scale perception capability. Subsequently, the fused features undergo a convolution process with a kernel size of 1×1 to compress the channel dimension of the features, reduce redundant channels, and improve feature compactness. Finally, add the channel-compressed features to the original features R3 and R4 respectively to fuse local details and global context, ensuring the integrity of the original information, and output features E3 and E4 respectively.

[0084] S104: Receive the first enhanced features E1 and E2 and the second enhanced features E3 and E4, perform deep feature decoding and fusion on the received features, and construct an auxiliary semantic loss through an auxiliary semantic prediction head. Finally, output the decoded features J4, J3, and J2, which specifically include:

[0085] The second enhanced feature of E4 is received and subjected to a DropOut operation with a probability of 0.1. Then, two sets of convolutions with a kernel size of 3×3, batch normalization, and ReLU operations are performed to achieve deep feature decoding. Finally, bilinear upsampling is performed twice to restore the feature to twice the size of the second enhanced feature of E4.

[0086] The upsampled features obtained above are subjected to a convolution with a kernel size of 3×3, and then upsampled to the same size as the original image through bilinear interpolation to obtain an output tensor of 2×300×300. This tensor is then used to construct an auxiliary semantic loss with the segmentation label image (this step is only used during model training and is discarded during model testing).

[0087] The obtained upsampled features are added element-wise with the received E1, E2, and E3 features to achieve the fusion of the decoded features and the original features, and the decoded features J4, J3, and J2 are output respectively.

[0088] The construction of the auxiliary semantic loss specifically includes the following:

[0089] Collect all PNG format segmentation label images from the training and validation sets;

[0090] The single-channel segmented label image is read using the OpenCV library, with pixel values ​​defined as: background = 0, disease area = 1.

[0091] Perform the following operations on each segmented label image: Use the np.unique() function to count the number of pixels in each category in the current image;

[0092] Calculate the temporary class probability of the current image. i represents the current image number, and class represents the temporary category index.

[0093]

[0094] The non-normalized weights of the current image are calculated as follows:

[0095]

[0096] in, This represents the temporary class weights for the i-th image (the current image); (unnormalized)

[0097] After summing the temporary class weights for all images, mean normalization is performed to obtain the final average weight of the temporary class across all images. final N is the total number of images, and the formula is as follows:

[0098]

[0099] Setting the temporary category to 0 and performing the above operation completes the calculation of the category weight for category 0; setting the temporary category to 1 and performing the above operation completes the calculation of the category weight for category 1; the category weights of category 0 and category 1 are represented by a one-dimensional tensor of length 2, i.e., the category splitting ratio weight. s ;

[0100] The auxiliary semantic loss is constructed as follows:

[0101] L aux_seg =BinaryCrossEntropyLoss(auxiliary segmentation result, segmentation label image, weight) s +Lovaszsoftmax (auxiliary segmentation result, segmentation label image)

[0102] Among them, L aux_seg To assist in semantic loss, the auxiliary segmentation result is represented as the output of the auxiliary semantic prediction head.

[0103] S105. Receive the decoded features J2, perform Dropout operation, convolutional neural network operation, and bilinear upsampling on them, and construct a semantic loss with the segmented label image to output the semantic prediction result, specifically including:

[0104] For the received decoded feature J2, a Dropout operation with a probability of 0.1 is first performed, followed by two sets of "3×3 convolution + batch normalization + ReLU" operations, and then a 2x bilinear upsampling process to restore the feature tensor to a size of 64×150×150. To filter redundant features and improve model accuracy, the 64×150×150 feature tensor is again processed by a set of "3×3 convolution + batch normalization + ReLU" operations, followed by another 2x bilinear upsampling process to restore the feature tensor to a size of 64×300×300. Finally, a 3×3 convolutional layer is used for semantic decoding to obtain a 2×300×300 output tensor, which is then used to construct a semantic loss with the segmentation label image.

[0105] The steps for constructing the semantic loss are the same as those for constructing the auxiliary semantic loss, and the formula is as follows:

[0106] L seg =BinaryCrossEntropyLoss(segmentation result, segmentation label image, weight) s +Lovaszsoftmax(segmentation result, segmentation label image)

[0107] Among them, L seg For semantic loss, the segmentation result is represented by the output of the segmentation head output module.

[0108] Steps S100-S105 are used in the model testing phase. These steps will be used during model training preparation and training, but will not be used in the model testing phase.

[0109] S106. Perform boundary extraction on the segmented label image to obtain a boundary label image, whose pixel values ​​include 0 and 1, specifically including:

[0110] S61. Specify the image path from which the boundary needs to be extracted and the path from which the boundary label image should be output;

[0111] S62. Read all segmentation label images under the image path from which the boundary needs to be extracted, iterate through each segmentation label image one by one, and create a single-channel PNG image with the same size as the segmentation label image and all pixel values ​​of 0, which will be used to fill in the boundary information to obtain the boundary label image.

[0112] S63. Fill the four sides of the segmentation label image with a 2-pixel-wide border with a pixel value of 0 to complete the boundary extraction of the target at the image boundary. Create a 5×5 sliding window and check the 5×5 neighborhood of each pixel. If there are more than two pixel values ​​in the window, the current pixel is considered to be the boundary and its corresponding pixel value in the boundary label image is set to 1.

[0113] S64. After the 5×5 sliding window has traversed the current image, the resulting boundary label image is saved in the path of the output boundary label image.

[0114] S107. Receive E1, E2 (first enhanced features), and E4 (second enhanced features), fuse the received features, and output the boundary prediction result of the rice disease area. Construct a boundary supervision loss by combining the boundary prediction result with the boundary label image, forcing the model to pay more attention to the target boundary. This specifically includes the following steps:

[0115] S71. The E2 feature in the first enhancement feature is subjected to a convolution operation with a kernel size of 1×1 to achieve channel dimension compression and cross-channel feature recombination. Then, it is upsampled to the same size as the E1 feature in the first enhancement feature, i.e., 64×75×75. Subsequently, the upsampled feature and the E1 feature are concatenated along the channel dimension to obtain the fused feature. The fused feature is then subjected to a "3×3 convolution + batch normalization + ReLU" operation to achieve feature dimensionality reduction and information extraction, resulting in the primary fused feature.

[0116] S72. The E4 feature in the second enhancement feature is subjected to a convolution with a kernel size of 1×1 and an 8x bilinear upsampling operation to obtain the same size as the E1 feature, i.e., 64×75×75. Then, this feature is concatenated with the primary fusion feature along the channel dimension, and then subjected to a "3×3 convolution + batch normalization + ReLU" operation to compress the number of feature channels from 128 to 64, fully fusing feature information to obtain the intermediate fusion feature.

[0117] S73. After global average pooling of the intermediate fusion features, a convolution operation with a kernel size of 1×1 is performed to reduce the dimensionality of the feature channels from 64 dimensions to 8 dimensions. Then, a convolution operation with a kernel size of 1×1 is performed to increase the dimensionality of the feature channels from 8 dimensions to 64 dimensions. At this time, the feature size is 64×75×75. The purpose is to make the model focus more on effective information and filter redundant information. The filtered features are processed by the Sigmoid function to activate the effective features. The effective features are multiplied with the original intermediate fusion features to obtain the features with enhanced boundary information.

[0118] S74. The enhanced features are subjected to a convolution operation with a kernel size of 3×3, and then subjected to 4 times bilinear upsampling to obtain a boundary prediction result tensor with a shape of 2×300×300.

[0119] The construction of the boundary supervision loss specifically includes:

[0120] Collect all PNG format boundary label images from the training and validation sets;

[0121] Read single-channel boundary label images using the OpenCV library, with pixel values ​​defined as: background = 0, disease area = 1;

[0122] Perform the following operations on each boundary label image: Use the np.unique() function to count the number of pixels in each category in the current image;

[0123] Calculate the temporary class probability of the current image. i represents the current image number, and class represents the temporary category index.

[0124]

[0125] The non-normalized weights of the current image are calculated as follows:

[0126]

[0127] in, This represents the temporary class weights for the i-th image (the current image); (unnormalized)

[0128] After summing the temporary class weights for all images, mean normalization is performed to obtain the final average weight of the temporary class across all images. final N is the total number of images, and the formula is as follows:

[0129]

[0130] Setting the temporary class to 0 and performing the above operation completes the calculation of the class weight for class 0; setting the temporary class to 1 and performing the above operation completes the calculation of the class weight for class 1; the class weights of class 0 and class 1 are represented by a one-dimensional tensor of length 2, i.e., the boundary class ratio weight. e ;

[0131] The boundary supervision loss is constructed as follows:

[0132] L edge =BinaryCrossEntropyLoss(boundary segmentation result, boundary label image, weight) e )

[0133] Among them, L edhe The boundary supervision loss is represented by the boundary segmentation result, which is the output of the boundary supervision module.

[0134] S108. Receive the boundary prediction result and semantic prediction result, then construct semantic-boundary loss and boundary-semantic loss by segmenting the label image and the boundary label image. Based on the semantic-boundary loss and boundary-semantic loss, construct a regularization loss. The regularization loss is used to mitigate the gap between semantic segmentation and boundary detection tasks, ensuring that the two tasks do not conflict and promote each other during model training. It is only used during training and specifically includes:

[0135] The boundary label image is binarized as follows:

[0136]

[0137] The softmax probability is calculated along the channel dimension of the semantic prediction result tensor output by the segmentation head output module. Then, a convolution operation with a kernel size of 5×5, a center point weight of 1, and weights of -1 in the 8 neighborhood directions is applied to each category channel to obtain the boundary response tensor. The boundary response tensor and the boundary label are used to construct the L2 loss, which is the semantic-boundary loss.

[0138] Pixels whose tensor median of the boundary prediction result is greater than a fixed threshold of 0.7 are set as predicted boundaries. The true boundaries in the boundary label image are extracted, and the predicted boundaries are merged with the true boundaries to generate a hard boundary mask. The original segmentation label image is cloned, and non-boundary areas are ignored through the hard boundary mask while boundary areas are retained to obtain hard semantic labels. The cross-entropy loss between the semantic prediction result and the hard semantic labels is constructed, which is the boundary-semantic loss.

[0139] The regularization loss is equal to the weighted sum of the semantic-boundary loss and the boundary-semantic loss, with weights of 1.5 and 1.5 respectively. The regularization loss makes the model training process more stable and controllable, and achieves mutual promotion between the semantic segmentation task and the boundary detection task.

[0140] S109. Construct a rice disease identification model, and apply the constructed semantic loss, auxiliary semantic loss, boundary supervision loss, and regularization term loss to the training of the rice disease identification model. Apply the rice disease identification model to identify diseases in rice images.

[0141] Table 1 Comparison of segmentation performance and model size of mainstream methods.

[0142]

[0143]

[0144] To verify the detection performance of this invention, the proposed method was compared with existing semantic segmentation methods, including FCN-8s, UNet, UNet++, PSPNet, Deeplabv3+, HRNet, and Segformer. The inference speed of the proposed method during the training phase was also compared to illustrate the difference in inference speed between training and testing. All methods were trained and tested on the same device using a uniform dataset of collected rice samples. The test results are shown in Table 1 above, which visually demonstrates the quantitative performance comparison between the proposed method and existing semantic segmentation methods. Four metrics were selected for quantitative evaluation: mean intersection-union ratio (mIoU), Dice coefficient, recall, and precision.

[0145] The comparison results of the quantitative indicators shown in Table 1 show that the method of the present invention has a great advantage in extraction accuracy compared with other existing methods, and can achieve better semantic segmentation performance.

[0146] Another embodiment of the present invention provides a rice disease identification system, the system comprising:

[0147] Image acquisition module: used to acquire images of diseased rice leaves and transmit the images of diseased rice leaves to the feature extraction module. The image acquisition module includes a mobile phone, etc.

[0148] Image annotation module: used to label the diseased areas in the collected rice leaf images to obtain an annotated rice leaf image, i.e., a segmented label image;

[0149] Feature extraction module: This module receives rice leaf images transmitted from the image acquisition module and extracts features from the rice leaf images to obtain multi-layer features of the rice leaf images. These multi-layer features are labeled as 4-layer features, which are respectively labeled as R1, R2, R3 and R4. The 4-layer features are then transmitted to the feature enhancement stage, namely the boundary generation module and the target search module.

[0150] Boundary generation module: This module receives features R1 and R2 from the feature extraction module and performs boundary information enhancement processing on them. It first extracts depth features and fuses boundary texture information, then retains the original features through residual connections. Finally, it outputs a first enhanced feature containing rich boundary and texture information, namely features E1 and E2, and transmits it to the feature aggregation module, specifically including:

[0151] The boundary generation module includes a depth information extraction branch, a boundary information acquisition branch, and a residual connection branch;

[0152] The deep information extraction branch uses two "3×3 convolution + batch normalization + ReLU" operations to mine deep information in shallow features and output the second boundary feature.

[0153] The boundary information acquisition branch is used to obtain rich boundary discrimination information through Sobel operator operation. After activating the boundary features through 1×1 convolution and Sigmoid operation, the first boundary feature is obtained, and irrelevant noise information is suppressed. Then, the second boundary feature is multiplied with the first boundary feature to obtain the boundary activation and enhancement feature. The feature is added to the received R1 and R2 features through the residual connection branch. After a "3×3 convolution + batch normalization + ReLU" operation, the feature is integrated to obtain the first enhanced feature containing boundary and texture information.

[0154] The target search module receives R3 and R4 features from the feature extraction module, performs target searches on the R3 and R4 features respectively, finds multi-scale target features, and obtains second enhanced features containing target features of various scales. The second enhanced features are E3 and E4 features. The enhanced features are then transmitted to the feature aggregation module, which specifically includes:

[0155] The target search module includes a multi-level receptive field feature acquisition branch and a residual connection branch;

[0156] The multi-level receptive field feature acquisition branch includes four sets of convolution operations at different scales. Each set consists of horizontal and vertical one-dimensional convolutions, standard two-dimensional convolutions, and dilated convolutions to capture local and global features at different scales. The local and global features at different scales are concatenated and fused using the channel dimension, then compressed using 1×1 convolutions, and enhanced using residual connection branches. This improves the model's ability to perceive target boundaries while maintaining global information.

[0157] Feature aggregation module: This module receives E1 and E2 features from the boundary generation module and E3 and E4 features from the target search module. First, it randomly discards E1, E2, E3, and E4 features to reduce redundancy and enhance robustness. Then, it restores the feature resolution through convolution and upsampling operations. The upsampled features are fused with the features from the previous stage and, on the other hand, generate intermediate supervision signals (used only during training) through an auxiliary semantic prediction head to optimize the model's learning ability. These signals are also used to construct an auxiliary semantic loss function with the segmentation label images. Finally, the fused multi-layer decoded features are output, specifically including:

[0158] The feature aggregation module first randomly discards E1, E2, E3 and E4 features to reduce feature redundancy and enhance model robustness. Then, it uses two sets of "3×3 convolution + batch normalization + ReLU" operations to decode depth information. After that, it performs double upsampling to obtain upsampled image features. The upsampled image features are then added to the features from the previous stage to achieve feature fusion.

[0159] On the other hand, the upsampled image features are subjected to an upsampling operation through a 3×3 convolution layer, i.e., an auxiliary semantic prediction head, to achieve auxiliary semantic supervision and construct an auxiliary semantic loss with the segmentation label image. The auxiliary semantic prediction head used by the feature aggregation module is only used during model training. Its purpose is to predict rice disease targets in the current layer, construct auxiliary semantic loss, and improve model performance. The auxiliary semantic prediction head is discarded during model testing and inference.

[0160] Boundary label generation module: used to obtain the target boundary from the segmentation label image to obtain an image with a binary mask of the target boundary, i.e., the boundary label image. During model training, the boundary label image is used together with the output of the boundary supervision module to construct the boundary supervision loss.

[0161] Boundary Supervision Module: This module receives E1 and E2 features from the boundary generation module and E4 enhanced features from the target search module. By fusing E1, E2, and E4 features, it outputs the boundary prediction results for rice disease areas. With the help of the spatial attention module, it enhances the model's attention to boundary information. The output of the boundary supervision module is used together with the boundary label image to construct the boundary supervision loss. The boundary supervision module is only used during the model training phase and is not enabled during the testing phase. Its purpose is to reduce the number of model parameters and improve the model inference speed.

[0162] The segmentation head output module is used to receive the output of the last layer feature aggregation module and output the semantic prediction result to construct the semantic loss. The segmentation head output module consists of a DropOut operation with a probability of 0.1, two sets of "3×3 convolution + batch normalization + ReLU" operations, double bilinear upsampling, a set of "3×3 convolution + batch normalization + ReLU", and a 3×3 convolution operation.

[0163] Regularization term loss module: It receives the output of the boundary supervision module and the semantic prediction results output by the segmentation head output module. It constructs semantic-boundary loss and boundary-semantic loss by segmenting the label image and the boundary label image. The weighted sum of the semantic-boundary loss and the boundary-semantic loss is the regularization term loss.

[0164] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying rice diseases, characterized in that, The method includes the following steps: S100. Collect images of rice leaves with diseases, and then label the diseased areas in the collected rice leaf images to obtain segmented labeled images. S101. Extract features from the collected rice leaf images to obtain multi-layer feature information of the rice leaf images; S102. Perform boundary information enhancement processing on some features in the extracted multi-layer features to obtain the first enhanced feature containing boundary and texture information; S103. Perform target search on some of the extracted multi-layer features to find multi-scale target features. Use convolution kernels of different sizes in multiple parallel branches to adaptively extract target features of different scales and obtain a second enhanced feature containing target features of various scales. S104. Receive the first enhanced feature and the second enhanced feature, perform deep feature decoding and fusion on the received features, construct an auxiliary semantic loss through the auxiliary semantic prediction head, and finally output multiple decoded features. S105. Receive partially decoded features, perform Dropout operation, convolutional neural network operation, and bilinear upsampling on them, construct semantic loss with the segmented label image, and output semantic prediction results; S106. Extract the boundaries of the segmented label image to obtain the boundary label image; S107. Receive the first enhanced feature and part of the second enhanced feature, fuse the received features and output the boundary prediction result of the rice disease area, and construct the boundary supervision loss with the boundary prediction result and the boundary label image. S108. Receive the boundary prediction result and the semantic prediction result, and then construct the semantic-boundary loss and the boundary-semantic loss by segmenting the label image and the boundary label image. Construct the regularization term loss based on the semantic-boundary loss and the boundary-semantic loss. S109. Construct a rice disease identification model, and apply the constructed semantic loss, auxiliary semantic loss, boundary supervision loss, and regularization term loss to the training of the rice disease identification model. Apply the rice disease identification model to identify diseases in rice images.

2. The method for identifying rice diseases according to claim 1, characterized in that, Step S102 specifically includes the following steps: S21. Perform boundary information enhancement processing, including a depth information extraction branch, a boundary information acquisition branch, and a residual connection branch. The boundary information acquisition branch receives some features from the extracted multi-layer features. First, it performs Sobel operator operation to calculate its gradient in the horizontal and vertical directions. After taking the absolute values, it is added to obtain the first boundary feature. Then, a convolutional layer with a kernel size of 1×1 is used to realize feature interaction and recombination of the first boundary feature. Finally, the first boundary feature is normalized by the Sigmoid activation function. S22. The deep information extraction branch receives partial features from the extracted multi-layer features respectively. After two "convolution + batch normalization + ReLU" operations, the deep information in the shallow features is mined, local features are extracted and spatial information is enhanced, and the second boundary feature is output. S23. Multiply the first boundary feature and the second boundary feature output by the boundary information acquisition branch and the depth information extraction branch to activate the boundary information contained therein. S24. The residual connection structure in the residual connection branch is added to the original extracted partial features. After a "convolution + batch normalization + ReLU" operation, the first enhanced feature containing boundary and texture information is obtained.

3. The method for identifying rice diseases according to claim 2, characterized in that, Step S103 specifically includes the following steps: S31. The first branch receives some features from the extracted multi-layer features. First, a convolution operation with kernel width and height of 1 and 3 is performed. Then, the local features in the horizontal direction are subjected to another convolution operation with kernel width and height of 3 and 1. The features in the vertical direction are subjected to a dilated convolution operation with kernel size of 3×3 and dilation rate of 3 to expand the receptive field to 7×7. S32. The second branch receives some features from the extracted multi-layer features. First, a convolution operation with kernel width and height of 1 and 5 is performed. Then, the local features in the horizontal direction are subjected to another convolution operation with kernel width and height of 5 and 1. The features in the vertical direction are subjected to a dilated convolution operation with kernel size of 5×5 and dilation rate of 5 to expand the receptive field to 21×21. S33. The third branch receives some features from the extracted multi-layer features. First, a convolution operation with kernel width and height of 1 and 7 is performed. Then, the local features in the horizontal direction are subjected to another convolution operation with kernel width and height of 7 and 1. The features in the vertical direction are subjected to a dilated convolution operation with kernel size of 7×7 and dilation rate of 7 to expand the receptive field to 43×43. S34. The fourth branch receives partial features from the extracted multi-layer features. First, a convolution operation with kernel width and height of 1 and 9 is performed, that is, convolution calculation is performed only in the width direction. Then, the local features in the horizontal direction are subjected to another convolution operation with kernel width and height of 9 and 1, that is, convolution calculation is performed only in the height direction. The features in the vertical direction are subjected to a dilated convolution operation with kernel size of 9×9 and dilation rate of 9, expanding the receptive field to 73×73. S35. Receive the features output from the four branches in steps S31-S34 and concatenate them along the channel dimension to generate fused features with multi-scale perception capability. Subsequently, the fused features undergo a convolution process with a kernel size of 1×1 to compress the channel dimension of the features, and feature enhancement is performed using residual connection branches. Finally, the features compressed by the channel dimension are added to some features in the original extracted multi-layer features to fuse local details and global context.

4. The method for identifying rice diseases according to claim 1, characterized in that, Step S105 specifically includes: For the received partially decoded features, a Dropout operation with a probability of 0.1 is first performed. Then, two sets of "convolution + batch normalization + ReLU" operations are performed, followed by a 2x bilinear upsampling process to restore the feature tensor to a size of 64×150×150. To filter redundant features, the 64×150×150 feature tensor is subjected to another set of "convolution + batch normalization + ReLU" operations, followed by another 2x bilinear upsampling process to restore the feature tensor to a size of 64×300×300. Finally, a convolutional layer is used for semantic decoding to obtain a 2×300×300 output tensor, which is then used to construct a semantic loss with the segmentation label image to output the semantic prediction result.

5. The method for identifying rice diseases according to claim 1, characterized in that, Step S106 specifically includes the following steps: S61. Specify the image path from which the boundary needs to be extracted and the path from which the boundary label image should be output; S62. Read all segmentation label images under the image path from which the boundary needs to be extracted, iterate through each segmentation label image one by one, and create a single-channel PNG image with the same size and pixel count as the segmentation label image. This image will be used to fill in the boundary information to obtain the boundary label image. S63. Fill the pixel boundaries around the segmented label image and create a 5×5 sliding window to check the 5×5 neighborhood of each pixel. S64. After the 5×5 sliding window has traversed the current image, the resulting boundary label image is saved in the path of the output boundary label image.

6. The method for identifying rice diseases according to claim 2, characterized in that, Step S107 specifically includes the following steps: S71. A portion of the first enhancement features is subjected to a convolution operation, and then upsampled to the same size as another portion of the first enhancement features. The upsampled features are concatenated with the other portion of the first enhancement features along the channel dimension to obtain a fused feature. This fused feature is then subjected to a "convolution + batch normalization + ReLU" operation to obtain a primary fused feature. S72. A portion of the second enhancement features is subjected to a convolution and bilinear upsampling operation to obtain a portion of the first enhancement features of the same size. This feature is then concatenated with the primary fusion feature along the channel dimension. After another "convolution + batch normalization + ReLU" operation, the number of feature channels is compressed to fully fuse feature information and obtain the intermediate fusion feature. S73. After global average pooling of the intermediate fusion features, a convolution operation is first performed to reduce the dimensionality of the feature channels, followed by another convolution operation to increase the dimensionality of the feature channels and filter redundant information. The filtered features are then processed by the Sigmoid function to activate the effective features. The effective features are then multiplied with the original intermediate fusion features to obtain the features with enhanced boundary information. S74. The enhanced features are subjected to a convolution operation and then bilinear upsampling to obtain the boundary prediction result tensor.

7. A rice disease identification system, characterized in that, The system for implementing any one of claims 1-6 comprises: Image acquisition module: used to acquire images of diseased rice leaves and transmit the images of diseased rice leaves to the feature extraction module; Image annotation module: used to label the diseased areas in the collected rice leaf images to obtain an annotated rice leaf image, i.e., a segmented label image; Feature extraction module: Used to receive rice leaf images transmitted by the image acquisition module, and to extract features from the rice leaf images to obtain multi-layer features of the rice leaf images; Boundary generation module: It is used to receive some features from the feature extraction module and perform boundary information enhancement processing on the received partial features to obtain the first enhanced feature containing boundary and texture information; Target search module: It is used to receive some features from the feature extraction module, and perform target search on each of the partial features to find multi-scale target features and obtain a second enhanced feature containing target features of various scales; Feature aggregation module: It is used to receive the first enhanced feature and the second enhanced feature, randomly discard the received features, and then restore the feature resolution through convolution and upsampling operations. The upsampled features are fused with the features of the previous stage on the one hand, and an intermediate supervision signal is generated through the auxiliary semantic prediction head on the other hand. It is also used to construct an auxiliary semantic loss with the segmentation label image, and finally output the fused multi-layer decoding features. Boundary label generation module: used to obtain the target boundary from the segmented label image to obtain the boundary label image; Boundary supervision module: It is used to receive the first enhanced features and part of the second enhanced features, fuse the received features and output the boundary prediction results of the rice disease area. The output of the boundary supervision module is used to construct the boundary supervision loss together with the boundary label image. The segmentation head output module is used to receive the output of the last layer feature aggregation module and output the semantic prediction result to construct the semantic loss. The segmentation head output module consists of DropOut operation, two sets of "convolution + batch normalization + ReLU" operation, double bilinear upsampling, one set of "convolution + batch normalization + ReLU" operation, and convolution operation. Regularization term loss module: It receives the semantic prediction results from the boundary supervision module and the segmentation head output module. It constructs semantic-boundary loss and boundary-semantic loss by segmenting the label image and the boundary label image, and constructs regularization term loss based on semantic-boundary loss and boundary-semantic loss.

8. A rice disease identification system according to claim 7, characterized in that, The boundary generation module includes a depth information extraction branch, a boundary information acquisition branch, and a residual connection branch; The deep information extraction branch uses two "convolution + batch normalization + ReLU" operations to mine deep information in shallow features and output the second boundary feature. The boundary information acquisition branch is used to obtain boundary discrimination information through Sobel operator operation, activate boundary features through convolution and Sigmoid operation to obtain the first boundary feature, and multiply the second boundary feature with the first boundary feature to obtain the boundary activated and enhanced feature. The feature is added to the received partial feature through the residual connection branch, and after one "convolution + batch normalization + ReLU" operation, the first enhanced feature containing boundary and texture information is obtained.

9. A rice disease identification system according to claim 7, characterized in that, The target search module includes a multi-level receptive field feature acquisition branch and a residual connection branch; The multi-level receptive field feature acquisition branch includes four sets of convolution operations at different scales. Each set consists of horizontal and vertical one-dimensional convolution, standard two-dimensional convolution, and dilated convolution to capture local and global features at different scales. The local and global features at different scales are spliced ​​and fused using the channel dimension, then compressed by convolution, and feature enhancement is performed using residual connection branches.

Images