AI-based identification method for plant diseases and pests based on MHMT-CLR

Abstract

This invention relates to the field of image processing technology, specifically to an AI-based method for plant disease and pest identification based on MHMT-CLR. The method includes: acquiring initial and verification images of plant leaves; analyzing the abnormal development behavior of abnormal regions in the images to determine disease feature values; and, when the disease feature values ​​exceed a disease feature threshold, processing the verification image based on a pre-trained MHMT-CLR model to obtain disease identification results indicating the disease category of the plant corresponding to the plant leaves. The MHMT-CLR model includes multiple head convolutional networks corresponding to multiple plant categories, a SimCLR backbone network, and multiple tail convolutional networks corresponding to multiple plant categories. The parameters of the SimCLR backbone network are frozen during the training of the multiple head and tail convolutional networks. This invention improves the accuracy and versatility of plant disease and pest identification schemes.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to an AI-based method for identifying plant diseases and pests based on MHMT-CLR. Background Technology

[0002] Plant disease and pest identification is a crucial aspect of agricultural production and forestry conservation. Traditional identification methods primarily rely on agricultural experts or experienced practitioners who judge pests by observing plant morphological characteristics, lesion patterns, color changes, and pest morphology. However, this method has significant limitations, including high subjectivity, low efficiency, over-reliance on personal experience, and the need for substantial human resources. These factors make large-scale application across extensive agricultural and forestry areas difficult.

[0003] With the continuous development of computer technology, early automated identification technologies based on image processing and machine learning emerged. These technologies typically combine traditional feature extraction algorithms (such as SIFT and HOG) with classifiers (such as decision trees and support vector machines) for pest and disease identification. This significantly reduced manual labor costs to some extent. However, these methods still have significant shortcomings in feature representation and generalization ability, especially in complex situations such as changes in lighting, background interference, and diverse pest and disease morphologies, where the identification accuracy often fails to meet the needs of practical applications.

[0004] In recent years, deep learning technology has developed rapidly, especially with the continuous breakthroughs in image recognition based on convolutional neural networks (CNN), which have gradually shown the great potential of deep learning in the identification of plant diseases and pests.

[0005] With the development of deep learning-based image recognition technology for plant diseases and pests, the main technical solutions for current AI-based plant disease and pest recognition algorithms focus on three aspects: end-to-end deep learning solutions, multimodal fusion solutions, and lightweight and embedded deployment solutions. These solutions each have their own advantages in image acquisition, feature extraction, model building, and practical applications.

[0006] However, in application, it was found that the plant disease and pest identification scheme did not perform as well as expected due to factors such as significant differences in the number of samples of different plant diseases and pests, similar disease manifestations of different plant diseases and pests, and continuous changes in plant diseases and pests. Summary of the Invention

[0007] The purpose of this invention is to provide an AI-based method for identifying plant diseases and pests based on MHMT-CLR, in order to solve the technical problem of poor application effect of existing plant disease and pest identification schemes.

[0008] In a first aspect, one embodiment of the present invention provides a plant disease and pest AI identification method based on MHMT-CLR, the method comprising: Acquire initial and verification images of the plant leaves; The abnormal development behavior of abnormal regions changing from the initial image to the verification image is analyzed to determine disease feature values, wherein the disease feature values ​​are used to indicate the probability that the plant leaves have suffered from diseases. When the disease feature value is greater than the disease feature threshold, the verification image is processed based on the pre-trained MHMT-CLR model to obtain the disease identification result indicating the disease category of the plant corresponding to the plant leaf. The MHMT-CLR model includes multiple head convolutional networks, a SimCLR backbone network, and multiple tail convolutional networks. The multiple head convolutional networks correspond one-to-one with multiple plant categories, and the multiple tail convolutional networks correspond one-to-one with the multiple plant categories. The parameters of the SimCLR backbone network are frozen during the training of the multiple head convolutional networks and the multiple tail convolutional networks.

[0009] In some embodiments, the plurality of head convolutional networks and the plurality of tail convolutional networks are deployed in an edge device, the SimCLR backbone network is deployed in a remote server, the edge device and the remote server are communicatively connected, and the edge device is located in the planting area of ​​the plant corresponding to the plant leaf.

[0010] In some embodiments, the step of analyzing the abnormal development behavior of abnormal regions changing from an initial image to a review image to determine disease characteristic values ​​includes: Geometric registration is performed on the verification image based on the initial image to obtain the registered image; The brightness distribution of the registered image is corrected based on the brightness distribution of the initial image to obtain a corrected image; The abnormal development behavior of abnormal regions in the initial image is analyzed to transform them into abnormal regions in the corrected image in order to determine the disease characteristic values.

[0011] In some embodiments, the step of analyzing the abnormal development behavior of abnormal regions changing from an initial image to a corrected image to determine disease characteristic values ​​includes: Texture analysis is performed on the initial image to obtain multiple resistance feature values, wherein the resistance feature values ​​are used to indicate the degree to which the corresponding pixel position in the initial image hinders the natural spread of the disease; Based on the multiple resistance characteristic values, the abnormal development behavior of the abnormal regions in the initial image changing to the abnormal regions in the corrected image is analyzed to determine the disease characteristic values.

[0012] In some embodiments, the step of performing texture analysis on an initial image to obtain multiple resistance feature values ​​includes: The gray-level gradient of each pixel in the initial image is analyzed to obtain multiple gradient feature values, which are used to indicate the degree of gray-level change at the corresponding pixel position in the initial image. The multiple gradient feature values ​​are corrected based on the ambient humidity to obtain multiple resistance feature values. The ambient humidity is used to indicate the average humidity of the environment in which the plant leaves are located during the interval period, and the interval period is the time period between the acquisition time of the initial image and the acquisition time of the verification image. The resistance feature value is used to indicate the degree to which the corresponding pixel position in the initial image hinders the natural spread of the disease.

[0013] In some embodiments, the step of analyzing the grayscale gradient of each pixel in the initial image to obtain multiple gradient feature values ​​includes: Eigenvalue decomposition is performed on the structure tensor corresponding to the target pixel to obtain the principal eigenvalue and secondary eigenvalue corresponding to the target pixel, wherein the target pixel is any one of the multiple pixels included in the initial image; Calculate the sum of the principal feature value and the secondary feature value corresponding to the target pixel to obtain the feature sum value corresponding to the target pixel, and calculate the difference between the principal feature value and the secondary feature value corresponding to the target pixel to obtain the feature difference value corresponding to the target pixel. By analyzing the difference between the feature value corresponding to the target pixel and its corresponding feature difference, the gradient feature value corresponding to the target pixel is obtained.

[0014] In some embodiments, the step of analyzing the abnormal development behavior of abnormal regions changing from the initial image to the corrected image based on the plurality of resistance characteristic values, in order to determine the disease characteristic values, includes: Anomaly detection is performed on the initial image to identify basic anomaly regions; Difference analysis is performed on the corrected image based on the initial image to identify newly added anomalous regions, where the basic anomalous regions and the newly added anomalous regions are different. Based on the multiple resistance characteristic values, the consistency between the regional flow channel and the texture structure of the plant leaves is analyzed to obtain the connectivity characteristic value. The regional flow channel is used to simulate the regional development channel between the basic abnormal region and the newly added abnormal region. Based on the multiple resistance characteristic values, the diffusion cost from the basic anomaly region to the newly added anomaly region is analyzed to obtain the cost characteristic value; Based on the connectivity feature value and the cost feature value, the disease feature value is determined.

[0015] In some embodiments, the step of performing difference analysis on the corrected image based on the initial image to determine newly added anomalous regions includes: Noise filtering is applied to the initial image and the corrected image respectively to obtain the initial smoothed image and the corrected smoothed image; The differences between the initial smoothed image and the corrected smoothed image in the LAB color space are analyzed to obtain a color difference map; Perform histogram equalization on the color difference map to obtain the difference histogram; Based on the difference histogram, the correction anomaly region is determined, and the region corresponding to the basic anomaly region is removed from the correction anomaly region to obtain the new anomaly region, wherein the correction anomaly region indicates the color deviation region between the corrected image and the initial image.

[0016] In some embodiments, the step of analyzing the consistency between the regional flow channels and the texture structure of plant leaves based on the plurality of resistance characteristic values ​​to obtain connectivity characteristic values ​​includes: Using multiple pixels in the initial image as graph nodes, the edge capacity between adjacent graph nodes is analyzed based on the multiple resistance feature values ​​to form a flow network graph, wherein the edge capacity is negatively correlated with the resistance feature value of the corresponding pixel. Based on the flow network diagram, the maximum flow from the virtual source point corresponding to the basic anomaly region to the virtual sink point corresponding to the newly added anomaly region is analyzed to obtain the anomaly development flow. The ratio of the abnormal development flow to the maximum inflow flow corresponding to the newly added abnormal region is determined as the connectivity characteristic value.

[0017] In some embodiments, the disease feature value is positively correlated with the connectivity feature value, and the disease feature value is negatively correlated with the cost feature value.

[0018] Secondly, another embodiment of the present invention provides a plant disease and pest AI identification system based on MHMT-CLR, the system comprising: The image acquisition module is used to acquire initial and verification images of plant leaves; The initial image inspection module is used to analyze the abnormal development behavior of abnormal areas changing from the initial image to the verification image, so as to determine the disease feature value, wherein the disease feature value is used to indicate the probability that the plant leaves have suffered from disease. The disease identification module is used to process the verification image based on a pre-trained MHMT-CLR model when the disease feature value is greater than the disease feature threshold, to obtain a disease identification result indicating the disease category of the plant corresponding to the plant leaf. The MHMT-CLR model includes multiple head convolutional networks, a SimCLR backbone network, and multiple tail convolutional networks. The multiple head convolutional networks correspond one-to-one with multiple plant categories, and the multiple tail convolutional networks correspond one-to-one with multiple plant categories. The parameters of the SimCLR backbone network are frozen during the training of the multiple head convolutional networks and the multiple tail convolutional networks.

[0019] Thirdly, in another embodiment of the present invention, an electronic device is provided, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method described in the first aspect.

[0020] Fourthly, in another embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0021] The present invention has the following beneficial effects: By analyzing the changes in abnormal regions of plant leaves over a period of time, the probability of plant leaves suffering from disease is initially assessed. When disease is detected, a pre-trained MHMT-CLR model is used to further process the corresponding verification images of the plant leaves to accurately identify the specific pests or diseases affecting the plant. The step of initially assessing the probability of plant leaves suffering from disease avoids sending normal images or images without pest or disease damage into the MHMT-CLR model for processing. This allows for the use of traditional vision and flow network algorithms to analyze the developmental connectivity of abnormal regions, filtering out non-infectious mechanical damage caused by wind and sand scratches, agricultural machinery abrasion, etc., thus improving the MHMT-CLR model's performance. To address the issue of CLR models misclassifying new mechanical scratches as diseases, the SimCLR backbone network's parameter freezing settings ensure its accurate and reliable feature extraction from various leaf images. Furthermore, the multiple head and tail convolutional networks effectively adapt to the characteristics of different plant categories and their corresponding pest and disease datasets. Each plant category is configured with a relatively independent head and tail convolutional network to accurately diagnose the pest and disease categories of each plant. This also supports flexible expansion to include newly added plants and pest and disease categories, significantly enhancing the accuracy and versatility of the plant disease and pest identification scheme. Attached Figure Description

[0022] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating a plant disease and pest AI identification method based on MHMT-CLR provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a disease identification process based on the MHMT-CLR model provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the architecture of an MHMT-CLR model provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a head convolutional network provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a plant disease and pest AI identification system based on MHMT-CLR provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0024] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a plant disease and pest AI identification method based on MHMT-CLR proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0026] The following description, in conjunction with the accompanying drawings, details the specific scheme of the plant disease and pest AI identification method based on MHMT-CLR provided by this invention.

[0027] In one embodiment, the present invention provides an AI-based method for identifying plant diseases and pests based on MHMT-CLR, such as... Figure 1 As shown, the method includes: Step S1: Obtain the initial image and the verification image of the plant leaves.

[0028] In this invention, the time interval between the acquisition time of the initial image and the acquisition time of the verification image is a set time. This set time is used to capture the disease development characteristics of the plant affected by pests and diseases, so as to ensure the accuracy of the disease characteristic values ​​obtained in subsequent analysis. This set time can be set to 24 hours based on experience.

[0029] The initial image and the verification image are obtained from the same image acquisition device (such as a camera or mobile phone camera), and the acquisition position, acquisition angle and acquisition height of the image acquisition device are required to be consistent to ensure the smooth progress of subsequent analysis.

[0030] Step S2: Analyze the abnormal development behavior of the abnormal areas in the initial image to the abnormal areas in the verification image in order to determine the disease characteristic values.

[0031] The disease characteristic value is used to indicate the probability that the plant leaves are affected by disease.

[0032] Analysis revealed that in real-world environments, when plant leaves are not affected by pests, diseases, or non-infectious mechanical damage, images show stagnant development of abnormal areas. When plant leaves suffer non-infectious mechanical damage (such as scratches from agricultural operations or wind and sand abrasion), images still show the development of abnormal areas, but the morphological development of these abnormal areas is more random and easily conflicts with the direction of the leaf's texture (i.e., the direction of the leaf veins). When plant leaves are infected by pests and diseases, abnormal areas will also develop, and the spread process is affected by the physical barrier of the leaf veins, often showing a development similar to the direction of the leaf's texture.

[0033] Therefore, by analyzing the abnormal developmental behavior of plant leaves through images and combining the degree of matching between their developmental trend and leaf texture, the probability of plant leaves suffering from diseases can be assessed to some extent.

[0034] Step S3: When the disease feature value is greater than the disease feature threshold, the verification image is processed based on the pre-trained MHMT-CLR model to obtain the disease identification result indicating the disease category of the plant corresponding to the plant leaf.

[0035] The MHMT-CLR model includes multiple head convolutional networks, a SimCLR backbone network, and multiple tail convolutional networks. The multiple head convolutional networks correspond one-to-one with multiple disease categories, and the multiple tail convolutional networks correspond one-to-one with the multiple disease categories. The parameters of the SimCLR backbone network are frozen during the training of the multiple head convolutional networks and the multiple tail convolutional networks.

[0036] In practical applications, when the numerical values ​​of disease characteristics are normalized to the 0-1 range, the above disease characteristic threshold can be set to 0.3 based on experience (determined by taking the 95th percentile of the distribution based on the statistical distribution of a large number of historical mechanical damage samples).

[0037] When the disease characteristic value is greater than the disease characteristic threshold, it can be roughly understood as the detection that the plant corresponding to the leaf is infected by pests and diseases.

[0038] This invention analyzes changes in abnormal regions of plant leaves over a period of time to preliminarily assess the probability of leaf disease. Upon detecting leaf disease, the MHMT-CLR leaf disease detection algorithm, based on a pre-trained contrastive learning-based multi-headed multi-tailed network model, further processes the corresponding verification image to accurately identify the specific pest or disease affecting the plant. The step of preliminarily assessing the probability of leaf disease avoids sending normal images or images without pest or disease damage to the MHMT-CLR model for processing. This allows for the use of traditional visual and flow network algorithms to analyze the connectivity of abnormal regions, filtering out non-infectious damage caused by wind and sand scratches, agricultural machinery abrasion, etc. To address the issue of mechanical damage, the MHMT-CLR model is prone to misclassifying new mechanical scratches as diseases, thus reducing the false alarm rate. The parameter freezing settings of the SimCLR backbone network ensure that it can accurately and reliably extract features from various leaf images. The multiple head and tail convolutional networks effectively adapt to the characteristics of different plant categories and their corresponding pest and disease datasets. A relatively independent head and tail convolutional network is configured for each plant category to accurately diagnose the pest and disease categories of each plant. Simultaneously, it supports flexible expansion to include newly added plants and pest and disease categories, significantly enhancing the accuracy and versatility of the plant disease and pest identification scheme.

[0039] The Multi-Head Multi-Tail with Contrastive Learning Representation for Leaf Disease Detection (MHMT-CLR) algorithm, based on a contrastive learning-based multi-head multi-tail network model, aims to improve the generalization ability and accuracy of disease identification across multiple plant species. Its workflow can be described as follows: Figure 2 As shown, the user selects a plant category and takes an image of the plant leaves. After preliminary detection confirms that the disease feature value of the plant leaves is greater than the disease feature threshold, the plant category and the verified image of the plant leaves are reported. Then, the head convolutional network and tail convolutional network corresponding to the plant category are called, and the SimCLR backbone network is used to extract intermediate features to output the corresponding disease identification result—indicating the disease category (such as brown spot, anthracnose, and early blight) of the plant and the confidence level of the disease identification result.

[0040] It should be noted that in the architecture of the MHMT-CLR model (such as...) Figure 3 As shown, the SimCLR backbone network is trained before the other parts. The SimCLR backbone network is specifically trained using a large source dataset to ensure that the MHMT-CLR model has the ability to extract features of leaves of various plants. After the SimCLR backbone network is trained, the parameters of the SimCLR backbone network (mainly the parameters of the encoder used for feature extraction) are frozen. Using the idea of ​​transfer learning, the frozen encoder is combined with additional head and tail convolutional network structures to predict the disease types in smaller sample datasets corresponding to various plant categories.

[0041] The inclusion of head and tail convolutional networks ensures that the algorithm can be easily applied to various datasets. Images of different plant categories will be fed into their respective head and tail convolutional networks for training and testing, thereby avoiding problems such as knowledge forgetting and interference between different tasks.

[0042] The SimCLR backbone network specifically uses SimCLR and selects ResNet18, which has fewer parameters, as the encoder to avoid overfitting and other problems caused by excessively large datasets. The features extracted by ResNet18 are mapped to a lower-dimensional space through a mapping head of a two-layer multilayer perceptron (MLP) with ReLU activation function (experiments showed that adding more than two layers had a negligible effect on performance improvement). Using a multilayer perceptron (MLP) as the mapping head increases the intra-class variance of features in the training dataset, thereby retaining more instance discrimination information and reducing feature redundancy, ensuring robust feature representation in high-dimensional space. This facilitates the calculation of the similarity matrix.

[0043] During the collection of large source datasets, the same disease may be photographed from different angles, with different lighting conditions and potential lens blur. Therefore, techniques including color jitter, horizontal flip, 30-degree rotation, grayscale conversion, and Gaussian blur are used in random combinations to better learn the intra-class and inter-class differences of lesions during the training phase of the SimCLR backbone network.

[0044] The SimCLR backbone network also employs the InfoNCE contrastive loss function to learn effective data representations through comparison (bringing the feature distance between similar samples (positive sample pairs) closer together, while pushing the feature distance between different samples (negative samples) further apart), in order to solve the problem of fine-grained, high-similarity recognition in pest and disease images.

[0045] In addition, for situations in practical applications where there may be insufficient data, unbalanced data, or the need to continuously add new plant categories and disease categories, a multi-head, multi-tail structure (i.e., multiple head convolutional networks and multiple tail convolutional networks) is used for flexible adaptation. Specifically, when a new plant category is introduced, only new head and tail convolutional networks corresponding to that plant category need to be added; and when a new disease category is introduced to an existing plant category, only the head and tail convolutional networks corresponding to that plant category need to be retrained.

[0046] Furthermore, since the parameters of the SimCLR backbone network are frozen, the channel dimension of the header convolutional network is also determined, specifically three in this invention. Therefore, MHMT-CLR introduces a bottleneck-like structure in the header convolutional network, such as... Figure 4 As shown, the number of channels is expanded proportionally to enable the network to capture complex information from the data and enhance the model's expressive power. Subsequently, the number of channels is reduced back to the original three channels, thereby eliminating redundant features while retaining basic information. At the same time, in order to ensure that the structure of the head convolutional network is as lightweight as possible, a certain degree of downsampling is performed on the specific features of the dataset.

[0047] Furthermore, due to the relatively small sample datasets corresponding to each plant category, to mitigate the risk of overfitting, each tail convolutional network is configured to consist of no more than three layers of convolutional neural networks, depending on the specific features of each dataset. Additionally, different degrees of dropout are introduced into the datasets corresponding to different plant categories to ensure that the output features are mapped to a probability space corresponding to the number of diseases in the corresponding plants.

[0048] In some embodiments, the plurality of head convolutional networks and the plurality of tail convolutional networks are deployed in an edge device, the SimCLR backbone network is deployed in a remote server, the edge device and the remote server are communicatively connected, and the edge device is located in the planting area of ​​the plant corresponding to the plant leaf.

[0049] Based on the above settings, it can be ensured that the data transmitted to the remote server is not the original image of the plant leaves, but the corresponding feature vector, which can protect the privacy of the data to a certain extent. At the same time, the deployment of multiple head convolutional networks and multiple tail convolutional networks, which are relatively lightweight (compared to the SimCLR backbone network), can also reduce the equipment deployment cost in the planting area.

[0050] In some embodiments, the step of analyzing the abnormal development behavior of abnormal regions changing from an initial image to a review image to determine disease characteristic values ​​includes: Geometric registration is performed on the verification image based on the initial image to obtain the registered image; The brightness distribution of the registered image is corrected based on the brightness distribution of the initial image to obtain a corrected image; The abnormal development behavior of abnormal regions in the initial image is analyzed to transform them into abnormal regions in the corrected image in order to determine the disease characteristic values.

[0051] In this embodiment, geometric registration is used to suppress spatial changes that may occur in the plant leaves during the two image acquisitions (such as positional shifts caused by wind). Brightness correction measures are taken to suppress brightness shifts that may occur in the plant leaves during the two image acquisitions (such as brightness deviations caused by sun angle or cloudy / sunny conditions), thereby ensuring the reliable execution of subsequent analysis.

[0052] In this invention, the Scale Invariant Feature Transform (SIFT) algorithm is used to detect key feature points in the initial image and the verification image respectively. By calculating the Euclidean distance between the feature descriptors of the key feature points of the two images, a set of matching point pairs is obtained. Then, the Random Sample Consensus (RANSAC) algorithm is applied to remove mismatched points from the point pair set. Based on the remaining point pairs, the homography matrix is ​​calculated from the coordinate system of the verification image to the coordinate system of the initial image. Finally, the matrix is ​​used to perform an inverse perspective transformation on the verification image to generate a registered image spatially aligned with the initial image.

[0053] The process of obtaining the corrected image is as follows: Calculate the gray-level histograms of the initial image and the registered image respectively, and obtain their respective cumulative distribution functions (CDF). Let the cumulative distribution function of the initial image be denoted as . The cumulative distribution function of the registered images is ; Create a grayscale mapping table ,in For each grayscale value (0-255), the grayscale level is specified. Find a mapping value , making closest Then, the mapping table is used to replace each pixel value of the registered image to obtain the corrected image.

[0054] In some embodiments, the step of analyzing the abnormal development behavior of abnormal regions changing from an initial image to a corrected image to determine disease characteristic values ​​includes: Texture analysis is performed on the initial image to obtain multiple resistance feature values, wherein the resistance feature values ​​are used to indicate the degree to which the corresponding pixel position in the initial image hinders the natural spread of the disease; Based on the multiple resistance characteristic values, the abnormal development behavior of the abnormal regions in the initial image changing to the abnormal regions in the corrected image is analyzed to determine the disease characteristic values.

[0055] Specifically, the step of performing texture analysis on the initial image to obtain multiple resistance feature values ​​includes: The gray-level gradient of each pixel in the initial image is analyzed to obtain multiple gradient feature values, which are used to indicate the degree of gray-level change at the corresponding pixel position in the initial image. The multiple gradient feature values ​​are corrected based on the ambient humidity to obtain multiple resistance feature values. The ambient humidity is used to indicate the average humidity of the environment in which the plant leaves are located during the interval period, and the interval period is the time period between the acquisition time of the initial image and the acquisition time of the verification image. The resistance feature value is used to indicate the degree to which the corresponding pixel position in the initial image hinders the natural spread of the disease.

[0056] Analysis revealed that veins and other textured structures in the image exhibit dramatic changes in grayscale values ​​(high gradient), while the grayscale values ​​in the leaf mesophyll area show relatively smooth changes. Compared to the leaf mesophyll area, veins and other textured structures have a stronger inhibitory effect on the spread of disease. Therefore, by analyzing the intensity of the grayscale gradient exhibited by each pixel in the initial image, the degree to which each pixel position hinders the natural spread of disease can be quantified to some extent.

[0057] Furthermore, further analysis revealed that environmental humidity also affects the physical properties of the cuticle, thereby altering the equivalent resistance of leaf tissue to the spread of pathogens. Specifically, pathogens are more likely to spread across leaf textures in humid environments. Therefore, in this invention, while analyzing the grayscale gradient of pixels, environmental humidity is further introduced to correct the grayscale gradient of each pixel to ensure accurate quantification of the degree to which the pixel position hinders the natural spread of disease.

[0058] During the interval period, the humidity of the environment where the plant leaves are located can be measured periodically (e.g., every hour) to obtain multiple humidity values. Then, the arithmetic mean of the multiple humidity values ​​is determined as the aforementioned environmental humidity (after numerical normalization processing (e.g., calculating the mean humidity value and the maximum humidity value under the pre-set ideal environment), so that the value range is (0,1]).

[0059] For example, location in the initial image resistance characteristic value It can be represented as: In the above formula, This is a texture enhancement factor (valued at 10 in this invention), used to amplify the numerical range of gradient feature values. Indicate location gradient eigenvalues, Indicates the aforementioned ambient humidity.

[0060] Furthermore, the step of analyzing the grayscale gradient of each pixel in the initial image to obtain multiple gradient feature values ​​includes: Eigenvalue decomposition is performed on the structure tensor corresponding to the target pixel to obtain the principal eigenvalue and secondary eigenvalue corresponding to the target pixel, wherein the target pixel is any one of the multiple pixels included in the initial image; Calculate the sum of the principal feature value and the secondary feature value corresponding to the target pixel to obtain the feature sum value corresponding to the target pixel, and calculate the difference between the principal feature value and the secondary feature value corresponding to the target pixel to obtain the feature difference value corresponding to the target pixel. By analyzing the difference between the feature value corresponding to the target pixel and its corresponding feature difference, the gradient feature value corresponding to the target pixel is obtained.

[0061] In this invention, the structure tensor corresponding to the target pixel is based on its surrounding... (In this invention) The neighborhood window of the target pixel is determined. The principal feature value corresponding to the target pixel indicates the direction of the most drastic gradient change within the neighborhood window of the target pixel, and the secondary feature value corresponding to the target pixel indicates the direction of the second most drastic gradient change within the neighborhood window of the target pixel. The principal feature value corresponding to the target pixel is greater than the secondary feature value corresponding to the target pixel.

[0062] For example, location in the initial image gradient eigenvalues It can be represented as: in, Indicate location The corresponding principal eigenvalues, Indicate location The corresponding secondary eigenvalues, To prevent extremely small positive numbers with a denominator of zero (such as...) ).

[0063] In this example, the contrast of high-texture areas is further enhanced by processing based on the square of the numerical values, thereby increasing the resistance to disease spread at the pixel locations that indicate leaf texture.

[0064] In some embodiments, the step of analyzing the abnormal development behavior of abnormal regions changing from the initial image to the corrected image based on the plurality of resistance characteristic values, in order to determine the disease characteristic values, includes: Anomaly detection is performed on the initial image to identify basic anomaly regions; Difference analysis is performed on the corrected image based on the initial image to identify newly added anomalous regions, where the basic anomalous regions and the newly added anomalous regions are different. Based on the multiple resistance characteristic values, the consistency between the regional flow channel and the texture structure of the plant leaves is analyzed to obtain the connectivity characteristic value. The regional flow channel is used to simulate the regional development channel between the basic abnormal region and the newly added abnormal region. Based on the multiple resistance characteristic values, the diffusion cost from the basic anomaly region to the newly added anomaly region is analyzed to obtain the cost characteristic value; Based on the connectivity feature value and the cost feature value, the disease feature value is determined.

[0065] Based on the foregoing analysis, it can be seen that the disease spread path on the leaves is not random, but usually consistent with the leaf texture structure. However, the abnormal spread path due to non-infectious mechanical damage often shows a random pattern and is usually inconsistent with the leaf texture structure. Therefore, by analyzing the consistency between the regional flow channels and the texture structure of the plant leaves, non-infectious mechanical damage and disease damage can be distinguished to a certain extent.

[0066] However, connectivity features can be affected by the shape of the region in practical applications (for example, the basic abnormal region and the newly added abnormal region are adjacent, but they are separated by large leaf veins). Therefore, this invention further introduces the analysis of diffusion cost to combine the connectivity of the diffusion of abnormal regions with the diffusion cost to comprehensively quantify the probability of plant leaves suffering from disease.

[0067] In this invention, a saliency detection algorithm based on spectral residuals or a simple Otsu adaptive threshold segmentation is used to detect and process the initial image to obtain basic abnormal regions.

[0068] It should be added that, in this invention, when the area of ​​the basic abnormal region is less than the set area threshold (set to 15 pixels in this invention, but in actual applications, it can also be set to 0.1% of the total number of pixels in the leaf mask area), the subsequent processing flow is skipped, and the prompt message that no abnormality was detected in the initial image is directly output. The user can then decide whether to upload the verification image to the MHMT-CLR model for further detailed diagnosis based on their own experience.

[0069] Specifically, the step of performing difference analysis on the corrected image based on the initial image to determine newly added anomalous regions includes: Noise filtering is applied to the initial image and the corrected image respectively to obtain the initial smoothed image and the corrected smoothed image; The differences between the initial smoothed image and the corrected smoothed image in the LAB color space are analyzed to obtain a color difference map; Perform histogram equalization on the color difference map to obtain the difference histogram; Based on the difference histogram, the correction anomaly region is determined, and the region corresponding to the basic anomaly region is removed from the correction anomaly region to obtain the new anomaly region, wherein the correction anomaly region indicates the color deviation region between the corrected image and the initial image.

[0070] In this invention, Gaussian blurring (using a 5×5 Gaussian kernel) is specifically used to complete the above noise filtering process.

[0071] Based on the above settings, firstly, filtering removes sensor noise, dust, or minor blade vibrations that introduce errors and anomalies. Then, LAB space is used to accurately simulate the human eye's perception of color changes to more accurately reflect the color changes in abnormal areas. Finally, histogram equalization is used to amplify weak abnormal signals and enhance the contrast of subtle differences, making them easier to segment in subsequent steps, thereby ensuring the accuracy and reliability of the identified new abnormal areas.

[0072] In this invention, the element value corresponding to the graph node in the color difference map can be understood as: the Euclidean distance between the corresponding pixel position in the LAB color image and the corrected smooth image.

[0073] In practical applications, to further suppress noise interference, after removing the area corresponding to the basic abnormal area from the corrected abnormal area, morphological opening operations (erosion followed by dilation) can be performed on the remaining area, and isolated noise points with an area smaller than a preset threshold (such as 5 pixels) can be removed. Finally, the remaining area is determined as the newly added abnormal area.

[0074] In some embodiments, the step of analyzing the consistency between the regional flow channels and the texture structure of plant leaves based on the plurality of resistance characteristic values ​​to obtain connectivity characteristic values ​​includes: Using multiple pixels in the initial image as graph nodes, the edge capacity between adjacent graph nodes is analyzed based on the multiple resistance feature values ​​to form a flow network graph, wherein the edge capacity is negatively correlated with the resistance feature value of the corresponding pixel. Based on the flow network diagram, the maximum flow from the virtual source point corresponding to the basic anomaly region to the virtual sink point corresponding to the newly added anomaly region is analyzed to obtain the anomaly development flow. The ratio of the abnormal development flow to the maximum inflow flow corresponding to the newly added abnormal region is determined as the connectivity characteristic value.

[0075] In this embodiment, each pixel in the initial image can be connected using four-neighbor or eight-neighbor connections, and adjacent graph nodes should be understood as adjacent pixels in the initial image.

[0076] Specifically, for adjacent positions in the initial image and In other words, the edge capacity corresponding to both It can be represented as: in, Indicates position The resistance characteristic value, Indicates position The resistance characteristic value.

[0077] In this invention, edge capacity is used to indicate the degree of flow between two adjacent pixels.

[0078] In the process of acquiring abnormal traffic, virtual source and virtual sink are introduced. The virtual source is connected to all pixel nodes within the basic abnormal region, and its edge capacity is set to infinity. Similarly, the virtual sink is connected to all pixel nodes within the newly added abnormal region, and its edge capacity is also set to infinity. Then, the Dinic algorithm or the Edmonds-Karp algorithm is used to solve for the maximum network traffic flowing from the virtual source to the virtual sink. , in order to pass Simulate the maximum flow rate from the original diseased area (i.e. the basic abnormal area) to the latest diseased area (i.e. the newly added abnormal area) under the current texture structure constraints.

[0079] The calculation of the ratio of the abnormal development traffic to the maximum inflow traffic corresponding to the newly added abnormal area is for the purpose of assessing the maximum network traffic. Numerical normalization is performed to ensure that the numerical ranges of connectivity feature values ​​and cost feature values ​​remain consistent, thereby ensuring the accuracy of disease feature values ​​calculated subsequently.

[0080] The maximum inflow corresponding to the newly added abnormal area can be understood as the theoretical maximum flow from other areas different from the newly added abnormal area into the newly added abnormal area.

[0081] The larger the connectivity feature value, the higher the consistency between the development channel of the abnormal region from the initial image to the abnormal region of the corrected image and the direction of the leaf texture, and the higher the possibility that the plant leaves are infected by diseases.

[0082] Furthermore, based on the aforementioned multiple resistance characteristic values, the steps for analyzing the diffusion cost from the basic anomaly region to the newly added anomaly region and obtaining the cost characteristic values ​​are as follows: Define that the quality of each pixel in the basic anomaly region remains consistent (uniform probability distribution), and define that the quality of each pixel in the newly added anomaly region remains consistent. Also define the pixel quality in the basic anomaly region. Move unit mass to the newly added anomaly area single transmission cost for: The optimal transmission scheme is then solved using the Sinkhorn algorithm. This minimizes the total transmission cost, and the minimum total cost is the cost characteristic value. .

[0083] The lower the cost eigenvalue, the smaller the cumulative developmental resistance overcome in the process of the abnormal region of the initial image changing into the abnormal region of the corrected image, and the higher the possibility that the plant leaves are infected by disease.

[0084] In this invention, the disease feature value is positively correlated with the connectivity feature value, and the disease feature value is negatively correlated with the cost feature value.

[0085] Specifically, disease characteristic values It can be represented as: in, Represents the connectivity feature value, This represents the cost feature value. The preset smoothing constant (can be set to 0.1 based on experience).

[0086] In some embodiments, the present invention also provides a plant disease and pest AI identification system based on MHMT-CLR, such as Figure 5 As shown, the system 200 includes: Image acquisition module 201 is used to acquire initial and verification images of plant leaves; The image initial inspection module 202 is used to analyze the abnormal development behavior of abnormal areas changing from the initial image to the verification image, so as to determine the disease feature value, wherein the disease feature value is used to indicate the probability that the plant leaves have suffered from disease. The disease identification module 203 is used to process the verification image based on a pre-trained MHMT-CLR model when the disease feature value is greater than the disease feature threshold, to obtain a disease identification result indicating the disease category of the plant corresponding to the plant leaf. The MHMT-CLR model includes multiple head convolutional networks, a SimCLR backbone network, and multiple tail convolutional networks. The multiple head convolutional networks correspond one-to-one with multiple plant categories, and the multiple tail convolutional networks correspond one-to-one with the multiple plant categories. The parameters of the SimCLR backbone network are frozen during the training of the multiple head convolutional networks and the multiple tail convolutional networks.

[0087] It should be noted that the system provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the plant disease and pest AI identification system based on MHMT-CLR and the plant disease and pest AI identification method based on MHMT-CLR provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiment, which will not be repeated here.

[0088] This invention also provides an electronic device. Please refer to [link to relevant documentation]. Figure 6 The electronic device may include a processor 301, a memory 302, and a program 3021 stored in the memory 302 and capable of running on the processor 301.

[0089] When program 3021 is executed by processor 301, it can achieve the following: Figure 1 Any steps in the corresponding method embodiments and the achievement of the same beneficial effects will not be repeated here.

[0090] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by hardware related to program instructions, and the program can be stored in a readable medium.

[0091] This invention also provides a readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described functions. Figure 1 Any step in the corresponding method embodiment can achieve the same technical effect, and will not be repeated here to avoid repetition.

[0092] The computer-readable storage medium of this invention can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0093] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0094] The program code contained on the storage medium can be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0095] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or terminal. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0096] This invention also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the MHMT-CLR-based AI identification method for plant diseases and pests provided in the above embodiments.

[0097] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0098] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A plant disease and pest AI identification method based on MHMT-CLR, characterized in that, The method includes: Acquire initial and verification images of the plant leaves; The abnormal development behavior of abnormal regions changing from the initial image to the verification image is analyzed to determine disease feature values, wherein the disease feature values ​​are used to indicate the probability that the plant leaves have suffered from diseases. When the disease feature value is greater than the disease feature threshold, the verification image is processed based on the pre-trained MHMT-CLR model to obtain the disease identification result indicating the disease category of the plant corresponding to the plant leaf. The MHMT-CLR model includes multiple head convolutional networks, a SimCLR backbone network, and multiple tail convolutional networks. The multiple head convolutional networks correspond one-to-one with multiple plant categories, and the multiple tail convolutional networks correspond one-to-one with the multiple plant categories. The parameters of the SimCLR backbone network are frozen during the training of the multiple head convolutional networks and the multiple tail convolutional networks.

2. The plant disease and pest AI identification method based on MHMT-CLR according to claim 1, characterized in that, The multiple head convolutional networks and the multiple tail convolutional networks are deployed in an edge device, the SimCLR backbone network is deployed in a remote server, the edge device and the remote server are communicatively connected, and the edge device is set in the planting area of ​​the plant corresponding to the plant leaf.

3. The plant disease and pest AI identification method based on MHMT-CLR according to claim 1, characterized in that, The steps for analyzing the abnormal development behavior of abnormal regions in the initial image to the verification image in order to determine the disease characteristic values ​​include: Geometric registration is performed on the verification image based on the initial image to obtain the registered image; The brightness distribution of the registered image is corrected based on the brightness distribution of the initial image to obtain a corrected image; The abnormal development behavior of abnormal regions in the initial image is analyzed to transform them into abnormal regions in the corrected image in order to determine the disease characteristic values.

4. The plant disease and pest AI identification method based on MHMT-CLR according to claim 3, characterized in that, The steps for analyzing the abnormal development behavior of abnormal regions in the initial image to the corrected image, in order to determine the disease characteristic values, include: Texture analysis is performed on the initial image to obtain multiple resistance feature values, wherein the resistance feature values ​​are used to indicate the degree to which the corresponding pixel position in the initial image hinders the natural spread of the disease; Based on the multiple resistance characteristic values, the abnormal development behavior of the abnormal regions in the initial image changing to the abnormal regions in the corrected image is analyzed to determine the disease characteristic values.

5. The plant disease and pest AI identification method based on MHMT-CLR according to claim 4, characterized in that, The steps of performing texture analysis on the initial image to obtain multiple resistance feature values ​​include: The gray-level gradient of each pixel in the initial image is analyzed to obtain multiple gradient feature values, which are used to indicate the degree of gray-level change at the corresponding pixel position in the initial image. The multiple gradient feature values ​​are corrected based on the ambient humidity to obtain multiple resistance feature values. The ambient humidity is used to indicate the average humidity of the environment in which the plant leaves are located during the interval period, and the interval period is the time period between the acquisition time of the initial image and the acquisition time of the verification image. The resistance feature value is used to indicate the degree to which the corresponding pixel position in the initial image hinders the natural spread of the disease.

6. The plant disease and pest AI identification method based on MHMT-CLR according to claim 5, characterized in that, The steps for analyzing the gray-level gradient of each pixel in the initial image to obtain multiple gradient feature values ​​include: Eigenvalue decomposition is performed on the structure tensor corresponding to the target pixel to obtain the principal eigenvalue and secondary eigenvalue corresponding to the target pixel, wherein the target pixel is any one of the multiple pixels included in the initial image; Calculate the sum of the principal feature value and the secondary feature value corresponding to the target pixel to obtain the feature sum value corresponding to the target pixel, and calculate the difference between the principal feature value and the secondary feature value corresponding to the target pixel to obtain the feature difference value corresponding to the target pixel. By analyzing the difference between the feature value corresponding to the target pixel and its corresponding feature difference, the gradient feature value corresponding to the target pixel is obtained.

7. The plant disease and pest AI identification method based on MHMT-CLR according to claim 5, characterized in that, The step of analyzing the abnormal development behavior of abnormal regions in the initial image to the corrected image based on the multiple resistance characteristic values, in order to determine the disease characteristic values, includes: Anomaly detection is performed on the initial image to identify basic anomaly regions; Difference analysis is performed on the corrected image based on the initial image to identify newly added anomalous regions, where the basic anomalous regions and the newly added anomalous regions are different. Based on the multiple resistance characteristic values, the consistency between the regional flow channel and the texture structure of the plant leaves is analyzed to obtain the connectivity characteristic value. The regional flow channel is used to simulate the regional development channel between the basic abnormal region and the newly added abnormal region. Based on the multiple resistance characteristic values, the diffusion cost from the basic anomaly region to the newly added anomaly region is analyzed to obtain the cost characteristic value; Based on the connectivity feature value and the cost feature value, the disease feature value is determined.

8. The plant disease and pest AI identification method based on MHMT-CLR according to claim 7, characterized in that, The steps of performing difference analysis on the corrected image based on the initial image to identify newly added anomalous regions include: Noise filtering is applied to the initial image and the corrected image respectively to obtain the initial smoothed image and the corrected smoothed image; The differences between the initial smoothed image and the corrected smoothed image in the LAB color space are analyzed to obtain a color difference map; Perform histogram equalization on the color difference map to obtain the difference histogram; Based on the difference histogram, the correction anomaly region is determined, and the region corresponding to the basic anomaly region is removed from the correction anomaly region to obtain the new anomaly region, wherein the correction anomaly region indicates the color deviation region between the corrected image and the initial image.

9. The plant disease and pest AI identification method based on MHMT-CLR according to claim 7, characterized in that, The step of analyzing the consistency between the regional flow channels and the texture structure of plant leaves based on the multiple resistance characteristic values ​​to obtain connectivity characteristic values ​​includes: Using multiple pixels in the initial image as graph nodes, the edge capacity between adjacent graph nodes is analyzed based on the multiple resistance feature values ​​to form a flow network graph, wherein the edge capacity is negatively correlated with the resistance feature value of the corresponding pixel. Based on the flow network diagram, the maximum flow from the virtual source point corresponding to the basic anomaly region to the virtual sink point corresponding to the newly added anomaly region is analyzed to obtain the anomaly development flow. The ratio of the abnormal development flow to the maximum inflow flow corresponding to the newly added abnormal region is determined as the connectivity characteristic value.

10. The plant disease and pest AI identification method based on MHMT-CLR according to claim 7, characterized in that, The disease feature value is positively correlated with the connectivity feature value, and the disease feature value is negatively correlated with the cost feature value.

Images