Crop disease and insect pest judgment method and system based on image feature recognition

Abstract

The present invention provides a crop pest and disease identification method and system based on image feature recognition, which relates to the technical field of image recognition and processing. The present invention uses a pre-trained pest and disease analysis model to identify pests and diseases in a crop image to be identified, and obtains a first pest and disease identification result; presets similar pest and disease combinations and corresponding joint spot features, and determines whether there is a similar pest and disease combination in the first pest and disease identification result; if there is no similar pest and disease combination, outputs the first pest and disease identification result; if there is the first pest and disease combination, extracts a spot area of ​​the crop image to be identified; traverses the spot area of ​​the crop image to be identified, and if a joint spot feature appears in the spot area, both pests and diseases in the similar pest and disease combination are used as output results; if there is no joint spot feature, eliminates the pest and disease with the lowest confidence in the similar pest and disease combination; and updates the first pest and disease identification result.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition and processing, and in particular to a method and system for determining crop pests and diseases based on image feature recognition. Background Art

[0002] With the development of smart agriculture, image recognition-based crop pest and disease detection technology has become a research hotspot. The main tomato diseases include early blight, late blight, leaf mold, powdery mildew, leaf spot, spider mite damage, and yellow leaf curl. Existing models can accurately identify powdery mildew, spider mite damage, and yellow leaf curl due to their distinct characteristics. However, for highly similar diseases such as early blight-leaf mold and late blight-leaf mold, existing models lack accuracy and often misidentify them.

[0003] In the prior art, publication number CN109002853A discloses a method and device for the joint identification of crop species and pest types, wherein a pest type feature map is run through a preset multi-instance learning fusion model to obtain a first pest type evaluation value, and a crop type Bayesian probability value is obtained based on a preset training set and the first pest type evaluation value. Subsequently, the crop type feature map is run through a preset fully connected neural network model to obtain a first crop type evaluation value, and a pest type Bayesian probability value is obtained based on a pre-stored training set and the first crop type evaluation value. A second pest type evaluation value is obtained based on the first pest type evaluation value and the pest type Bayesian probability value, and then a second crop type evaluation value is obtained based on the first crop type evaluation value and the crop type Bayesian probability value. While achieving high-accuracy identification of crop species and pest types, the demand for training data is greatly reduced. However, this existing technology still has flaws. While its holistic recognition framework can handle general pest and disease identification, it fails to design a specialized joint feature analysis mechanism for specific pest and disease combinations with similar symptoms, such as early blight and leaf spot, or late blight and leaf mold, common on tomato leaves. This coarse-grained approach makes it difficult for the system to accurately distinguish true complex diseases from misidentified single diseases when faced with common comorbidities in actual agricultural production, thus affecting the accuracy of diagnostic results.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and system for determining crop pests and diseases based on image feature recognition, so as to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A method for determining crop pests and diseases based on image feature recognition, comprising the following steps:

[0008] Step 1: The crop image to be judged is grayscaled and converted to color space to obtain a grayscale image and a CIELAB color space image. The leaf area is extracted based on the grayscale image using the maximum inter-class variance method, and the image spot area is extracted from it using the Sobel operator. The image spot area is then pixel-mapped with the CIELAB image to obtain the grayscale spot area and color spot area corresponding to the crop image to be judged.

[0009] Step 2: Use the pre-trained pest and disease analysis model to identify pests and diseases in the crop image to be identified, where grayscale and color spots are identified. Confidence scores for each type of pest and disease are obtained, and a first image recognition label is constructed based on the confidence scores and the corresponding pest and disease types.

[0010] Step 3: Preset similar pest and disease combinations and joint spot features corresponding to each similar pest and disease combination, and determine whether a similar pest and disease combination exists in the first pest and disease identification result. The joint spot features include a first joint spot feature reflecting the phenomenon of water and pigment anisotropy when early blight and leaf spot are combined, and a second joint spot feature reflecting the three-layer color band structure of the joint spot when late blight and leaf mold are combined.

[0011] Step 4: If no similar pest and disease combination exists, the first pest and disease recognition result is used as the final pest and disease recognition result; if the first pest and disease combination exists, the spot area of ​​the crop image to be judged is traversed. If the spot area has a joint spot feature, both pests and diseases in the similar pest and disease combination are used as the output result; if the joint spot feature does not exist, the pest and disease with the lowest confidence in the similar pest and disease combination is eliminated, and the first pest and disease recognition result is updated as the final pest and disease recognition result;

[0012] Step 5: Determine the pests and diseases based on the final pest and disease identification results, and output the pest and disease status of the crop image to be determined.

[0013] Furthermore, the logic for obtaining the spot area of ​​the crop image to be judged is as follows: the crop image to be judged is copied twice, one is grayscaled, and the other is converted from the RGB color space to the CIELAB color space. For the grayscaled crop image to be judged, the maximum inter-class variance method is used to extract the leaf part of the crop image to be judged, and then the Sobel operator is used to determine the spot area of ​​the leaf part of the crop image to be judged, which is called the grayscale spot area; based on the grayscale spot area, the spot area of ​​the crop image to be judged that has undergone color space conversion is framed, and the framed spot area is called the color spot area.

[0014] Furthermore, based on the identified grayscale spot areas and color spot areas, the unprocessed crop image to be judged in the RGB color space is framed to obtain the crop image to be judged in the RGB color space with the spot areas identified, and mask processing is performed to retain the R, G, B values ​​of the spot areas of the crop image to be judged in the RGB color space, and the R, G, B values ​​of the non-spot areas of the crop image to be judged in the RGB color space are set to 0. The crop image to be judged after mask processing is called the crop image to be judged with the grayscale spot areas and color spot areas identified for pest and disease identification.

[0015] Furthermore, the logic for obtaining the first joint lesion feature is: using the gray level co-occurrence matrix to extract the texture features of each gray spot area and construct a texture feature vector ,in, For the Texture feature vector of block grayscale spot area, For the The contrast of the gray-level co-occurrence matrix of the block gray spot area, For the The energy of the gray-level co-occurrence matrix of the block gray-level spot region, For the Homogeneity of the gray-level co-occurrence matrix of the block gray-level spot region, For the The entropy of the gray-level co-occurrence matrix of the block gray-level spot area; the gray-level spot area is divided into several spot sub-areas of equal area, and the local texture feature vector of each spot sub-area is obtained ,in, For the In the gray spot area, Texture feature vector of the spot sub-region, For the In the gray spot area, The contrast of the gray-level co-occurrence matrix of each spot sub-region, For the In the gray spot area, The energy of the gray-level co-occurrence matrix of each spot sub-region, For the In the gray spot area, The homogeneity of the gray level co-occurrence matrix of each spot sub-region, For the In the gray spot area, The entropy of the gray-level co-occurrence matrix of each spot sub-region is calculated; the Euclidean distance of the texture feature vector between the gray-level spot region and each of its spot sub-regions is calculated, and the spot sub-region with the largest Euclidean distance is called the target sub-region. is the index of the grayscale spot area, is the index of the spot sub-region in the spot region;

[0016] Based on the target sub-region, the crop image to be judged is framed and processed for color space conversion to obtain the key area. The brightness gradient direction and color gradient direction of each pixel in the key area are obtained respectively, and the angle between the brightness gradient direction and the color gradient direction is calculated. The angle is counted. The number of pixels within the range is divided by the total number of pixels in the target sub-region to obtain the fused adversarial feature;

[0017] A fusion adversarial feature threshold is preset. If the fusion adversarial feature is greater than the fusion adversarial feature threshold, the first joint lesion feature is deemed to exist; otherwise, it is deemed not to exist.

[0018] Furthermore, starting from the center of each color spot area, an extension direction is determined every 15 degrees in the clockwise direction;

[0019] In any determined extension direction, a pixel interval is preset, and the pixels in the extension direction are divided into multiple pixel segments based on the preset pixel interval. The color difference between adjacent pixels in the pixel segment is analyzed. To determine whether a color mutation occurs, the Euclidean distance of the pixel attribute of the color difference CIELAB color space is used. If at least two color mutations occur within a pixel segment in any extension direction of any spot, it is considered that the second joint lesion feature exists; otherwise, it is considered that the second joint lesion feature does not exist.

[0020] Furthermore, the color difference is the Euclidean distance of three channels of pixel points in the CIELAB color space. If the color difference between two adjacent pixels is greater than a preset color difference threshold, it is marked that a color mutation occurs.

[0021] Furthermore, similar pest and disease combinations include an early blight-spotted disease combination and a late blight-leaf mold combination; a confidence threshold is preset, and if the confidences of both pests and diseases in a similar pest and disease combination are greater than the confidence threshold, it is determined that a similar pest and disease combination exists; otherwise, it is determined that a similar pest and disease combination does not exist.

[0022] Furthermore, the confidence level of each pest in the updated first pest identification result is compared with the confidence threshold, and all pests with a confidence level greater than the confidence threshold are taken as the judgment result of the image.

[0023] The present invention further provides a crop disease and insect pest identification system based on image feature recognition, wherein the system is used to implement the crop disease and insect pest identification method based on image feature recognition, and specifically comprises:

[0024] The spot recognition module is used to grayscale and color-space convert the crop image to be judged, obtaining a grayscale image and a CIELAB color space image. The maximum inter-class variance method is used to extract the leaf area based on the grayscale image, and the Sobel operator is used to extract the image spot area from it. The image spot area is then pixel-mapped with the CIELAB image to obtain the grayscale spot area and color spot area corresponding to the crop image to be judged.

[0025] A preliminary recognition module is configured to use a pre-trained pest and disease analysis model to identify pests and diseases in the crop image to be identified, where grayscale and color spots are identified, to obtain confidence levels that the crop to be identified has various pests and diseases, and to construct a first image recognition label based on the confidence levels and the corresponding pest and disease types;

[0026] A similarity analysis module is configured to preset similar pest and disease combinations and corresponding joint spot features for each similar pest and disease combination, and determine whether a similar pest and disease combination exists in the first pest and disease identification result. The joint spot features include a first joint spot feature reflecting the phenomenon of heterogeneous permeability of water and pigment when early blight and leaf spot are combined, and a second joint spot feature reflecting the three-layer color band structure of the joint spot when late blight and leaf mold are combined;

[0027] The comprehensive judgment module is used to use the first pest identification result as the final pest identification result if no similar pest combination exists; if the first pest combination exists, traverse the spot area of ​​the crop image to be judged, and if the spot area has a joint spot feature, use both pests in the similar pest combination as the output result; if the joint spot feature does not exist, eliminate the pest with the lowest confidence in the similar pest combination and update the first pest identification result as the final pest identification result;

[0028] The final judgment module is used to judge the pests and diseases based on the final pest and disease identification results and output the pest and disease conditions of the crop images to be judged.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] This method constructs joint image features for specific, similar pest and disease combinations to determine their status, effectively distinguishing between co-morbidities and misidentified conditions. This significantly improves the accuracy of identifying easily confused combinations, such as early blight / spot blight and late blight / leaf mold. Ultimately, the final pest and disease identification result is derived by combining the primary crop pest and disease identification results with the status of similar pest and disease combinations, enabling more comprehensive pest and disease identification and providing a reliable basis for disease prevention and control in precision agriculture. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Schematic diagram of the overall method flow of the present invention;

[0032] Figure 2 This is a comparison chart of early blight identification in the present invention;

[0033] Figure 3 This is a comparison diagram of the identification of leaf spot disease of the present invention;

[0034] Figure 4 This is a comparison chart of the late blight identification situation of the present invention;

[0035] Figure 5 This is a comparison chart of leaf mold identification in the present invention;

[0036] Figure 6 Schematic diagram of the overall system structure of the present invention. DETAILED DESCRIPTION

[0037] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to specific embodiments.

[0038] It should be noted that, unless otherwise defined, the technical or scientific terms used in the present invention should have the usual meanings understood by people with ordinary skills in the field to which the present invention belongs. The "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative position relationships. When the absolute position of the object being described changes, the relative position relationship may also change accordingly.

[0039] Example:

[0040] See also Figure 1-Figure 5 , the present invention provides a technical solution:

[0041] A method for determining crop pests and diseases based on image feature recognition, comprising the following steps:

[0042] Step 1: The crop image to be judged is grayscaled and converted to color space to obtain a grayscale image and a CIELAB color space image. The leaf area is extracted based on the grayscale image using the maximum inter-class variance method, and the image spot area is extracted from it using the Sobel operator. The image spot area is then pixel-mapped with the CIELAB image to obtain the grayscale spot area and color spot area corresponding to the crop image to be judged.

[0043] Identified pests and diseases include early blight, late blight, leaf mold, powdery mildew, leaf spot, spider mite damage, and yellow leaf curl;

[0044] Tomato early blight: If tomatoes are infected with pests and diseases during the seedling stage, dark brown lesions will form at the base of the stems, showing slightly concave whorls. During the adult stage, when the leaves are infected by pests and diseases, it usually starts from the leaves at the bottom of the plant and gradually spreads upwards. In the early stage, small water-soaked dark brown spots will appear on the leaves. As the pests and diseases develop, these spots will become irregular, oval, or circular lesions, and the color will gradually change from brown to dark brown. A small circle of yellow or light green halo will appear on the edge of the lesion, and the lesion will have large raised concentric whorls. In a humid environment, black mold will grow on the lesions. If the severity of the pests and diseases increases, many lesions may merge into large, irregular lesions, causing the leaves to die prematurely.

[0045] Tomato late blight: Lesions typically begin at the leaf margins or tips, initially appearing as irregular, water-soaked patches of dark green. As the disease progresses, the lesions gradually enlarge and turn brown. In humid weather or high humidity, the disease spreads more rapidly, covering the entire leaf or even the entire plant. Lesions spread along leaf veins toward the stem and petiole, forming brown, streaky patches. Drought limits the spread of lesions, causing water loss in the affected area, resulting in a bluish-white lesion and ultimately, dry, brittle leaves.

[0046] Tomato leaf mold: The initial symptoms of tomato leaf mold are circular or irregular, pale yellow or light green, chlorotic spots on the underside of the leaves. Later, a grayish-white, grayish-purple, or dark brown velvety mold develops over the lesions, while the front of the leaves is light yellow with indistinct edges. On rainy days or in environments with high humidity, mold may also develop over the lesions on the leaves. The disease typically begins on older leaves and gradually spreads to newer leaves. In severe cases, the leaves curl from top to bottom, and the entire plant turns a yellowish-brown color and dies.

[0047] Tomato powdery mildew: Tomato leaf mold is a common pest and disease. In the early stages of the disease, many small white mold spots can be seen on the leaves. These mold spots will then continue to expand and connect with each other to form white powdery spots of varying sizes. When the disease is severe, the entire leaf will be covered with a white powdery mold layer. In the later stages, the diseased leaves will curl and dry up, causing the plant to age prematurely. In addition to the leaves, the petioles and stems of tomatoes can also be infected, and the white mold layer will also appear. In a humid environment, a new layer of mold may grow on the surface of the lesions. Therefore, when growing tomatoes, special attention should be paid to preventing the occurrence of leaf mold.

[0048] Tomato leaf spot: Tomato leaf spot can occur at all stages of tomato growth and on all parts of the plant, but is particularly common after flowering and fruiting. Initially, the lesions appear as small, water-soaked spots on the underside of the leaves. Later, numerous circular or nearly circular lesions of varying sizes appear on both sides of the leaves. These lesions have dark brown edges, a slightly sunken grayish-white center, and small black spots on the surface. As the lesions persist, these small spots gradually merge into larger, dry spots, sometimes with perforations. In severe cases, the lower and middle leaves dry up and fall off, leaving only a few leaves at the top.

[0049] Spider mite damage: Adult and larval spider mites pierce the skin of tomato leaves, green branches, and fruit, sucking out the juice. This causes yellowing, leaf drop, and fruit drop, reducing yields. The most severe damage to tomato leaves is caused by spider mites. Small yellow-white spots form on the surface of affected leaves. In severe cases, the spots can become so dense that the entire leaf turns grayish-white, loses its luster, becomes brittle, and easily drops.

[0050] Tomato yellow leaf curl: Tomato plants infected with yellow leaf curl experience stunted growth, slowing or stagnating growth. Top leaves often become chlorotic, yellow, and shrunken, with leaf edges curling upward. Leaves become thicker and harder, often with purple veins on the undersides. Infected plants experience severe stunting in the early stages of growth and development, preventing them from flowering and fruiting. Later in development, symptoms are limited to the upper leaves and new shoots, resulting in fewer fruits, smaller fruits, and uneven fruit coloring (a translucent red), essentially eliminating their commercial value. Severely infected plants fail to flower or produce fruit.

[0051] Furthermore, the logic for obtaining the spot area of ​​the crop image to be judged is as follows: the crop image to be judged is copied twice, one is grayscaled, and the other is converted from the RGB color space to the CIELAB color space. For the grayscaled crop image to be judged, the maximum inter-class variance method is used to extract the leaf part of the crop image to be judged, and then the Sobel operator is used to determine the spot area of ​​the leaf part of the crop image to be judged, which is called the grayscale spot area; based on the grayscale spot area, the spot area of ​​the crop image to be judged that has undergone color space conversion is framed, and the framed spot area is called the color spot area.

[0052] The maximum inter-class variance method is an existing technology for extracting the leaf portion of the crop image to be judged. Specifically, the frequency of pixels at each level in the 256 grayscale levels is counted to form a grayscale histogram, and the inter-class variance of each threshold in the histogram is calculated. The optimal separation threshold is selected based on the inter-class variance. The optimal separation threshold is used to perform threshold segmentation on the crop image, identify the leaf portion and the background portion, and extract the leaf portion.

[0053] The specific logic for calculating the optimal separation threshold is as follows: traverse all possible thresholds, segment the crop image into background and foreground parts according to the value of each threshold, count the number of pixels in the background and foreground parts, and calculate the mean grayscale value of the pixels in the background and foreground parts. Calculate the inter-class variance of the corresponding threshold according to the following formula:

[0054]

[0055] in, The threshold is The between-class variance when ,and , 、 The thresholds are The number of pixels in the background and foreground when 、 The thresholds are The mean grayscale value of the background and foreground pixels at the time;

[0056] Calculate the threshold value when the between-class variance reaches the maximum, and define this value as the optimal separation threshold.

[0057] When segmenting into background and foreground parts, the grayscale value of the pixel is less than or equal to Then the pixel is judged to belong to the background part, and it is greater than It is determined that the pixel point belongs to the foreground part, and the foreground part is the leaf part.

[0058] The Sobel operator is used to calculate the horizontal and vertical gradient values ​​of each pixel in the crop image, and a first gradient amplitude data set including the gradient amplitude data of all pixels is generated. The error rate between the gradient amplitude of the current pixel and the gradient amplitude of adjacent pixels is determined. If the error rate is less than a set error rate threshold, the pixel value is retained; otherwise, it is set to 0. The portion with the retained pixel value is defined as the edge area of ​​the spot, and the closed area surrounded by the edge area is defined as the spot area.

[0059] When calculating the horizontal and vertical gradient values ​​of each pixel, the grayscale value of the pixel and its adjacent pixels are multiplied with the horizontal gradient template of the Sobel operator, and all the product results are added to obtain the horizontal gradient value of the pixel. The grayscale value of the pixel and its adjacent pixels are multiplied with the vertical gradient template of the Sobel operator, and all the product results are added to obtain the vertical gradient value of the pixel. The formula for calculating the horizontal and vertical gradient values ​​is as follows:

[0060]

[0061]

[0062] in, and Respectively Row, No. The horizontal gradient value and vertical gradient value of the column pixel points; For the Row, No. Grayscale value of the column pixel;

[0063] The formula for generating the gradient magnitude of a pixel is:

[0064]

[0065] in, For the Row, No. The gradient magnitude of the pixel column.

[0066] The formula for generating the error rate between the gradient magnitude of the current pixel and the gradient magnitude of the adjacent pixel is:

[0067]

[0068] in, For the Row, No. The gradient magnitude of the column pixel, The current pixel Row, No. The gradient magnitude of the pixel point in the column and the adjacent pixel point , No. The error rate between columns of pixels.

[0069] Step 2: Use the pre-trained pest and disease analysis model to identify pests and diseases in the crop image to be identified, where grayscale and color spots are identified. Confidence scores for each type of pest and disease are obtained, and a first image recognition label is constructed based on the confidence scores and the corresponding pest and disease types.

[0070] According to the identified grayscale spot areas and color spot areas, the unprocessed crop image to be judged in the RGB color space is framed to obtain the crop image to be judged in the RGB color space with the spot areas identified. Mask processing is performed to retain the R, G, and B values ​​of the spot areas of the crop image to be judged in the RGB color space, and the R, G, and B values ​​of the non-spot areas of the crop image to be judged in the RGB color space are set to 0. The crop image to be judged after mask processing is called the crop image to be judged with the grayscale spot areas and color spot areas identified for pest and disease identification.

[0071] Using a pre-trained pest and disease analysis model to identify pests and diseases in crop images to be identified, existing techniques can be employed to obtain historical crop images, process them according to step 1, and obtain images of the crop to be identified, with historically identified grayscale and color spots. These historical crop images are then manually annotated, with the annotation results including the aforementioned pest and disease status. The annotated crop images are then fed into a deep learning model, such as a VGG convolutional neural network, to extract features. The confidence scores for each type of pest and disease are then output using a fully connected layer and a softmax function. This is a conventional technique employed by those skilled in the art and will not be elaborated upon here.

[0072] Step 3: Preset similar pest and disease combinations and joint spot features corresponding to each similar pest and disease combination, and determine whether a similar pest and disease combination exists in the first pest and disease identification result. The joint spot features include a first joint spot feature reflecting the phenomenon of water and pigment anisotropy when early blight and leaf spot are combined, and a second joint spot feature reflecting the three-layer color band structure of the joint spot when late blight and leaf mold are combined.

[0073] Similar pest and disease combinations include the early blight-spotted blight combination and the late blight-leaf mold combination;

[0074] Furthermore, the logic for obtaining the first joint lesion feature is: using the gray level co-occurrence matrix to extract the texture features of each gray spot area and construct a texture feature vector ,in, For the Texture feature vector of block grayscale spot area, For the The contrast of the gray-level co-occurrence matrix of the block gray spot area, For the The energy of the gray-level co-occurrence matrix of the block gray-level spot region, For the Homogeneity of the gray-level co-occurrence matrix of the block gray-level spot region, For the The entropy of the gray-level co-occurrence matrix of the block gray-level spot area; the gray-level spot area is divided into several spot sub-areas of equal area, and the local texture feature vector of each spot sub-area is obtained ,in, For the In the gray spot area, Texture feature vector of the spot sub-region, For the In the gray spot area, The contrast of the gray-level co-occurrence matrix of each spot sub-region, For the In the gray spot area, The energy of the gray-level co-occurrence matrix of each spot sub-region, For the In the gray spot area, The homogeneity of the gray level co-occurrence matrix of each spot sub-region, For the In the gray spot area, The entropy of the gray-level co-occurrence matrix of each spot sub-region is calculated; the Euclidean distance of the texture feature vector between the gray-level spot region and each of its spot sub-regions is calculated, and the spot sub-region with the largest Euclidean distance is called the target sub-region. is the index of the grayscale spot area, is the index of the spot sub-region in the spot region;

[0075] The gray level co-occurrence matrix is ​​used to extract the texture features of each spot area, and the gray level co-occurrence matrix is ​​used to perform texture analysis on the texture features of the target spot area. The gray level co-occurrence matrix is ​​expressed as:

[0076]

[0077] in, is the gray-level co-occurrence matrix, The gray level in the spot area is The pixels and gray levels are The number of times adjacent pixel pairs appear, and are grayscale, and , .

[0078] The specific formulas for calculating homogeneity, entropy, and energy are:

[0079]

[0080]

[0081]

[0082]

[0083] in, After normalization , is the correction number, in order to avoid , its value is very small, and can be a decimal such as 0.0001, 0.001, etc., and is not limited here; is the contrast of the gray-level co-occurrence matrix, is the energy of the gray-level co-occurrence matrix, is the homogeneity of the gray-level co-occurrence matrix, is the entropy of the gray-level co-occurrence matrix.

[0084] This method is used to calculate the texture features of each subsequent spot and spot sub-region.

[0085] Based on the target sub-region, the crop image to be judged is framed and processed for color space conversion to obtain the key area. The brightness gradient direction and color gradient direction of each pixel in the key area are obtained respectively, and the angle between the brightness gradient direction and the color gradient direction is calculated. The angle is counted. The number of pixels within the range is divided by the total number of pixels in the target sub-region to obtain the fused adversarial feature;

[0086] The distribution of water will systematically affect the intensity of light reflection in the image, leaving a regular spatial variation trajectory in the brightness channel. Therefore, the direction of the brightness gradient can represent the direction of water diffusion. Color is a direct reflection of pigment, and the direction of the color gradient can represent the direction of the pigment gradient.

[0087] The brightness gradient can refer to the gradient calculation method of the Sobel operator mentioned above. Replace the grayscale in the previous text with brightness to obtain the horizontal brightness gradient value and the vertical brightness gradient value;

[0088] By formula: , get the brightness gradient direction, similarly, replace the gray value in the previous text with the a value or b value of the CIELAB color space, through the formula Get the color gradient direction, where and are the brightness gradient direction and the color gradient direction respectively, , and are the vertical gradients of brightness, red-green channel value, and blue-yellow channel value, respectively. , and are the horizontal gradients of brightness, red-green channel values, and blue-yellow channel values, respectively.

[0089] In tomato leaves infected with both early blight and leaf spot, the directions of their action in the tissue layer conflict: early blight spots cause the cell fluid to seep outwards through the degradation of cell wall enzymes, forming a typical water-soaked, expanding halo; while leaf spot lesions quickly cause local cell necrosis and cracking, causing the tissue to collapse inwards and the pigment to retract. When the two lesions overlap, water still attempts to seep out, but is blocked or turned back by the cracked structure at the edge, while the pigment is retained in the center due to invagination, eventually forming an anisotropic infiltration phenomenon in which water and pigment "diffuse in opposite directions" in the image. This manifests as irreversible and unforgeable visual abnormal structures such as dark inside and bright outside, and inverted color halo structure. Fusion adversarial features are used to characterize the anisotropic infiltration phenomenon; the angle between Within the range, it means that the direction of the brightness gradient representing the moisture direction and the color gradient representing the pigment direction at the analyzed pixel point are different, or even opposite, and there is a certain degree of heterogeneous penetration; the fusion confrontation feature is the ratio of the number of pixels with heterogeneous penetration to the whole, which shows the proportion of pixels with heterogeneous penetration relative to the analysis area. The larger the value, the greater the proportion, and the greater the probability of the combination of the two diseases.

[0090] A fusion adversarial feature threshold is preset. If the fusion adversarial feature is greater than the fusion adversarial feature threshold, the first joint lesion feature is deemed to exist; otherwise, it is deemed not to exist.

[0091] Furthermore, starting from the center of each color spot area, an extension direction is determined every 15 degrees in the clockwise direction;

[0092] In any determined extension direction, a pixel interval is preset, and the pixels in the extension direction are divided into multiple pixel segments based on the preset pixel interval. The color difference between adjacent pixels in the pixel segment is analyzed. To determine whether a color mutation occurs, the Euclidean distance of the pixel attribute of the color difference CIELAB color space is used. If at least two color mutations occur within a pixel segment in any extension direction of any spot, it is considered that the second joint lesion feature exists; otherwise, it is considered that the second joint lesion feature does not exist.

[0093] Furthermore, the color difference is the Euclidean distance of the three channels of the pixel points in the CIELAB color space. If the color difference between two adjacent pixels is greater than a preset color difference threshold, a color mutation is marked. The color difference is expressed as:

[0094]

[0095] in, For color differences, For the CIELAB color space information of each pixel, For the CIELAB color space information of each pixel, is the index of the pixel points in the pixel segment, and the pixel points in the pixel segment are assigned indexes in sequence based on the extension direction.

[0096] Color differences directly reflect the color change between two pixels. For a single spot of late blight or leaf mold, there's typically no noticeable color change within the spot. Even if a color change occurs, it's gradual. When late blight and leaf mold co-occur, the combined patterns of the two diseases will form a three-layered banding structure: "yellow → green → dark brown" within a relatively small pixel interval. Each sudden change represents a color change. Because this three-layered banding structure occurs within a small pixel interval, a smaller pixel interval can be preset. If two sudden changes occur within that pixel interval, it's considered a three-layered banding structure within the spot, indicating the presence of both late blight and leaf mold.

[0097] Take 1, otherwise, let the edge multi-layer color band feature value be 0;

[0098] The number of pixel segments is:

[0099]

[0100] in, For the The number of pixel segments divided in the extension direction is For the The total number of pixels from the center of the spot to the edge of the spot in the extension direction, is the pixel spacing, For the rounding operation, is the index of the extension direction;

[0101] The color difference threshold and pixel interval are determined by experts based on the specific situation of tomatoes. Experts in this field can be invited to demonstrate the color difference and pixel interval of spots with fusion of late blight and leaf mold spots to determine the color difference threshold and pixel interval. This is existing technology and will not be elaborated here.

[0102] Step 4: If there is no similar pest and disease combination, the first pest and disease recognition result is used as the final pest and disease recognition result; if there is a first pest and disease combination, the spot area of ​​the crop image to be judged is traversed. If the spot area has a joint spot feature, both pests in the similar pest and disease combination are used as the output result; if there is no joint spot feature, the pest and disease with the lowest confidence in the similar pest and disease combination is eliminated, and the first pest and disease recognition result is updated as the final pest and disease recognition result;

[0103] Furthermore, similar pest and disease combinations include an early blight-spotted disease combination and a late blight-leaf mold combination; a confidence threshold is preset, and if the confidences of both pests and diseases in a similar pest and disease combination are greater than the confidence threshold, it is determined that a similar pest and disease combination exists; otherwise, it is determined that a similar pest and disease combination does not exist.

[0104] Step 5: Determine the pests and diseases based on the final pest and disease identification results, and output the pest and disease status of the crop image to be determined.

[0105] Furthermore, the confidence level of each pest in the updated first pest identification result is compared with the confidence threshold, and all pests with a confidence level greater than the confidence threshold are taken as the judgment result of the image.

[0106] See also Figure 2-Figure 5 , Figure 2-Figure 5 The following table shows the comparison of the recognition accuracy of early blight, leaf spot, late blight and leaf mold by only performing model analysis and identification and simultaneously performing model identification and analysis of the combined status of similar pests and diseases.

[0107] The following table summarizes the recognition results of multiple groups of images for early blight, leaf spot, late blight, and leaf mold respectively;

[0108] Table 1 Statistics of early blight identification

[0109]

[0110] The 15 comparative data sets in Table 1 show that, in most cases, the identification accuracy of combined pest and disease analysis surpassed that of model analysis alone, with an average improvement of +2.3 percentage points. However, there were three outliers (Groups 1, 3, and 11) where the combined analysis yielded results that were actually lower than those of the model analysis alone, with the largest decrease reaching -2.7 percentage points (Group 11). When the combined analysis outperformed the model analysis, the margin of improvement (+1.8 to +3.4 percentage points) was significantly greater than the decrease (-0.9 to -2.7 percentage points) when it underperformed, indicating that this method has a positive impact on early blight identification.

[0111] Table 2 Statistics of identification of leaf spot

[0112]

[0113] In Table 2, the average accuracy of "model analysis only" was 83.94%. After adding "analysis of similar pest and disease combination status", it increased to 85.16%, an average improvement of 1.22 percentage points, indicating that this method has certain positive benefits in identifying leaf spot disease. In terms of improvement, the accuracy of 11 of the 15 data sets increased, while that of 2 decreased (for example, the fourth group decreased from 84.5% to 84.3%, and the sixth group decreased from 86.4% to 84.7%), while the ninth and tenth groups remained almost unchanged. This shows that this method can bring slight to moderate improvements in most samples, but some samples still have problems with misjudgment or feature extraction failure.

[0114] Table 3 Statistics of late blight identification

[0115]

[0116] In this dataset of late blight identification data, the average accuracy of "model analysis alone" was 80.31%. This increased to 83.03% after the introduction of "analysis of similar pest and disease combinations," an average improvement of 2.72 percentage points, demonstrating a significant positive effect of this method on late blight identification. Eleven of the 15 groups saw improvements, with Group 1 (80.5 → 86.1), Group 10 (80.4 → 86.2), and Group 13 (80.0 → 85.8) all seeing increases of more than 5 percentage points. However, four groups experienced declines, but overall, most groups saw significant benefits.

[0117] Table 4 Statistics of leaf mold identification

[0118]

[0119] Table 4 shows that the combined analysis method improved leaf mold identification accuracy in most cases, with an average improvement of +3.2 percentage points and a maximum increase of +5.3 percentage points (Group 5). However, in six data sets (40%), performance was poor, with five showing a decrease in accuracy (the maximum decrease was -4.8 percentage points in Group 7) and one remaining essentially unchanged (Group 8). Overall, most groups benefited significantly from analyzing similar pest and disease combinations.

[0120] See also Figure 6 The present invention further provides a crop disease and insect pest identification system based on image feature recognition, wherein the system is used to implement the crop disease and insect pest identification method based on image feature recognition, specifically comprising:

[0121] The spot recognition module is used to grayscale and color-space convert the crop image to be judged, obtaining a grayscale image and a CIELAB color space image. The maximum inter-class variance method is used to extract the leaf area based on the grayscale image, and the Sobel operator is used to extract the image spot area from it. The image spot area is then pixel-mapped with the CIELAB image to obtain the grayscale spot area and color spot area corresponding to the crop image to be judged.

[0122] A preliminary recognition module is configured to use a pre-trained pest and disease analysis model to identify pests and diseases in the crop image to be identified, where grayscale and color spots are identified, to obtain confidence levels that the crop to be identified has various pests and diseases, and to construct a first image recognition label based on the confidence levels and the corresponding pest and disease types;

[0123] A similarity analysis module is configured to preset similar pest and disease combinations and corresponding joint spot features for each similar pest and disease combination, and determine whether a similar pest and disease combination exists in the first pest and disease identification result. The joint spot features include a first joint spot feature reflecting the phenomenon of heterogeneous permeability of water and pigment when early blight and leaf spot are combined, and a second joint spot feature reflecting the three-layer color band structure of the joint spot when late blight and leaf mold are combined;

[0124] The comprehensive judgment module is used to use the first pest identification result as the final pest identification result if no similar pest combination exists; if the first pest combination exists, traverse the spot area of ​​the crop image to be judged, and if the spot area has a joint spot feature, use both pests in the similar pest combination as the output result; if the joint spot feature does not exist, eliminate the pest with the lowest confidence in the similar pest combination and update the first pest identification result as the final pest identification result;

[0125] The final judgment module is used to judge the pests and diseases based on the final pest and disease identification results and output the pest and disease conditions of the crop images to be judged.

[0126] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters in the formulas are set by technicians in this field according to actual conditions.

[0127] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software depends on the specific application and design constraints of the technical solution.

[0128] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, and may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment as needed.

[0129] The above is only a specific implementation method of the present application, but the scope of protection of the present application is not limited thereto. Any technical personnel familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, and they should all be covered by the scope of protection of the present application.

Claims

1. A method for determining crop diseases and insect pests based on image feature recognition, characterized in that: The specific steps include: Step 1: The crop image to be judged is grayscaled and converted to color space to obtain a grayscale image and a CIELAB color space image. The leaf area is extracted based on the grayscale image using the maximum inter-class variance method, and the image spot area is extracted from it using the Sobel operator. The image spot area is then pixel-mapped with the CIELAB image to obtain the grayscale spot area and color spot area corresponding to the crop image to be judged. Step 2: Use the pre-trained pest and disease analysis model to identify pests and diseases in the crop image to be identified, where grayscale and color spots are identified. Confidence scores for each type of pest and disease are obtained, and a first image recognition label is constructed based on the confidence scores and the corresponding pest and disease types. Step 3: Preset similar pest and disease combinations and joint spot features corresponding to each similar pest and disease combination, and determine whether a similar pest and disease combination exists in the first pest and disease identification result. The joint spot features include a first joint spot feature reflecting the phenomenon of water and pigment anisotropy when early blight and leaf spot are combined, and a second joint spot feature reflecting the three-layer color band structure of the joint spot when late blight and leaf mold are combined. Step 4: If there is no similar pest and disease combination, the first pest and disease recognition result is used as the final pest and disease recognition result; if there is a first pest and disease combination, the spot area of ​​the crop image to be judged is traversed. If the spot area has a joint spot feature, both pests in the similar pest and disease combination are used as the output result; if there is no joint spot feature, the pest and disease with the lowest confidence in the similar pest and disease combination is eliminated, and the first pest and disease recognition result is updated as the final pest and disease recognition result; Step 5: Determine the pests and diseases based on the final pest and disease identification results, and output the pest and disease status of the crop image to be determined.

2. The method for determining crop pests and diseases based on image feature recognition according to claim 1, characterized in that: The logic for obtaining the spot area of ​​the crop image to be judged is as follows: the crop image to be judged is copied twice, one copy is grayscaled, and the other copy is converted from RGB color space to CIELAB color space. For the grayscaled crop image to be judged, the maximum inter-class variance method is used to extract the leaf part of the crop image to be judged, and then the Sobel operator is used to determine the spot area of ​​the leaf part of the crop image to be judged, which is called the grayscale spot area; based on the grayscale spot area, the spot area of ​​the crop image to be judged that has undergone color space conversion is framed, and the framed spot area is called the color spot area.

3. The method for determining crop pests and diseases based on image feature recognition according to claim 2, characterized in that: According to the identified grayscale spot areas and color spot areas, the unprocessed crop image to be judged in the RGB color space is framed to obtain the crop image to be judged in the RGB color space with the spot areas identified. Mask processing is performed to retain the R, G, and B values ​​of the spot areas of the crop image to be judged in the RGB color space, and the R, G, and B values ​​of the non-spot areas of the crop image to be judged in the RGB color space are set to 0. The crop image to be judged after mask processing is called the crop image to be judged with the grayscale spot areas and color spot areas identified for pest and disease identification.

4. The method for determining crop pests and diseases based on image feature recognition according to claim 2, characterized in that: The logic of obtaining the first joint lesion feature is: use the gray level co-occurrence matrix to extract the texture features of each gray spot area and form a texture feature vector ,in, For the Texture feature vector of block grayscale spot area, For the The contrast of the gray-level co-occurrence matrix of the block gray spot area, For the The energy of the gray-level co-occurrence matrix of the block gray-level spot region, For the Homogeneity of the gray-level co-occurrence matrix of the block gray-level spot region, For the The entropy of the gray-level co-occurrence matrix of the block gray-level spot area; the gray-level spot area is divided into several spot sub-areas of equal area, and the local texture feature vector of each spot sub-area is obtained ,in, For the In the gray spot area, Texture feature vector of the spot sub-region, For the In the gray spot area, The contrast of the gray-level co-occurrence matrix of each spot sub-region, For the In the gray spot area, The energy of the gray-level co-occurrence matrix of each spot sub-region, For the In the gray spot area, The homogeneity of the gray level co-occurrence matrix of each spot sub-region, For the In the gray spot area, The entropy of the gray-level co-occurrence matrix of each spot sub-region is calculated; the Euclidean distance of the texture feature vector between the gray-level spot region and each of its spot sub-regions is calculated, and the spot sub-region with the largest Euclidean distance is called the target sub-region. is the index of the grayscale spot area, is the index of the spot sub-region in the spot region; Based on the target sub-region, the crop image to be judged is framed and processed for color space conversion to obtain the key area. The brightness gradient direction and color gradient direction of each pixel in the key area are obtained respectively, and the angle between the brightness gradient direction and the color gradient direction is calculated. The angle is counted. The number of pixels within the range is divided by the total number of pixels in the target sub-region to obtain the fused adversarial feature; A fusion adversarial feature threshold is preset. If the fusion adversarial feature is greater than the fusion adversarial feature threshold, the first joint lesion feature is deemed to exist; otherwise, it is deemed not to exist.

5. The method for determining crop pests and diseases based on image feature recognition according to claim 2, characterized in that: Starting from the center of each color spot area, determine an extension direction every 15 degrees in the clockwise direction; In any determined extension direction, a pixel interval is preset, and the pixels in the extension direction are divided into multiple pixel segments based on the preset pixel interval. The color difference between adjacent pixels in the pixel segment is analyzed. To determine whether a color mutation occurs, the Euclidean distance of the pixel attribute of the color difference CIELAB color space is used. If at least two color mutations occur within a pixel segment in any extension direction of any spot, it is considered that the second joint lesion feature exists; otherwise, it is considered that the second joint lesion feature does not exist.

6. The method for determining crop pests and diseases based on image feature recognition according to claim 5, characterized in that: The color difference is the Euclidean distance of the three channels of the pixel points in the CIELAB color space. If the color difference between two adjacent pixels is greater than a preset color difference threshold, a color mutation is marked.

7. The method for determining crop pests and diseases based on image feature recognition according to claim 6, characterized in that: Similar pest and disease combinations include an early blight-spotted disease combination and a late blight-leaf mold combination. A confidence threshold is preset. If the confidence of both pests and diseases in a similar pest and disease combination is greater than the confidence threshold, it is determined that a similar pest and disease combination exists; otherwise, it is determined that no similar pest and disease combination exists.

8. The method for determining crop pests and diseases based on image feature recognition according to claim 7, characterized in that: The confidence of each pest and disease in the updated first pest and disease identification result is compared with the confidence threshold, and all pests and diseases with a confidence greater than the confidence threshold are taken as the judgment results of the image.

9. A crop disease and insect pest identification system based on image feature recognition, characterized by: The system is used to implement the crop disease and insect pest identification method based on image feature recognition according to any one of claims 1 to 8, specifically comprising: The spot recognition module is used to grayscale and color-space convert the crop image to be judged, obtaining a grayscale image and a CIELAB color space image. The maximum inter-class variance method is used to extract the leaf area based on the grayscale image, and the Sobel operator is used to extract the image spot area from it. The image spot area is then pixel-mapped with the CIELAB image to obtain the grayscale spot area and color spot area corresponding to the crop image to be judged. A preliminary recognition module is configured to use a pre-trained pest and disease analysis model to identify pests and diseases in the crop image to be identified, where grayscale and color spots are identified, to obtain confidence levels that the crop to be identified has various pests and diseases, and to construct a first image recognition label based on the confidence levels and the corresponding pest and disease types; A similarity analysis module is configured to preset similar pest and disease combinations and corresponding joint spot features for each similar pest and disease combination, and determine whether a similar pest and disease combination exists in the first pest and disease identification result. The joint spot features include a first joint spot feature reflecting the phenomenon of heterogeneous permeability of water and pigment when early blight and leaf spot are combined, and a second joint spot feature reflecting the three-layer color band structure of the joint spot when late blight and leaf mold are combined; The comprehensive judgment module is used to use the first pest identification result as the final pest identification result if no similar pest combination exists; if the first pest combination exists, traverse the spot area of ​​the crop image to be judged, and if the spot area has a joint spot feature, use both pests in the similar pest combination as the output result; if the joint spot feature does not exist, eliminate the pest with the lowest confidence in the similar pest combination and update the first pest identification result as the final pest identification result; The final judgment module is used to judge the pests and diseases based on the final pest and disease identification results and output the pest and disease conditions of the crop images to be judged.

Images