Vegetable pest image recognition and classification method and device based on feature detection

Abstract

The application discloses a vegetable disease and pest image recognition and classification method and device based on feature detection, and the method comprises the following steps: acquiring a vegetable leaf scene image of a vegetable planting area, wherein the vegetable leaf scene image represents an image with a single leaf or multiple leaves as the main body and containing a scene background; identifying a disease and pest feature based on the vegetable leaf scene image, and acquiring a disease and pest candidate area; identifying a leaf edge where the disease and pest candidate area is located based on the position of the disease and pest candidate area in the whole vegetable leaf scene image, intercepting a leaf image where the disease and pest candidate area is located based on the leaf edge; identifying a disease and pest type based on the leaf image, and determining a vegetable disease and pest type. The application avoids the complexity of detection and classification identification of multiple types and multiple position disease and pest areas in a multi-leaf overlapping image, and reduces the missed detection and false detection of detection and classification identification of multiple types and multiple position disease and pest areas in a multi-leaf image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a method and apparatus for identifying and classifying vegetable pest and disease images based on feature detection. Background Technology

[0002] Vegetable cultivation is fraught with numerous pests and diseases, and their causes are complex. Most vegetable growers rely on subjective judgment regarding pest control, but a high rate of misjudgment leads to uncontrolled and inaccurate pest and disease management, directly impacting vegetable quality and yield, resulting in significant economic losses. Traditional methods rely on manual observation and experience, which are inefficient and susceptible to individual differences and subjective factors. Existing disease detection methods are also inefficient and cannot flexibly adapt to the complex and varied monitoring needs of vegetable cultivation environments. Summary of the Invention

[0003] To address the problems existing in the prior art, this invention provides a method and apparatus for vegetable pest and disease image recognition and classification based on feature detection. This method is applicable to both complex and simple backgrounds in vegetable growing environments, improving detection efficiency and accuracy. The technical solution is as follows:

[0004] Firstly, a method for image recognition and classification of vegetable diseases and pests based on feature detection is provided, including the following steps:

[0005] Acquire vegetable leaf scene images of a vegetable planting area, wherein the vegetable leaf scene images represent images with a single leaf or multiple leaves as the main body and also include a scene background;

[0006] Based on the identification of pest and disease characteristics in vegetable leaf scene images, candidate regions for pests and diseases are obtained.

[0007] Based on the location of the disease and pest candidate region in the entire vegetable leaf scene image, identify the leaf edge where the disease and pest candidate region is located, and based on the leaf edge, crop the leaf image where the disease and pest candidate region is located;

[0008] Based on the leaf images, the types of diseases and pests are identified to determine the types of vegetable diseases and pests.

[0009] In some implementations, the step of identifying pest and disease features based on vegetable leaf scene images and obtaining candidate regions for pests and diseases includes:

[0010] Acquire a first image set, train a preset first model, and acquire a first model. The first image in the first image set is an image of a single leaf, and the first model is a model that can identify whether an image of a single leaf has pest or disease characteristics.

[0011] Obtain the depth image corresponding to the vegetable leaf scene image. Based on the coordinate position and depth value of each pixel in the depth image, perform cluster analysis on the depth image pixels to obtain several cluster regions. The image pixel positions and depth values ​​in each cluster region are similar.

[0012] Based on the position of each cluster region in the depth image, the corresponding cluster region is obtained in the vegetable leaf scene image. For each cluster region in the vegetable leaf scene image, the first model is used to identify whether it has pest and disease characteristics. If so, the cluster region in the vegetable leaf scene image is the first pest and disease candidate region.

[0013] In some implementations, the training process of the first model includes:

[0014] Based on the first image, several cropped partial images are obtained. The cropped partial images include healthy areas of the leaves, or include both healthy areas and areas affected by pests and diseases.

[0015] The first model is trained by using cropped local images as training samples. The first model is a model that can identify whether a local area of ​​an image collected from a single leaf has pest or disease characteristics.

[0016] The method for identifying whether there are candidate regions for pests and diseases based on vegetable leaf scene images also includes:

[0017] For images of vegetable leaves, a window of a preset size is slid across the currently acquired image to obtain an image of each sliding window;

[0018] For each sliding window image, the first model is used to identify and detect whether it has pest and disease characteristics. Sliding window images with pest and disease characteristics are marked as second pest and disease candidate regions.

[0019] The candidate regions for pests and diseases are determined by combining the first and second candidate regions.

[0020] In some implementations, identifying the leaf edge where the candidate pest / disease region is located based on the position of the candidate region in the entire vegetable leaf scene image includes:

[0021] Based on the first edge recognition method, edge lines are identified in the entire vegetable leaf scene image, and edge lines belonging to the vegetable leaf edge are selected and recorded as the first vegetable leaf recognized edge. The first edge recognition method is used to detect and identify edge lines whose recognition significance exceeds a preset threshold.

[0022] Based on the location of the pest and disease candidate regions in the entire acquired image, analyze whether each pest and disease candidate region location matches a first vegetable leaf identification edge. If a match is found, determine the first leaf edge where the pest and disease candidate region is located. The matching includes the closest pest and disease candidate region and the location of the pest and disease candidate region within the bounded area of ​​the first vegetable leaf identification edge or within the bounded area of ​​the fitted first vegetable leaf identification edge.

[0023] In some embodiments, the screening of edge lines belonging to the edge of vegetable leaves includes:

[0024] Based on the identified edge lines, combined with the depth image, it is determined whether the depth values ​​of pixels on a single edge line are continuous, and the similarity between the shape of a single edge line and the edge of a leaf is analyzed; when both continuity and similarity meet preset conditions, the edge lines belonging to the edge of a vegetable leaf are determined.

[0025] In some embodiments, identifying the leaf edge where the candidate pest / disease region is located based on the position of the candidate region in the entire vegetable leaf scene image further includes:

[0026] Based on the edge recognition of the first vegetable leaf, the size of the leaf is estimated, and the estimated size of the leaf is obtained. The estimated size of the leaf includes the maximum value, mean value, and mode value of the leaf size.

[0027] For the candidate regions of pests and diseases in the entire acquired image, based on the candidate regions of pests and diseases as the center, an area is expanded outward to a range larger than the preset value of the estimated leaf size. A second edge recognition method is used to identify whether there is a leaf edge within the expanded area. If there is, the candidate regions of pests and diseases are determined to be the second leaf edge. If there is no leaf edge, the candidate regions of pests and diseases are determined to be the second candidate non-pest and disease area. The second edge recognition method is used to detect and identify edge lines whose clarity does not exceed a preset threshold.

[0028] For the location of a candidate area for pests and diseases where there is a first leaf edge and / or a second leaf edge, the leaf edge where the candidate area for pests and diseases is located is determined based on the union of the first leaf edge and the second leaf edge;

[0029] For disease and pest candidate areas where there are no first or second leaf edges, the disease and pest candidate areas are determined as candidate non-disease and pest areas.

[0030] In some embodiments, the second edge recognition method includes:

[0031] Edge enhancement is performed on pixels in regions where there are no pest or disease candidate regions in the extended region image;

[0032] Edge detection is performed on the extended region image after edge enhancement.

[0033] Secondly, a feature detection-based image recognition and classification device for vegetable diseases and pests is provided, comprising:

[0034] An image acquisition unit is used to acquire vegetable leaf scene images of a vegetable planting area. The vegetable leaf scene images represent images with a single leaf or multiple leaves as the main body and also include a scene background.

[0035] The pest and disease identification unit is used to identify candidate regions for pests and diseases based on vegetable leaf scene images;

[0036] The leaf image acquisition unit for pests and diseases is used to identify the leaf edge where the pest and disease candidate region is located based on the location of the pest and disease candidate region in the whole vegetable leaf scene image, and to extract the leaf image where the pest and disease candidate region is located based on the leaf edge.

[0037] The pest and disease classification unit identifies the types of pests and diseases in vegetables based on the leaf images.

[0038] Thirdly, an electronic device is provided, the electronic device comprising:

[0039] processor;

[0040] Memory used to store processor-executable instructions;

[0041] The processor executes the executable instructions to implement the feature detection-based image recognition and classification method for vegetable diseases and pests as described in the first aspect above.

[0042] Fourthly, a computer-readable storage medium is provided, on which computer instructions are stored, which, when executed by a processor, implement the steps of the feature detection-based image recognition and classification method for vegetable diseases and pests as described in the first aspect above.

[0043] This invention provides a method and apparatus for vegetable pest and disease image recognition and classification based on feature detection, which has the following beneficial effects: The method provided by this invention is flexibly applicable to various scenes of vegetable leaf images under various image acquisition methods. It can be an image of a large area captured with a complex background, or an image of a single leaf captured with a simple background. For vegetable leaf images captured in large scenes, this invention first identifies candidate regions for pests and diseases. Using each candidate region as the center, it identifies the leaf edge where the candidate region is located. Subsequently, it identifies the type of pest and disease only for a single leaf, improving detection efficiency and accuracy. Attached Figure Description

[0044] Figure 1 This is a flowchart of the vegetable disease and pest image recognition and classification method based on feature detection in the embodiments of this application;

[0045] Figure 2 This is a schematic diagram of the method for obtaining candidate regions of pests and diseases in an embodiment of this application;

[0046] Figure 3 This is a schematic diagram of a vegetable disease and pest image recognition and classification device system based on feature detection in an embodiment of this application. Detailed Implementation

[0047] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0048] See Figure 1 This application provides a method for image recognition and classification of vegetable diseases and pests based on feature detection, including the following steps:

[0049] Step 1: Obtain a scene image of vegetable leaves in the vegetable planting area. The vegetable leaf scene image represents an image with a single leaf or multiple leaves as the main body and also includes a scene background.

[0050] Step 2: Identify pest and disease features based on vegetable leaf scene images to obtain candidate regions for pests and diseases;

[0051] Step 3: Based on the location of the disease and pest candidate region in the entire vegetable leaf scene image, identify the leaf edge where the disease and pest candidate region is located, and based on the leaf edge, crop the leaf image where the disease and pest candidate region is located;

[0052] Step 4: Based on the leaf images, identify the types of diseases and pests to determine the types of vegetable diseases and pests.

[0053] The vegetable leaf scene image in the vegetable planting area includes one or more overlapping leaves, as well as the planting ground, vegetable stems and vines, vegetable planting racks, etc. In this embodiment, for the captured images of vegetable leaves in a large scene, pest and disease candidate regions are first identified. It can be understood that the identification result filters out areas of completely healthy leaves on the one hand, and non-leaf areas in the scene captured image on the other hand. Considering that the captured image contains overlapping images of multiple leaves and background images of non-leaf, this application initially filters out areas of completely healthy leaves and non-leaf background areas based on pest and disease characteristics. Of course, when performing pest and disease candidate region identification, there may be a possibility of misidentifying non-leaf background areas as pest and disease candidate regions. However, this will be further eliminated in the next step of single leaf edge recognition, where images of non-leaf background areas are excluded, and only leaf images containing pest and disease candidate regions are retained.

[0054] Furthermore, in this embodiment, each pest / disease candidate region is used as the center to identify the leaf edge where the candidate region is located. Subsequent pest / disease type identification is performed only on a single leaf, improving detection efficiency and accuracy. In this case, during the training of the pest / disease type identification model, it is not necessary to consider image sets with complex backgrounds consistent with or similar to those in the leaf images of this embodiment as training samples. Only images of vegetable leaves of the same type are needed to create training samples, improving model training efficiency. Moreover, using leaf images with simple backgrounds for training the pest / disease type identification model in this embodiment also improves model training convergence efficiency, increases model recognition accuracy, avoids the complexity of detecting and classifying multiple types and locations of pests / diseases in multi-leaf images, and avoids missed detections and false detections in multi-leaf images.

[0055] Of course, the method provided in this application embodiment is also applicable to the identification and classification of pest and disease images in simple background images containing only a single leaf. In this case, when identifying the candidate regions of pests and diseases, only the candidate regions of pests and diseases on the same leaf will be identified. By identifying the leaf edge where the candidate regions of pests and diseases are located, the single leaf image region is extracted, and the pest and disease type is identified in the single leaf image region.

[0056] The method provided in this application embodiment is flexibly applicable to various vegetable leaf image acquisition methods, which can be large-area image acquisition (complex background image) or local single leaf image acquisition (single background image).

[0057] In one embodiment, step 2 above, which involves identifying pest and disease features based on vegetable leaf scene images to obtain candidate regions for pests and diseases, includes the following steps:

[0058] Step 21: Obtain the first image set, train the preset first model, and obtain the first model. The first image in the first image set is an image of a single leaf, and the first model is a model that can identify whether a single leaf has pest or disease characteristics based on the collected images.

[0059] Step 22: Obtain the depth image corresponding to the vegetable leaf scene image. Based on the coordinate position and depth value of each pixel in the depth image, perform cluster analysis on the depth image pixels to obtain several cluster regions. The image pixel positions and depth values ​​in each cluster region are similar.

[0060] Step 23: Based on the position of each cluster region in the depth image, obtain the corresponding cluster region in the vegetable leaf scene image. For each cluster region in the vegetable leaf scene image, use the first model to identify whether it has pest and disease characteristics. If so, the cluster region in the vegetable leaf scene image is the first pest and disease candidate region.

[0061] In this embodiment, images of a single leaf are used as the first image set to train the first model, making the training image set for the first model readily available and allowing for the direct use of existing image sets. Furthermore, images of a single leaf effectively avoid the complex background differences of different scenes. This application avoids the problem that a first model trained using images from current complex background scenes is not suitable for identifying leaf diseases and pests in images from other background scenes.

[0062] In this embodiment, a depth map is used to obtain clustered regions, and pixel regions that may be leaf regions are obtained. Furthermore, pest and disease feature recognition is performed on the clustered regions to avoid misidentification caused by background interference when directly recognizing pest and disease features in the entire acquired image.

[0063] See Figure 2 In one implementation, the training process of the first model in step 21 above includes:

[0064] Step 211: Based on the first image, crop to obtain several cropped partial images. The cropped partial images include healthy areas of the leaves, or include both healthy areas and areas affected by pests and diseases.

[0065] Step 212: Based on the cropped local image as training sample, train the first model to obtain the first model. The first model is a model that can identify whether a local area of ​​an image collected from a single leaf has pest or disease characteristics.

[0066] Furthermore, step 2 above, which involves identifying whether there are candidate regions for pests and diseases based on the vegetable leaf scene image, also includes:

[0067] Step 24: For the vegetable leaf scene image, slide a window of a preset size on the currently acquired image to obtain the image of each sliding window;

[0068] Step 25: For each sliding window image, use the first model to identify and detect whether it has pest and disease characteristics, and mark the sliding window images with pest and disease characteristics as the second pest and disease candidate regions.

[0069] Step 26: Determine the disease and pest candidate areas based on the first and second disease and pest candidate areas.

[0070] It is understood that the first model in this application can distinguish between images with and without pest and disease characteristics at a fine-grained level. Therefore, this application can use the first model to locally identify pest and disease characteristics in multiple sliding windows of collected vegetable leaf scene images. In this embodiment, the cropped local image is set to include both pest and disease areas and healthy areas. It is understood that the identification of pest and disease characteristic pixels requires combining the positional distribution of the pest and disease area and the neighboring healthy leaf pixel areas within the leaf; that is, it requires combining global feature information. If identification is based solely on pest and disease pixels, some interfering background pixels may be misidentified as pest and disease characteristics due to their similarity to the pest and disease pixels. The size of the cropped local image cannot be too small; if it is too small, a large amount of global feature information of pest and disease characteristics in the entire leaf will be lost. The size of the cropped local image should be determined based on the size of the outer border that can simultaneously include the positions of the pest and disease areas and the healthy areas. Similarly, the size of the sliding window cannot be too small. The method of comprehensively determining the disease and pest candidate region based on the first and second disease and pest candidate regions can, in order to avoid missed detection, use both the first and second disease and pest candidate regions as disease and pest candidate regions.

[0071] In one implementation, step 23 above, based on the location of the candidate pest and disease region in the entire vegetable leaf scene image, identifies the leaf edge where the candidate pest and disease region is located, including:

[0072] Step 231: Based on the first edge recognition method, identify edge lines in the entire vegetable leaf scene image and filter out edge lines belonging to the vegetable leaf edge, which are recorded as the first vegetable leaf recognized edge. The first edge recognition method is used to detect and identify edge lines whose obviousness exceeds a preset threshold.

[0073] Step 232: Based on the location of the pest and disease candidate region in the entire acquired image, analyze whether each pest and disease candidate region location matches a first vegetable leaf identification edge. If a match is found, determine the first leaf edge where the pest and disease candidate region location is located. The matching includes the closest to the pest and disease candidate region and the location of the pest and disease candidate region within the bounded area of ​​the first vegetable leaf identification edge or within the fitted bounded area of ​​the first vegetable leaf identification edge. If no match is found, determine the pest and disease candidate region location as the first candidate non-pest and disease region location.

[0074] It is understandable that the first edge recognition algorithm is used to detect and identify relatively obvious and clear edge lines. For example, when using an edge detection operator, a higher threshold can be used to ensure that the detected edge lines are real edge lines. Considering that the background of the vegetable leaf scene image is complex and has a large number of interfering lines, the first edge recognition algorithm in this embodiment can be implemented using an edge detection operator.

[0075] In one embodiment, step 231 above, which involves filtering edge lines belonging to the edge of a vegetable leaf, includes:

[0076] Step 2311: Based on the identified edge lines and combined with the depth image, determine whether the depth values ​​of the pixels on a single edge line are continuous, and analyze the similarity between the shape of the single edge line and the edge of the leaf; when both continuity and similarity meet the preset conditions, determine the edge lines belonging to the edge of the vegetable leaf.

[0077] In this embodiment, the continuity of depth values ​​and shape analysis of pixels on the edge line are combined to determine whether the edge belongs to a vegetable leaf, thus avoiding false detections and missed detections of vegetable leaf edges caused by overlapping leaves. It is understood that a complete leaf edge may exhibit multiple shapes under different acquisition angles, and multiple possible shapes need to be considered when detecting shape similarity. It is also understood that the depth values ​​of pixels on the same edge line should change uniformly and continuously from one end to the other. If the depth value of a pixel on an edge line changes abruptly from one end to the other, the pixels on either side of the abrupt change location may not be on the same edge line.

[0078] It should be noted that, for vegetable leaf edge lines that have been filtered based on both continuity and similarity meeting the preset conditions, the following further applies:

[0079] Based on the selected vegetable leaf edge lines, a complete leaf edge is fitted according to the curvature and curvature changes at both ends of the edge lines;

[0080] Based on the area enclosed by the fitted complete leaf edge, analyze whether it satisfies the rule that the area enclosed by the fitted complete leaf edge decreases as the depth of the leaf edge lines of different vegetables increases. If not, mark the edge lines as suspected misjudged vegetable leaf edge lines.

[0081] It is understandable that as the depth changes, i.e. the distance from the camera increases, the image of an object of the same size will become smaller and smaller. Even considering the different sizes of the blades, the blades at a shallower depth will most likely be larger than the blades at a deeper depth.

[0082] It should be noted that in step 232 above, the matching includes matching that is closest to the candidate region of pests and diseases and the candidate region of pests and diseases is located within the area enclosed by the first vegetable leaf recognition edge or within the area enclosed by the first vegetable leaf recognition edge fitting, including:

[0083] Step 2321: Based on the identified first vegetable leaf identification edge, determine several first vegetable leaf identification edges that are relatively close to each other;

[0084] Step 2322: For several closely spaced first vegetable leaf identification edges, analyze whether they are closed. If they are closed, analyze whether the disease and pest candidate area is within the closed range of the first vegetable leaf identification edge. If they are not closed, fit the closed range of the first vegetable leaf identification edge along the curvature and curvature change of the two ends of the identified first vegetable leaf identification edge, and analyze whether the disease and pest candidate area is within the closed range of the first vegetable leaf identification edge.

[0085] In one embodiment, step 23 above, which identifies the leaf edge where the candidate pest / disease region is located based on the location of the candidate pest / disease region in the entire vegetable leaf scene image, further includes:

[0086] Step 233: Based on the edge recognition of the first vegetable leaf, estimate the size of the leaf and obtain the estimated size value of the leaf, which includes the maximum value, mean value and mode value of the leaf size;

[0087] Step 234: For the candidate disease and pest region locations in the entire acquired image, based on the candidate disease and pest region locations as the center, expand outward to a region larger than the preset value of the estimated leaf size (this preset value is small, the purpose is to obtain an expanded region slightly larger than the estimated leaf size). Use the second edge recognition method to identify whether there is a leaf edge within the expanded region. If it exists, determine that the candidate disease and pest region location is the second leaf edge. If it does not exist, determine that the candidate disease and pest region location is the second candidate non-disease and pest region location. The second edge recognition method is used to detect and identify edge lines whose clarity does not exceed a preset threshold.

[0088] Step 235: For the location of a candidate area for pests and diseases where there is a first leaf edge and / or a second leaf edge, the leaf edge where the candidate area for pests and diseases is located is determined based on the union of the first leaf edge and the second leaf edge.

[0089] Step 236: For disease and pest candidate areas where there are no first leaf edges or second leaf edges, determine the disease and pest candidate area as a candidate non-disease and pest area.

[0090] It is understood that the second edge recognition algorithm is used to identify edge lines with indistinct boundaries. In this embodiment, the leaf edge detection is performed in a specified area within the extended region based on the pest and disease area, thereby improving the accuracy of leaf edge detection. In this embodiment, edge detection of indistinct edge lines is only required within the extended region, and enhancement processing can be performed only on this specific area, avoiding the introduction of a lot of noise and resulting in false edge detection results due to enhancement processing of the entire vegetable leaf scene image.

[0091] In one embodiment, the second edge recognition method in step 234 above includes:

[0092] Step 23401: Perform edge enhancement based on the pixels of the non-pest candidate region locations in the extended region image;

[0093] Step 23402: Perform edge detection on the extended region image after edge enhancement.

[0094] In this embodiment of the application, in order to accurately identify whether the extended region image contains edge lines, edge enhancement is performed on pixels in areas outside the pest and disease candidate regions of the extended region image to enhance the edge contours. Further, this edge enhancement process includes:

[0095] (1) Based on the extended region image as the second image, the second image is convolved by four different convolution kernels. The four convolution kernels are all 3*3 in size. The first convolution kernel uses the three elements in the second row of the 3*3 matrix as non-zero elements and the rest of the 3*3 matrix as zero elements. The second convolution kernel uses the three elements in the second column of the 3*3 matrix as non-zero elements and the rest of the 3*3 matrix as zero elements. The third convolution kernel uses the three elements on one diagonal of the 3*3 matrix as non-zero elements and the rest of the 3*3 matrix as zero elements. The fourth convolution kernel uses the three elements on the other diagonal of the 3*3 matrix as non-zero elements and the rest of the 3*3 matrix as zero elements. The first edge feature map, the second edge feature map, the third edge feature map, and the fourth edge feature map are obtained respectively.

[0096] (2) Subtract the pixel value of each pixel from 255 to obtain the inverse pixel value of each pixel in the second image, and obtain the inverse image of the second image; perform convolution calculation on the inverse image by the first convolution kernel, the second convolution kernel, the third convolution kernel and the fourth convolution kernel respectively to obtain four convolution feature maps; pass each convolution feature map through the activation function to obtain the weight feature map of each convolution feature map, which is denoted as the first weight feature map, the second weight feature map, the third weight feature map and the fourth weight feature map; the size design and non-zero element position design of the four convolution kernels in this step are the same as those of the four convolution kernels in step (1);

[0097] (3) Multiply the first edge feature map with the remaining weight feature maps that are not the first weight feature map, multiply the second edge feature map with the remaining weight feature maps that are not the second weight feature map, multiply the third edge feature map with the remaining weight feature maps that are not the third weight feature map, multiply the fourth edge feature map with the remaining weight feature maps that are not the fourth weight feature map, and fuse the results of multiplying each edge feature map with the weight feature map to obtain the edge feature fusion map;

[0098] (4) The edge feature fusion map is added to the second image to obtain the edge enhancement feature map.

[0099] In this embodiment, the non-zero elements on the four convolutional kernels in step (1) are learnable parameters. By using the extended region image of vegetable leaves as samples and the edges of vegetable leaves as labeled data, the learnable parameters on the four convolutional kernels can be trained through model training. In this embodiment, four different operators are used to achieve edge feature detection in multiple directions by expressing the operators on the horizontal, vertical, and two diagonals. In this embodiment, in step (2), the reverse image of the second image is obtained by calculating the reverse pixel value of each pixel. Based on the reverse image, the reverse information features of the second image are fully expressed. Step (1) is used to extract the edge feature information of the second image, and step (2) is used to extract the reverse information features of the second image. The combination of steps (1) and (2) fully utilizes the information features of the second image and improves the expression of edge information. In this embodiment, in step (3), each edge feature map is multiplied with the weight feature map in an alternating manner to fuse edge features in multiple directions and strengthen the edge feature information. In step (4), the edge feature fusion map enhanced by edge feature information is added to the original second image and fused to form the edge enhancement feature map.

[0100] It should be noted that, in this embodiment of the application, the above-mentioned identification of pest and disease types based on the leaf images to determine the types of vegetable pests and diseases includes:

[0101] The first image in the first image set, which is a single leaf image, is used as the training sample. The types of vegetable diseases and pests in the first image are used as the sample annotation data to train the second model. The trained second model is then used to identify the types of vegetable diseases and pests in the first image.

[0102] See Figure 3 This application provides a vegetable disease and pest image recognition and classification device based on feature detection, comprising:

[0103] An image acquisition unit is used to acquire vegetable leaf scene images of a vegetable planting area. The vegetable leaf scene images represent images with a single leaf or multiple leaves as the main body and also include a scene background.

[0104] The pest and disease identification unit is used to identify candidate regions for pests and diseases based on vegetable leaf scene images;

[0105] The leaf image acquisition unit for pests and diseases is used to identify the leaf edge where the pest and disease candidate region is located based on the location of the pest and disease candidate region in the whole vegetable leaf scene image, and to extract the leaf image where the pest and disease candidate region is located based on the leaf edge.

[0106] The pest and disease classification unit identifies the types of pests and diseases in vegetables based on the leaf images.

[0107] Specific limitations regarding the feature detection-based vegetable pest and disease image recognition and classification device can be found in the limitations of the feature detection-based vegetable pest and disease image recognition and classification method described above, and will not be repeated here. Each unit in the aforementioned feature detection-based vegetable pest and disease image recognition and classification device can be implemented entirely or partially through software, hardware, or a combination thereof. These units can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each unit.

[0108] This application provides an electronic device, the electronic device comprising:

[0109] processor;

[0110] Memory used to store processor-executable instructions;

[0111] The processor executes the executable instructions to implement the feature detection-based image recognition and classification method for vegetable diseases and pests as described above.

[0112] It is understood that the processor in the embodiments of this application may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). In some embodiments, the computer device may also optionally include an input interface and an output interface. The processor, memory, and input / output interfaces can be connected via a bus or signal lines. Various peripheral devices can be connected to the input / output interfaces via buses, signal lines, or circuit boards. The input / output interfaces can be used to connect at least one input / output related peripheral device to the processor and memory. In some embodiments, the processor, memory, and input / output interfaces are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor, memory, and input / output interfaces can be implemented on separate chips or circuit boards, and this application embodiment does not limit this.

[0113] This application provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of the feature detection-based image recognition and classification method for vegetable diseases and pests described above. Those skilled in the art will recognize that the functions described in the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable storage medium or transmitted as one or more instructions or code on the computer-readable storage medium. Computer-readable storage media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of computer programs from one location to another. Storage media can be any available medium accessible to general-purpose or special-purpose computers.

[0114] This invention is not limited to the specific embodiments described above. Any modifications made by those skilled in the art based on the above concept without creative effort are within the scope of protection of this invention.

Claims

1. A method for image recognition and classification of vegetable diseases and pests based on feature detection, characterized in that, include: Acquire vegetable leaf scene images of a vegetable planting area, wherein the vegetable leaf scene images represent images with a single leaf or multiple leaves as the main body and also include a scene background; Based on the identification of pest and disease characteristics in vegetable leaf scene images, candidate regions for pests and diseases are obtained. Based on the location of the disease and pest candidate region in the entire vegetable leaf scene image, identify the leaf edge where the disease and pest candidate region is located, and based on the leaf edge, crop the leaf image where the disease and pest candidate region is located; Based on the leaf images, identify the types of diseases and pests to determine the types of vegetable diseases and pests; The method for identifying pest and disease features based on vegetable leaf scene images and obtaining pest and disease candidate regions includes: acquiring a first image set; training a preset first model to obtain a first model, wherein the first image in the first image set is an image of a single leaf, and the first model is a model capable of identifying whether an image of a single leaf has pest and disease features; acquiring a depth image corresponding to the vegetable leaf scene image; performing cluster analysis on the pixels of the depth image based on the coordinate position and depth value of each pixel in the depth image to obtain several cluster regions, wherein the image pixel positions and depth values ​​in each cluster region are similar; obtaining a corresponding cluster region in the vegetable leaf scene image based on the position of each cluster region in the depth image; and using the first model to identify whether each cluster region in the vegetable leaf scene image has pest and disease features. If so, the cluster region in the vegetable leaf scene image is a first pest and disease candidate region. Based on the location of candidate regions for pests and diseases in the entire vegetable leaf scene image, the leaf edge where the candidate regions for pests and diseases are located is identified, including: based on a first edge recognition method, edge lines are identified in the entire vegetable leaf scene image, and edge lines belonging to the vegetable leaf edge are filtered out. The filtering of edge lines belonging to the vegetable leaf edge includes: based on the identified edge lines, combined with the depth image, determining whether the depth values ​​of pixels on a single edge line are continuous, and analyzing the similarity between the shape of a single edge line and the leaf edge; when both continuity and similarity meet preset conditions, the edge lines belonging to the vegetable leaf edge are determined.

2. The method for vegetable disease and pest image recognition and classification based on feature detection according to claim 1, characterized in that, The training process of the first model includes: Based on the first image, several cropped partial images are obtained. The cropped partial images include healthy areas of the leaves, or include both healthy areas and areas affected by pests and diseases. The first model is trained by using cropped local images as training samples. The first model is a model that can identify whether a local area of ​​an image collected from a single leaf has pest or disease characteristics. The method for identifying whether there are candidate regions for pests and diseases based on vegetable leaf scene images also includes: For images of vegetable leaves, a window of a preset size is slid across the currently acquired image to obtain an image of each sliding window; For each sliding window image, the first model is used to identify and detect whether it has pest and disease characteristics. Sliding window images with pest and disease characteristics are marked as second pest and disease candidate regions. The candidate regions for pests and diseases are determined by combining the first and second candidate regions.

3. The method for vegetable disease and pest image recognition and classification based on feature detection according to claim 1, characterized in that, The step of identifying the leaf edge where the candidate regions for pests and diseases are located based on the position of the candidate regions in the entire vegetable leaf scene image includes: Based on the first edge recognition method, edge lines are identified in the entire vegetable leaf scene image, and edge lines belonging to the vegetable leaf edge are selected and recorded as the first vegetable leaf recognized edge. The first edge recognition method is used to detect and identify edge lines whose recognition significance exceeds a preset threshold. Based on the location of the pest and disease candidate regions in the entire acquired image, analyze whether each pest and disease candidate region location matches a first vegetable leaf identification edge. If a match is found, determine the first leaf edge where the pest and disease candidate region is located. The matching includes the closest pest and disease candidate region and the location of the pest and disease candidate region within the bounded area of ​​the first vegetable leaf identification edge or within the bounded area of ​​the fitted first vegetable leaf identification edge.

4. The vegetable disease and pest image recognition and classification method based on feature detection according to claim 3, characterized in that, The step of identifying the leaf edge where the candidate regions for pests and diseases are located based on the position of the candidate regions in the entire vegetable leaf scene image also includes: Based on the edge recognition of the first vegetable leaf, the size of the leaf is estimated, and the estimated size of the leaf is obtained. The estimated size of the leaf includes the maximum value, mean value, and mode value of the leaf size. For the candidate regions of pests and diseases in the entire acquired image, based on the candidate regions of pests and diseases as the center, an area is expanded outward to a range larger than the preset value of the estimated leaf size. A second edge recognition method is used to identify whether there is a leaf edge within the expanded area. If there is, the candidate regions of pests and diseases are determined to be the second leaf edge. If there is no leaf edge, the candidate regions of pests and diseases are determined to be the second candidate non-pest and disease area. The second edge recognition method is used to detect and identify edge lines whose clarity does not exceed a preset threshold. For the location of a candidate area for pests and diseases where there is a first leaf edge and / or a second leaf edge, the leaf edge where the candidate area for pests and diseases is located is determined based on the union of the first leaf edge and the second leaf edge; For disease and pest candidate areas where there are no first or second leaf edges, the disease and pest candidate areas are determined as candidate non-disease and pest areas.

5. The vegetable disease and pest image recognition and classification method based on feature detection according to claim 4, characterized in that, The second edge recognition method includes: Edge enhancement is performed on pixels in regions where there are no pest or disease candidate regions in the extended region image; Edge detection is performed on the extended region image after edge enhancement.

6. A vegetable disease and pest image recognition and classification device based on feature detection, characterized in that, include: An image acquisition unit is used to acquire vegetable leaf scene images of a vegetable planting area. The vegetable leaf scene images represent images with a single leaf or multiple leaves as the main body and also include a scene background. The pest and disease identification unit is used to identify candidate regions for pests and diseases based on vegetable leaf scene images; The leaf image acquisition unit for pests and diseases is used to identify the leaf edge where the pest and disease candidate region is located based on the location of the pest and disease candidate region in the whole vegetable leaf scene image, and to extract the leaf image where the pest and disease candidate region is located based on the leaf edge. The pest and disease classification unit identifies pest and disease types based on the leaf images to determine the types of vegetable pests and diseases. The method for identifying pest and disease candidate regions based on vegetable leaf scene images includes: acquiring a first image set; training a preset first model to acquire a first model, wherein the first image in the first image set is an image of a single leaf, and the first model is a model capable of identifying whether an image of a single leaf has pest and disease characteristics; acquiring a depth image corresponding to the vegetable leaf scene image; performing cluster analysis on the pixels of the depth image based on the coordinate position and depth value of each pixel in the depth image to acquire several cluster regions, wherein the image pixel positions and depth values ​​within each cluster region are similar; acquiring a corresponding cluster region in the vegetable leaf scene image based on the position of each cluster region in the depth image; and using the first model to identify whether each cluster region in the vegetable leaf scene image has pest and disease characteristics. If so, the cluster region in the vegetable leaf scene image is a first pest and disease candidate region. The step of identifying the leaf edge where the candidate regions for pests and diseases are located based on the position of the candidate regions in the entire vegetable leaf scene image includes: identifying edge lines in the entire vegetable leaf scene image based on a first edge recognition method, and filtering edge lines belonging to the vegetable leaf edge. The filtering of edge lines belonging to the vegetable leaf edge includes: determining whether the depth values ​​of the pixels on a single edge line are continuous based on the identified edge lines and the depth image, and analyzing the similarity between the shape of a single edge line and the leaf edge; when both continuity and similarity meet preset conditions, the edge lines belonging to the vegetable leaf edge are determined.

7. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store processor-executable instructions; The processor implements the method as described in any one of claims 1-5 by executing the executable instructions.

8. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1-5.

Images