Methods for acquiring images of pests for the prevention and control of vegetable diseases and pests

Abstract

This invention relates to the field of image acquisition technology, specifically to a method for acquiring images of pests and diseases in vegetable pest control. This invention precisely extracts individual leaves from leaf images using the vein characteristics of vegetable leaves, obtaining both complete and occluded leaves. Then, based on the color characteristics of the leaf mesophyll and the continuity of the veins within a single leaf, the pest-infested area within that leaf is determined. Next, based on the number and area of ​​the pest-infested areas, the degree of pest infestation on both complete and occluded leaves is quantified. Finally, different image acquisition methods are used for different leaves depending on the degree of pest infestation. This method enables more precise location and acquisition of pest-infested areas, avoiding detection errors caused by leaf occlusion in traditional methods. Furthermore, through dynamic resource allocation, it provides more targeted data support for subsequent assessment of pest infestation levels and the development of control strategies.

Description

Technical Field

[0001] This invention relates to the field of image acquisition technology, specifically to a method for acquiring pest images for the prevention and control of vegetable diseases and pests. Background Technology

[0002] Pests and diseases damage vegetables throughout their entire growth cycle, affecting all parts from leaves to roots and fruits. Leaves are the primary target of pests, with cabbage caterpillars, aphids, and leaf beetles destroying leaf tissue through chewing or piercing-sucking mouthparts. Mild damage leaves irregular notches, holes, or discolored spots, affecting photosynthesis; severe damage can reduce leaves to just veins, causing seedlings to die and ultimately reducing vegetable yield. Furthermore, damaged fruits and vegetables suffer from appearance defects, residual insect excrement, or excessive pesticide residues, significantly reducing their commercial value and severely impacting farmers' income.

[0003] Existing technologies typically employ manual visual observation or image recognition-based methods to identify pests and diseases, but both have limitations. Relying on manual observation requires professionals to judge the type and severity of pests and diseases based on experience, resulting in low efficiency, susceptibility to subjective factors, and high labor costs, making it particularly unsuitable for large-scale vegetable pest and disease control. While traditional image recognition methods overcome some limitations of manual observation and improve recognition efficiency, they suffer from weaknesses in handling overlapping or occluded leaf scenes, and color or contour segmentation algorithms struggle to accurately extract individual leaves, leading to deviations in pest feature localization and quantification, thus affecting recognition accuracy. Summary of the Invention

[0004] To address the above technical problems, the present invention aims to provide a method for acquiring pest images for the prevention and control of vegetable diseases and pests.

[0005] According to a first aspect of the present invention, a method for acquiring pest images for the prevention and control of vegetable pests and diseases is provided, the specific technical solution of which is as follows:

[0006] Collect images of vegetable leaves;

[0007] Analyze the vein features of the leaf in the leaf image, perform image segmentation on the leaf image to obtain a single leaf, and determine whether the single leaf is a complete leaf or an occluded leaf;

[0008] Analyze the color characteristics of the leaf mesophyll and the continuity of the leaf veins in the single leaf to determine the area of ​​insect damage within the single leaf;

[0009] For each intact leaf, the number and area characteristics of the insect-eaten areas are analyzed to obtain the degree of insect infestation on each intact leaf.

[0010] For the covered leaves, the differences in vein distribution between the uncovered area and the complete leaves are analyzed to estimate the total leaf area of ​​the covered leaves. Then, the number and area characteristics of the insect-eaten areas in the uncovered area are analyzed. Combined with the area of ​​the uncovered area, the degree of insect infestation of each covered leaf is obtained.

[0011] The leaf image acquisition method is dynamically adjusted based on the degree of insect damage to different individual leaves.

[0012] In some embodiments of the present invention, analyzing the vein features of the leaf in the leaf image and performing image segmentation on the leaf image to obtain a single leaf includes:

[0013] Based on the color features of the leaf image, the leaf vein region of the leaf image is separated;

[0014] Based on the topological connection relationship of the leaf veins in the leaf vein region, the starting point of the leaf vein is identified, wherein the starting point includes the starting point of the main vein and the starting point of the lateral vein.

[0015] Starting from the aforementioned starting point, the distribution characteristics of pixels in the leaf vein region are searched and analyzed to determine the main vein and lateral veins in the leaf vein region, and a complete leaf vein network of a single leaf is constructed.

[0016] Edge detection is performed on the leaf image, and effective edge points are searched by combining the distribution characteristics of the leaf vein network to form a closed contour. The connected region containing the leaf vein region is selected as a single leaf.

[0017] In some embodiments of the present invention, the starting point of the leaf vein is identified based on the topological connection relationship of the leaf veins in the leaf vein region, wherein the starting point includes the starting point of the main vein and the starting point of the lateral vein, including:

[0018] The leaf vein region is processed into a skeletonized form, which is converted into a skeleton pixel with a width of one pixel, while preserving the topological connection relationship between the main vein and the lateral veins.

[0019] Based on the aforementioned topological connection, the direction of each leaf vein is transformed into a unit direction vector, and the unit direction vectors are clustered to obtain multiple clusters, each cluster corresponding to a single leaf.

[0020] The unit direction vector of all veins in each cluster is extended to form multiple straight lines, and the intersection of the fitted straight lines is taken as the starting point of the main vein.

[0021] Search for all skeleton pixels in the neighborhood of the main vein starting point until a node is encountered. If the number of skeleton pixels in the neighborhood of the node is greater than or equal to 3, then the node is the starting point of the side vein.

[0022] In some embodiments of the present invention, starting from the starting point, the distribution characteristics of pixels in the leaf vein region are searched and analyzed to determine the main vein and lateral veins in the leaf vein region, and a complete leaf vein network of a single leaf is constructed, including:

[0023] Search all skeleton pixels in the neighborhood of the main vein starting point, determine the starting direction of the adjacent line segment, extend the skeleton pixels along each starting direction to the far end, record all skeleton pixels on the path, until a node is encountered.

[0024] If the number of skeleton pixels in the neighborhood of a node is equal to 2, then continue to extend the tracking and recording of skeleton pixels;

[0025] If the number of skeleton pixels in the neighborhood of a node is greater than or equal to 3, then the current node is the starting point of a side vein. The starting point of the side vein is recorded, and the skeleton pixels are continued to be tracked and recorded along the branch with an angle of less than 45° with the current path direction.

[0026] If the number of neighboring skeleton pixels of a node is 1, the current node is a breakpoint. Search for another breakpoint around the breakpoint. If a matching breakpoint is found, morphological bridging is applied to the two breakpoints to repair the path. If no matching breakpoint is found, the current breakpoint is determined to be the leaf tip, tracking and recording are stopped, and the recorded skeleton pixel sequence is merged into the main vein.

[0027] Based on the lateral vein origins recorded on the main vein, all lateral veins of the leaf are extracted, and each lateral vein is connected to the main vein to construct a complete leaf vein network for a single leaf.

[0028] In some embodiments of the present invention, edge detection is performed on the leaf image, and effective edge points are searched and connected in combination with the distribution characteristics of the leaf vein network to form a closed contour. A connected region containing the leaf vein region is selected as a single leaf, including:

[0029] Edge detection is performed on the blade image to obtain a preliminary edge map;

[0030] The preliminary edge map is morphologically expanded to generate a leaf vein buffer mask, which filters out the edge points of the leaf vein region and obtains the effective edge points of the outer edge of the leaf.

[0031] Extract the outer pixels of the leaf vein region as seed points, and search and connect effective edge points along the normal direction perpendicular to the leaf vein to form a closed contour;

[0032] Select the connected region containing the leaf veins as a single leaf.

[0033] In some embodiments of the present invention, determining whether the single blade is a complete blade or a blocking blade includes:

[0034] Calculate the minimum distance between the breakpoint and all outer edge contour pixels of the single blade;

[0035] Set a minimum distance threshold, filter out near-edge breakpoints based on the minimum distance, and count the number of near-edge breakpoints;

[0036] Set a quantity threshold, determine whether the number of near-edge breakpoints is greater than or equal to the quantity threshold, and determine whether the near-edge breakpoints are continuously distributed;

[0037] If so, the single blade is determined to be an obstructing blade;

[0038] Otherwise, the single blade is determined to be a complete blade.

[0039] In some embodiments of the present invention, analyzing the color characteristics of the leaf mesophyll and the continuity of the veins in a single leaf to determine the insect-eaten area within the single leaf includes:

[0040] The image of the single leaf is projected into the HSV space, the color features of the leaf mesophyll in the single leaf are analyzed, the candidate pest-eating areas in the single leaf are obtained, and the edge contours of the candidate pest-eating areas are extracted.

[0041] The breakpoints of the leaf veins are obtained by analyzing the pixel count feature in the neighborhood of each skeleton pixel.

[0042] Extract the outer edge contour pixels of the single blade, analyze the positional relationship between each breakpoint and the outer edge contour pixels, and obtain unnatural breakpoints;

[0043] Analyze the positional relationship between the edge contour of the candidate pest-eaten area and the non-natural interruption point to determine the pest-eaten area within the single leaf.

[0044] In some embodiments of the present invention, for the intact leaf, the number and area characteristics of the insect-eaten areas are analyzed to obtain the degree of insect infestation on each intact leaf, including:

[0045] For the intact leaf, based on the number and area of ​​the insect-eaten areas, combined with the total area of ​​the intact leaf, the relative number and relative area of ​​the insect-eaten areas are obtained;

[0046] By setting weighting coefficients and summing the relative quantity and relative area, the degree of insect damage to each complete leaf is obtained. The weighting coefficients are dynamically adjusted according to the type of insect damage.

[0047] In some embodiments of the present invention, for the shaded leaf, analyzing the difference in vein distribution between the unshaded area and the intact leaf, and estimating the total leaf area of ​​the shaded leaf, includes:

[0048] Select The complete blade described in the image is used as a reference blade;

[0049] In the reference blade, based on the number and total length of the lateral veins, and combined with the area of ​​the reference blade, the average lateral vein area ratio coefficient and average lateral vein density of all the reference blades are obtained.

[0050] In the shielded blade, the lateral vein density of the unshielded area is obtained by combining the total length of all lateral veins in the unshielded area with the area of ​​the unshielded area.

[0051] The degree of difference in vein density between the unshaded area and the reference leaf is obtained based on the lateral vein density of the unshaded area and the average lateral vein density of the reference leaf.

[0052] Set a difference threshold, and determine whether the difference in leaf vein density is less than the difference threshold based on the degree of difference in leaf vein density;

[0053] If so, the total leaf area of ​​the shaded leaf is obtained based on the average lateral vein area ratio coefficient and the number of lateral veins in the unshaded area.

[0054] If not, the total leaf area of ​​the shaded leaf is obtained based on the average lateral vein area ratio coefficient and the number of lateral veins in the unshaded area, combined with the degree of difference in leaf vein density.

[0055] In some embodiments of the present invention, the method for dynamically adjusting the leaf image acquisition based on the degree of insect damage to different individual leaves includes:

[0056] Set a severity threshold;

[0057] Determine whether the extent of insect damage exceeds the severity threshold;

[0058] If so, then a focused high-resolution image is taken of the single blade;

[0059] Based on images of two adjacent leaves, calculate the growth rate of insect damage on the single leaf;

[0060] Set a threshold for the severity growth rate;

[0061] Determine whether the rate of increase in the degree of insect damage exceeds the threshold for the rate of increase in the severity of the damage;

[0062] If so, then high-frequency sampling is performed on the single blade.

[0063] Compared with existing technologies, the pest image acquisition method for vegetable pest control provided by this invention has the following beneficial effects:

[0064] This invention first uses the vein features of vegetable leaves to precisely segment numerous leaves in the acquired images, extracting individual leaves and obtaining both complete and occluded leaves. Then, based on the color features of the leaf tissue and the continuity of the veins within a single leaf, a dual determination is made to identify the insect-eaten areas within that leaf. Next, based on the number and area of ​​these insect-eaten areas, the degree of insect damage is quantitatively analyzed for both complete and occluded leaves. Finally, different image acquisition methods are used for different leaves depending on the degree of damage. This invention achieves more precise location and acquisition of insect-eaten areas, avoiding detection errors caused by leaf occlusion in traditional methods. Furthermore, through dynamic resource allocation, it provides more targeted data support for subsequent assessment of pest damage and the development of control strategies. Attached Figure Description

[0065] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 This is a schematic diagram of the basic process of a pest image acquisition method for vegetable pest control provided in an embodiment of the present invention;

[0067] Figure 2 This is a schematic diagram of the basic process of a method for segmenting a single leaf based on leaf vein features, provided as an embodiment of the present invention. Detailed Implementation

[0068] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of the pest image acquisition method for vegetable pest control proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0069] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Terms such as “comprising,” “including,” or any other variations thereof are intended to cover a non-exclusive inclusion, such that a circuit structure, article, or device comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such article or device. Without further limitation, an element defined by the phrase “comprising one…” does not exclude the presence of additional identical elements in the article or device that includes the element.

[0070] The following description, in conjunction with the accompanying drawings, details a specific scheme for a pest image acquisition method for vegetable pest control provided by the present invention.

[0071] Please see Figure 1 This illustrates the basic process of a pest image acquisition method for vegetable pest control provided by an embodiment of the present invention.

[0072] like Figure 1 As shown, an embodiment of the present invention provides a method for acquiring pest images for the prevention and control of vegetable pests and diseases, which specifically includes:

[0073] S100: Capture images of vegetable leaves.

[0074] Inside the vegetable greenhouse, cameras are used to monitor the leaves of the vegetable plants. When setting up the cameras, it's crucial to ensure that the camera lens's field of view completely covers the leaves of all the vegetable plants to clearly capture insect bite marks on the leaves. Furthermore, the cameras must be equipped with intelligent zoom lenses and high frame rate shooting capabilities: the zoom lens allows for close-up shots of individual leaves, and the high frame rate shooting function accurately records the movement patterns of pests. In addition, a dedicated network transmission link should be deployed for the cameras to ensure zero latency and distortion-free data transmission.

[0075] The RGB format image of the vegetable field captured by the camera is converted into the HSV color space to enhance the saturation difference between the vegetables and the soil in the background. A threshold range is set in the HSV space (such as H∈[20,80], S∈[30%,80%], V∈[30%,100%], which is calibrated according to the leaf color characteristics of different vegetables) to generate a binary mask. The green area is edited as "1" and the non-green area is marked as "0". The outline of the vegetable area is separated to obtain the leaf image of the vegetables.

[0076] S200: Analyze the vein features of the leaf in the leaf image, perform image segmentation on the leaf image to obtain a single leaf, and determine whether the single leaf is a complete leaf or an occluded leaf.

[0077] The topological structure of leaf veins, including the direction of the main vein and the branching of lateral veins, not only provides the skeletal support for the leaf's morphology but also constitutes the unique expression of each leaf: the differences in the location of vein branching points and the spacing between lateral veins among different leaves form their unique structures. These structures remain stable under environmental changes and can serve as the basis for leaf segmentation in complex scenarios. Therefore, segmentation of a single leaf can be achieved by extracting the unique vein features within the leaf.

[0078] Based on the above analysis, in some embodiments of the present invention, by analyzing the leaf vein features of the leaf in the leaf image, the leaf image is segmented to obtain a single leaf, and the single leaf is determined to be a complete leaf or an occluded leaf.

[0079] Please see Figure 2 This illustrates the basic process of a single leaf segmentation method based on leaf vein features provided by an embodiment of the present invention.

[0080] like Figure 2 As shown, an embodiment of the present invention provides a method for segmenting a single leaf based on leaf vein features, comprising:

[0081] S201: Based on the color features of the leaf image, separate the leaf vein region of the leaf image.

[0082] Leaf vein cells primarily function for transport and support, have relatively few chloroplasts, and are rich in non-photosynthetic substances such as cellulose and lignin. This results in a lower proportion of green light reflected from vein cells compared to mesophyll cells, making them typically lighter in color. In contrast, the non-vein region (mesophyll), as the core site of photosynthesis, exhibits a bright green color. Based on this characteristic, the Otsu algorithm can be used to automatically determine the optimal grayscale threshold for grayscale images of leaves, separating the vein and mesophyll regions to obtain the vein region of the leaf image.

[0083] S202: Based on the topological connectivity of leaf veins in the leaf vein region, identify the starting point of the leaf vein, where the starting point includes the starting point of the main vein and the starting point of the lateral vein.

[0084] The base of the main vein of the leaf is directly connected to the leaf stem, forming a radial structure centered on the leaf stem. Even if the leaf is truncated in the image, the direction of the main vein still points to the position of the stem. The lateral veins branch off from the main vein on one side and radiate outward with the main vein as the only "source point", without converging to other centers.

[0085] Based on the above analysis, in an embodiment of the present invention, the starting point of the leaf vein is identified based on the topological connection relationship of the leaf veins in the leaf vein region, wherein the starting point includes the starting point of the main vein and the starting point of the lateral veins. Further, it includes:

[0086] First, the leaf vein region is processed into a skeletonized form, converting the leaf vein region into a skeleton pixel with a width of one pixel, while preserving the topological connection relationship between the main vein and the lateral veins.

[0087] Then, based on the topological connection relationship, the direction of each leaf vein is transformed into a unit direction vector. The direction of the vector is defined as "the direction from the end of the leaf vein to the potential stem point", that is, the vector direction of the main vein points to the stem, and the vector direction of the lateral vein points to the main vein. The DBSCAN algorithm is then used to cluster the unit direction vectors to obtain multiple clusters. Each cluster corresponds to a "radiation center" of a stem, that is, the leaf veins contained in each cluster correspond to the leaf veins of a single leaf.

[0088] In addition, the unit direction vector of all veins in each cluster is extended (with a maximum extension length L). max , such as L max为 Multiple straight lines are formed by tracing the average width of the leaves (to avoid erroneous convergence across stems). The intersection of all these lines is fitted as the starting point of the main vein. The method for fitting all the lines to obtain the intersection point is as follows: a least squares line intersection fitting algorithm is used. By calculating the distance from each line to the candidate point, an optimal intersection point is found through iterative optimization, which minimizes the sum of the squared distances from this intersection point to all lines. The final intersection point is the starting point of the main vein corresponding to this cluster.

[0089] Finally, search all skeleton pixels in the 16-neighborhood of the main vein starting point to determine the starting direction of the adjacent segment. Extend the skeleton pixels along each starting direction to the far end and record all skeleton pixels on the path until a node is encountered (the starting point of a lateral vein, a break point caused by insect damage, or a leaf tip). Determine the number of skeleton pixels in the 16-neighborhood of the node. If the number of skeleton pixels in the 16-neighborhood of the node is greater than or equal to 3, it indicates that there are at least 2 branch directions at the node position, and the node is the starting point of a lateral vein.

[0090] S203: Starting from the starting point, find and analyze the distribution characteristics of pixels in the leaf vein region, determine the main vein and lateral veins in the leaf vein region, and construct a complete leaf vein network for a single leaf.

[0091] After identifying the starting points of the midrib and lateral veins, the distribution characteristics of pixels within the vein region are searched and analyzed to determine the midrib and lateral veins, thus constructing a complete vein network for a single leaf. Further steps include:

[0092] First, search for all skeleton pixels in the 16-neighborhood of the main vein starting point to determine the starting direction of the adjacent line segment. Extend the skeleton pixels along each starting direction to the far end and record all skeleton pixel points on the path until a node is encountered.

[0093] Then, determine the number of skeleton pixels in the 16-neighborhood of the node. If the number of skeleton pixels in the neighboring neighborhood of the node is equal to 2, it means that there is only one extension direction at this position. Then continue to extend, track and record the skeleton pixels. If the number of skeleton pixels in the neighboring neighborhood of the node is greater than or equal to 3, it means that there are at least 2 branch directions at this position. Then the current node is the starting point of the lateral vein. Record the starting point of the lateral vein and continue to extend, track and record the skeleton pixels along the branch with an angle of less than 45° with the current path direction.

[0094] If the number of skeleton pixels in the 16-neighborhood of a node is detected to be 1, then the current node is a breakpoint (a breakpoint caused by insect damage or a leaf tip); in this case, based on topological rationality, along the extension direction at this breakpoint, the distance from the breakpoint is ≤D. max (D) max Based on the vegetable growth status, another breakpoint is searched within a range of 3-8 pixels and with an directional angle ≤60°. If a matching breakpoint is found, the breakpoint is considered a broken breakpoint, and morphological bridging is used to repair the path between the two breakpoints. If no matching breakpoint is found, the current breakpoint is determined to be the leaf tip, tracking and recording are stopped, and the recorded skeleton pixel sequence is merged into the main vein.

[0095] Finally, all lateral veins of the leaf are extracted based on the lateral vein origins recorded on the main vein. Specifically, for each lateral vein origin recorded on the main vein, the skeleton pixels within its 16-neighborhood are detected. The skeleton pixels are tracked and recorded along the extension direction outside the main vein until a breakpoint node is encountered (i.e., the number of pixels in its 16-neighborhood is 1). At this point, it is considered that a breakpoint has been encountered or the end of the lateral vein has been reached. Similarly, at this breakpoint, along the extension direction, the distance from the breakpoint is ≤D. max (D) max Search for another breakpoint within a range slightly below the main vein (1-5 pixels) and with an directional angle ≤30°. If a matching breakpoint is found, the current breakpoint is determined to be a broken breakpoint, and morphological bridging is used to generate connected components to repair the path between the two breakpoints. If no matching breakpoint is found, the current breakpoint is determined to be the end of a lateral vein, tracking and recording are stopped, and the recorded pixel sequence is merged into a single lateral vein. Perform the above operation sequentially on the starting point of each lateral vein on the main vein to obtain multiple lateral veins, and connect each lateral vein to the main vein to construct a complete leaf vein network for a single leaf.

[0096] S204: Perform edge detection on the leaf image, combine the distribution characteristics of the leaf vein network, search for effective edge points, form a closed contour, and select the connected region containing the leaf vein region as a single leaf.

[0097] After obtaining the complete vein network of a single leaf, edge detection is performed on the leaf image. Based on the distribution characteristics of the vein network, valid edge points are searched to form a closed contour, and the connected regions containing the vein areas are selected as the single leaf. Further steps include:

[0098] First, edge detection is performed on the leaf image (e.g., using the Canny algorithm) to obtain a preliminary edge map containing the outer contour of the leaf and the edges of the veins.

[0099] Then, a leaf vein buffer mask is generated by morphological dilation of the preliminary edge map to filter out the edge points of the leaf vein region and obtain the effective edge points of the outer edge of the leaf.

[0100] Next, the outer pixels of the leaf vein region are extracted as seed points. Valid edge points are searched along the normal direction perpendicular to the leaf veins (main vein and lateral veins) with a step size of 1 to 5 pixels. These valid edge points are connected to form a closed contour. Among them, valid edge points with high edge response intensity and spatial proximity are connected first.

[0101] Finally, the connected domain containing the vein region is selected as a single leaf.

[0102] The single leaf obtained above may be a complete leaf or a partially obscured leaf. For the convenience of subsequent analysis, it is necessary to determine whether the single leaf is a complete leaf or an obscured leaf.

[0103] Compared to intact leaves, obscured leaves often have incomplete edges and abrupt breaks in the veins at the points of incompleteness. Therefore, intact and obscured leaves can be identified by examining the outer edge contour and vein network of a single leaf.

[0104] Based on the above analysis, in some embodiments of the present invention, the method for determining whether a single blade is a complete blade or an occluded blade includes: extracting the outer edge contour pixels of the single blade using the OpenCV contour detection algorithm, calculating the minimum distance between the breakpoint (the breakpoint obtained in step S203) and all the outer edge contour pixels of the single blade; setting a minimum distance threshold (which can be 5 pixels), filtering out near-edge breakpoints based on the minimum distance, i.e., filtering out breakpoints with a minimum distance less than or equal to 5 pixels, and counting the number of near-edge breakpoints; setting a number threshold (which can be 3), determining whether the number of near-edge breakpoints is greater than or equal to the number threshold, and determining whether the near-edge breakpoints are continuously distributed; if yes, i.e., the number of near-edge breakpoints is greater than or equal to 3, and the near-edge breakpoints are continuously distributed, then the single blade is determined to be an occluded blade; otherwise, the single blade is determined to be a complete blade.

[0105] Since the shapes of whole leaves of vegetables are relatively similar, while the shapes of shading leaves are uncertain and diverse, whole leaves and shading leaves can be identified by analyzing the shape characteristics of individual leaves. In other embodiments of the present invention, cluster analysis of the shapes of individual leaves (such as using the DBSCAN algorithm) is performed, and the individual leaves in the cluster containing the most individual leaves are regarded as whole leaves, while the individual leaves in the remaining clusters are regarded as shading leaves.

[0106] S300: Analyze the color characteristics of the leaf mesophyll and the continuity of the leaf veins in a single leaf to determine the area of ​​insect damage within a single leaf.

[0107] In terms of color, the edges of areas damaged by insects will turn yellowish-brown due to cell damage and chlorophyll loss, while healthy leaves will mainly remain green. Structurally, leaf veins, as the "skeleton" of the leaf, have strong structural toughness. Abiotic disturbances (such as wind or collisions) often only create gaps or cracks at the leaf edges, and the veins will continue to extend naturally or gradually taper at the damaged edges. However, when insects bite, the mechanical damage from their mouthparts or the randomness of their feeding paths can cause the veins to suddenly stop at the bite marks.

[0108] Based on the above analysis, in this embodiment of the invention, the insect-eaten area within a single leaf is determined by analyzing the color characteristics of the leaf mesophyll and the continuity of the leaf veins. Further, this includes:

[0109] First, the image of a single leaf is projected into the HSV space, the color features of the leaf mesophyll in the single leaf are analyzed, the region H∈[20, 40] (yellow) is extracted, the candidate pest-eaten areas in the single leaf are obtained, and the edge contours of the candidate pest-eaten areas are extracted by an edge detection algorithm (such as the Canny algorithm).

[0110] Furthermore, the number of pixels in the neighborhood of each skeleton pixel in a leaf vein is analyzed to obtain the breakpoints of the vein. Specifically, for each skeleton pixel p(x,y) in a leaf vein, the number of pixels in its 16-neighborhood is calculated as follows:

[0111]

[0112] In the formula, S represents the number of pixels within the 16-neighborhood of the skeletal pixel p(x,y) in a leaf vein; skeleton A binarized image representing leaf veins; and These represent the offsets of the skeleton pixel p(x,y) in the vertical and horizontal directions, respectively. Defined as 16 directions including horizontal, vertical, diagonal, and second-neighbor areas; The coordinates of the binary image representing leaf veins are The pixel value.

[0113] Because S skeleton A binary image representing leaf veins, in which pixel values ​​have only two possibilities: 1 or 0. Essentially, it represents the number of pixels with a statistical value of 1. If C(p) = 1, it means that the skeleton pixel p(x,y) is connected to only one neighboring skeleton pixel, which matches the morphological characteristics of the breakpoint of a leaf vein. That is, the skeleton pixel corresponding to C(p) = 1 is the breakpoint of a leaf vein.

[0114] Then, OpenCV's contour detection algorithm is used to extract the outer edge contour pixels of a single leaf, forming an outer edge contour pixel set called EdgePixels. The positional relationship between each breakpoint and the outer edge contour pixels is analyzed. Specifically, for each breakpoint... Calculate its distance to all outer edge contour pixels. The minimum value of the Euclidean distance is used to obtain the non-natural breakpoint. The minimum Euclidean distance to all outer edge contour pixels is calculated using the following formula:

[0115]

[0116] In the formula, Indicates breakpoint The minimum Euclidean distance to all outer edge contour pixels; Represents the set of pixels representing the outer edge contour; Indicates breakpoint The x-coordinate; Indicates breakpoint The ordinate; Represents the outer edge contour pixels The x-coordinate; Represents the outer edge contour pixels The ordinate; This represents the function that takes the minimum value.

[0117] Since the natural breaks in leaf veins are usually close to the leaf edge, while breaks caused by insect damage are mostly located in the middle of the leaf, a distance threshold is set. 10 pixels (adjustable according to resolution), if If so, the breakpoint is determined to be a non-natural breakpoint.

[0118] Spatially, leaf vein truncation caused by insect bites must occur in the damaged area of ​​the leaf. Therefore, the interruption point far away from the damaged area is more likely to be a natural variation or noise in the leaf vein. Thus, the insect-eaten area can be determined by detecting the correlation between non-natural interruption nodes in the leaf vein and candidate insect-eaten areas (leaf damaged areas).

[0119] Therefore, finally, by analyzing the positional relationship between the edge contours of candidate pest-eaten areas and non-natural break points, the pest-eaten areas within a single leaf are determined. Specifically, non-natural break points are calculated. All points on the edge contour of the candidate pest feeding area Euclidean distance Traverse all points on the edge contour of the candidate pest-eaten area to find points that are not naturally interrupted. minimum distance ,like If the pixel value is less than 5 pixels (the pixel value can be adjusted according to the actual resolution), then the non-natural interruption point and the candidate pest-eaten area are determined to meet the spatial characteristics of pest eating, and the candidate pest-eaten area is determined to be a pest-eaten area.

[0120] In summary, the regions initially screened by color features are defined as candidate pest-eating regions. Further verification of the candidate pest-eating regions is conducted from two dimensions: structural features and spatial relationships. Finally, the regions that pass the dual-judgment process are identified as pest-eating regions within a single leaf.

[0121] S400: For complete leaves, analyze the number and area characteristics of insect-eaten areas to obtain the degree of insect infestation on each complete leaf.

[0122] The physical damage caused by pests and diseases to vegetable leaves, the area and number of insect holes are the most direct indicators of the severity of the pest infestation: the higher the proportion of a single or cumulative infested area, the more serious the damage to the leaf function; the number of independent infested areas reflects the degree of spread of pests and diseases on the leaves.

[0123] Therefore, for intact leaves, the degree of insect damage is determined by analyzing the number and area of ​​areas damaged by pests. Further, this includes:

[0124] First, for intact leaves, based on the number and area of ​​insect-eaten regions, combined with the total area of ​​the intact leaf, the relative number and relative area of ​​insect-eaten regions are obtained. Specifically, the number of outlines of insect-eaten regions is denoted as N. To eliminate the influence of leaf size on the number of outlines of insect-eaten regions, the relative number of insect-eaten regions per unit leaf area is set as:

[0125]

[0126] In the formula, This indicates the relative number of insect-eaten areas per unit leaf area, expressed as individuals per pixel. Indicates the total area of ​​the blades; This indicates the number of outlines of the area within a single leaf that has been eaten by insects.

[0127] The area of ​​each pest-eaten zone is calculated as follows: The total area of ​​all insect-eaten areas on a single leaf is:

[0128]

[0129] In the formula, This represents the total area of ​​all insect-eaten areas within a single leaf. Indicates the first in a single leaf The area of ​​the region gnawed by the pests; This indicates the total number of areas on a single leaf that have been eaten by insects.

[0130] The percentage of the area damaged by insects relative to the total area of ​​a single leaf is:

[0131]

[0132] In the formula, This indicates the percentage of the area damaged by insects relative to the total area of ​​a single leaf. This represents the total area of ​​all insect-eaten areas within a single leaf. This represents the total area of ​​a single leaf.

[0133] Then, weighting coefficients are set, and the relative quantity and relative area are summed with weights to obtain the degree of insect damage to each intact leaf. The weighting coefficients are dynamically adjusted according to the type of insect damage. The formula for calculating the degree of insect damage to an intact leaf is as follows:

[0134]

[0135] In the formula, Indicates the extent of insect damage to intact leaves; , The weighting coefficients for quantity and area are respectively ( ); This indicates the relative amount of area within a unit leaf area that has been eaten by insects. This indicates the percentage of the area damaged by insects relative to the total area of ​​a single leaf.

[0136] and The weighting needs to be dynamically adjusted based on the type of pest. For example, if boring pests focus more on a single area, then the weighting should be adjusted accordingly. Higher; for diffuse diseases, more attention is paid to the quantity, so the weight is higher. higher.

[0137] S500: For shaded leaves, analyze the differences in vein distribution between the unshaded area and the complete leaf, estimate the total leaf area of ​​the shaded leaf, and then analyze the number and area characteristics of the insect-eaten area in the unshaded area. Combined with the area of ​​the unshaded area, obtain the degree of insect infestation on each shaded leaf.

[0138] In the special case of a single leaf being shaded, the area of ​​the entire shaded leaf can be estimated first based on the distribution characteristics of the veins in the unshaded area of ​​the shaded leaf. Then, the degree of insect damage in the unshaded area can be counted. The degree of insect damage in the entire shaded leaf can be calculated based on the proportion of the unshaded area to the total area of ​​the shaded leaf.

[0139] Therefore, in embodiments of the present invention, for shaded leaves, the area of ​​the entire shaded leaf is estimated by analyzing the difference in vein distribution between the unshaded area and the intact leaf. Then, the number and area characteristics of insect-eaten areas in the unshaded area are analyzed. Combined with the area of ​​the unshaded area, the degree of insect damage to each shaded leaf is obtained. Further, this includes:

[0140] First, select Use 3-5 intact leaves as reference leaves. Specifically, for a partially shaded leaf (the shaded leaf), select 3-5 intact leaves from the leaves growing on the same stem as the shaded leaf.

[0141] Then, in the reference leaves, based on the number and total length of the lateral veins, and combined with the area of ​​the reference leaves, the average lateral vein area ratio coefficient and average lateral vein density of all reference leaves are obtained. Specifically, this includes:

[0142] Based on the number and total length of lateral veins in the reference leaf, and in conjunction with the area of ​​the reference leaf, the lateral vein area ratio coefficient of the reference leaf is defined as follows:

[0143]

[0144] In the formula, Indicates the first The lateral vein area ratio coefficient of a reference leaf, that is, the average area corresponding to each lateral vein; Indicates the first The area of ​​a reference blade; Indicates the first The number of lateral veins in a reference leaf.

[0145] The average lateral vein area ratio coefficient of all reference leaves is then obtained as follows:

[0146]

[0147] In the formula, This represents the average lateral vein area ratio coefficient of all reference leaves; Indicates the first The lateral vein area ratio coefficient of a reference leaf; Indicates the number of reference blades (3~5).

[0148] Based on the total length of all lateral veins in the reference leaf and the area of ​​the reference leaf, the lateral vein density of the reference leaf is defined as:

[0149]

[0150] In the formula, Indicates the first Lateral vein density of a reference leaf; Indicates the first The total length of all lateral veins in a reference leaf; Indicates the first The area of ​​a reference blade.

[0151] The average lateral vein density of all reference leaves is then obtained as follows:

[0152]

[0153] In the formula, This represents the average lateral vein density of all reference leaves; Indicates the first Lateral vein density of a reference leaf; Indicates the number of reference blades (3~5).

[0154] Meanwhile, in the shaded leaf, based on the total length of all lateral veins in the unshaded area and the area of ​​the unshaded area, the lateral vein density in the unshaded area is obtained as follows:

[0155]

[0156] In the formula, This indicates the density of lateral veins in the unshaded area of ​​a shaded leaf; This represents the total length of all lateral veins in the unshaded area of ​​a shaded leaf. This indicates the area of ​​the unshaded region within the shaded blade.

[0157] Then, based on the lateral vein density of the unshaded area and the average lateral vein density of the reference leaf, the degree of difference in vein density between the unshaded area and the reference leaf is obtained as follows:

[0158]

[0159] In the formula, This indicates the degree of difference in vein density between the unshaded area of ​​the shaded leaf and the reference leaf; This indicates the density of lateral veins in the unshaded area of ​​a shaded leaf; This represents the average lateral vein density of all reference leaves.

[0160] Next, set the difference threshold (the value can be...). Based on the degree of difference in leaf vein density, determine whether the degree of difference in leaf vein density is less than a difference threshold, that is, determine whether the uniformity of leaf vein distribution in the unshaded area of ​​the reference leaf is consistent; if so, that is... If the vein distribution of the unshaded area is uniform, then based on the average lateral vein area ratio coefficient and the number of lateral veins in the unshaded area, the total leaf area of ​​the shaded leaf can be obtained as follows:

[0161]

[0162] In the formula, express The total area of ​​the leaf blades that are blocked at the time; This indicates the number of all lateral veins in the unshaded area of ​​a shaded leaf; This represents the average lateral vein area ratio coefficient for all reference leaves.

[0163] If not, that is This indicates a density difference between the shaded leaf and the reference leaf. A calibration factor is introduced for correction. Specifically, based on the average lateral vein area ratio coefficient and the number of lateral veins in the unshaded area, combined with the degree of difference in vein density, the total leaf area of ​​the shaded leaf is obtained as follows:

[0164]

[0165] In the formula, express The entire leaf area is blocked at times; This indicates the number of all lateral veins in the unshaded area of ​​a shaded leaf; This represents the average lateral vein area ratio coefficient of all reference leaves; This indicates the degree of difference in vein density between the unshaded area of ​​the shaded leaf and the reference leaf.

[0166] when hour, If the density of lateral veins in the unshaded area of ​​the leaf is high (>1), then the area should be increased; otherwise, it should be decreased.

[0167] Finally, the quantity and area characteristics of pest-eaten areas in the unshaded region are analyzed to obtain the degree of pest infestation in the unshaded region. Combined with the area of ​​the unshaded region, the degree of pest infestation for each shaded leaf is obtained. Specifically, the analysis method for the quantity and area characteristics of pest-eaten areas in the unshaded region is the same as that for the unshaded leaves (only the analysis object is replaced by the unshaded area of ​​the shaded leaf), which will not be repeated here. Combining the area of ​​the unshaded region, the degree of pest infestation for each shaded leaf is obtained as follows:

[0168]

[0169] In the formula, This indicates the extent of insect damage to the entire leaf being covered; This indicates the extent of insect damage in the uncovered areas of the covered leaves; This indicates the total area of ​​the leaf that is being covered. This represents the area of ​​the unshaded region within the shaded blade. It should be noted that, to ensure the calculation results are meaningful, in this embodiment of the invention, when performing fractional operations, if the denominator is 0, a parameter adjustment factor greater than 0 needs to be added to the denominator before summing to prevent the denominator from being 0. The value of the parameter adjustment factor is set by the implementer according to the actual situation; in this application, it is set to 0.1.

[0170] S600: Dynamically adjusts the leaf image acquisition method according to the degree of insect damage to different individual leaves.

[0171] Based on the degree of pest damage on different individual leaves, the leaf image acquisition method is dynamically adjusted. Specifically, a basic sampling frequency is set to acquire time-series images of the vegetable area. A severity threshold is set (value can be 10-12); it is determined whether the degree of pest damage exceeds the severity threshold; if so, a focused high-resolution image is taken of the individual leaf; after each fixed shooting interval, the images of two adjacent periods are compared and analyzed, and the growth rate of pest damage on an individual leaf is calculated based on the degree of pest damage in the images of two adjacent leaves; a severity growth rate threshold is set (value can be 20%); it is determined whether the growth rate of pest damage exceeds the severity growth rate threshold; if so, it indicates a sudden increase in the degree of pest damage within a fixed interval, and the pests in the area are determined to be in an active period. Subsequently, high-frequency sampling is performed on individual leaves, increasing the sampling frequency to 2-4 times the original frequency (e.g., taking pictures once every 3-6 hours) to track the development trajectory of pests in real time.

[0172] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0173] The various embodiments in this specification are described in a progressive manner. For the same or similar parts between the various embodiments, please refer to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for acquiring pest images for the prevention and control of vegetable diseases and pests, characterized in that, The method includes: Collect images of vegetable leaves; Analyze the vein features of the leaf in the leaf image, perform image segmentation on the leaf image to obtain a single leaf, and determine whether the single leaf is a complete leaf or an occluded leaf; Analyze the color characteristics of the leaf mesophyll and the continuity of the leaf veins in the single leaf to determine the area of ​​insect damage within the single leaf; For each intact leaf, the number and area characteristics of the insect-eaten areas are analyzed to obtain the degree of insect infestation on each intact leaf. For the covered leaves, the differences in vein distribution between the uncovered area and the complete leaves are analyzed to estimate the total leaf area of ​​the covered leaves. Then, the number and area characteristics of the insect-eaten areas in the uncovered area are analyzed. Combined with the area of ​​the uncovered area, the degree of insect infestation of each covered leaf is obtained. The leaf image acquisition method is dynamically adjusted based on the degree of insect damage to different individual leaves.

2. The method for acquiring pest images for vegetable pest control according to claim 1, characterized in that, Analyze the vein features of the leaf in the leaf image, perform image segmentation on the leaf image to obtain a single leaf, including: Based on the color features of the leaf image, the leaf vein region of the leaf image is separated; Based on the topological connection relationship of the leaf veins in the leaf vein region, the starting point of the leaf vein is identified, wherein the starting point includes the starting point of the main vein and the starting point of the lateral vein. Starting from the aforementioned starting point, the distribution characteristics of pixels in the leaf vein region are searched and analyzed to determine the main vein and lateral veins in the leaf vein region, and a complete leaf vein network of a single leaf is constructed. Edge detection is performed on the leaf image, and effective edge points are searched by combining the distribution characteristics of the leaf vein network to form a closed contour. The connected region containing the leaf vein region is selected as a single leaf.

3. The method for acquiring pest images for vegetable pest control according to claim 2, characterized in that, Based on the topological connectivity of the veins in the leaf vein region, the starting points of the veins are identified, wherein the starting points include the starting points of the main vein and the starting points of the lateral veins, including: The leaf vein region is processed into a skeletonized form, which is converted into a skeleton pixel with a width of one pixel, while preserving the topological connection relationship between the main vein and the lateral veins. Based on the aforementioned topological connection, the direction of each leaf vein is transformed into a unit direction vector, and the unit direction vectors are clustered to obtain multiple clusters, each cluster corresponding to a single leaf. The unit direction vector of all veins in each cluster is extended to form multiple straight lines, and the intersection of the fitted straight lines is taken as the starting point of the main vein. Search for all skeleton pixels in the neighborhood of the main vein starting point until a node is encountered. If the number of skeleton pixels in the neighborhood of the node is greater than or equal to 3, then the node is the starting point of the side vein.

4. The method for acquiring pest images for vegetable pest control according to claim 3, characterized in that, Starting from the aforementioned starting point, the distribution characteristics of pixels in the leaf vein region are searched and analyzed to determine the main vein and lateral veins in the leaf vein region, and a complete leaf vein network for a single leaf is constructed, including: Search all skeleton pixels in the neighborhood of the main vein starting point, determine the starting direction of the adjacent line segment, extend the skeleton pixels along each starting direction to the far end, record all skeleton pixels on the path, until a node is encountered. If the number of skeleton pixels in the neighborhood of a node is equal to 2, then continue to extend the tracking and recording of skeleton pixels; If the number of skeleton pixels in the neighborhood of a node is greater than or equal to 3, then the current node is the starting point of a side vein. The starting point of the side vein is recorded, and the skeleton pixels are continued to be tracked and recorded along the branch with an angle of less than 45° with the current path direction. If the number of neighboring skeleton pixels of a node is 1, the current node is a breakpoint. Search for another breakpoint around the breakpoint. If a matching breakpoint is found, morphological bridging is applied to the two breakpoints to repair the path. If no matching breakpoint is found, the current breakpoint is determined to be the leaf tip, tracking and recording are stopped, and the recorded skeleton pixel sequence is merged into the main vein. Based on the lateral vein origins recorded on the main vein, all lateral veins of the leaf are extracted, and each lateral vein is connected to the main vein to construct a complete leaf vein network for a single leaf.

5. The method for acquiring pest images for vegetable pest control according to claim 2, characterized in that, Edge detection is performed on the leaf image. Based on the distribution characteristics of the leaf vein network, valid edge points are searched and connected to form a closed contour. A connected region containing the leaf vein area is selected as a single leaf, including: Edge detection is performed on the blade image to obtain a preliminary edge map; The preliminary edge map is morphologically expanded to generate a leaf vein buffer mask, which filters out the edge points of the leaf vein region and obtains the effective edge points of the outer edge of the leaf. Extract the outer pixels of the leaf vein region as seed points, and search and connect effective edge points along the normal direction perpendicular to the leaf vein to form a closed contour; Select the connected region containing the leaf veins as a single leaf.

6. The method for acquiring pest images for vegetable pest control according to claim 4, characterized in that, Determining whether a single blade is a complete blade or a blocked blade includes: Calculate the minimum distance between the breakpoint and all outer edge contour pixels of the single blade; Set a minimum distance threshold, filter out near-edge breakpoints based on the minimum distance, and count the number of near-edge breakpoints; Set a quantity threshold, determine whether the number of near-edge breakpoints is greater than or equal to the quantity threshold, and determine whether the near-edge breakpoints are continuously distributed; If so, the single blade is determined to be an obstructing blade; Otherwise, the single blade is determined to be a complete blade.

7. The method for acquiring pest images for vegetable pest control according to claim 1, characterized in that, Analyzing the color characteristics of the leaf mesophyll and the continuity of the veins within the single leaf, the insect-eaten area within the single leaf is determined, including: The image of the single leaf is projected into the HSV space, the color features of the leaf mesophyll in the single leaf are analyzed, the candidate pest-eating areas in the single leaf are obtained, and the edge contours of the candidate pest-eating areas are extracted. The breakpoints of the leaf veins are obtained by analyzing the pixel count feature in the neighborhood of each skeleton pixel. Extract the outer edge contour pixels of the single blade, analyze the positional relationship between each breakpoint and the outer edge contour pixels, and obtain unnatural breakpoints; Analyze the positional relationship between the edge contour of the candidate pest-eaten area and the non-natural interruption point to determine the pest-eaten area within the single leaf.

8. The method for acquiring pest images for vegetable pest control according to claim 1, characterized in that, For the intact leaves, the number and area characteristics of the insect-eaten areas are analyzed to obtain the degree of insect infestation on each intact leaf, including: For the intact leaf, based on the number and area of ​​the insect-eaten areas, combined with the total area of ​​the intact leaf, the relative number and relative area of ​​the insect-eaten areas are obtained; By setting weighting coefficients and summing the relative quantity and relative area, the degree of insect damage to each complete leaf is obtained. The weighting coefficients are dynamically adjusted according to the type of insect damage.

9. The method for acquiring pest images for vegetable pest control according to claim 1, characterized in that, For the shaded leaf, the differences in vein distribution between the unshaded area and the intact leaf are analyzed to estimate the total leaf area of ​​the shaded leaf, including: Select The complete blade described in the image is used as a reference blade; In the reference blade, based on the number and total length of the lateral veins, and combined with the area of ​​the reference blade, the average lateral vein area ratio coefficient and average lateral vein density of all the reference blades are obtained. In the shielded blade, the lateral vein density of the unshielded area is obtained by combining the total length of all lateral veins in the unshielded area with the area of ​​the unshielded area. The degree of difference in vein density between the unshaded area and the reference leaf is obtained based on the lateral vein density of the unshaded area and the average lateral vein density of the reference leaf. Set a difference threshold, and determine whether the difference in leaf vein density is less than the difference threshold based on the degree of difference in leaf vein density; If so, the total leaf area of ​​the shaded leaf is obtained based on the average lateral vein area ratio coefficient and the number of lateral veins in the unshaded area. If not, the total leaf area of ​​the shaded leaf is obtained based on the average lateral vein area ratio coefficient and the number of lateral veins in the unshaded area, combined with the degree of difference in leaf vein density.

10. The method for acquiring pest images for vegetable pest control according to claim 1, characterized in that, The leaf image acquisition method is dynamically adjusted based on the degree of insect damage to different individual leaves, including: Set a severity threshold; Determine whether the extent of insect damage exceeds the severity threshold; If so, then a focused high-resolution image is taken of the single blade; Based on images of two adjacent leaves, calculate the growth rate of insect damage on the single leaf; Set a severity growth rate threshold; Determine whether the rate of increase in the degree of insect damage exceeds the threshold for the rate of increase in the severity of the damage; If so, then high-frequency sampling is performed on the single blade.

Images