System and method for determining the optimal time for controlling grape smooth ambrosia beetle based on image recognition

Abstract

The application discloses a system and method for determining the prevention and control suitable period of grape Dendroctonus valens based on image recognition, and belongs to the cross technical field of plant protection and computer vision. A plurality of two-dimensional images are continuously acquired by an image acquisition device arranged in a grape planting area; image recognition processing is performed, and index data of adult emergence hole crawling base, male to female ratio, trunk surface boring hole density, sap flow characteristic, grape tree foot area wood powder accumulation thickness, and fallen or suspended cylindrical boring debris distribution density are respectively counted and generated; each index is compared with a corresponding preset prevention and control starting threshold value, and when at least three indexes reach or exceed the threshold value, it is determined that the current time is a suitable time node for starting prevention and control measures. The application automatically captures the early representation of multiple dimensions of the bark beetle damage through the image recognition technology, realizes the accurate and objective determination of the prevention and control suitable period of the grape Dendroctonus valens, and provides a scientific basis for the intelligent prevention and control of the grape Dendroctonus valens.

Description

Technical Field

[0001] This invention discloses a system and method for determining the optimal control period for the grape smooth-foot bark beetle based on image recognition, belonging to the interdisciplinary field of plant protection and computer vision. Background Technology

[0002] With the continuous advancement of modern smart agriculture technology, real-time monitoring and precise control of biological disasters using computer vision and image recognition technologies have become key approaches to improving fruit yield and quality. In the grape growing industry, effective control of borer pests directly affects the maintenance of grapevine vigor and the overall economic benefits of the orchard. Especially for specific pests characterized by their high concealment and destructive power, the use of automation to achieve dynamic perception and early warning of pest conditions not only supports the implementation of pesticide reduction and pest control, reducing pesticide residues and heavy metal pollution, improving fruit quality and safety, and increasing farmers' income, but also plays a significant role in building a scientific, safe, and efficient plant protection system.

[0003] Among them, the technology for determining the optimal control period for the grape smooth-footed bark beetle based on image recognition is a current research hotspot in the field of plant protection. It aims to achieve precise understanding of the pest's life habits and dispersal patterns by digitally extracting and quantitatively analyzing the visual features of the affected plant surface. The basic principle of this technology is to use high-definition imaging equipment to capture the damage characteristics of grapevine trunks bored by the smooth-footed bark beetle, and then combine this with an evaluation model to classify the pest infestation index, thereby determining the optimal control period. This provides decision support for the precise selection of targeted control methods and pesticides.

[0004] However, existing technologies still have significant limitations when targeting tiny and elusive borers like the smooth-footed bark beetle. Traditional manual inspection methods are limited by the pest's extremely small size and short emergence time, making it difficult to detect early signs of damage at low population densities, easily missing the optimal prevention window for centrally infested plants. Furthermore, due to the severe generational overlap and long-term internal habitation of this pest within the wood, conventional monitoring methods lack the integrated analytical capabilities to analyze heterogeneous characterization information such as the number of adult smooth-footed bark beetles, morphological characteristics, sex ratio, resin exudation patterns in the affected grapevine trunks, sawdust accumulation, and borehole locations. This makes it difficult to accurately determine the occurrence level and spatial distribution patterns of the smooth-footed bark beetle in the field. In addition, the lack of a standardized image evaluation system leads to excessive reliance on subjective experience in control decisions, increasing the cost of purchasing and applying pesticides. Furthermore, indiscriminate and excessive pesticide use across the entire orchard can lead to increased pest resistance and environmental pollution, hindering the achievement of low-cost, high-efficiency, and precise control.

[0005] Therefore, a method for determining the appropriate control period for the grape smooth foot bark beetle based on image recognition is desired. Summary of the Invention

[0006] According to a first aspect of the present invention, the present invention claims protection for a method for determining the optimal control period for the grape bark beetle based on image recognition, characterized by comprising the following steps:

[0007] S1, using an image acquisition device set up within the grapevine planting area, acquire multiple frames of two-dimensional images containing the grapevine trunk and base area;

[0008] S2, perform image recognition processing on the multi-frame two-dimensional images to identify and count the number of adult grape smooth-legged bark beetles crawling on the tree trunk and the morphological characteristics of the adults, and generate the number of adult beetles crawling through the exit hole and the male-female ratio that cross the preset baseline on the tree trunk.

[0009] S3, perform image recognition processing on the tree trunk surface area in the multi-frame two-dimensional images, extract the circular or elliptical dark spot features in the tree trunk surface image, filter out the pixel areas that meet the preset morphological features of borer holes, and calculate and generate the density of borer holes on the tree trunk surface.

[0010] S4, perform image recognition processing on the resin adhering to the tree trunk surface in the multi-frame two-dimensional images, convert the image color space to a preset chroma-saturation-brightness color space, and extract a set of pixels that meet the preset conditions in the color space as resin morphological features.

[0011] S5, perform image recognition processing on the ground in the tree foot area of ​​the multi-frame two-dimensional image, extract the wood powder accumulation area, and calculate the wood powder accumulation thickness in the tree foot area by analyzing the gray-level gradient change of the accumulation area in the vertical direction.

[0012] S6, perform image recognition processing on the cylindrical wood dust, bark beetle excrement and resin mixture that fall or hang around the tree trunk in the multi-frame two-dimensional images, and statistically generate the wood dust distribution density;

[0013] S7. The number of adult insects crawling out of holes, the male-female ratio, the density of borer holes on the trunk surface, the resin exudation characteristics, the thickness of wood powder accumulation in the tree foot area, and the density of borer debris distribution are compared with their respective preset prevention and control activation thresholds. When at least three items reach or exceed their corresponding preset prevention and control activation thresholds, the current time is determined to be a suitable time node for initiating prevention and control measures.

[0014] Further, S2 includes:

[0015] S21, extract the region of interest containing the main trunk of the grapevine from the multi-frame two-dimensional images, and perform background subtraction processing on the region of interest to separate the moving target foreground;

[0016] S22, Connectivity analysis is performed on the separated moving target foreground to screen out candidate targets whose connected area, perimeter, and aspect ratio conform to the morphological characteristics of the adult bark beetle with smooth legs;

[0017] S23. Extract the color histogram features and local binary pattern texture features of the candidate targets, and compare them with the features of the pre-stored standard sample of smooth-legged bark beetle adults to confirm the identity of the adult individual.

[0018] S24. In the continuous frame images, the Kalman filter algorithm is used to predict and track the centroid movement trajectory of the identified adult insect. When the tracked trajectory crosses the preset horizontal baseline on the tree trunk, a crawling event of the adult insect is recorded.

[0019] S25, count the number of all crawling events recorded within a predetermined unit of time, and determine the number as the adult insect's crawling baseline for exiting the hole.

[0020] Further, S3 includes:

[0021] S31, perform image enhancement processing on the tree trunk surface area in the multi-frame two-dimensional images, using a combination of top-hat transform and bottom-hat transform to highlight the contrast between the dark spots on the tree trunk surface and the background, while suppressing the interference of bark texture.

[0022] S32, perform local adaptive threshold segmentation on the enhanced image to divide the tree trunk surface pixels into candidate blob pixels and non-blob pixels, and obtain a binarized candidate blob image.

[0023] S33, perform morphological filtering on each connected region in the binarized candidate spot image, calculate the roundness, eccentricity and area of ​​each connected region, and retain connected regions with roundness greater than the first preset threshold, eccentricity less than the second preset threshold and area within the preset wormhole area range as initial wormholes.

[0024] S34, perform gray-level distribution analysis on the internal pixels of the original image region corresponding to each initially selected wormhole, calculate the gray-level mean and standard deviation of the pixels in the region, and construct a gray-level co-occurrence matrix to extract energy, correlation and homogeneity texture parameters. The initially selected wormholes with a gray-level mean lower than the average gray-level value of the tree trunk surface, a standard deviation greater than the third preset threshold, and texture parameters that meet the rough texture characteristics inside the wormhole are determined as the final wormholes.

[0025] S35, count the number of all final boreholes in the region of interest on the tree trunk surface, divide the number by the area of ​​the region of interest on the tree trunk surface to obtain the borehole density on the tree trunk surface, wherein the area of ​​the region of interest on the tree trunk surface is obtained by conversion through the calibration relationship between image pixel size and actual physical size.

[0026] Further, S4 includes:

[0027] S41, the suspected resinous image region attached to the tree trunk surface is segmented from the multi-frame two-dimensional images, and the color space of the suspected resinous image region is converted from the original red-green-blue color space to the preset chroma-saturation-brightness color space;

[0028] S42, in the chroma-saturation-luminance color space, extract the chroma channel image, saturation channel image and luminance channel image respectively, perform statistical analysis on the chroma channel image, and determine the chroma value distribution range of all pixels in the entire chroma channel image;

[0029] S43, Gaussian filtering is applied to the saturation channel image for smoothing, and then watershed segmentation is performed on the processed saturation channel image to segment out continuous regions with high saturation values ​​as high saturation candidate regions.

[0030] S44, Spatial position matching is performed between the high saturation candidate region and the pixel region in the chroma channel image whose chroma value falls within the preset fat chroma threshold range, and the intersection region of the two is taken as the fat candidate pixel set;

[0031] S45, perform shape analysis on the corresponding region of the candidate pixel set of lipid flow in the brightness channel image, calculate the aspect ratio of the bounding rectangle of the region, the Fourier descriptor and the number of branch points after skeletonization, and extract shape feature parameters that reflect the natural flow state of lipid flow.

[0032] S46, compare the shape feature parameters with the pre-stored standard shape feature parameter range of lipid flow, filter out pixel regions that conform to the natural flow morphology of lipid flow, and encapsulate the average chroma, average saturation of these pixel regions in the chroma-saturation-brightness color space and the shape feature parameters as the lipid flow characteristics.

[0033] Further, S5 includes:

[0034] S51, the ground area around the base of the grapevine is extracted from the multi-frame two-dimensional image as the analysis area, and geometric correction is performed on the analysis area to eliminate perspective distortion, so that the size ratio of objects on the ground in the corrected image conforms to the actual physical size ratio.

[0035] S52, perform color space conversion on the corrected analysis area image, extract color feature vectors, and use a semantic segmentation method based on color and texture to classify image pixels into soil background pixels and wood flour deposit pixels, generating a wood flour deposit mask image.

[0036] S53, in the wood flour accumulation mask image, identify each connected region, and mark the connected region with an area greater than the preset minimum accumulation area as an effective wood flour accumulation region.

[0037] S54. For each effective wood flour accumulation area, select multiple sampling points along the edge of the area towards the center in the original image or the corresponding 3D reconstructed depth map. For each sampling point, obtain the relative height value between the surface of the wood flour accumulation and the surface of the soil below at that point. The arithmetic mean of the relative height values ​​of all sampling points is taken as the average accumulation thickness of the effective wood flour accumulation area.

[0038] S55, the average thickness of all effective wood flour accumulation areas within the image acquisition area is weighted and averaged, where the weight is the area of ​​each effective wood flour accumulation area, to calculate the wood flour accumulation thickness in the tree foot area.

[0039] Further, S6 includes:

[0040] S61, preprocess the multi-frame two-dimensional images to enhance the edge information of the cylindrical target in the image, and use the Canny edge detection operator to extract the edge contours of all objects in the image;

[0041] S62, perform polygon approximation and line segment detection on the extracted edge contours, and identify two parallel long side contours with similar lengths as candidates for the generatrix of the cylinder.

[0042] S63, between two candidate contours of the two generatrices, detect whether there is an arc-shaped or elliptical arc-shaped contour connecting the two generatrices, or based on the image gradient direction analysis, determine whether the gradient direction of the region between the two generatrices presents a rotationally symmetric distribution around a certain central axis.

[0043] S64, count the number of all cylindrical woodworm individuals identified within the predetermined ground area, and divide the number by the predetermined ground area to obtain the cylindrical woodworm distribution density.

[0044] Furthermore, S22 includes:

[0045] Calculate the pixel area of ​​the connected component, requiring that the area is between the first adult insect area threshold and the second adult insect area threshold;

[0046] Calculate the perimeter of the connected component, requiring that the perimeter is between the first adult perimeter threshold and the second adult perimeter threshold;

[0047] Calculate the aspect ratio of the smallest bounding rectangle of the connected component, requiring that the aspect ratio is between the first and second adult aspect ratio thresholds.

[0048] Further, S33 includes:

[0049] For each candidate connected region in the tree trunk surface image, the roundness index of the region is calculated. The formula for calculating the roundness index is 4π multiplied by the area of ​​the region divided by the square of the perimeter of the region. When the roundness index is greater than 0.7, the roundness screening condition is met.

[0050] Calculate the eccentricity of the region, which is based on the eccentricity of the ellipse calculated from the second moment of the region. When the eccentricity is less than 0.5, the eccentricity screening condition is met.

[0051] The total number of pixels contained in the region is counted as the region area. The region area is then compared with the actual wormhole area range obtained by pre-converting the pixel size. If the actual area after conversion is within the range of 0.5 square millimeters to 3 square millimeters, the area screening condition is met.

[0052] Candidate connected regions that meet the above roundness screening conditions, eccentricity screening conditions, and area screening conditions are identified as preliminary wormholes.

[0053] Further, S42 includes:

[0054] The chroma value of each pixel in the chroma channel image is quantized from 0 to 360 degrees. A histogram of the chroma values ​​of all pixels is plotted, the positions of the main peak and the secondary peak in the histogram are identified, and the chroma value range corresponding to the main peak is recorded.

[0055] In S43, the Gaussian kernel size used for Gaussian filtering and smoothing of the saturation channel image is 5×5, and the standard deviation is 1.0.

[0056] When performing watershed segmentation on the filtered saturation channel image, local maxima are used as seed points, and the threshold height for watershed segmentation is set to 20% of the maximum saturation value of the saturation channel image.

[0057] In S44, the preset lipid chromaticity threshold range is between 15 degrees and 45 degrees. The intersection of the high saturation candidate region and the pixel region in the chromaticity channel image whose chromaticity value falls within the preset lipid chromaticity threshold range is taken as the lipid candidate pixel set.

[0058] The shape feature parameters reflecting the natural flow state of the lipid in S45 are extracted, specifically including:

[0059] Calculate the ratio of the length to the width of the bounding rectangle of the region formed by the candidate pixel set of the lipid flow, and require that the ratio be greater than 1.2 and less than 3.5;

[0060] Calculate the Fourier descriptor of the region, and take the normalized values ​​of the first 10 Fourier coefficients as the shape contour features.

[0061] The region is skeletonized by extracting skeleton lines and counting the number of branch points of the skeleton lines. The number of branch points must be greater than 0 and less than 5.

[0062] In step S46, pixel regions that conform to the natural flow morphology characteristics of lipids are selected. Specifically, these refer to a set of candidate pixels that simultaneously meet the following conditions: the aspect ratio of the circumscribed rectangle is between 1.2 and 3.5; the Euclidean distance between the first 10 normalized Fourier descriptors and the pre-stored standard lipid Fourier descriptor vectors is less than a fifth preset threshold; and the number of skeleton line branch points is between 1 and 4. The average chroma and average saturation of the selected pixel regions, along with the aforementioned aspect ratio of the circumscribed rectangle, the first 10 Fourier descriptors, and the number of skeleton line branch points, are combined and encapsulated as the lipid morphological characteristics.

[0063] According to a second aspect of the present invention, the present invention claims protection for a system for determining the optimal control period for the grape smooth-foot bark beetle based on image recognition, comprising:

[0064] One or more processors;

[0065] A memory storing one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the image recognition-based method for determining the optimal control period of grape bark beetle.

[0066] This invention discloses a system and method for determining the optimal control period for the grape smooth-footed bark beetle based on image recognition, belonging to the interdisciplinary field of plant protection and computer vision. Multiple frames of two-dimensional images are continuously acquired by image acquisition devices installed in the grape-growing area. The images are processed to statistically analyze and generate data on the number of adult beetles crawling through holes, the female-to-male ratio, the density of borer holes on the trunk surface, the characteristics of resin exudation in the chromaticity-saturation-brightness color space, the thickness of wood powder accumulation in the tree foot area, and the distribution density of cylindrical borer debris. Each indicator is compared with a corresponding preset control activation threshold. When at least three indicators reach or exceed their thresholds, the current moment is determined to be the appropriate time to initiate control measures. This invention automatically captures early characteristics of beetle damage from multiple dimensions using image recognition technology, achieving accurate and objective determination of the control period for the smooth-footed bark beetle, and providing a scientific basis for the intelligent control of the grape smooth-footed bark beetle. Attached Figure Description

[0067] Figure 1 A flowchart illustrating the workflow of a method for determining the optimal control period for grape bark beetle based on image recognition, as claimed in an embodiment of the present invention.

[0068] Figure 2 A second flowchart of a method for determining the optimal control period of grape bark beetle based on image recognition, as claimed in an embodiment of the present invention;

[0069] Figure 3 The third flowchart of a method for determining the optimal control period of grape bark beetle based on image recognition, as claimed in an embodiment of the present invention;

[0070] Figure 4 The fourth flowchart is a method for determining the optimal control period of grape bark beetle based on image recognition, as claimed in an embodiment of the present invention. Detailed Implementation

[0071] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0072] The terms "first," "second," and "third" in this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of those features. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. All directional indications in the embodiments of this application, such as up, down, left, right, front, back, etc., are only used to explain the relative positional relationships and movements between components in a specific orientation as shown in the accompanying drawings. If the specific orientation changes, the directional indications will change accordingly. Furthermore, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0073] References to embodiments herein mean that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0074] According to a first embodiment of the present invention, the present invention claims protection for a method for determining the appropriate control period of grape bark beetle based on image recognition, referring to... Figure 1 This includes the following steps:

[0075] S1, using an image acquisition device set up within the grapevine planting area, acquire multiple frames of two-dimensional images containing the grapevine trunk and base area;

[0076] S2, perform image recognition processing on multiple frames of two-dimensional images to identify and count the number and morphological characteristics of adult grape smooth-legged bark beetles crawling on the trunk of the tree, and generate the number of adult beetles crawling through the pre-set baseline on the trunk and the ratio of males and females.

[0077] S3, perform image recognition processing on the tree trunk surface area in multiple frames of two-dimensional images, extract the circular or elliptical dark spot features in the tree trunk surface image, filter out the pixel area that meets the preset morphological features of the borer holes, and calculate and generate the borer hole density on the tree trunk surface.

[0078] S4. Perform image recognition processing on the resin adhering to the tree trunk surface in multi-frame two-dimensional images, convert the image color space to the preset chroma-saturation-brightness color space, and extract the set of pixels that meet the preset conditions in the color space as resin morphological features.

[0079] S5. Perform image recognition processing on the ground in the tree foot area of ​​multi-frame two-dimensional images, extract the wood powder accumulation area, and calculate the wood powder accumulation thickness in the tree foot area by analyzing the gray-level gradient change of the accumulation area in the vertical direction.

[0080] S6, perform image recognition processing on cylindrical woodworm debris that has fallen or is hanging around the tree trunk in multiple frames of two-dimensional images, and statistically generate the distribution density of cylindrical woodworm debris.

[0081] S7. The number of adult insects crawling out of holes, the ratio of male to female, the density of borer holes on the trunk surface, the characteristics of resin exudation, the thickness of wood powder accumulation in the tree foot area, and the distribution density of cylindrical borer debris are compared with their respective preset control activation thresholds. When at least three of them reach or exceed their corresponding preset control activation thresholds, the current time is determined to be an appropriate time node for initiating control measures.

[0082] In this embodiment, the first step is to deploy an image acquisition device within the grape-growing area. This device is specifically installed on fixed monitoring posts, with the lens aimed at the selected grapevine trunk, ensuring the field of view covers an area from the ground at the base of the vine up to approximately 1.5 meters above the trunk, as well as a ground area with a diameter of about half a meter around the base of the vine. The image acquisition device is set to automatically capture and store a high-resolution two-dimensional image every fifteen minutes, from 6:00 AM to 7:00 PM daily, acquiring image sequences over several consecutive days.

[0083] Subsequently, the calculator processes the acquired multi-frame two-dimensional images. When identifying adult grape bark beetles, a rectangular region containing the tree trunk is first extracted from each frame. By comparing the grayscale changes of pixels at the same location in several consecutive frames, the static bark background is removed, leaving only the displaced pixel groups, i.e., the moving target foreground. These moving foregrounds are labeled with connected components, and the number of pixels covered by each connected component, its edge length, and the aspect ratio of its minimum bounding rectangle are calculated to filter out candidate targets with approximately elliptical shapes and sizes comparable to the bark beetle. For candidate targets that pass the shape screening, the average color within their region is further extracted and compared with pre-stored typical body colors of adult grape bark beetles, such as dark brown or black, to confirm their identity as target adults. In consecutive frames, a movement trajectory is established for each confirmed adult individual, tracking its centroid position. When the centroid of an adult crosses a pre-set virtual horizontal baseline on the tree trunk image during movement, a crawling event is recorded. By counting the total number of times all tracked adult insects cross the baseline within a unit of time in one hour, the baseline number of adult insects crawling out of the hole during that time period is finally generated.

[0084] Simultaneously, the surface of the tree trunk in the image was analyzed. Within the region of interest on the trunk, an image enhancement technique that highlights the difference between local dark spots and the surrounding bright background was used to make small wormholes on the bark more prominent. Next, a segmentation threshold was automatically set based on the dynamic grayscale changes of local image regions to separate dark pixels suspected of wormholes from the bark texture. For these separated dark areas, the roundness of their shape and the ratio of their major and minor axes were calculated. Irregularly shaped cracks or bark fissures were discarded, and areas with near-circular or elliptical shapes and areas within a specific range were retained as preliminary wormholes. For further confirmation, the pixel grayscale values ​​of these preliminary wormholes were checked to see if they were significantly lower than the grayscale values ​​of the surrounding healthy bark, and whether the texture inside the wormholes was rough and uneven, consistent with the characteristics of being eaten by bark beetles. These areas were ultimately confirmed as genuine wormholes. The total number of wormholes confirmed within the entire monitored area of ​​the tree trunk was counted, and the actual area of ​​the monitored area was calculated based on the calibration relationship between image pixels and actual size, thereby calculating the wormhole density per unit area.

[0085] For resin secreted from tree trunks, the resin-adhering regions attached to the trunks are first segmented from the image, and the colors of these regions are converted from the common red-green-blue color space to a chroma, saturation, and brightness color space. Within this color space, pixels with chroma values ​​falling within the yellow to brown range and high saturation are selected; these pixels constitute candidate resin regions. Further analysis of the geometry of these candidate regions is performed, calculating the aspect ratio of their bounding rectangles, describing the curvature of their edges, and extracting their skeletal structure to determine whether they exhibit the natural, downward-flowing morphological characteristics of resin, rather than other circular or regular shapes. The mean chroma, mean saturation, and shape parameters describing the flow pattern of the selected regions collectively constitute the current resin characteristics.

[0086] When processing the tree foot region, the ground portion is extracted from the image. By analyzing the color and texture of pixels in the image, the yellowish-white, loosely textured wood flour deposits are separated from the dark, coarse-textured soil background, generating a mask image of the wood flour deposits. For each connected region marked as a wood flour deposit, multiple sampling points are selected in the original image along the edge of the region towards the center. By analyzing the grayscale gradient changes at each sampling point, or using 3D depth information reconstructed from multiple frames, the height of the wood flour deposit surface relative to the underlying soil surface at that point is estimated. The average relative height of all sampling points within a region is taken as the average wood flour deposit thickness for that region. Finally, the thicknesses of all wood flour deposit regions within the field of view are weighted and averaged according to their area size to obtain the final wood flour deposit thickness of the tree foot region.

[0087] In addition, it is crucial to identify cylindrical wood-boring debris, formed by the compression and mixing of wood dust, bark beetle excrement, and resin, that has fallen or hung around tree trunks. This type of material typically has a specific columnar shape. First, an edge detection operator is used to extract the edges of all objects in the image. Then, two parallel straight line segments of similar length are identified within these edges and used as candidate generatrices for the cylinder. Between these generatrices, the presence of an arc connecting the two ends is detected, or the distribution pattern of the image gradient direction within the region is analyzed to confirm its columnar structure. For regions matching the columnar structure, each individual entity is counted once, and the frequency of occurrence of this type of material within a predetermined ground area is finally calculated, i.e., the distribution density of cylindrical wood-boring debris.

[0088] Finally, the six indicators mentioned above—the number of adult insects crawling through holes, the male-to-female ratio, the density of borer holes on the trunk surface, the characteristics of resin exudation, the thickness of wood powder accumulation in the tree base area, and the distribution density of cylindrical borer debris—were compared one by one with the control initiation thresholds summarized from extensive field experimental data. When three or more of the six indicators simultaneously reach or exceed their respective thresholds, it is determined that the current time point has entered the critical control period requiring the initiation of scientific pesticide application or biological control measures.

[0089] Furthermore, referring to Figure 2 S2 includes:

[0090] S21, Extract the region of interest containing the main trunk of the grapevine from multiple frames of two-dimensional images, and perform background subtraction on the region of interest to separate the moving target foreground;

[0091] S22, Connectivity analysis is performed on the separated moving target foreground to screen out candidate targets whose connected area, perimeter, and aspect ratio conform to the morphological characteristics of the adult bark beetle with smooth legs;

[0092] S23. Extract the color histogram features and local binary pattern texture features of the candidate targets, and compare them with the features of the pre-stored standard sample of smooth-legged bark beetle adults to confirm the identity of the adult individual.

[0093] S24. In the continuous frame images, the Kalman filter algorithm is used to predict and track the centroid movement trajectory of the identified adult insect. When the tracked trajectory crosses the preset horizontal baseline on the tree trunk, a crawling event of the adult insect is recorded.

[0094] S25, count the number of all crawling events recorded within a predetermined unit of time, and determine the number as the adult insect's crawling baseline for exiting the hole.

[0095] In this embodiment, in S21, the region containing the main body of the grapevine is accurately located and extracted from the acquired original image. To separate the crawling adult insect, a background modeling method based on a Gaussian mixture model is used. This method establishes multiple Gaussian distributions for each pixel in the image, continuously learning and updating the background over time. This effectively adapts to slow changes in lighting and slight swaying of branches, ultimately accurately segmenting the pixels of the moving adult insect to form a foreground image containing only the moving target.

[0096] In step S22, connected component analysis is performed on the foreground image. This step is not just a simple screening, but rather considers multiple morphological indicators. The actual pixel area of ​​each connected component is calculated, requiring that it be no less than the projected area of ​​the smallest insect body and no greater than the projected area of ​​the largest insect body. Simultaneously, the perimeter of the connected component's boundary and the ratio of the longer to shorter side of its smallest bounding rectangle are calculated. Only when the area, perimeter, and aspect ratio all fall within a preset interval representing the morphological characteristics of the adult *Bartholin's slickfoot* beetle are the connected components marked as candidate targets.

[0097] In step S23, to further eliminate interference from other insects or debris, the candidate target is precisely identified. Color information of all pixels within the candidate target region is extracted to generate a color histogram for that region, describing its overall color composition. Simultaneously, a local binary mode operator is used to calculate the grayscale relationship between each pixel within the target region and its neighboring pixels, generating a texture feature vector that reflects the roughness or fineness of the target surface texture. The extracted color histogram and texture feature vector are then compared with standard features pre-stored in a database, calculated from a large number of standard *Bartholin's spurred beetle* adult samples, to determine their similarity. Only when the similarity exceeds a set matching threshold is the moving target finally confirmed as an adult *Bartholin's spurred beetle*.

[0098] S24 involves complex multi-target tracking. A Kalman filter algorithm is used to predict the possible location of each identified adult insect in the next frame. This algorithm provides a predicted location and a range of uncertainty based on the insect's current speed and direction. After acquiring a new frame, the best-matching target is searched within the neighborhood of the predicted location, thus enabling continuous tracking of the insect. Simultaneously, one or more virtual horizontal counting lines are defined in the tree-like structure of the image. When the centroid coordinates of a tracked adult insect move from one side of the counting line to the other in two consecutive frames, the insect is considered to have completed a valid crawl, and the count is accumulated.

[0099] Finally, in S25, the number of times all adult insects cross the counting line will be accumulated within a default unit of time, such as one hour. This final accumulated number is the "adult emergence crawling base number," which reflects the intensity of adult insect activity on the tree trunk at that point in time.

[0100] Furthermore, referring to Figure 3 S3 includes:

[0101] S31, Image enhancement processing is performed on the tree trunk surface area in multiple frames of two-dimensional images. The combination of top-hat transformation and bottom-hat transformation is used to highlight the contrast between the dark spots on the tree trunk surface and the background, while suppressing the interference of bark texture.

[0102] S32, perform local adaptive threshold segmentation on the enhanced image to divide the tree trunk surface pixels into candidate blob pixels and non-blob pixels, and obtain a binarized candidate blob image.

[0103] S33, perform morphological filtering on each connected region in the binarized candidate spot image, calculate the roundness, eccentricity and area of ​​each connected region, and retain connected regions with roundness greater than the first preset threshold, eccentricity less than the second preset threshold and area within the preset wormhole area range as the initial wormholes.

[0104] S34, perform gray-level distribution analysis on the internal pixels of the original image region corresponding to each initially selected wormhole, calculate the gray-level mean and standard deviation of the pixels in the region, and construct a gray-level co-occurrence matrix to extract energy, correlation and homogeneity texture parameters. The initially selected wormholes with a gray-level mean lower than the average gray-level value of the tree trunk surface, a standard deviation greater than the third preset threshold, and texture parameters that meet the rough texture characteristics inside the wormhole are determined as the final wormholes.

[0105] S35, count the number of all final boreholes in the region of interest on the tree trunk surface, divide the number by the area of ​​the region of interest on the tree trunk surface to obtain the borehole density on the tree trunk surface, where the area of ​​the region of interest on the tree trunk surface is obtained by conversion through the calibration relationship between image pixel size and actual physical size.

[0106] In this embodiment, in step S31, to overcome the interference of the complex texture of the bark itself, the extracted image of the tree trunk surface is first subjected to a combination of morphological top-hat and bottom-hat transformations. The top-hat transformation subtracts the image after opening from the original image, which can highlight bright details in the image; while the bottom-hat transformation subtracts the original image from the image after closing, which can highlight dark details. By adding the top-hat transformation result to the original image and then subtracting the bottom-hat transformation result, the image contrast can be effectively enhanced, making the dark wormholes more clearly distinguishable against the relatively uniform bark background, while suppressing unnecessary texture information such as bark cracks.

[0107] In S32, a local adaptive thresholding technique is used for the enhanced image. Unlike global thresholding, this method dynamically calculates a local threshold based on the gray-level distribution characteristics of each pixel's neighborhood in the image. The advantage of this approach is that even if the tree trunk surface has uneven lighting or varying bark colors, it can accurately segment potential small holes in either shadow or highlight areas, ultimately generating a binary image where white pixels represent candidate holes or spots, and black pixels represent the background.

[0108] S33 is the morphological screening process for candidate spots. Each white connected region in the binary image is analyzed one by one. First, the roundness of the region is calculated. The roundness calculation is based on the relationship between the region's area and perimeter; a perfect circle has a roundness value close to 1. Wormholes are usually close to circular, so only regions with a roundness value higher than 0.7 are retained. Second, the eccentricity of the region is calculated, which is the eccentricity of an ellipse with the same standard second moment. An eccentricity of 0 represents a perfect circle, and the closer it is to 1, the more elongated it is. Only regions with an eccentricity less than 0.5 and a relatively regular shape are retained. Finally, the pixel area of ​​this region is converted to the actual physical area using a pre-calibrated scale, such as 0.1 mm per pixel, to determine if it falls within the typical area range of a smooth-footed bark beetle wormhole, such as 0.5 square millimeters to 3 square millimeters. Only regions that simultaneously meet the three conditions of roundness, eccentricity, and area are considered as initial wormholes.

[0109] S34 performs a deeper grayscale and texture analysis on the initially selected borer holes to eliminate bark dents or lenticels that are similar in shape but not caused by insect infestation. Pixels in the corresponding region of each initially selected hole in the original, unenhanced image are extracted. The grayscale mean of these pixels is calculated; if it is significantly lower than the grayscale mean of the healthy bark region surrounding the borer hole, it indicates that there is a depression or material change in that area. The grayscale standard deviation is calculated; a large standard deviation indicates that the grayscale values ​​of pixels inside the borer hole fluctuate drastically, which is consistent with the characteristics of a rough and uneven surface after insect infestation. A grayscale co-occurrence matrix for this region is also constructed, which describes the frequency of co-occurrence of grayscale values ​​of two pixels at a specific distance and direction in the image. Based on this matrix, the energy value, also known as the angular second moment, is calculated; the lower the energy value, the more uneven and complex the image texture. A correlation value is also calculated; the lower the correlation, the weaker the spatial dependence of the texture. Finally, an isomorphism value is calculated; the lower the isomorphism, the more drastic the texture changes. When a preliminary wormhole meets a series of conditions, such as a low mean gray value, a high standard deviation of gray value, a low energy value, a low correlation value, and a low isomorphism value, its rough and disordered internal texture is confirmed to be consistent with the characteristics of a wormhole and is finally identified as a real wormhole.

[0110] S35 performs the final statistics and calculations. It counts the total number of borer holes finally determined within the entire region of interest (ROI) on the trunk. Simultaneously, based on the calibration relationship between the image pixel size and the actual physical size, it calculates the actual area of ​​the ROI on the trunk. Dividing the total number of borer holes by the area yields the "bottom hole density on the trunk surface," which quantifies the degree of damage to the trunk; the unit is typically "holes / square centimeter."

[0111] Furthermore, S4 includes:

[0112] S41, Segment the suspected resinous image region attached to the tree trunk surface from multiple frames of two-dimensional images, and convert the color space of the suspected resinous image region from the original red-green-blue color space to the preset chroma-saturation-brightness color space;

[0113] S42, in the chroma-saturation-luminance color space, extract the chroma channel image, saturation channel image and luminance channel image respectively, perform statistical analysis on the chroma channel image, and determine the distribution range of chroma values ​​of all pixels in the entire chroma channel image;

[0114] S43, Gaussian filtering is applied to the saturation channel image for smoothing, and then watershed segmentation is performed on the processed saturation channel image to segment out continuous regions with high saturation values ​​as high saturation candidate regions.

[0115] S44, Spatial position matching is performed between the high saturation candidate region and the pixel region in the chroma channel image whose chroma value falls within the preset fat chroma threshold range, and the intersection region of the two is taken as the fat candidate pixel set;

[0116] S45, perform shape analysis on the corresponding region of the candidate pixel set of lipid flow in the brightness channel image, calculate the aspect ratio of the bounding rectangle of the region, the Fourier descriptor and the number of branch points after skeletonization, and extract the shape feature parameters that reflect the natural flow state of lipid flow.

[0117] S46. The shape feature parameters are compared with the pre-stored standard shape feature parameter range of lipids to select pixel areas that conform to the natural flow morphology of lipids. The average chroma, average saturation and shape feature parameters of these pixel areas in the chroma-saturation-brightness color space are combined and encapsulated into lipid morphology features.

[0118] In this embodiment, step S41 first separates the tree trunk region from the original image and, using prior knowledge of color and position, preliminarily segments out regions suspected to be resinous substances. A crucial step is to convert the color space of these regions from the original red-green-blue color space to a chroma-saturation-luminance color space. This conversion separates the essential color attribute of chroma from its vividness, saturation, and brightness, thereby enabling more stable color-based analysis and reducing the impact of lighting changes.

[0119] In steps S42 and S43, the converted chroma and saturation channel images are processed, respectively. For the chroma channel, the chroma values ​​of pixels within all suspected lipid regions are statistically analyzed to form a distribution histogram, and the chroma interval corresponding to the main peak is determined. This interval represents the dominant hue of the lipid region. For the saturation channel, Gaussian filtering is first applied to smooth the image and eliminate minor noise interference. Subsequently, a watershed segmentation algorithm is used to process the saturation image. This algorithm treats the image as a topographic map, where the pixel saturation value represents altitude. It then "fills" the image with water from the local lowest or highest point, separating different "watersheds." In this way, regions with high saturation and contiguous areas can be accurately segmented; these regions correspond to areas with dense lipids and vibrant colors.

[0120] The core of S44 is the fusion of chroma and saturation information. The high-saturation regions segmented in S43 are spatially compared with pixel regions in S42 whose chroma values ​​fall within a preset typical chroma range for lipid flow, such as 15 to 45 degrees, corresponding to the yellow to orange range. Only pixels that simultaneously possess high saturation and chroma values ​​matching the characteristics of lipid flow are collectively labeled as candidate pixels for lipid flow. This step significantly improves the accuracy of identification.

[0121] S45 performs an in-depth analysis of the geometric morphology of the candidate pixel set. It calculates the ratio of the length to the width of the bounding rectangle of the region. A typical flowing colloid usually has a longer length than its width, resulting in an aspect ratio greater than 1. Fourier descriptors are also used to describe the contour shape of the region. By performing a Fourier transform on the region boundary, the first few frequency components are taken as feature vectors describing the shape; these components effectively represent the overall shape and the degree of curvature of the contour. Furthermore, the region is skeletonized, i.e., edge pixels are continuously stripped until a single-pixel-wide skeleton representing the geometric center line of the region is obtained. Counting the number of branch points along this skeleton line determines whether the region is a simple strip or a complex, bifurcated shape, consistent with the bifurcation phenomenon that may occur when colloid flows naturally.

[0122] Finally, in S46, the shape features calculated in S45, such as aspect ratio, Fourier descriptor, and number of skeleton branch points, are compared with the standard morphological feature range learned in advance from a large number of resin flow samples from healthy and damaged grapevines. Only areas where all shape features conform to the natural flow pattern of resin are finally confirmed. The average chromaticity value and average saturation value of these confirmed areas in the chromaticity-saturation-luminance color space, along with the aforementioned parameters such as aspect ratio, Fourier descriptor, and number of skeleton branch points, are collectively encapsulated into a comprehensive feature vector, namely the "resin flow morphology feature," for subsequent prevention and control assessment.

[0123] Furthermore, referring to Figure 4 S5 includes:

[0124] S51, the ground area around the base of the grapevine is extracted from multiple frames of two-dimensional images as the analysis area, and geometric correction is performed on the analysis area to eliminate perspective distortion, so that the size ratio of objects on the ground in the corrected image conforms to the actual physical size ratio.

[0125] S52, perform color space conversion on the corrected analysis area image, extract color feature vectors, and use a semantic segmentation method based on color and texture to classify image pixels into soil background pixels and wood flour deposit pixels, generating a wood flour deposit mask image.

[0126] S53, in the mask image of wood flour accumulation, identify each connected region, and mark the connected region with an area greater than the preset minimum accumulation area as an effective wood flour accumulation region.

[0127] S54. For each effective wood flour accumulation area, select multiple sampling points along the edge of the area towards the center in the original image or the corresponding 3D reconstructed depth map. For each sampling point, obtain the relative height value between the surface of the wood flour accumulation and the surface of the soil below at that point. The arithmetic mean of the relative height values ​​of all sampling points is taken as the average accumulation thickness of the effective wood flour accumulation area.

[0128] S55, the average thickness of all effective wood flour accumulation areas within the image acquisition area is weighted and averaged, where the weight is the area of ​​each effective wood flour accumulation area, to calculate the wood flour accumulation thickness in the tree foot area.

[0129] In this embodiment, in step S51, the ground area around the base of the grapevines is extracted from the original image. Because the camera's shooting angle causes perspective distortion (objects appear larger when closer and smaller when farther away), the same object at different locations in the image can appear to have different sizes. Therefore, geometric correction is needed for this area. By identifying a reference object of known size in the image, or using camera calibration parameters, a perspective transformation matrix is ​​established to correct the tilted ground image into a top-down view. This ensures that pixels at any location in the corrected image represent the same actual physical size, laying the foundation for subsequent thickness calculations.

[0130] S52 performs semantic segmentation on the corrected image. It doesn't rely solely on a single color threshold but comprehensively considers color and texture information. For each pixel, it extracts multiple channel values ​​in the red-green-blue color space, as well as texture features within its neighborhood, such as texture spectra based on local binary patterns and energy responses obtained at different directions and scales using Gabor filter banks. These texture features effectively distinguish between granular wood flour and blocky soil particles. A pre-trained random forest classifier is used, taking the pixel's multidimensional color and texture features as input and outputting the probability that the pixel belongs to "wood flour deposits" or "soil background." Finally, a binary mask image is generated, where pixels representing wood flour deposits are marked white, and the background is black.

[0131] In S53, connected component analysis is performed on the generated mask image. For each connected white region, its pixel area is calculated and converted into an actual area. Only when the actual area of ​​a region is greater than a pre-set minimum accumulation area, such as 1 square centimeter, is it considered a valid wood flour accumulation region, thus eliminating the interference of a small number of scattered wood flour particles.

[0132] S54 is the core step in calculating thickness. For each valid wood flour accumulation area, the height of its surface relative to the underlying soil needs to be obtained. This can be achieved in two ways: if a 3D imaging device such as binocular vision or structured light is configured, the height information of each pixel can be directly obtained from the reconstructed depth map. Without a 3D device, estimation can also be performed based on a 2D image. Within the wood flour accumulation area, multiple sampling points are selected from the edge to the center. For each sampling point, its grayscale gradient change in the vertical direction is analyzed. Since there is a slope from the accumulation surface to the ground at the edge of the wood flour accumulation, the image grayscale will show a continuous gradient change at the slope. By analyzing the length and slope of the gradient, the relative thickness of the accumulation at that point can be indirectly estimated. Alternatively, the height of the accumulation can also be calculated using the length of the shadow cast by the accumulation under sunlight, combined with the solar altitude angle information. The arithmetic mean of the relative heights calculated from all sampling points is taken as the average accumulation thickness of the valid wood flour accumulation area.

[0133] S55 calculates the final overall wood dust accumulation thickness. Since multiple wood dust accumulation areas of varying sizes may exist within a single monitoring field, a simple average would be distorted by the influence of smaller areas. Therefore, a weighted average method is used. The average accumulation thickness Hi of each effective wood dust accumulation area is multiplied by its area Ai, and the products of all areas are summed to obtain a weighted sum of thickness and area. This weighted sum is then divided by the total area ΣAi of all effective wood dust accumulation areas. The final result is the "wood dust accumulation thickness at the base of the tree," which represents the overall damage level of the tree base area. This value is actually the average accumulation thickness per unit area, and better reflects the severity of the beetle infestation.

[0134] Furthermore, S6 includes:

[0135] S61, preprocess multiple frames of two-dimensional images to enhance the edge information of cylindrical targets in the images, and use the Canny edge detection operator to extract the edge contours of all objects in the images;

[0136] S62, perform polygon approximation and line segment detection on the extracted edge contours, and identify two parallel long side contours with similar lengths as candidates for the generatrix of the cylinder.

[0137] S63, between two candidate contours of the two generatrices, detect whether there is an arc-shaped or elliptical arc-shaped contour connecting the two generatrices, or based on the image gradient direction analysis, determine whether the gradient direction of the region between the two generatrices presents a rotationally symmetric distribution around a certain central axis.

[0138] S64. Count the number of all cylindrical woodworm individuals identified within the predetermined ground area, and divide the number by the predetermined ground area to obtain the distribution density of cylindrical woodworms.

[0139] In this embodiment, in step S61, the acquired ground image is first preprocessed to enhance the edge information of objects in the image. The Canny edge detection operator is used. This operator calculates the magnitude and direction of the image grayscale gradient and, through steps such as non-maximum suppression and double thresholding, can accurately extract the single-pixel wide edges of all object contours in the image, providing high-quality input for subsequent shape analysis.

[0140] S62 focuses on detecting the most prominent feature of a cylinder—two parallel generatrices. Among all extracted edge contours, the Hough transform line detection method is used to find line segments in the image. For each detected pair of line segments, it is determined whether they are parallel to each other and whether their lengths are similar within an acceptable error range. Only line segment pairs that meet both conditions are considered as the two generatrices of a potential cylinder and are retained for further verification.

[0141] S63 aims to verify whether the two generatrices constitute a complete cylinder. Analysis is performed in the region between the two generatrices. One approach is to directly search this region for arcuate or elliptical edges connecting the endpoints of the two generatrices, representing the ends of the cylinder. A more general approach is to analyze the image gradient directions within this region. For an ideal cylinder, the brightness variations on its surface would exhibit a symmetrical distribution around a central axis. This means that the gradient direction of pixels within the region, pointing towards the direction of the fastest brightness change, should roughly point towards or away from an imaginary central axis. Calculating the histogram of the gradient directions within this region, if it exhibits a clear bimodal symmetrical distribution, further supports the columnar structure of the region.

[0142] S64. Count the total number of all identified cylindrical woodworm debris individuals within the predetermined ground area corresponding to the entire monitoring field of view. Then divide this total number by the predetermined ground area to obtain the "cylindrical woodworm debris distribution density," expressed in "instruments / square meter." This indicator quantifies the abundance of gelatinous excrement or secretions that have fallen from or remain on trees per unit area.

[0143] Furthermore, S22 includes:

[0144] Calculate the pixel area of ​​the connected component, requiring that the area is between the first adult insect area threshold and the second adult insect area threshold;

[0145] Calculate the perimeter of the connected component, requiring that the perimeter is between the first adult perimeter threshold and the second adult perimeter threshold;

[0146] Calculate the aspect ratio of the smallest bounding rectangle of the connected component, requiring that the aspect ratio is between the first and second adult aspect ratio thresholds.

[0147] In this embodiment, in S22, a strict numerical filtering process is performed on each connected component separated from the foreground image.

[0148] Specifically, the first step is to calculate the total number of pixels contained in the connected region, i.e., the area of ​​that region. This area value needs to fall within a preset range, which is derived by statistically analyzing the lateral or dorsal projected areas of a large number of smooth-legged bark beetle adult samples. Connected regions with too small an area may be dust or image noise, while those with too large an area may be multiple insects stuck together or other large insects, and both will be excluded.

[0149] Next, calculate the boundary length of the connected component, i.e., the perimeter. Similarly, the perimeter must fall within a predetermined range corresponding to the size of the insect. The perimeter can serve as a supplement to the area, used to distinguish objects with similar areas but significantly different shapes.

[0150] Finally, the ratio of the length of the longer side to the shorter side of the smallest rectangle enclosing the connected region is calculated, i.e., the aspect ratio. The body of an adult *Bartholinae sclerotiorum* is typically elliptical, with its aspect ratio remaining stable within a specific range; it is neither a perfect circle with an aspect ratio close to 1, nor a slender shape with a very large aspect ratio. Only when the aspect ratio of the connected region also falls within the corresponding preset threshold range, for example, between 1.5 and 2.2, is the connected region considered morphologically consistent with adult characteristics.

[0151] Only when a connected component simultaneously satisfies the three numerical constraints of area, perimeter, and aspect ratio can it successfully advance to the "candidate target" stage and enter the subsequent color and texture recognition stages. This multi-parameter joint screening method greatly improves the purity of target detection, laying a solid foundation for subsequent accurate recognition.

[0152] Furthermore, S33 includes:

[0153] For each candidate connected region in the tree trunk surface image, calculate the roundness index of the region. The formula for calculating the roundness index is 4π multiplied by the area of ​​the region divided by the square of the perimeter of the region. When the roundness index is greater than 0.7, the roundness screening condition is met.

[0154] Calculate the eccentricity of the region. The eccentricity is based on the ellipse eccentricity calculated from the second moment of the region. When the eccentricity is less than 0.5, the eccentricity screening condition is met.

[0155] The total number of pixels contained in the region is counted as the region area. The region area is then compared with the actual wormhole area range obtained by pre-converting the pixel size. If the actual area after conversion is within the range of 0.5 square millimeters to 3 square millimeters, the area screening condition is met.

[0156] Candidate connected regions that meet the above roundness screening conditions, eccentricity screening conditions, and area screening conditions are identified as preliminary wormholes.

[0157] In this embodiment, in step S33, for each candidate dark connected region, its circularity is first calculated. Circularity is calculated by multiplying the region's area by four times pi and then dividing by the square of the region's perimeter. The closer this ratio is to 1, the closer the region's shape is to a standard circle. To filter out approximately circular holes formed by borers, it is set that a region only meets the circularity screening condition if its circularity is greater than 0.7.

[0158] Next, calculate the eccentricity of the region. Eccentricity is calculated based on the region's moment of inertia, describing its flatness. Its value is between 0 and 1, where 0 indicates the region is a perfect circle, and 1 indicates it has degenerated into a straight line segment. Since wormholes are usually not long, thin cracks, the eccentricity must be set to less than 0.5 to ensure the selected regions have a compact and regular shape.

[0159] Next, the area of ​​the region is physically screened. By calibrating the pixel equivalent of the image, the pixel area of ​​the region is converted into an actual area in square millimeters. According to entomological research, the hole size drilled by the adult smooth-legged bark beetle is relatively fixed, and the area of ​​the resulting borehole is usually between 0.5 and 3 square millimeters. Therefore, only regions whose converted actual area falls within this range meet the area screening criteria. Any candidate region must simultaneously meet three conditions: a circularity greater than 0.7, an eccentricity less than 0.5, and an actual area between 0.5 and 3 square millimeters, to be identified as a "preliminary borehole".

[0160] In S34, a more rigorous internal grayscale and texture analysis is performed on the initially selected wormholes. All pixels in the corresponding region of the initially selected wormhole in the original image are extracted, and its average grayscale value is calculated. A true wormhole typically appears darker than the surrounding healthy bark because light cannot easily penetrate or the internal material is porous. The average grayscale value inside the wormhole is required to be less than 70% of the average grayscale value of its neighboring tree trunk surface. Simultaneously, the interior of the wormhole appears rough due to the residue from the borer, resulting in large fluctuations in grayscale values; therefore, its grayscale standard deviation must be greater than 1.5 times that of the tree trunk surface grayscale standard deviation.

[0161] Next, the gray-level co-occurrence matrix (GLCM) for this region is constructed, and three key texture parameters are calculated from it. The first is energy, which reflects the uniformity of the image's gray-level distribution and the coarseness of the texture. The texture inside the wormhole is complex and disordered, so the energy value should be less than a low texture threshold. The second is correlation, which measures the similarity of the GLCM in the row or column direction, reflecting the directionality of the texture. The texture inside the wormhole lacks a clear directionality, so its correlation value should fall within a low range. The third isomorphism, which measures the amount of local texture variation in the image; the greater the variation, the lower the isomorphism. Therefore, the isomorphism value of the image inside the wormhole should also be less than a low threshold.

[0162] Only when a preliminary wormhole meets the above conditions of mean gray level and standard deviation of gray level, and its three texture parameters of energy, correlation and isomorphism also fall within the preset stringent range, will it be finally determined as a real "final wormhole" caused by the activity of wood-boring insects.

[0163] Furthermore, S42 includes:

[0164] The chroma value of each pixel in the chroma channel image is quantized from 0 to 360 degrees. A histogram of the chroma values ​​of all pixels is plotted, the positions of the main peak and the secondary peak in the histogram are identified, and the chroma value range corresponding to the main peak is recorded.

[0165] In S43, the Gaussian kernel size used for Gaussian filtering smoothing of the saturation channel image is 5×5, and the standard deviation is 1.0.

[0166] When performing watershed segmentation on the filtered saturation channel image, local maxima are used as seed points, and the threshold height for watershed segmentation is set to 20% of the maximum saturation value of the saturation channel image.

[0167] In S44, the preset chromaticity threshold range for chromaticity values ​​is between 15 and 45 degrees. The intersection of the high saturation candidate region and the pixel region in the chromaticity channel image whose chromaticity values ​​fall within the preset chromaticity threshold range for chromaticity is taken as the chromaticity candidate pixel set.

[0168] Shape feature parameters reflecting the natural flow state of lipids were extracted from S45, specifically including:

[0169] Calculate the ratio of the length to the width of the bounding rectangle of the region formed by the candidate pixel set of the lipid flow, and require that the ratio be greater than 1.2 and less than 3.5;

[0170] Calculate the Fourier descriptor of the region, and take the normalized values ​​of the first 10 Fourier coefficients as the shape contour features.

[0171] The region is skeletonized by extracting skeleton lines and counting the number of branch points of the skeleton lines. The number of branch points must be greater than 0 and less than 5.

[0172] In S46, pixel regions that conform to the natural flow morphology characteristics of lipid are selected. Specifically, these refer to a set of candidate pixels that simultaneously meet the following conditions: the aspect ratio of the circumscribed rectangle is between 1.2 and 3.5; the Euclidean distance between the first 10 normalized Fourier descriptors and the pre-stored standard lipid Fourier descriptor vector is less than the fifth preset threshold; and the number of skeleton line branch points is between 1 and 4. The mean chroma and mean saturation of the selected pixel regions, along with the aforementioned aspect ratio of the circumscribed rectangle, the first 10 Fourier descriptors, and the number of skeleton line branch points, are combined and encapsulated into lipid morphological characteristics.

[0173] In this embodiment, in step S42, after identifying a suspected area of ​​lipid leakage, its chromaticity channel image undergoes refined analysis. This involves quantizing the chromaticity value of each pixel on a color wheel ranging from 0 to 360 degrees and generating a histogram of chromaticity values ​​for all pixels. By identifying the highest and second-highest peaks in the histogram, the dominant hue range of the color in that area can be accurately determined, which is crucial for subsequent screening.

[0174] In S43, when performing Gaussian filtering on the saturation channel image, a Gaussian kernel with a size of 5×5 pixels and a standard deviation of 1.0 was explicitly used. This parameter combination effectively smooths high-frequency noise in the image while preserving saturation variation information at flow edges. The subsequent watershed segmentation algorithm sets a key parameter—the segmentation threshold height. This threshold is set to 20% of the maximum saturation value in the entire saturation channel image. This means that only when the saturation of a pixel drops more than 20% from a local maximum is it considered a boundary between different "watersheds," thus accurately segmenting connected regions with similar saturation.

[0175] In step S44, the highly saturated regions segmented above are overlaid with the chroma channel image for analysis. A default chroma threshold range for "fat flow" is set, determined based on extensive actual observation data to be between 15 and 45 degrees, corresponding to a color range from pale yellow to orange-brown, which is typical of fat flow. Only pixels that are simultaneously located within the highly saturated regions and whose chroma values ​​fall within the range of 15 to 45 degrees are ultimately included in the "fat flow candidate pixel set".

[0176] S45 performs a detailed shape analysis on the candidate pixel set. When calculating the aspect ratio of the circumscribed rectangle, the ratio must be greater than 1.2 and less than 3.5. A ratio less than 1.2 indicates a shape close to a circle, which does not conform to flow characteristics; a ratio greater than 3.5 indicates an overly elongated shape, which may be confused with interfering elements such as tree branches. For the Fourier descriptor, the first 10 Fourier coefficients of the region contour are extracted and normalized. This normalized numerical vector is used as the "fingerprint" describing the shape. In the skeletonization analysis, the region is iteratively eroded to extract its centerline skeleton, and the number of branch points along the skeleton is counted. A typical flowing colloid may produce one to four simple branches due to gravity; therefore, the number of branch points is limited to a range greater than 0 and less than 5.

[0177] Finally, in S46, a comprehensive judgment is made. For a candidate pixel set to be confirmed as representing true lipid flow, it must simultaneously meet all of the following conditions: the aspect ratio of its bounding rectangle is between 1.2 and 3.5; the Euclidean distance between its first 10 normalized Fourier descriptor vectors and the pre-stored standard lipid flow Fourier descriptor vectors is less than a fifth preset threshold; and the number of skeleton line branch points is between 1 and 4. All pixel regions that meet these stringent conditions, their average chroma value, average saturation value, aspect ratio, first 10 Fourier descriptors, and number of skeleton line branch points together constitute a multi-dimensional "lipid flow morphology feature." This feature vector comprehensively and quantitatively describes the color, vibrancy, and unique flow pattern of the lipid flow.

[0178] According to a second embodiment of the present invention, the present invention claims protection for a system for determining the optimal control period of the grape smooth-foot bark beetle based on image recognition, comprising:

[0179] One or more processors;

[0180] A memory that stores one or more programs, which, when executed by one or more processors, enable one or more processors to implement an image recognition-based method for determining the optimal control period of the grape bark beetle.

[0181] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0182] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units. The above are merely embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made based on the description and drawings of this application, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

[0183] The specific embodiments of the invention have been described in detail above, but they are only examples, and this application is not limited to the specific embodiments described above. For those skilled in the art, any equivalent modifications or substitutions to the invention are also within the scope of this application. Therefore, all equivalent changes, modifications, and improvements made without departing from the spirit and principles of this application should be covered within the scope of this application.

Claims

1. A method for determining the optimal control period for grape bark beetle based on image recognition, characterized in that, Includes the following steps: S1, using an image acquisition device set up within the grapevine planting area, acquire multiple frames of two-dimensional images containing the grapevine trunk and base area; S2, perform image recognition processing on the multi-frame two-dimensional images to identify and count the number of adult grape smooth-legged bark beetles crawling on the tree trunk and the morphological characteristics of the adults, and generate the number of adult beetles crawling through the exit hole and the male-female ratio that cross the preset baseline on the tree trunk. S3, perform image recognition processing on the tree trunk surface area in the multi-frame two-dimensional images, extract the circular or elliptical dark spot features in the tree trunk surface image, filter out the pixel areas that meet the preset morphological features of boreholes, and calculate and generate the borehole density on the tree trunk surface. S4, perform image recognition processing on the resin adhering to the tree trunk surface in the multi-frame two-dimensional images, convert the image color space to a preset chroma-saturation-brightness color space, and extract a set of pixels that meet the preset conditions in the color space as resin morphological features. S5, perform image recognition processing on the ground in the tree foot area of ​​the multi-frame two-dimensional image, extract the wood powder accumulation area, and calculate the wood powder accumulation thickness in the tree foot area by analyzing the gray-level gradient change of the accumulation area in the vertical direction. S6, perform image recognition processing on the cylindrical wood dust, bark beetle excrement and resin mixture that fall or hang around the tree trunk in the multi-frame two-dimensional images, and statistically generate the wood dust distribution density; S7. The number of adult insects crawling out of holes, the male-female ratio, the density of borer holes on the trunk surface, the resin exudation characteristics, the thickness of wood powder accumulation in the tree foot area, and the density of borer debris distribution are compared with their respective preset prevention and control activation thresholds. When at least three items reach or exceed their corresponding preset prevention and control activation thresholds, the current time is determined to be a suitable time node for initiating prevention and control measures.

2. The method for determining the optimal control period for grape bark beetle based on image recognition according to claim 1, characterized in that, The S2 method generates the number of adult insects crawling through the exit holes of the tree trunk along a preset baseline, including: S21, extract the region of interest containing the main trunk of the grapevine from the multi-frame two-dimensional images, and perform background subtraction processing on the region of interest to separate the moving target foreground; S22, Connectivity analysis is performed on the separated moving target foreground to screen out candidate targets whose connected area, perimeter, and aspect ratio conform to the morphological characteristics of the adult bark beetle with smooth legs; S23. Extract the color histogram features and local binary pattern texture features of the candidate targets, and compare them with the features of the pre-stored standard sample of smooth-legged bark beetle adults to confirm the identity of the adult individual. S24. In the continuous frame images, the Kalman filter algorithm is used to predict and track the centroid movement trajectory of the identified adult insect. When the tracked trajectory crosses the preset horizontal baseline on the tree trunk, a crawling event of the adult insect is recorded. S25, count the number of all crawling events recorded within a predetermined unit of time, and determine the number as the adult insect's crawling baseline for exiting the hole.

3. The method for determining the optimal control period for grape bark beetle based on image recognition according to claim 1, characterized in that, S3 includes: S31, perform image enhancement processing on the tree trunk surface area in the multi-frame two-dimensional images, using a combination of top-hat transform and bottom-hat transform to highlight the contrast between the dark spots on the tree trunk surface and the background, while suppressing the interference of bark texture. S32, perform local adaptive threshold segmentation on the enhanced image to divide the tree trunk surface pixels into candidate blob pixels and non-blob pixels, and obtain a binarized candidate blob image. S33, perform morphological filtering on each connected region in the binarized candidate spot image, calculate the roundness, eccentricity and area of ​​each connected region, and retain connected regions whose roundness is greater than the first preset threshold, whose eccentricity is less than the second preset threshold and whose area is within the preset area range of the borehole as the initial selected borehole. S34, perform gray-level distribution analysis on the internal pixels of the original image area corresponding to each initially selected borehole, calculate the gray-level mean and standard deviation of the pixels in the area, and construct a gray-level co-occurrence matrix to extract energy, correlation and homogeneity texture parameters. The initially selected boreholes with a gray-level mean lower than the average gray-level value of the tree trunk surface, a standard deviation greater than the third preset threshold, and texture parameters that meet the rough texture characteristics inside the borehole are determined as the final boreholes. S35, count the number of all final boreholes in the region of interest on the tree trunk surface, divide the number by the area of ​​the region of interest on the tree trunk surface to obtain the borehole density on the tree trunk surface, wherein the area of ​​the region of interest on the tree trunk surface is obtained by conversion through the calibration relationship between image pixel size and actual physical size.

4. The method for determining the optimal control period for grape bark beetle based on image recognition according to claim 1, characterized in that, S4 includes: S41, the suspected resinous image region attached to the tree trunk surface is segmented from the multi-frame two-dimensional images, and the color space of the suspected resinous image region is converted from the original red-green-blue color space to the preset chroma-saturation-brightness color space; S42, in the chroma-saturation-luminance color space, extract the chroma channel image, saturation channel image and luminance channel image respectively, perform statistical analysis on the chroma channel image, and determine the chroma value distribution range of all pixels in the entire chroma channel image; S43, Gaussian filtering is applied to the saturation channel image for smoothing, and then watershed segmentation is performed on the processed saturation channel image to segment out continuous regions with high saturation values ​​as high saturation candidate regions. S44, Spatial position matching is performed between the high saturation candidate region and the pixel region in the chroma channel image whose chroma value falls within the preset fat chroma threshold range, and the intersection region of the two is taken as the fat candidate pixel set; S45, perform shape analysis on the corresponding region of the candidate pixel set of lipid flow in the brightness channel image, calculate the aspect ratio of the bounding rectangle of the region, the Fourier descriptor and the number of branch points after skeletonization, and extract shape feature parameters that reflect the natural flow state of lipid flow. S46, compare the shape feature parameters with the pre-stored standard shape feature parameter range of lipid flow, filter out pixel regions that conform to the natural flow morphology of lipid flow, and encapsulate the average chroma, average saturation of these pixel regions in the chroma-saturation-brightness color space and the shape feature parameters as the lipid flow characteristics.

5. The method for determining the optimal control period for grape bark beetle based on image recognition according to claim 1, characterized in that, S5 includes: S51, the ground area around the base of the grapevine is extracted from the multi-frame two-dimensional image as the analysis area, and geometric correction is performed on the analysis area to eliminate perspective distortion, so that the size ratio of objects on the ground in the corrected image conforms to the actual physical size ratio. S52, perform color space conversion on the corrected analysis area image, extract color feature vectors, and use a semantic segmentation method based on color and texture to classify image pixels into soil background pixels and wood flour deposit pixels, generating a wood flour deposit mask image. S53, in the wood flour accumulation mask image, identify each connected region, and mark the connected region with an area greater than the preset minimum accumulation area as an effective wood flour accumulation region. S54. For each effective wood flour accumulation area, select multiple sampling points along the edge of the area towards the center in the original image or the corresponding 3D reconstructed depth map. For each sampling point, obtain the relative height value between the surface of the wood flour accumulation and the surface of the soil below at that point. The arithmetic mean of the relative height values ​​of all sampling points is taken as the average accumulation thickness of the effective wood flour accumulation area. S55, the average thickness of all effective wood flour accumulation areas within the image acquisition area is weighted and averaged, where the weight is the area of ​​each effective wood flour accumulation area, to calculate the wood flour accumulation thickness in the tree foot area.

6. The method for determining the optimal control period for grape bark beetle based on image recognition according to claim 1, characterized in that, S6 includes: S61, preprocess the multi-frame two-dimensional images to enhance the edge information of the cylindrical target in the images, and use the Canny edge detection operator to extract the edge contours of all objects in the images; S62, perform polygon approximation and line segment detection on the extracted edge contours, and identify two parallel long side contours with similar lengths as candidates for the generatrix of the cylinder. S63, between two candidate contours of the two generatrices, detect whether there is an arc-shaped or elliptical arc-shaped contour connecting the two generatrices, or based on the image gradient direction analysis, determine whether the gradient direction of the region between the two generatrices presents a rotationally symmetric distribution around a certain central axis. S64, count the number of all cylindrical woodworm individuals identified within the predetermined ground area, and divide the number by the predetermined ground area to obtain the cylindrical woodworm distribution density.

7. The method for determining the optimal control period for grape bark beetle based on image recognition according to claim 2, characterized in that, S22 includes: Calculate the pixel area of ​​the connected component, requiring that the area is between the first adult insect area threshold and the second adult insect area threshold; Calculate the perimeter of the connected component, requiring that the perimeter is between the first adult perimeter threshold and the second adult perimeter threshold; Calculate the aspect ratio of the smallest bounding rectangle of the connected component, requiring that the aspect ratio is between the first and second adult aspect ratio thresholds.

8. The method for determining the optimal control period for grape bark beetle based on image recognition according to claim 3, characterized in that, S33 includes: For each candidate connected region in the tree trunk surface image, the roundness index of the region is calculated. The formula for calculating the roundness index is 4π multiplied by the area of ​​the region divided by the square of the perimeter of the region. When the roundness index is greater than 0.7, the roundness screening condition is met. Calculate the eccentricity of the region, which is based on the eccentricity of the ellipse calculated from the second moment of the region. When the eccentricity is less than 0.5, the eccentricity screening condition is met. The total number of pixels contained in the region is counted as the region area. The region area is then compared with the actual wormhole area range obtained by pre-converting the pixel size. If the actual area after conversion is within the range of 0.5 square millimeters to 3 square millimeters, the area screening condition is met. Candidate connected regions that meet the above roundness screening conditions, eccentricity screening conditions, and area screening conditions are identified as preliminary wormholes.

9. The method for determining the optimal control period for grape bark beetle based on image recognition according to claim 4, characterized in that, S42 includes: The chroma value of each pixel in the chroma channel image is quantized from 0 to 360 degrees. A histogram of the chroma values ​​of all pixels is plotted, the positions of the main peak and the secondary peak in the histogram are identified, and the chroma value range corresponding to the main peak is recorded. In S43, the Gaussian kernel size used for Gaussian filtering and smoothing of the saturation channel image is 5×5, and the standard deviation is 1.

0. When performing watershed segmentation on the filtered saturation channel image, local maxima are used as seed points, and the threshold height for watershed segmentation is set to 20% of the maximum saturation value of the saturation channel image. In S44, the preset lipid chromaticity threshold range is between 15 degrees and 45 degrees. The intersection of the high saturation candidate region and the pixel region in the chromaticity channel image whose chromaticity value falls within the preset lipid chromaticity threshold range is taken as the lipid candidate pixel set. The shape feature parameters reflecting the natural flow state of the lipid in S45 are extracted, specifically including: Calculate the ratio of the length to the width of the bounding rectangle of the region formed by the candidate pixel set of the lipid flow, and require that the ratio be greater than 1.2 and less than 3.5; Calculate the Fourier descriptor of the region, and take the normalized values ​​of the first 10 Fourier coefficients as the shape contour features. The region is skeletonized by extracting skeleton lines and counting the number of branch points of the skeleton lines. The number of branch points must be greater than 0 and less than 5. In step S46, pixel regions that conform to the natural flow morphology characteristics of lipids are selected. Specifically, these refer to a set of candidate pixels that simultaneously meet the following conditions: the aspect ratio of the circumscribed rectangle is between 1.2 and 3.5; the Euclidean distance between the first 10 normalized Fourier descriptors and the pre-stored standard lipid Fourier descriptor vectors is less than a fifth preset threshold; and the number of skeleton line branch points is between 1 and 4. The average chroma and average saturation of the selected pixel regions, along with the aforementioned aspect ratio of the circumscribed rectangle, the first 10 Fourier descriptors, and the number of skeleton line branch points, are combined and encapsulated as the lipid morphological characteristics.

10. A system for determining the optimal control period for the grape bark beetle based on image recognition, characterized in that: include: One or more processors; A memory having stored one or more programs that, when executed by one or more processors, cause the one or more processors to implement the method for determining the appropriate control period of grape bark beetle based on image recognition according to any one of claims 1 to 9.

Images