A method and system for identifying and assessing damage of leafworms

Abstract

The present application provides a kind of blade pest identification and damage assessment method and system, the method comprises: extracting residual leaf area of target leaf image, the target leaf image includes to be evaluated leaf and insect body;Identify the insect body in the target leaf image, construct the uncertain area representing the range of insect body obstruction and interference;Based on the reference leaf area obtained in advance, the residual leaf area corresponding to the residual leaf area and the uncertain area corresponding to the uncertain area, determine the leaf loss area, and generate leaf damage assessment result according to the leaf loss area, by introducing reference area as a benchmark, and combining the interference of local obstruction blind area is excluded, the leaf area that has been chewed and eaten by larva is reasonably inferred, the defects of insufficient precision of artificial statistics of leaf pest at present stage are effectively overcome, and the error quantization result that the damage rate of pest given by the existing algorithm is extremely low is corrected.

Description

Technical Field

[0001] This invention relates to the fields of agricultural image processing and machine vision technology, and in particular to a method and system for identifying leaf pests and assessing damage. Background Technology

[0002] Plant insect resistance assessment is a crucial experimental step in crop variety breeding, biopesticide screening, and gene function verification. In typical high-throughput bioassays of detached leaves, researchers usually place the leaves of the plant under test in a transparent multi-well plate with agar and inoculate them with larvae of a specific insect species for co-culture. The insect resistance effect is assessed by observing and quantifying the feeding damage to the leaves at the experimental endpoint. Unlike fungal or bacterial diseases, which mainly cause discoloration and necrosis of leaf tissue but retain the overall outline, the core damage phenotype of chewing insect pests is the physical loss of leaf tissue. That is, the leaf tissue chewed and consumed by the larvae disappears directly and is no longer present in the final image field of view.

[0003] Currently, the assessment of leaf damage mainly relies on manual interpretation, threshold segmentation based on traditional image processing, or image segmentation schemes partially based on deep learning. Existing publicly available technologies generally employ adaptive threshold segmentation to distinguish the background, contour extraction based on edge detection, or deep learning models such as convolutional neural networks to identify residual damaged areas. The core logic of these existing technical solutions focuses heavily on segmenting and identifying visible lesions such as discoloration and necrosis on the surface of residual leaves. Essentially, it is a process of finding abnormal pixels on existing residual tissue, that is, generally using a disease identification paradigm to quantify and assess the severity of biological damage.

[0004] Existing methods lack a mechanism to infer the physical area of ​​missing leaves under complex interferences such as insect occlusion, thus severely underestimating the actual damage caused by chewing pests. Since leaves missing due to chewing have physically disappeared, and the live insects remaining in high-throughput experimental wells (especially green larvae) often closely resemble leaves in color and texture, traditional segmentation models are highly susceptible to mistaking insects for part of healthy leaves. Current techniques neither eliminate this pseudo-leaf effect nor establish a logic for inferring the lost area by incorporating spatial geometric priors. Consequently, when dealing with chewing pests, they can only rely on surface anomalies of residual tissue, inevitably leading to erroneous quantitative results that overestimate leaf area and underestimate pest damage rates, failing to meet the reliability requirements of high-throughput experiments for quantitative assessment. Summary of the Invention

[0005] This invention provides a method and system for leaf pest identification and damage assessment, which addresses the shortcomings of existing technologies that lack a mechanism to infer the physical missing area of ​​leaves under complex interferences such as insect occlusion, thus seriously underestimating the actual damage caused by chewing pests. This invention enables automated, objective, and accurate quantitative assessment of chewing pest damage in high-throughput multi-well plate experiments.

[0006] This invention provides a method for identifying leaf pests and assessing damage, including: Extract the residual leaf region from the target leaf image, which includes the leaf to be evaluated and the insect body; Identify the insects in the target leaf image and construct an uncertain region characterizing the range of insect occlusion and interference; Based on the pre-obtained reference leaf area, the residual leaf area corresponding to the residual leaf region, and the uncertain area corresponding to the uncertain region, the missing leaf area is determined, and a leaf damage assessment result is generated based on the missing leaf area.

[0007] According to the leaf pest identification and damage assessment method provided by the present invention, after extracting the residual leaf area of ​​the target leaf image, the method further includes: Extract the contour morphology features and brightness distribution features of the leaf to be evaluated from the target leaf image and the residual leaf region; The degree of curling of the leaf to be evaluated is determined based on the outline morphology features and the brightness distribution features. When the degree of curling exceeds a set threshold, a quality mark requiring manual review is generated in the blade damage assessment results.

[0008] According to the leaf pest identification and damage assessment method provided by the present invention, the step of identifying insects in the target leaf image and constructing an uncertain region characterizing the extent of insect occlusion and interference includes: Obtain the residual leaf area mask corresponding to the residual leaf area, and obtain the insect area mask corresponding to the identified insect body; perform local hole filling and repair on the residual leaf area mask to obtain the leaf support area mask; The insect body region mask is intersected with the leaf support region mask or a pre-obtained geometric template reference region mask to obtain the direct occlusion region mask. A morphological dilation operation is performed on the insect body region mask to obtain a neighborhood expansion uncertain region mask; The uncertain region mask is obtained by taking the union of the direct occlusion region mask and the neighborhood extended uncertain region mask, and the region corresponding to the uncertain region mask is constructed as the uncertain region.

[0009] According to the leaf pest identification and damage assessment method provided by the present invention, the method for obtaining the reference leaf area includes: deterministically constructing a standardized template using known geometric shapes, and using the area corresponding to the standardized template as the reference leaf area; or Obtain the target area and use the target area as the reference leaf area; wherein, the target area includes any one of the following: the leaf segmentation area in the image of the same hole position before insect inoculation, the theoretical area of ​​the standard leaf disc, and the statistical area of ​​the control sample without insects in the same batch.

[0010] According to the leaf pest identification and damage assessment method provided by the present invention, the step of determining the missing leaf area based on the pre-acquired reference leaf area, the residual leaf area corresponding to the residual leaf region, and the uncertain area corresponding to the uncertain region includes: An area deformation correction coefficient is estimated by combining the area change data of the insect-free control group, and the pre-acquired reference leaf area is corrected based on the area deformation correction coefficient to obtain the corrected reference area; the residual leaf area and the uncertain area in the support domain are subtracted from the corrected reference area to obtain the evaluation difference; the evaluation difference is compared with zero value, and the larger value is determined as the missing leaf area.

[0011] According to the leaf pest identification and damage assessment method provided by the present invention, after determining the missing area of ​​the leaf, the method further includes: subtracting the remaining leaf area from the corrected reference area to obtain a boundary difference; comparing the boundary difference with zero and taking the larger value as the upper bound estimate of the missing area; and using the missing leaf area and the upper bound estimate of the missing area together to form a missing proportion interval, and outputting it to the leaf damage assessment result.

[0012] According to the leaf pest identification and damage assessment method provided by the present invention, the process of generating leaf damage assessment results based on the missing leaf area further includes: The difference between the residual blade region and the uncertain region is determined as the identifiable effective blade region; when the area of ​​the identifiable effective blade region mask is greater than a preset minimum threshold, visible abnormal features are identified for the pixels in the residual blade region, and visible abnormal regions are output; the visible abnormality ratio calculated based on the visible abnormal regions is merged into the blade damage assessment result.

[0013] According to the leaf pest identification and damage assessment method provided by the present invention, the process of acquiring the target leaf image before extracting the residual leaf area of ​​the target leaf image includes: Receive experimental plate images containing multiple wells; The hole location module identifies the position of each hole in the experimental board image and performs perspective correction based on the plane of the experimental board in the experimental board image to obtain a perspective-corrected image. Based on the positions of each hole, the perspective correction image is cropped to output a standardized, approximately orthogonalized single-hole image as the target blade image.

[0014] The leaf pest identification and damage assessment method provided by the present invention further includes, before identifying the position of each hole in the experimental board image through the hole positioning module and performing perspective correction based on the plane of the experimental board in the experimental board image: performing lens distortion correction on the experimental board image based on camera calibration parameters or the geometric mesh prior of the experimental board. After outputting the standardized single-aperture approximately orthogonalized image as the target leaf image, the method further includes: performing illumination normalization and color normalization on the target leaf image; the illumination normalization process compensates for illumination unevenness by estimating the illumination field and subtracting the slowly varying components, and the color normalization process performs empirical colorimetric normalization based on the mean of the background region.

[0015] According to the leaf pest identification and damage assessment method provided by the present invention, the extraction of residual leaf regions from a target leaf image includes: inputting the target leaf image into a first deep learning network for feature extraction and segmentation, and outputting a preliminary residual leaf region; wherein, the input features of the first deep learning network include the red, green, and blue channels of the target leaf image, and the saturation and brightness channels in the hue-saturation-brightness space obtained by converting the red, green, and blue channels; the training loss function of the first deep learning network adopts a weighted combination of pixel-level cross-entropy loss and region overlap loss; The process of identifying insects in the target leaf image includes: The target leaf image is input into a second deep learning network to perform instance segmentation or target detection, and the region corresponding to the insect body is output. The region corresponding to the insect body is used to check for overlap in the preliminary residual leaf region, and the part of the preliminary residual leaf region that overlaps with the region corresponding to the insect body is removed to obtain the final residual leaf region.

[0016] This invention also provides a leaf pest identification and damage assessment system, comprising: An extraction module is used to extract the residual leaf region of a target leaf image, the target leaf image containing the leaf to be evaluated and the insect body; A construction module is used to identify insects in the target leaf image and construct an uncertain region characterizing the range of insect occlusion and interference. The determination module is used to determine the missing area of ​​the leaf based on the pre-acquired reference leaf area, the residual leaf area corresponding to the residual leaf region, and the uncertain area corresponding to the uncertain region, and to generate a leaf damage assessment result based on the missing area of ​​the leaf.

[0017] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the leaf pest identification and damage assessment method as described above.

[0018] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the leaf pest identification and damage assessment method as described above.

[0019] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the leaf pest identification and damage assessment method as described above.

[0020] The leaf pest identification and damage assessment method and system provided by this invention extracts the residual leaf area from the target leaf image and identifies the insects in the target leaf image containing both the leaf to be assessed and the insects. An uncertain region characterizing the range of insect occlusion and interference is constructed. This feature effectively distinguishes real residual leaf tissue from live insects, reducing the interference of the pseudo-leaf effect—where traditional image processing models easily mistake insects with similar colors and textures for healthy leaves—and avoiding overestimation of the residual leaf area. Simultaneously, a mechanism for quantifying insect occlusion interference is established at the image processing level. Furthermore, based on pre-acquired reference leaf area, the residual leaf area corresponding to the residual leaf region, and the uncertain area corresponding to the uncertain region, the invention determines the missing leaf area and generates a leaf damage assessment result based on the missing leaf area. This feature overcomes the limitation of traditional assessment methods that rely solely on surface anomalies of residual tissue. By introducing a reference area as a benchmark and combining it with the uncertain region area to eliminate interference from local occlusion blind spots, it reasonably infers the leaf area that has been physically lost due to larval chewing and ingestion. This effectively overcomes the shortcomings of insufficient accuracy in current manual leaf pest statistics and corrects the erroneous quantification results of extremely low pest damage rates given by existing algorithms. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 This is one of the flowcharts of the leaf pest identification and damage assessment method provided by the present invention.

[0023] Figure 2 This is the second flowchart of the leaf pest identification and damage assessment method provided by the present invention.

[0024] Figure 3 This is a schematic diagram of the leaf pest identification and damage assessment system provided by the present invention.

[0025] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0027] It should be noted that existing leaf damage assessment methods, whether based on traditional image processing (such as thresholding and edge detection) or deep learning-based image segmentation, all share the common assessment logic of using visible anomalies (such as discoloration, necrosis, and browning) on ​​the surface of residual leaves as identification targets, essentially following a disease assessment paradigm. However, for pests caused by chewing insects (such as lepidopteran larvae), the typical damage characteristic is the physical loss of leaf tissue due to direct ingestion, rather than just color or texture changes on the residual tissue. Existing methods lack a mechanism to infer the area of ​​physically lost leaves, leading to a systematic underestimation of actual damage in samples with heavy feeding. Furthermore, in laboratory bioassay scenarios, live insects often coexist with residual leaves in the same container or culture well. Some insects (especially young green larvae) are highly similar to healthy leaves in color, saturation, and local texture. When using conventional segmentation models, insects are easily misidentified as leaf areas, resulting in a pseudo-leaf effect, further leading to an overestimation of leaf area and an underestimation of damage rate. A search revealed that existing technologies do not specifically address the uncertainties caused by insect occlusion, nor do they provide a complete technical solution for inferring the missing area based on reference leaf information and combining it with visible anomalies for joint evaluation.

[0028] To more clearly illustrate the application environment and technical challenges addressed in this application, the following supplementary description provides a typical bioassay experiment for insect resistance on detached leaves. In conventional experiments, researchers place the leaf to be tested (either a whole leaf or a standard leaf disc prepared using a perforator) and chewing insects of a certain age together in a culture container and culture them for several days (usually 5 to 7 days) under controlled temperature and humidity conditions. At the end of the experiment, an overhead image of the leaf and insects coexisting in the container is acquired for subsequent quantitative damage assessment. In actual imaging, in addition to interference from the similar color of the insects and leaves, additional interference factors may arise, such as uneven lighting, lens distortion, perspective distortion, leaf curling or lifting due to water loss or mechanical damage, and molting or remains of the insects. If these factors are not considered when designing the assessment method, they will significantly affect the stability and repeatability of the damage quantification results. Most publicly available solutions currently focus on a single algorithm module (such as leaf segmentation or lesion recognition), and there is no end-to-end process that covers image input, discrimination of interfering targets, management of uncertain regions, inference of missing area and output of structured reports.

[0029] To address the aforementioned technical problems, this invention provides a method for identifying and assessing leaf pests. The method first extracts the residual leaf region from a target leaf image, which includes the leaf to be assessed and the insect. Next, the insect in the target leaf image is identified, and an uncertain region characterizing the extent of insect occlusion and interference is constructed. Then, based on a pre-obtained reference leaf area, the residual leaf area corresponding to the residual leaf region, and the uncertain region corresponding to the uncertain area, the missing leaf area is determined, and a leaf damage assessment result is generated based on the missing leaf area.

[0030] Through the above methods, the embodiments of the present invention effectively distinguish between real residual leaf tissue and live insects, reducing the interference of the "pseudo-leaf effect." Simultaneously, by introducing a reference benchmark to infer the area of ​​physically disappeared leaves and defining and isolating the insect-occupied area as an uncertain region, it overcomes the limitation of traditional methods that rely solely on surface anomalies of residual tissue. This improves the accuracy, objectivity, and automation of the quantitative assessment of insect damage loss area, effectively correcting the erroneous quantitative results of extremely low insect damage rates obtained by existing technologies due to ignoring physical loss. It is particularly suitable for high-throughput experimental scenarios requiring batch analysis of large numbers of samples. The technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0031] Before describing the technical solutions of the embodiments of the present invention, the terminology used in the embodiments of the present invention will be explained illustratively.

[0032] Target leaf image: refers to a single-frame digital image containing both the leaf to be evaluated and the insect, which can be derived from a single-well overhead image from a multi-well plate experiment, a petri dish image, or an image acquired from other imaging containers. This image must at least contain residual leaf tissue and the insect target coexisting in the same field of view.

[0033] Deep learning networks refer to machine learning models based on deep neural network architectures, including but not limited to convolutional neural networks (CNNs), visual transformers (ViTs), and their variants, which can be used to perform visual perception tasks such as semantic segmentation, instance segmentation, or object detection. In this embodiment of the invention, a deep learning network is used to extract and output leaf region masks and / or insect body region masks from a target leaf image.

[0034] Residual leaf region mask: This refers to a binary matrix with the same spatial resolution as the target leaf image. Pixels with a value of 1 represent leaf tissue remaining at the end of the experiment (i.e., the part remaining after being consumed by the insect), while pixels with a value of 0 represent background, missing areas, or other non-leaf objects. This mask is obtained by semantic segmentation of the target leaf image using a deep learning network.

[0035] Insect region mask: This refers to a binary matrix with the same spatial resolution as the target leaf image. Pixels with a value of 1 represent insect bodies (including complete insect bodies, insect remains, molted skin, etc.), while pixels with a value of 0 represent non-insect regions. This mask can be obtained by combining an instance segmentation network or an object detection network with pixel-level segmentation output.

[0036] Mask set operation: refers to pixel-level logical operations performed on two or more binary masks, including but not limited to union (logical OR), intersection (logical AND), difference (logical NAND), and complement (logical NOT). In this embodiment of the invention, mask set operation is used to combine leaf masks, insect masks, support region masks, etc., to construct uncertain region masks or determineable effective regions.

[0037] Region expansion operation: This refers to the morphological expansion of the target region in a binary mask, usually achieved through morphological dilation. This involves sliding a structuring element across the mask, setting the output pixel to 1 when any pixel within the structuring element's coverage area is 1. The region expansion operation is used to extend the occluded area of ​​the insect body outwards by a certain distance, covering the insect's boundary positioning error and uncertain areas when the insect is close to the leaf edge.

[0038] Uncertainty region mask: This refers to a binary mask representing the spatial region where the leaf damage state cannot be reliably determined due to insect occlusion, insect neighbor interference, or uncertainty in the segmentation boundary. This mask typically includes a directly occluded region and a neighbor expansion uncertainty region obtained through a region expansion operation. Within this region, the leaf state (whether healthy, damaged, or exhibiting visible abnormalities) is not included in the denominator or numerator of the damage assessment but is recorded as an independent quality indicator.

[0039] Reference leaf area: This refers to a reference quantity used to characterize the leaf area that a leaf should have before being affected by pests or experimental treatment (or under ideal conditions). This reference quantity can be the pixel area (after scaling) or the actual physical area, and its sources include, but are not limited to: the theoretical area of ​​a standard leaf disc, the segmented area of ​​leaves at the same hole position before insect infestation, and the statistical value of the leaf area of ​​the same batch of insect-free control samples. It can be adjusted in conjunction with a natural deformation correction factor before use.

[0040] Leaf loss area: refers to the area of ​​leaf tissue that has physically disappeared due to feeding by chewing insects. This area is inferred by the calculation relationship between the reference leaf area, the area of ​​the remaining leaf at the endpoint, and the area of ​​the uncertain region, and can be expressed as a conservative estimate (lower bound) or an interval estimate.

[0041] Direct occlusion area mask: This refers to the binary mask corresponding to the direct occlusion range of the insect body projected onto the potential leaf support area. This mask is obtained by performing an intersection operation between the insect body area mask and the leaf support area mask (or a geometric template-type reference area mask), representing the leaf area that might have existed below the insect body but was occluded and could not be directly observed.

[0042] Morphological dilation is a fundamental operation in image morphological processing. It involves sliding a structuring element (such as a circle or rectangle) of a preset shape and size onto a binary mask, setting any output pixel within the area covered by the structuring element that is initially set to 1, thereby expanding the target region outwards. In this embodiment of the invention, morphological dilation is used to expand the insect body region mask to obtain an uncertain neighborhood expansion region.

[0043] Evaluation difference: This refers to the difference obtained by subtracting the remaining blade area and the uncertain area within the support domain from the corrected reference area when calculating a conservative estimate of the missing blade area. This difference is compared with zero, and the larger value is taken as the conservative estimate of the missing blade area.

[0044] Boundary difference: This refers to the difference between the corrected reference area and the remaining leaf area when calculating the upper bound estimate of the missing leaf area. This difference is compared with zero, and the larger value is taken as the upper bound estimate of the missing leaf area.

[0045] Perspective-corrected image: This refers to the image obtained after correcting the perspective distortion of the original acquired image based on a four-point homography transformation on the plate surface (or reference plane). In the corrected image, the rectangular plate surface, which was originally caused by the tilt of the shooting angle, is mapped to an approximate rectangle, and the relative positional relationship of the centers of each hole in the image is closer to the ideal layout under orthographic projection.

[0046] The first deep learning network refers to the deep learning network used for leaf region segmentation. Its input is a target leaf image or a preprocessed normalized image, and its output is a mask of the remaining leaf region. The first deep learning network can adopt an encoder-decoder semantic segmentation architecture. The encoder can use a visual Transformer or a convolutional neural network backbone, and the decoder can use a multi-scale feature fusion structure. For ease of description, in this embodiment of the invention, the network performing leaf segmentation is referred to as the first deep learning network, while the network performing insect detection and segmentation is referred to as the second deep learning network.

[0047] The leaf pest identification and damage assessment method provided in this invention can be executed by a computing device or system with image processing and deep learning inference capabilities. This executing entity includes, but is not limited to, personal computers (PCs), workstations, servers (e.g., cloud servers or local physical servers), embedded edge computing devices (e.g., industrial cameras or edge computing boxes equipped with graphics processing units (GPUs) or neural network processing units (NPUs), and distributed combinations of the above devices. The executing entity is typically configured with at least one processor (CPU), at least one memory (for storing image data, model parameters, and temporary calculation results), and optional acceleration hardware (e.g., GPUs, TPUs, NPUs). At the software level, the executing entity runs an operating system (e.g., Windows, Linux, macOS, or an embedded real-time operating system) and a deep learning inference framework (e.g., PyTorch, TensorFlow, ONNX Runtime, TensorRT, etc.), loads pre-trained deep learning network weights, and sequentially performs image preprocessing, deep learning inference, post-processing operations, and output generation on the input target leaf image. In addition, the executing entity can also communicate with image acquisition devices (such as industrial cameras, document scanners, scanners, or mobile terminals with video recording capabilities) to receive raw images, or directly integrate an image acquisition module.

[0048] In this embodiment of the invention, the executing entity can also be an integrated detection device that combines image acquisition, image processing, and result display functions. The specific form of the executing entity described above does not constitute a limitation on the invention, as long as it can achieve the functional steps defined in this embodiment.

[0049] Figure 1This is one of the flowcharts illustrating the leaf pest identification and damage assessment method provided by the present invention. The method includes: Step 101: Extract the residual leaf region of the target leaf image, wherein the target leaf image contains the leaf to be evaluated and the insect body.

[0050] In this step, the computing device first needs to acquire an image of the target leaf that meets the analysis requirements and then accurately extract the region of the remaining leaf from it. In actual high-throughput experimental scenarios, this process specifically includes several sub-modules such as image acquisition, standardization processing, and deep learning-based region extraction.

[0051] First, in the target blade image acquisition stage, the computing device receives an original image of the experimental plate containing multiple holes (e.g., a top-down image of a 6-well, 12-well, 24-well, or 48-well plate), preferably with a resolution of no less than 2048×1536 pixels and an RGB three-channel color space. Before performing perspective correction, to eliminate nonlinear deformation of the edge holes caused by close-up shooting, the computing device preferentially performs lens distortion correction on the experimental plate image based on camera calibration parameters or prior geometric mesh features of the experimental plate (such as the linear features of the longitudinal and transverse partitions on the plate surface). Subsequently, the hole location module (e.g., based on Hough circle transform and row and column topological constraints) identifies the position of each hole in the experimental plate image, and performs perspective correction based on the plate surface plane to obtain a perspective-corrected image. Finally, based on the identified hole positions, the perspective-corrected image is cropped into circular holes, and the scale parameters during the cropping and scaling process are recorded. A standardized, approximately orthogonalized image of a single hole is output as the target blade image.

[0052] Secondly, in the image standardization stage, to reduce the interference of lighting and color differences between different shooting batches on subsequent network recognition, the computing device performs lighting normalization and color normalization processing on the target leaf image. Lighting normalization specifically involves converting the image to the Lab color space, performing a large-scale morphological closing operation on the luminance (L) channel to estimate the illumination field, and subtracting this slowly varying component to compensate for uneven lighting. Adaptive histogram equalization can then be performed to enhance contrast. Color normalization specifically involves evaluating non-leaf background areas in the image (such as agar culture medium areas), and performing empirical global chromaticity shift normalization based on the difference between the mean chromaticity of this background area and the batch-level reference value. After processing, the image is converted back to the RGB color space.

[0053] Finally, in the stage of extracting the residual leaf region, the computing device inputs the standardized target leaf image into a first deep learning network for feature extraction and segmentation, outputting a preliminary residual leaf region. To improve the network's ability to distinguish between healthy leaves, damaged leaves, and the background, the input features of the first deep learning network include not only the red, green, and blue (RGB) channels of the target leaf image, but also the saturation (S) and lightness (V) channels in the hue-saturation-lightness (HSV) space obtained by converting the RGB channels. The first deep learning network adopts an encoder-decoder semantic segmentation architecture, and its training loss function uses a weighted combination of pixel-level cross-entropy loss and region overlap (Dice) loss, enabling the network to accurately depict the broken edges of the damaged leaf. In addition, the computing device can also extract the contour morphology features (such as roundness and convexity) and brightness distribution features (such as brightness standard deviation and gradient magnitude) of the leaf to be evaluated from the extracted residual leaf region, and determine the degree of curling of the leaf to be evaluated accordingly. When the degree of curling exceeds a set threshold, a quality mark requiring manual review is generated in the final leaf damage assessment result.

[0054] Step 102: Identify the insects in the target leaf image and construct an uncertain region characterizing the range of insect occlusion and interference.

[0055] Because live insects (such as green lepidopteran larvae) are highly similar in color and texture to healthy leaves in high-throughput experiments, the aforementioned first deep learning network is prone to misclassification (i.e., the pseudo-leaf effect), and insects lying on the leaves can cause visual occlusion. Therefore, this step aims to independently identify insects, eliminate misclassifications, and scientifically quantify the range of occlusion interference.

[0056] First, the computing device synchronously inputs the target leaf image into a second deep learning network (such as an instance segmentation network or an object detection network) to perform insect detection and outputs the region corresponding to the insect. In the low-level computer processing, the preliminary residual leaf region is represented as a preliminary residual leaf region mask, and the identified region corresponding to the insect is represented as an insect region mask.

[0057] To address the false leaf effect, the computing device performs an overlap check on the preliminary residual leaf region based on the region of the corresponding insect (i.e., the insect region mask). The portion of the preliminary residual leaf region that overlaps with the region of the corresponding insect (e.g., the insect misjudgment region that appears as an isolated connected domain) is removed, resulting in the final residual leaf region after error correction (i.e., the residual leaf region mask). This effectively separates the real leaf tissue from the insect with similar color at the pixel level.

[0058] Subsequently, the computing device performs mask set operations and region expansion operations based on the insect body region mask and the final residual leaf region mask to construct an uncertain region mask (i.e., the corresponding uncertain region). The specific construction process is as follows: The first step is to perform local hole filling, closure operation, or convex hull repair on the residual leaf area mask within the local neighborhood of the insect body to obtain a continuous leaf support area mask.

[0059] The second step involves performing an intersection operation between the insect body region mask and the leaf support region mask (or a pre-obtained geometric template-type reference region mask) to obtain the directly occluded region mask. This region clearly defines the area of ​​the leaf that might have existed under the projection of the insect body but is directly occluded.

[0060] Third, considering that the insect may be feeding on the edge of the leaf and there may be boundary positioning error, a morphological dilation operation (expanding outward by a preset pixel distance) is performed on the insect area mask to obtain a neighborhood expansion uncertain area mask.

[0061] Fourth, the union of the directly occluded region mask and the neighborhood extended uncertain region mask is taken to obtain the uncertain region mask, and the region corresponding to this mask is constructed as the uncertain region. This region will be isolated in subsequent calculations to avoid evaluation bias caused by forcibly assuming it to be healthy or missing.

[0062] Step 103: Based on the pre-acquired reference blade area, the residual blade area corresponding to the residual blade region, and the uncertain area corresponding to the uncertain region, determine the missing blade area, and generate a blade damage assessment result based on the missing blade area.

[0063] The core damage phenotype of chewing pests is the physical loss of leaves due to being eaten. This step overcomes the limitation of relying solely on abnormalities on the surface of residual tissue, and makes a reasonable inference of the area of ​​physical loss based on a reference benchmark.

[0064] First, the computing device obtains the pre-acquired reference blade area (A). ref The acquisition process includes: constructing a standardized template (such as a theoretical circular template generated by a standard leaf disc of known diameter) deterministically using known geometric shapes, and using the area corresponding to the standardized template as a pixel-level reference leaf area; or, obtaining a target area as an area-level reference leaf area, which may be derived from the leaf segmentation area in the same hole position image before insect release (Day 1), or the endpoint statistical area of ​​the same batch of insect-free control samples.

[0065] Before inferring the missing area, considering the water loss and shrinkage during the detached leaf culture process, the computing device estimates the area deformation correction coefficient (η) by combining the area change data of the insect-free control group, and corrects the pre-acquired reference leaf area based on this coefficient to obtain the corrected reference area (A). ref_corr =A ref ×η).

[0066] Subsequently, the computing device calculated the final residual blade area (A). leaf ) and the uncertain area (A) within the supporting domain uncertain_support The residual blade area and the uncertain area within the support domain are subtracted from the corrected reference area to obtain the evaluation difference. This evaluation difference is compared with zero, and the larger value is determined as the missing blade area (A). missing_min This value is the conservative estimate of the missing area after eliminating the uncertainty of occlusion. Optionally, the computing device also directly subtracts the residual leaf area from the corrected reference area to obtain the boundary difference, and takes the larger value between this and zero as the upper bound estimate of the missing area (A). missing_max ), together they constitute the output of the missing proportion range.

[0067] As a supplementary evaluation method, for visible damage on residual blades (such as mechanical edge yellowing or necrosis), the computing device performs a difference operation between the residual blade region and the uncertain region to determine the identifiable valid blade region. When the area of ​​the identifiable valid blade region is greater than a preset minimum threshold, visible abnormality features are identified in the pixels within the residual blade region, the visible abnormality region is output, and the visible abnormality ratio is calculated.

[0068] Finally, the computing device generates a structured leaf damage assessment report based on data such as the determined leaf missing area, the proportion of visible anomalies, and the percentage of uncertain areas. When a geometric template-type reference area mask is available, the approximate spatial location of the missing area can also be marked on the visualization overlay for intuitive reference. This assessment effectively overcomes the problem of extremely low pest damage rates caused by ignoring physical defects in existing technologies, meeting the reliability requirements of high-throughput quantitative assessment.

[0069] Through steps 101 to 103 above, the leaf pest identification and damage assessment method provided in this embodiment of the invention achieves end-to-end automated processing from input image to damage mass production, overcomes the inherent defects of the prior art in chewing pest scenarios, and significantly improves the accuracy, repeatability and automation level of damage assessment.

[0070] Furthermore, in this embodiment, step 101, the step of obtaining a target leaf image containing the leaf to be evaluated and the insect, further includes: First, the computing device receives the original top-down image of the perforated plate as input, denoted as... I raw ∈R H0×W0×3 ; Its resolution is preferably no less than 2048×1536 pixels, and the color space is RGB three-channel. The final output of this module includes: a list of center coordinates of each hole {(x i ,y i )|i=1,…,N} (where N is the total number of holes), and the cutting radius r of each hole. i Single-hole approximate orthogonalized image sequence {S i ∈R Hs×Ws×3 |i=1,…,N} and the scale parameter k corresponding to each single-hole image i .

[0071] Before performing perspective correction, to eliminate nonlinear distortion of edge holes caused by close-up shooting (e.g., camera distance 30-80cm from the board surface), the computing device prioritizes processing the original image I. raw Lens distortion correction is performed, specifically including: when the camera calibration parameters are known, directly performing distortion correction processing on the original image based on the intrinsic parameter matrix K and the distortion coefficient vector D to obtain the distortion-corrected image I. undist The calculation formula is: I undist =undistort(I raw ,K,D); When camera calibration parameters are unavailable, the regular geometric structure of the perforated plate surface is used as a self-calibration prior to extract the elliptical parameters of the outer contour of the hole and the linear features of the longitudinal and transverse partitions between the holes. Nonlinear optimization is performed based on geometric constraints to estimate the radial distortion coefficient and perform correction. For shooting conditions where the degree of distortion is negligible (such as when using a telecentric lens), this step can be skipped.

[0072] Subsequently, the distortion-corrected image is converted into a grayscale image I. gray Gaussian blur is applied to suppress noise. Next, Hough circle transform is performed to detect circular holes, and non-maximum suppression (NMS) is applied to the detected circles to remove overlap, outputting a preliminary set of circle centers. Holes are sorted and their integrity is checked according to the row and column topology constraints of the perforated plate specifications. If any holes are missed, they are filled in by spatial interpolation of adjacent known holes.

[0073] Next, the computing device performs perspective correction based on the plane of the experimental board in the image of the experimental board. Specifically, the hole positions at the four corners of the perforated board are selected from the sorted set of hole positions as the perspective calibration point set P. src ={p1,p2,p3,p4}; Calculate the target coordinate set P under ideal orthogonal projection based on the physical dimensions of the plate. dst={q1,q2,q3,q4}, so that the holes are evenly spaced; solve the 3×3 perspective transformation matrix H by the correspondence of the four points, that is, H=getPerspectiveTransform(P src ,P dst Finally, a perspective transformation is performed on the distorted image to obtain the corrected image I. corrected The calculation formula is: I corrected =warpPerspective(I undist H).

[0074] It should be noted that the perspective transformation matrix H is strictly applicable to the plane of the plate. For blade objects located at the depth of the hole bottom (with a depth difference Δd from the plate surface), residual perspective deviation exists. This deviation is proportional to Δd / Z (where Z is the camera working distance). Under typical shooting conditions (e.g., Z≥300mm, Δd≤20mm), the deviation magnitude Δd / Z≤7%, therefore I can be used as a reference. corrected It is considered an approximately orthogonalized image for area statistics.

[0075] Finally, for the corrected image I corrected Perform hole cutting and dimensional adaptation. Using the center coordinates of each hole as the center and the cutting radius r as the radius... i Perform circular cropping, and denot the size of the original single-hole image obtained by cropping as H. crop ×W crop To accommodate the input size requirements of subsequent deep learning networks (such as target size H) target ×W target ), scale it to the target size, and strictly record the scaling ratios in the horizontal and vertical directions: k h =H crop / H target k w =W crop / W target .

[0076] In the final area calculation, the pixel area needs to be multiplied by the corresponding scaling factor (k). h ×k w This restores the physical pixel scale of the corrected image, ensuring that the correspondence between the area and the actual scale is not distorted due to network input adaptation.

[0077] Furthermore, the output of the single-aperture approximately orthogonalized image (denoted as S) i After using an RGB three-channel color space, to reduce lighting and color differences between different shooting batches and improve the consistency of input to subsequent deep learning perception modules, the computing device processes the image S... i Perform illumination and color normalization processing, and output the normalized single-aperture image S.i Specifically, it includes the following detailed calculation mechanisms: The illumination normalization process compensates for illumination unevenness by estimating the illumination field and subtracting the slowly changing component. The specific steps are as follows: First, input the RGB three-channel image S i Converted to CIE Lab color space, the L channel (luminance) and a channel (a) are separated. ∗ Channel and b ∗ aisle; Next, a large-scale morphological closing operation is performed on the L channel to estimate the illumination field. The calculation formula is: L bg =morphClose(L,kernel=disk(R)), where R is a disk structure element with a large radius (preferably, R=50 pixels), thereby obtaining an estimate of the spatially slowly varying illumination distribution; Subsequently, the luminance channel after illumination compensation is calculated using the following formula: L comp =LL bg +mean(L bg This operation, while subtracting the slowly changing spatial illumination components, strictly maintains the global average brightness. Finally, the compensated luminance channel L comp Adaptive histogram equalization (CLAHE) is further performed to effectively suppress noise amplification while enhancing local contrast.

[0078] The color normalization process performs empirical colorimetric normalization based on the mean of the background region. The specific steps are as follows: For the separated a ∗ and b ∗ The system automatically evaluates the area percentage of non-leaf background regions (such as culture medium areas or plate surfaces) in a single-well image. When the background region has a sufficient area (e.g., more than 10% of the total image area) and its color distribution is relatively stable, the computing device extracts the a value of that background region. ∗ and b ∗ The channel mean is calculated, and its difference from the preset batch-level reference value is determined, thus affecting a. ∗ and b ∗ The channels undergo a global mean shift. This first-order statistical normalization operation makes the background chromaticity statistics between different batches more consistent. When this prerequisite is not met or the background area is insufficient, the color normalization process degenerates into retaining only the aforementioned luminance normalization process, or directly using preset batch-level chromaticity reference parameters. After completing the above luminance and chromaticity channel processing, the normalized Lab image is converted back to the RGB color space to obtain the final normalized output image S. i ′.

[0079] As an optional method for fine calibration, when experimental conditions permit, the computing device can also automatically detect the color card area from the image by placing a standard color card (such as a 24-color standard color card) within the field of view, extracting the measured Lab values ​​of each color patch and comparing them with the standard reference values ​​of the color card, thus adjusting the a... ∗ and b ∗ The channel establishes a polynomial regression correction model to achieve lighting and color compensation that more closely approximates real physical colors.

[0080] In another specific embodiment of the present invention, before identifying the positions of each hole in the experimental board image through the hole location module and performing perspective correction, the computing device further includes a step of performing lens distortion correction on the experimental board image. Specifically, when the camera calibration parameters (including the intrinsic parameter matrix and distortion coefficients) are known, the computing device directly performs distortion correction processing on the original image based on the calibration parameters; when the camera calibration parameters are unavailable, the computing device uses the regular geometric structure of the experimental board surface itself as a self-correction prior, extracts the longitudinal and transverse partitions or grid straight line features in the board surface, performs nonlinear optimization based on the geometric constraint that "the regular grid should remain straight and equally spaced under distortion-free projection," estimates the radial distortion coefficient, and performs correction. This lens distortion correction step can eliminate the positional offset and shape deformation of edge holes during close-up shooting, providing a more accurate input image for subsequent hole location and perspective correction.

[0081] In addition, after outputting a standardized, approximately orthogonalized single-well image as the target leaf image, the computing device also performs illumination normalization and color normalization processing on the target leaf image. The illumination normalization processing compensates for uneven illumination by estimating the illumination field and subtracting slowly varying components: specifically, the computing device converts the single-well image to the CIELab color space, performs a large-scale morphological closing operation on the L channel to estimate the illumination background field, then subtracts this background field from the original L channel and adds the global mean, thereby eliminating spatially varying differences in illumination distribution. Optionally, adaptive histogram equalization is then performed to enhance local contrast. The color normalization processing performs empirical chromaticity normalization based on the background region mean: the computing device filters out non-leaf background regions (such as culture medium regions) in the single-well image, extracts the mean values ​​of these regions in channels a and b, compares them with batch-level reference values, and performs mean shifting to make the background chromaticity statistics between batches more consistent; when the background region is insufficient or the conditions are not met, this step can degenerate into performing only luminance normalization.

[0082] Furthermore, this paper details the specific implementation process for extracting residual leaf regions from target leaf images and identifying insects within them. Since live insects (especially young green lepidopteran larvae) remaining in high-throughput experimental wells often closely resemble healthy leaves in color and texture, conventional algorithms are prone to misjudging due to the "pseudo-leaf effect." Therefore, this embodiment employs a dual-branch deep learning network combined with a cross-elimination mechanism to ensure the purity of region extraction.

[0083] First, the residual leaf region of the target leaf image is extracted. The specific process includes: the computing device inputs the target leaf image into the first deep learning network for feature extraction and segmentation, and outputs the preliminary residual leaf region.

[0084] To maximize the network's ability to distinguish between healthy leaves, damaged tissue, and complex backgrounds, the first deep learning network employs a targeted multi-channel fusion design for its input features: its input features not only include the original red, green, and blue (RGB) channels of the target leaf image, but also the saturation (S) and lightness (V) channels from the hue-saturation-lightness (HSV) color space converted from the RGB channels. By concatenating these five channels to form a 5-channel input tensor, the network can more sensitively capture subtle differences in saturation and lightness distribution between the leaf region and the agar medium or insect body.

[0085] During the model training phase of the first deep learning network, its training loss function employs a weighted combination of pixel-level cross-entropy loss and region overlap (Dice) loss. The cross-entropy loss optimizes the accuracy of pixel-level classification, while the region overlap loss specifically optimizes the integrity of the overall region. This allows the network to output mask results with extremely high boundary fit even when faced with complex morphologies where severely damaged leaf edges are extremely fragmented. However, it's important to note that due to the pseudo-leaf effect, the output at this stage represents a preliminary residual leaf region, which may incorrectly include some insect pixels of similar color.

[0086] The first deep learning network adopts an encoder-decoder semantic segmentation architecture based on deep neural networks, which specifically includes the following detailed computation and processing mechanisms: First, to enhance the network's ability to distinguish between healthy leaves, damaged tissue, and the background when constructing network input features, the computing device introduces the saturation (S) and brightness (V) channels of the HSV color space as supplementary inputs, based on the RGB three channels of the standardized single-hole image. Through channel concatenation, a 5-channel input tensor X=[R,G,B,S,V]∈R is constructed. H×W×5 , where H=W is the spatial size of the network input.

[0087] During the network initialization phase, if the encoder uses ImageNet-based pre-trained weights, the computing device loads the pre-trained weights into the first three channels corresponding to RGB, while the input layer weights corresponding to the newly added S and V channels are randomly initialized or zero-initialized; or a 1×1 convolutional input adaptation layer is set at the front end of the encoder to map the 5-channel features to 3 channels before connecting them to the pre-trained encoder.

[0088] Subsequently, the input tensor X is fed into the encoder of the first deep learning network. The encoder employs a pre-trained visual Transformer backbone network or a convolutional backbone network, extracting multi-scale hierarchical features through multiple downsampling stages, with each stage outputting a multi-scale feature map F. enc (l) (l=1,…,L). Next, the decoder adopts a multi-scale feature fusion structure, which fuses high-level semantic features and low-level detail features step by step through a top-down path and restores them to the original resolution. Finally, the classification head outputs a pixel-by-pixel predicted probability map, denoted as P. leaf ∈[0,1] H×W This probability map represents the posterior probability that each pixel belongs to the residual leaf category.

[0089] Next, the computing device binarizes the probability map based on a set segmentation threshold to generate the final residual leaf region mask M. leaf Its generation formula is: M leaf =1(P leaf >τ leaf ).

[0090] Where, τ leaf To determine the threshold (preferably, τ) leaf =0.5), 1(·) is an indicator function.

[0091] Based on this mask, the computing device can directly calculate the actual visible residual leaf pixel area A at the end of the experiment. leaf The calculation formula is: A leaf =∣M leaf |=∑(x,y)M leaf (x,y).

[0092] Furthermore, during the model training phase of the first deep learning network, in order to simultaneously optimize pixel-level classification accuracy and region-level overlap, its training loss function L... seg Using pixel-level binary cross-entropy loss L CE And regional overlap loss L Dice The weighted combination. The specific loss function is defined as follows: The total loss function is: L seg =λ1·LCE +λ2·L Dice .

[0093] The pixel-level binary cross-entropy loss formula is as follows: ; In the formula, y(x,y) is the true label of the pixel, and p(x,y) is the network prediction probability.

[0094] The formula for regional overlap loss (Dice loss) is: In the formula, To prevent smooth terms with a denominator of zero, for example, take =1e-5. Preferably, both the weight hyperparameters λ1 and λ2 are set to 1.0. In an optional implementation, the total loss function may further incorporate a boundary-aware loss term L based on boundary distance transformation. boundary The total loss at this point is: , By designing this joint loss function, the network can still output masking results with extremely high boundary fit even under complex morphologies where severe damage to the leaf edges leads to broken leaf edges.

[0095] Secondly, for insect identification in the target leaf image, the specific process includes: inputting the target leaf image into a second deep learning network, whereby the second deep learning network employs an instance segmentation architecture or a target detection architecture with instance segmentation capabilities, and the network output is a set of detected insect instance information, denoted as {(bbox)}. j ,cls j score j ,mask j ) |j=1,…,K}. Where: bbox j ∈R 4 To detect bounding box coordinates, cls j For category labels (including insect targets such as complete insect bodies, remains, molted skin, etc.), score j ∈[0,1] represents the confidence score, and mask j ∈{0,1} H×W This is the pixel-level binary mask for this instance.

[0096] Subsequently, the computing device is based on the set confidence threshold τ det Take the union of all instances of the mask identified as insect targets to generate a unified insect region mask M. insect Its calculation formula is: M insect =∪ j:clsj=insect,scorej>τdet mask j .

[0097] Mask M for obtaining the insect body region insect Subsequently, to completely eliminate the pseudo-leaf effect, the computing device performs overlap screening and removal operations on the preliminary residual leaf regions based on the corresponding insect body regions: obtaining the preliminary mask (denoted as M) corresponding to the preliminary residual leaf regions output by the first deep learning network. leaf_initial ), M leaf_initial Mask M in the middle of the insect body area insect Overlapping pixel values ​​are forcibly set to 0, thereby eliminating falsely identified areas and obtaining the final residual leaf area mask (denoted as M) corresponding to the residual leaf area. leaf Its underlying calculation formula can be expressed as: M leaf = M leaf_initial - (M leaf_initial ∩ M insect ).

[0098] After obtaining an absolutely pure mask M of the residual leaf area leaf M, the mask for the insect body area insect The system then constructs the uncertain region through explicit pixel-level set operations, as follows: The first step is to construct a mask for the directly obscured area corresponding to the insect body. To accurately quantify the potential leaf area obscured by the insect body, the computing device first applies a mask M to the remaining leaf area. leaf By performing local hole-filling repair, closure operation, or local convex hull repair within the local neighborhood of the insect body, a continuous leaf support area mask M is obtained. support (When there is a pre-obtained known geometric template type reference region mask M) ref At that time, M can also be taken directly. support = M ref Subsequently, the insect body region is masked with M. insect With the blade support area mask M support Perform intersection operations to obtain the mask M for the directly occluded region. occ_direct The calculation formula is as follows: M occ_direct = M insect ∩ M support .

[0099] This area clearly defines the actual leaf area that might have existed under the projection of the insect body, but was directly physically obscured and could not be observed.

[0100] The second step involves constructing a mask for the uncertain region of the insect's neighborhood. Considering that the insect may be feeding on the edge of a leaf (at which point there may be uneaten leaf fragments or forming gaps below the insect), and that the insect mask output by the network has certain boundary errors, the computing device performs a morphological dilation operation on the insect region mask, further expanding the uncertain region from the directly occluded area to the insect's nearest neighbor, resulting in a neighborhood-extended uncertain region mask M. occ_expand The calculation formula is as follows: M occ_expand =[Dilate(M insect kernel=disk(r expand ))∩(M support ∪N boundary (M support ,d boundary ))-M occ_direct ]-M confirmed_healthy .

[0101] Where Dilate is the image morphological dilation operation, r expand The set expansion radius (preferably a pixel value much smaller than the aperture radius to ensure the expansion range is limited to the vicinity of the insect); N boundary (M support, d boundary The outer width of the support region is d. boundary The neighborhood band of a pixel; M confirmed_healthy This is a mask for definitively healthy leaf regions far from insects, identified through methods such as connected component analysis. This expansion operation can reasonably incorporate uncertain areas where insects are close to leaf margin notches, while also subtracting M... confirmed_healthy Avoid incorrectly marking confirmed leaf areas far from insects as uncertain.

[0102] The third step is to synthesize a total mask for the uncertain region and define the area of ​​the blades that can be determined as valid.

[0103] The computing device will mask the directly obscured area M occ_direct With the neighborhood extended uncertain region mask M occ_expand Taking the union of the sets yields the mask M for the uncertain region. uncertain The formula is: M uncertain =M occ_direct ∪M occ_expand .

[0104] Subsequently, the computing device masks the uncertain region M. uncertainThe corresponding spatial pixel matrix is ​​constructed as the uncertain region characterizing the extent of insect occlusion and interference. This region encompasses the entire area where the true damage state of the leaf cannot be reliably determined due to insect occlusion and the physical influence of its neighborhood.

[0105] Meanwhile, to support subsequent optional supplementary evaluation paths for visible anomalies, the computing device will evaluate the residual blade region (corresponding to mask M). leaf ) and the uncertain region (corresponding to mask M) uncertain The difference set is used to determine the determinate effective blade region (corresponding to the determinate effective blade region mask M). valid_visible The formula is: M valid_visible = M leaf - M uncertain .

[0106] Within this effective area, the leaf tissue is fully visible and unobstructed by insects or their surroundings, providing an extremely pure regional constraint for subsequent reliable identification of visible abnormalities (such as mechanical lesions, yellowing, etc.).

[0107] Finally, to completely eliminate the area overestimation caused by the pseudo-leaf effect, the system performs an error correction and reorganization process: the computing device performs pixel-level overlap checks on the preliminary residual leaf regions output by the first deep learning network based on the corresponding insect body region output by the second deep learning network. Specifically, the system performs explicit set logic operations to remove the parts of the preliminary residual leaf regions that overlap with the corresponding insect body regions (i.e., isolated connected regions or intersecting pixels that were misidentified as leaves by the first network but are actually insect bodies). After the above cross-filtering and removal operations, all interfering pixels containing insect body features are removed, and the final residual leaf regions are obtained.

[0108] This embodiment introduces saturation and brightness channels through a first deep learning network, enhancing the network's ability to distinguish between leaves and the background. The combination of cross-entropy and Dice loss ensures that the segmentation results have both pixel accuracy and region integrity. The second deep learning network achieves instance-level detection and segmentation of the insect, providing accurate insect location and contour information for subsequent construction of uncertain regions.

[0109] Furthermore, in some optional embodiments, because detached leaves are prone to three-dimensional non-rigid deformations such as edge curling, central bulging, or overall warping after being eaten by insects or losing water during cultivation, their two-dimensional projected area deviates from their actual flattened area. To ensure the reliability of the final quantitative data, after completing the above mask extraction, the computing device also includes a process of curling state recognition and data quality assessment, specifically including the following detailed feature extraction and judgment mechanisms: First, the computing device uses the target blade image and the mask M of the residual blade region. leaf In the process, the multidimensional quality characteristics of the residual leaves are extracted, specifically including: (1) Edge morphological characteristics: Calculate the circularity and solidity of the residual blade profile. The formula for calculating circularity is: Circularity = 4π·Area / Perimeter 2 The formula for calculating convexity is Solidity = Area / ConvexHullArea (i.e., the ratio of the residual area to its minimum convex hull area), and it is combined with the degree of curvature change of the profile to quantify abnormal wrinkles at the blade edge.

[0110] (2) Brightness distribution characteristics: The target leaf image is converted to the Lab color space, and the pixel standard deviation σ of the L channel (brightness) is calculated in the residual leaf area. L And the mean gradient magnitude mean(∣▽L∣). The physical basis is that when the blade undergoes three-dimensional curling, the inconsistent orientation of its surface normals leads to severe differences in light reflection, resulting in a significant increase in the variance of the brightness distribution and the local gradient.

[0111] (3) Characteristics of the proportion of uncertain areas: In order to quantify the assessment risk caused by insect occlusion, the uncertain area A within the support domain is calculated. uncertain_support (i.e. |M) uncertain ∩M support The relative proportion of | is calculated using the following formula: UncertaintyRatio=A uncertain_support / max (A leaf +A uncertain_support , ) in, To prevent extremely small positive numbers with a denominator of zero.

[0112] Subsequently, the computing device, based on the extracted contour morphology features, brightness distribution features, and uncertain region proportion features, uses a preset rule threshold or lightweight classification network to determine the degree of curling and occlusion of the residual leaf. For example, the samples are classified into states such as flat, slightly curled, moderately curled, or severely curled / unreliable.

[0113] When the degree of curling exceeds a set threshold (e.g., it is determined to be severe curling), or when the UncertaintyRatio exceeds a preset risk threshold (e.g., 20%), the computing device adds a quality warning mark requiring manual review to the final generated blade damage assessment result.

[0114] It is important to note that, due to the limitations of information loss at the physical level in single-view 2D overhead images, the true area and shape of the curled-up portion cannot be recovered from a single 2D image. Therefore, the curling state recognition process is only used to output the quality assessment status, not to output quantitative coefficients for area correction. This system proactively abandons the introduction of unreliable quantitative mathematical compensation and instead adopts an objective hierarchical labeling mechanism to maximize the scientific rigor and objectivity of the high-throughput quantitative assessment results.

[0115] In one specific embodiment of the present invention, after extracting the residual leaf region of the target leaf image, the computing device further performs the following quality assessment operation: The computing device extracts the contour morphology features and brightness distribution features of the leaf to be evaluated from the target leaf image and the residual leaf region. The contour morphology features include, but are not limited to, the roundness of the residual leaf region (4π × area / perimeter). 2 The features include convexity (area / convex hull area) and the degree of change in contour curvature; brightness distribution features include converting the target leaf image to the Lab color space and calculating the pixel standard deviation and the mean gradient amplitude in the L channel of the residual leaf area. These features can reflect the three-dimensional deformation of the leaf caused by water loss or mechanical damage, such as curling and warping.

[0116] Subsequently, the computing device determines the degree of curling of the blade to be evaluated based on the contour morphology features and the brightness distribution features. For example, when the roundness of the blade area deviates significantly from 1, the convexity is significantly less than 1, the contour curvature changes drastically, and the standard deviation of the L channel and the mean gradient amplitude exceed the corresponding thresholds, it is determined that there is obvious curling. The computing device can classify the degree of curling into levels such as flattened, slightly curled, moderately curled, and severely curled.

[0117] When the degree of curling exceeds a set threshold (e.g., determined to be severe or moderate curling or above), the computing device generates a quality marker requiring manual review in the leaf damage assessment results. This quality marker is output as an independent field, indicating to researchers that the projected area of ​​the sample deviates significantly from the actual flattened area due to leaf curling, and that the automatic quantitative result is for reference only, suggesting verification in conjunction with manual observation. It should be noted that this embodiment only outputs the quality grade and prompts, and does not output the quantitative coefficients used for area correction, because a single overhead image cannot recover the true area of ​​the curled-obscured portion.

[0118] Furthermore, in a specific embodiment of the present invention, step 102 specifically includes: First, prepare the basic mask data. The computing device obtains the residual leaf region mask corresponding to the residual leaf region (i.e., the absolutely pure residual leaf mask after eliminating the misjudgment of insects, denoted as M). leaf), and obtain the mask of the insect body region corresponding to the identified insect body (denoted as M). insect These two masks form the basis for the binarization of subsequent spatial operations.

[0119] Secondly, the potential support base for the leaf is determined. The computing device performs localized hole-filling repair on the mask of the remaining leaf area to obtain the leaf support area mask. Specifically, the computing device only masks the insect body area M. insect Within the covered local neighborhood, the mask M covers the residual blade area. leaf The system performs morphological closing operations or contour-based hole filling, and locally repairs the convex hull of depressions at the leaf edges. Through this operation, the system simulates the physical base of the leaf that should be continuous beneath the insect body, generating a mask for the leaf support area (denoted as M). support ).

[0120] Next, the physical absolute occlusion is defined. The computing device performs an intersection operation on the insect body region mask and the leaf support region mask or a pre-acquired geometric template-type reference region mask to obtain the direct occlusion region mask. Specifically, when the experiment uses a standard perforator to prepare leaf discs with known geometry, the system can directly call the pre-constructed geometric template-type reference region mask (denoted as M). ref In other natural leaf scenarios, the M obtained by filling and repairing holes as described above is used. support Combine it with the insect body area mask M insect Find the intersection: M occ_direct = M insect ∩ M support (or M) ref The intersection operation (M) occ_direct It clearly and conservatively defines the leaf area that must exist under the vertical projection of the insect body but is physically obscured and cannot be observed.

[0121] Subsequently, the uncertainties arising from the boundary and feeding behavior are addressed. Considering that the insect may be feeding on the edge of a leaf (at which point there may be uneaten leaf fragments or forming gaps in the insect's vicinity), and that the insect mask output by the deep learning network itself has a certain pixel-level boundary positioning error, the computing device performs a morphological dilation operation on the insect region mask to obtain a neighborhood expansion uncertainty mask. Specifically, a circular structural element with a preset radius (preferably 3 to 10 pixels to ensure that the expansion range is limited to the insect's vicinity) is selected to expand the insect region mask M. insect Expanding outwards, the expanded area intersects with the outer perimeter of the leaf support area, excluding the directly obscured portion and the healthy leaf area clearly far from the insect, thereby accurately extracting the mask of the uncertain neighborhood expansion area (denoted as M). occ_expand ).

[0122] Finally, the final interference boundary is generated through fusion. The computing device takes the union of the directly occluded region mask and the neighborhood extended uncertain region mask to obtain the uncertain region mask, and constructs the uncertain region corresponding to the uncertain region mask. The comprehensive set operation process is as follows: M uncertain =M occ_direct ∪M occ_expand At this point, the computing device has masked the uncertain region M. uncertain The spatial pixel matrix mapped in the two-dimensional coordinate system of the image formally delineates and constructs the uncertain region characterizing the extent of insect occlusion and interference. This region will be extracted and isolated separately in the subsequent leaf area inference, thereby avoiding the fatal flaw in the prior art of misjudging the occluded area as healthy tissue or true absence, and greatly improving the scientific rigor and accuracy of the quantitative assessment.

[0123] In one specific embodiment of the present invention, the reference blade area is obtained in the following manner.

[0124] One method is to use a computing device to definitively construct a standardized template based on known geometry, and then use the area corresponding to this standardized template as the reference blade area. Specifically, when a standard punch is used to prepare a blade disc, the diameter of the punch is known (e.g., 2 cm), and the theoretical area of ​​the blade disc can be directly calculated using the formula for the area of ​​a circle. In a calibrated image coordinate system, the computing device can definitively construct a circular template based on the center coordinates of the punch and known physical dimensions; the theoretical area corresponding to this template is the reference blade area. This method is suitable for scenarios where the initial blade morphology can be determined during the experimental design phase.

[0125] Another method of obtaining the target area is: the computing device acquires the target area and uses the target area as the reference blade area. The target area can be any of the following sources: 1) Leaf segmentation area in the same hole image before insect release: The computing device performs the same geometric correction and leaf segmentation process on the same hole image collected on the experimental start day (e.g., Day 1) as on the end image, extracting the pixel area of ​​the leaves before insect release, which is then scaled back and used as the reference leaf area. It should be noted that, since detached leaves undergo non-rigid deformation during cultivation, there is no reliable pixel-level spatial correspondence between the leaf contour before insect release and the leaf contour in the end image. Therefore, this acquisition method only extracts the area value as a reference, and does not use its contour as a pixel-level reference mask.

[0126] 2) Theoretical area of ​​standard leaf disc: Similar to the first method, the theoretical area is calculated directly based on the known diameter.

[0127] 3) Statistical area of ​​control samples without insects in the same batch: The computing device collects leaf images of the endpoints of the control wells without insects in the same experimental batch and performs segmentation. The mean or median of the leaf area of ​​each control well is taken as the statistical estimate of the initial leaf area of ​​the batch.

[0128] In a specific embodiment of the present invention, step 103 is implemented in the following manner: First, the computing device estimates the area deformation correction coefficient by combining the area change data of the insect-free control group. Specifically, detached leaves will naturally deform during the culture process due to water loss, shrinkage, or expansion. Even in the control group without insect inoculation, the final leaf area may differ from the initial area. The computing device obtains the initial area (e.g., Day 1 segmentation area) and final area (e.g., Day 7 segmentation area) of the same batch of insect-free control group leaves under the same culture conditions, calculates the area change ratio (final area / initial area) for each control well, and takes the mean or median as the area deformation correction coefficient η. Then, the computing device corrects the pre-obtained reference leaf area based on this correction coefficient to obtain the corrected reference area: A ref_corr = A ref × η. When data from the insect-free control group is unavailable, η is set to 1, meaning no correction is made, and this is noted in the results.

[0129] Next, the computing device obtains the uncertain area within the support domain. This uncertain area is the pixel area of ​​the intersection between the uncertain region and the leaf support region (or the geometric template-type reference region), representing the area within the leaf support range that cannot be reliably determined due to occlusion.

[0130] Then, the computing device subtracts the residual blade area and the uncertain area within the support domain from the corrected reference area to obtain the evaluation difference: Evaluation difference = A ref_corr -A leaf - A uncertain_support .

[0131] Finally, the computing device compares the evaluation difference with zero and takes the larger value as the missing area of ​​the leaf: A missing = max(evaluation difference, 0). This approach ensures that the missing leaf area is non-negative and provides a conservative estimate assuming that leaves exist below the uncertain region.

[0132] In addition, leaf damage assessment results are generated based on the missing leaf area, specifically including: merging the missing leaf area with the area ratio of uncertain regions to generate a structured data report; when the pre-acquired reference leaf area contains pixel-level geometric template reference information, the approximate spatial location of the missing area is marked in the visualization overlay map, and the approximate visualization spatial location is not used as the basis for quantitative area measurement.

[0133] Specifically, the computing device combines the determined missing leaf area with the area ratio of the uncertain region to generate a structured data report. Specifically, the computing device treats the missing leaf area (and its ratio to the corrected reference area, i.e., the missing proportion) and the area of ​​the uncertain region (especially the proportion of the uncertain area within the support domain to the leaf support area) as independent fields, along with indicators such as the remaining leaf area and the identifiable effective area, and summarizes them into a structured data table. The output format includes, but is not limited to, CSV, Excel, or JSON. This structured report can be directly imported into subsequent statistical analysis software, facilitating batch comparisons by processing group.

[0134] Furthermore, when the pre-acquired reference blade area includes pixel-level geometric template-type reference information (e.g., a circular theoretical template corresponding to a standard blade disc of known diameter), the computing device marks the approximate spatial location of the missing region in the visualization overlay. Specifically, in the aperture coordinate system, the geometric template-type reference is overlaid with the union of the endpoint residual blade area and the uncertain area, and the difference between the two is calculated as the approximate visual annotation of the missing region, which is then highlighted in the output image with a specific color (e.g., semi-transparent red). It should be clearly stated that this approximate visual spatial location is only for researchers' intuitive reference and result display, and is not used as the basis for quantitative area calculation. The quantitative result of the missing area is still based on the aforementioned value based on area calculation.

[0135] In one specific embodiment of the present invention, after determining the missing area of ​​the blade, the computing device further performs the following operations to generate an upper bound estimate and interval output of the missing area.

[0136] The computing device subtracts the residual leaf area from the corrected reference area to obtain the boundary difference: Boundary difference = A_ref_corr - A_leaf. Then, this boundary difference is compared with zero, and the larger value is determined as the upper bound estimate of the missing area: A_missing_max = max(boundary difference, 0). This upper bound estimate assumes that a true missing area has formed below the uncertain region, i.e., the uncertain area is not deducted.

[0137] Subsequently, the computing device combines the aforementioned missing area of ​​the blade (i.e., the conservatively estimated lower bound A_missing_min) with the upper bound estimate to form the missing ratio interval. Specifically, the lower bound missing ratio and the upper bound missing ratio are calculated separately: MissingRatio_min = A_missing_min / A_ref_corr × 100%, MissingRatio_max = A_missing_max / A_ref_corr × 100%, and this interval [MissingRatio_min, MissingRatio_max] is used as the output of the missing ratio and merged into the blade damage assessment result. When a single numerical value is required, the lower bound value can be selected and noted as a conservative estimate.

[0138] In one specific embodiment of the present invention, during the process of generating a leaf damage assessment result based on the missing leaf area, the computing device also performs the following operations to incorporate visible anomaly information.

[0139] First, the computing device determines the valid blade region by the difference between the residual blade region and the uncertain region. That is, M valid_visible = M leaf - M uncertain This area represents the effective region of the remaining leaf that is not obscured by insects and whose leaf condition can be reliably determined.

[0140] Secondly, when the area of ​​the definable effective leaf region is greater than a preset minimum threshold, the computing device performs visible abnormality feature identification on the pixels within the residual leaf region and outputs the visible abnormal region. Specifically, the preset minimum threshold is used to avoid statistical unreliability due to an excessively small effective region. Visible abnormality feature identification can employ a deep learning semantic segmentation network to perform binary classification (normal tissue / visible abnormal tissue) (or multi-classification such as necrosis, yellowing, browning, etc.) on the residual leaf pixels under the constraint of the definable effective region, outputting the visible abnormal region M. lesion When the effective area is determined to be insufficient, no visible anomaly detection is performed, and a quality warning is output.

[0141] Finally, the computing device incorporates the visible anomaly ratio calculated based on the visible anomaly area into the leaf damage assessment result. The formula for calculating the visible anomaly ratio is: VisibleLesionRatio = A_lesion / A_valid_visible × 100%, where A_lesion is the area of ​​the visible anomaly region, and A_valid_visible is the area that can be determined as valid. This indicator, as part of the leaf damage assessment result, together with the missing leaf area, constitutes a comprehensive damage evaluation system.

[0142] Furthermore, in some optional embodiments, the step of determining the missing leaf area and performing visible anomaly identification based on the pre-acquired reference leaf area, the residual leaf area corresponding to the residual leaf region mask, and the uncertain area corresponding to the uncertain region mask specifically includes the following detailed joint evaluation and calculation derivation mechanism: For chewing pests such as lepidopteran larvae, the core damage phenotype is the direct loss of leaf tissue. To reasonably infer the area of ​​this physical loss, the computing device executes a loss area inference path (main evaluation path): First, considering that detached leaves will naturally shrink or expand due to water loss during the several-day cultivation process, the computing device estimates an area deformation correction coefficient η (e.g., η = Day_7_average area of ​​control group / Day_1_average area of ​​control group) by combining the area change data of the insect-free control group in the same batch. Subsequently, this correction coefficient is used to calculate the area A of the pre-acquired reference leaf. ref By performing natural deformation correction, the corrected reference area A is obtained. ref_corr The calculation formula is: A ref_corr =A ref ×η.

[0143] Next, the system calculates the pixel area A of the residual leaf mask. leaf =∣M leaf | and calculate the area of ​​the uncertain region within the support domain. A uncertain_support =∣M uncertain ∩M support |

[0144] Subsequently, the computing device calculates the evaluation difference, compares this difference with zero to obtain a conservative estimate (lower bound) of the missing area, denoted as A. missing_min Its derivation formula is: A missing_min =max (A ref_corr -A leaf -A uncertain_support ,0).

[0145] The physical meaning of this formula is: the corrected reference area minus the observable residual leaf area at the endpoint, and then subtracting the undetermined area due to insect obstruction, the remaining part is the area that has been absolutely consumed and is missing. The max operation is used to effectively handle negative value anomalies that may be caused by statistical bias in the reference area or deformation correction errors. In the optimal embodiment of this invention, this conservative estimate A is... missing_min This is directly determined as the final area of ​​missing leaf area.

[0146] As a more complete interval estimate output, the computing device can further calculate the boundary difference and derive the upper bound estimate A of the missing area. missing_maxIts formula is: A missing_max =max (A ref_corr -A leaf ,0).

[0147] The upper bound estimate assumes that true missing values ​​have formed below all occlusions and uncertain regions. Ultimately, the system combines the conservative estimate with the upper bound estimate to form the missing value ratio interval [A]. missing_min / A ref_corr A missing_max / A ref_corr Output.

[0148] As a visible abnormality identification pathway (supplementary evaluation pathway) for physiological lesions in residual leaf tissue: On residual leaf tissue still present in the endpoint image, yellowing, necrosis, or browning at the edges of mechanical damage may occur. The computing device calculates the area A of the definitively identifiable leaf region. valid_visible =∣M valid_visible ∣, when A valid_visible Greater than the preset minimum threshold A min_threshold At that time, the standardized single-aperture image is input into an independent visible anomaly recognition decoding head, and the visible anomaly probability map P is output. lesion ∈[0,1] H×W .

[0149] In M valid_visible Under strict constraints, a mask M for visible abnormal regions is generated. lesion The formula is: M lesion =1(P lesion >τ lesion )∩M valid_visible .

[0150] Where τ lesion To determine the threshold. During the training phase of this network branch, its loss function L lesion =α·L CE_lesion +β·L Dice_lesion The gradient calculation only occurs in M valid_visible The learning is performed within the region (by masking the background, missing regions, and occluded uncertain regions), thus ensuring that the network only learns anomalous features of observable real tissue.

[0151] Finally, the computing device merges the missing assessment and the visible anomaly assessment to generate a comprehensive blade damage assessment result: The system separately counts the visible anomaly area A. lesion =∣M lesion | Calculate the visible anomaly ratio: VisibleLesionRatio = A lesion / A valid_visible ×100%.

[0152] By combining the missing area range, the extreme value range of the overall damage ratio [DamageRatio] is calculated. total_min DamageRatio total_max The proportion of conservative overall injury is calculated as follows: DamageRatio total_min =(A missing_min +A lesion ) / A ref_corr ×100%.

[0153] Through the above multi-dimensional joint calculation, the system not only eliminates the risk of repeated calculation or misjudgment of the shading area, but also accurately integrates the physical missing area of ​​the leaf with the visible abnormal area on the surface, and outputs a structured data report that is closest to the actual severity of the damage caused by chewing insect pests.

[0154] Furthermore, in some optional embodiments, for practical applications requiring batch analysis in high-throughput multi-well plate experiments, after completing the quantitative evaluation of each well, the computing device also performs a batch statistical analysis and report output process to generate structured analysis results for the entire experimental batch. This process specifically includes the following detailed data processing and visualization mechanisms: First, in the batch data scheduling and processing stage, after receiving a batch of raw images of multi-well plates, the computing device automatically parses the image file name or embedded metadata to extract information such as the plate number (PlateID), the shooting date, and the corresponding treatment group (TreatmentGroup). Then, the system organizes the data structure using "plate number-well ID (WellID)" as a unique composite index. Next, the computing device sequentially calls the aforementioned image standardization, mask extraction, uncertain region construction, and area inference steps for each well through concurrent or streaming processing mechanisms, and records all intermediate results.

[0155] Subsequently, in the structured data table aggregation stage, the computing device aggregated the calculated indicators for all boreholes into a comprehensive structured data table. This table not only includes basic information such as plate number, borehole number, row and column coordinates, and treatment group, but also fully maps all the previously calculated physical quantities and evaluation indicators, specifically including: residual blade area (A... leaf ), and determine the effective area (A) valid_visible ), uncertain area (A) uncertain ), uncertain area within the support domain (A) uncertain_support ), visible anomaly area (A) lesion ), lower bound of missing area (A) missing_min ), upper bound of missing area (A) missing_max ), corrected reference area (A)ref_corr In addition, it also outputs the visible anomaly ratio (VisibleLesionRatio), the missing ratio range (MissingRatio_min / MissingRatio_max), and the overall damage ratio range (DamageRatio) calculated based on the above physical quantities. total_min / DamageRatio total_max ), as well as the quality flag and confidence flag triggered by curl recognition and effective area determination.

[0156] Finally, in the multidimensional statistical analysis and visualization report generation stage, the computing device performs deep data mining based on the above structured data tables, and the specific output formats include: (1) Descriptive statistics and comparative charts: The mean, standard deviation and other statistics of each damage index are automatically calculated according to different treatment groups, and a bar chart with error bars is generated to intuitively compare the relative insect resistance effects of different biological pesticides or insect-resistant genes.

[0157] (2) Plate-level spatial thermal map: Using the physical rows and columns of the porous plate as the coordinate system, the comprehensive damage ratio of the pores is mapped to the color gradient to generate a thermal map, which makes it easier for researchers to quickly check whether there are spatial systematic deviations caused by uneven lighting, airflow or edge effects in the incubator.

[0158] (3) High-dimensional overlay visualization: To facilitate manual tracing and verification, the system generates a visualization overlay of a single hole. The specific fusion method is as follows: the residual leaf region M is overlaid on the standardized original target leaf image. leaf The outline, the insect detection box and category label output by the second deep learning network, and the visible abnormal area M marked with a highlight block (such as semi-transparent yellow). lesion Mark the uncertain area M with a warning color block (such as semi-transparent gray or diagonal shadow). uncertain Furthermore, when pixel-level geometric template-type reference information exists, the missing M will also be approximately visualized. missing_vis Superimposed on the image.

[0159] (4) Standardized data export: The above structured data tables can be directly exported as CSV or Excel format data files, which can be seamlessly supported for import and subsequent analysis of variance (ANOVA) in downstream biostatistics software (such as SPSS, R or GraphPad Prism).

[0160] The following is a detailed description of a deep learning-based leaf pest identification and damage assessment method provided by an embodiment of the present invention, with reference to the accompanying drawings.

[0161] Combination Figure 2 The main process of this method specifically includes the following steps: 201. Image reception and standardization.

[0162] Acquire an image of the target leaf containing both the leaf to be evaluated and the insect. In actual high-throughput experiments, the specific acquisition process includes: first, receiving an original image of the experimental board containing multiple holes; then, based on camera calibration parameters or the geometric mesh prior of the experimental board, performing lens distortion correction on the experimental board image to eliminate edge distortion caused by close-up shooting; next, identifying the position of each hole in the experimental board image through a hole location module, and performing four-point homography perspective correction based on the plane of the experimental board in the experimental board image to obtain a perspective-corrected image; finally, performing circular cropping of the perspective-corrected image based on the positions of each hole, outputting a standardized single-hole approximately orthogonalized image as the target leaf image, and recording the scale parameters during the cropping and scaling process for subsequent area restoration.

[0163] 202. Lighting and color normalization processing.

[0164] To reduce the interference of lighting and color differences between different shooting batches on subsequent network recognition, after outputting standardized target leaf images, lighting normalization and color normalization processing are performed. Specifically, the lighting normalization processing converts the image to a luminance-chrominance color space, performs morphological closing operations on the luminance channel to estimate the spatially slowly varying lighting field, and subtracts this slowly varying component to compensate for uneven lighting; the color normalization processing extracts the mean value of the background region from the chrominance channel, and performs empirical global chrominance translation normalization based on this mean value.

[0165] 203. Semantic segmentation of leaf regions.

[0166] The standardized target leaf image is input into a first deep learning network for feature extraction and segmentation, outputting a mask of the remaining leaf region. To improve the network's ability to distinguish between healthy leaves, damaged leaves, and the background, the input features of the first deep learning network include not only the red, green, and blue (RGB) channels of the target leaf image, but also the saturation and lightness channels in the hue-saturation-lightness (HSV) space obtained by converting the RGB channels. During the model training phase, the training loss function of the first deep learning network adopts a weighted combination of pixel-level cross-entropy loss and region overlap (Dice) loss.

[0167] 204. Insect body target instance segmentation.

[0168] To identify the interference source causing the missing leaf, the target leaf image is synchronously input into a second deep learning network to perform instance segmentation or target detection (combined with in-box mask generation), and the pixel-level range of the detected live insect, remains, or molted skin is output, i.e., the insect region mask. Steps 203 and 204 above use a deep learning network to thoroughly separate real leaf residue tissue from insects with extremely similar color and texture at the pixel level, fundamentally eliminating the "pseudo-leaf effect".

[0169] 205. Construction of uncertain regions.

[0170] Because insects may lie prone on the surface of remaining leaves or close to the gaps in the leaves, the actual condition of the leaves beneath them cannot be directly observed visually. Therefore, mask set operations and region expansion operations are performed based on the insect region mask and the remaining leaf region mask to construct an uncertain region mask representing the occlusion range.

[0171] The specific process is as follows: Locally fill and repair the residual leaf area mask to obtain a continuous leaf support area; project the insect body onto the overlapping area of ​​the leaf support area or a pre-obtained geometric template-type reference area, and determine this as the direct occlusion area mask; considering the insect body edge positioning error and feeding gap characteristics, perform a morphological dilation operation on the insect body area mask to obtain a neighborhood expansion uncertain area mask; finally, take the union of the direct occlusion area mask and the neighborhood expansion uncertain area mask to obtain the uncertain area mask; simultaneously, take the difference between the residual leaf area mask and the uncertain area mask to obtain a identifiable valid leaf area mask.

[0172] 206. Inference of missing leaf area.

[0173] This step is the core assessment path for dealing with chewing pests. Based on the pre-obtained reference leaf area, the residual leaf area corresponding to the mask of the residual leaf area, and the uncertain area corresponding to the mask of the uncertain area, the missing leaf area is jointly determined.

[0174] The specific sources of the pre-obtained reference leaf area include: obtaining pixel-level reference information by using a standardized template that is deterministically constructed with known geometric shapes; or obtaining area-level reference information by extracting the leaf segmentation area from the image of the same hole position before insect release, obtaining the theoretical area of ​​the standard leaf disc, or extracting the statistical area of ​​the same batch of insect-free control samples.

[0175] When inferring the missing area, an area deformation correction coefficient is estimated by combining the area change data of the insect-free control group. Based on this correction coefficient, the pre-acquired reference leaf area is corrected to obtain the corrected reference area. Subsequently, the remaining leaf area and the uncertain area within the support domain are subtracted from the corrected reference area to obtain an evaluation difference. This evaluation difference is compared with zero, and the larger value is determined as the missing leaf area (conservative estimate). Optionally, the difference between the corrected reference area and the remaining leaf area can also be calculated as an upper bound estimate of the missing area, thus forming a missing proportion interval.

[0176] 207. Visible anomaly identification.

[0177] This step is a supplementary evaluation path. Under the constraint of the effective leaf area mask and when its area is greater than the preset minimum threshold, the pixels in the residual leaf area mask are identified for visible abnormal features (such as yellowing, browning, necrosis, etc.), and the visible abnormal area mask is output to capture the area of ​​physiological lesions of residual tissue other than those directly ingested.

[0178] 208. Joint assessment and report output.

[0179] The blade damage assessment results are generated by combining the calculation results from the above dimensions. Specifically, the missing area (or missing proportion range) of the blade is combined with the visible anomaly proportion calculated based on the visible anomaly region mask, and combined with the area proportion of the uncertain region mask to generate a structured data report. When the pre-acquired reference area contains pixel-level geometric template-type reference information, the approximate spatial location of the missing area can also be marked in the visualization overlay map (the approximate visualization spatial location is not used as a quantitative basis for area).

[0180] In addition, curling state recognition can be introduced as a quality control method in the whole process: the degree of curling of the blade is judged based on the blade outline morphology and brightness distribution characteristics. When the degree of curling exceeds the set threshold, a quality warning mark that needs to be manually reviewed is generated in the blade damage assessment result. It should be noted that, due to the physical limitations of two-dimensional image projection, this curling state recognition process does not output a correction coefficient for directly modifying the area, so as to ensure the objectivity and authenticity of the final quantitative data.

[0181] The leaf pest identification and damage assessment method provided in this invention achieves the following technical effects compared to existing technologies: 1) Precise quantification for chewing pests: Breaking through the limitations of the traditional "disease identification paradigm" which relies solely on visible abnormalities on the surface of residual leaves, this paper introduces for the first time a reference leaf area and uncertain area management mechanism, which enables quantitative inference of the leaf area that has physically disappeared due to chewing and feeding, thus avoiding the systematic underestimation of damage.

[0182] 2) Effectively eliminate insect interference: By simultaneously outputting leaf masks and insect masks through a deep learning network, and combining mask set operations and morphological expansion to construct masks for uncertain regions, the interference of live insects with similar color and texture (pseudo-leaf effect) on leaf area statistics and damage assessment can be eliminated, thus improving the accuracy of assessment.

[0183] 3) Automated end-to-end processing: From inputting a target leaf image containing both leaf and insect body to the final output of a structured damage assessment report, no manual delineation or parameter adjustment is required throughout the process, significantly improving processing efficiency and repeatability in high-throughput experimental scenarios.

[0184] 4) Transparent management of uncertainty: Clearly distinguish between uncertain areas (insect occlusion and adjacent areas) and areas that can be determined as valid areas. Do not make strong assumptions in occluded areas, and output the missing area in the form of conservative estimates, upper bound estimates or intervals, so that the evaluation results have scientific interpretability.

[0185] 5) Joint assessment comprehensively covers damage phenotypes: Physical leaf defects and visible abnormalities (necrosis, yellowing, etc.) on remaining leaves are calculated separately and output together, fully reflecting the two different types of damage to leaves caused by chewing pests.

[0186] 6) Quality control and reliability prompts: The system identifies three-dimensional deformations such as blade curling. When the effective area is insufficient, the proportion of uncertain areas is too high, or the curling is severe, a manual verification mark is output to avoid outputting unreliable and misleading values, thereby improving the prudence and practical value of the evaluation results.

[0187] The leaf pest identification and damage assessment system provided in the embodiments of the present invention is described below. The leaf pest identification and damage assessment system described below can be referred to in correspondence with the leaf pest identification and damage assessment method described above.

[0188] This invention provides a leaf pest identification and damage assessment system, see [link to documentation]. Figure 3 ,include: Extraction module 310 is used to extract the residual leaf area of ​​the target leaf image, the target leaf image containing the leaf to be evaluated and the insect body; The construction module 320 is used to identify insects in the target leaf image and construct an uncertain region characterizing the range of insect occlusion and interference. The determination module 330 is used to determine the missing area of ​​the leaf based on the pre-acquired reference leaf area, the residual leaf area corresponding to the residual leaf region, and the uncertain area corresponding to the uncertain region, and to generate a leaf damage assessment result based on the missing area of ​​the leaf.

[0189] Figure 4An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a leaf pest identification and damage assessment method. The method includes: extracting the residual leaf region of a target leaf image, wherein the target leaf image contains the leaf to be assessed and the insect; identifying the insect in the target leaf image and constructing an uncertain region characterizing the range of insect occlusion and interference; determining the missing leaf area based on a pre-acquired reference leaf area, the residual leaf area corresponding to the residual leaf region, and the uncertain area corresponding to the uncertain region; and generating a leaf damage assessment result based on the missing leaf area.

[0190] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0191] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the leaf pest identification and damage assessment methods provided by the above methods. The method includes: extracting the residual leaf region of a target leaf image, the target leaf image containing the leaf to be assessed and the insect; identifying the insect in the target leaf image and constructing an uncertain region characterizing the range of insect occlusion and interference; determining the missing leaf area based on a pre-acquired reference leaf area, the residual leaf area corresponding to the residual leaf region, and the uncertain area corresponding to the uncertain region, and generating a leaf damage assessment result based on the missing leaf area.

[0192] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the leaf pest identification and damage assessment method provided by the above methods. The method includes: extracting the residual leaf region of a target leaf image, the target leaf image including the leaf to be assessed and the insect; identifying the insect in the target leaf image and constructing an uncertain region characterizing the range of insect occlusion and interference; determining the missing leaf area based on a pre-acquired reference leaf area, the residual leaf area corresponding to the residual leaf region, and the uncertain area corresponding to the uncertain region, and generating a leaf damage assessment result based on the missing leaf area.

[0193] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0194] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0195] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying leaf pests and assessing damage, characterized in that, include: Extract the residual leaf region from the target leaf image, which includes the leaf to be evaluated and the insect body; Identify the insects in the target leaf image and construct an uncertain region characterizing the range of insect occlusion and interference; Based on the pre-obtained reference leaf area, the residual leaf area corresponding to the residual leaf region, and the uncertain area corresponding to the uncertain region, the missing leaf area is determined, and a leaf damage assessment result is generated based on the missing leaf area.

2. The method according to claim 1, characterized in that, After extracting the residual leaf region of the target leaf image, the method further includes: Extract the contour morphology features and brightness distribution features of the leaf to be evaluated from the target leaf image and the residual leaf region; The degree of curling of the leaf to be evaluated is determined based on the outline morphology features and the brightness distribution features. When the degree of curling exceeds a set threshold, a quality mark requiring manual review is generated in the blade damage assessment results.

3. The method according to claim 1 or 2, characterized in that, The process of identifying insects in the target leaf image and constructing an uncertain region characterizing the extent of insect occlusion and interference includes: Obtain the residual leaf area mask corresponding to the residual leaf area, and obtain the insect area mask corresponding to the identified insect body; perform local hole filling and repair on the residual leaf area mask to obtain the leaf support area mask; The insect body region mask is intersected with the leaf support region mask or a pre-obtained geometric template reference region mask to obtain the direct occlusion region mask. A morphological dilation operation is performed on the insect body region mask to obtain a neighborhood expansion uncertain region mask; The uncertain region mask is obtained by taking the union of the direct occlusion region mask and the neighborhood extended uncertain region mask, and the region corresponding to the uncertain region mask is constructed as the uncertain region.

4. The method according to claim 1 or 2, characterized in that, The method for obtaining the reference blade area includes: A standardized template is deterministically constructed using known geometric shapes, and the area corresponding to the standardized template is used as the reference blade area; or Obtain the target area and use the target area as the reference leaf area; wherein, the target area includes any one of the following: the leaf segmentation area in the image of the same hole position before insect inoculation, the theoretical area of ​​the standard leaf disc, and the statistical area of ​​the control sample without insects in the same batch.

5. The method according to claim 1 or 2, characterized in that, The step of determining the missing leaf area based on the pre-acquired reference leaf area, the remaining leaf area corresponding to the remaining leaf region, and the uncertain area corresponding to the uncertain region includes: The area deformation correction coefficient is estimated by combining the area change data of the insect-free control group, and the pre-acquired reference leaf area is corrected based on the area deformation correction coefficient to obtain the corrected reference area. The evaluation difference is obtained by subtracting the residual blade area and the uncertain area within the support domain from the corrected reference area. The evaluation difference is compared with zero, and the larger value is determined as the missing area of ​​the leaf.

6. The method according to claim 5, characterized in that, After determining the area of ​​the missing leaf, the process also includes: Subtract the residual blade area from the corrected reference area to obtain the boundary difference value; The boundary difference is compared with the zero value, and the larger value is determined as the upper bound estimate of the missing area. The missing area of ​​the leaf and the upper bound estimate of the missing area together constitute the missing proportion range, which is then output to the leaf damage assessment result.

7. The method according to claim 1 or 2, characterized in that, The process of generating leaf damage assessment results based on the missing leaf area also includes: The difference between the residual leaf region and the uncertain region is determined as the region of valid leaf that can be identified. When the area of ​​the identifiable effective leaf region is greater than a preset minimum threshold, visible abnormal features are identified in the pixels within the residual leaf region, and the visible abnormal region is output. The proportion of visible abnormalities calculated based on the visible abnormal areas is incorporated into the leaf damage assessment results.

8. The method according to claim 1 or 2, characterized in that, The process of acquiring the target leaf image before extracting the residual leaf region of the target leaf image includes: Receive experimental plate images containing multiple wells; The hole location module identifies the position of each hole in the experimental board image and performs perspective correction based on the plane of the experimental board in the experimental board image to obtain a perspective-corrected image. Based on the positions of each hole, the perspective correction image is cropped to output a standardized, approximately orthogonalized single-hole image as the target blade image.

9. The method according to claim 8, characterized in that, Before identifying the positions of each hole in the experimental board image through the hole positioning module and performing perspective correction based on the plane of the experimental board in the experimental board image, the method further includes: performing lens distortion correction on the experimental board image based on camera calibration parameters or the geometric mesh prior of the experimental board. After outputting the standardized single-aperture approximately orthogonalized image as the target leaf image, the method further includes: performing illumination normalization and color normalization on the target leaf image; the illumination normalization process compensates for illumination unevenness by estimating the illumination field and subtracting the slowly varying components, and the color normalization process performs empirical colorimetric normalization based on the mean of the background region.

10. The method according to claim 1 or 2, characterized in that, The extraction of the residual leaf region from the target leaf image includes: The target leaf image is input into a first deep learning network for feature extraction and segmentation, and a preliminary residual leaf region is output. The input features of the first deep learning network include the red, green and blue channels of the target leaf image, as well as the saturation and brightness channels in the hue-saturation-brightness space obtained by converting the red, green and blue channels. The training loss function of the first deep learning network adopts a weighted combination of pixel-level cross-entropy loss and region overlap loss. The process of identifying insects in the target leaf image includes: The target leaf image is input into a second deep learning network to perform instance segmentation or target detection, and the corresponding insect body region is output. The preliminary residual leaf area is overlapped with the corresponding insect body area. The part of the preliminary residual leaf area that overlaps with the corresponding insect body area is removed to obtain the final residual leaf area.

11. A leaf pest identification and damage assessment system, characterized in that, include: An extraction module is used to extract the residual leaf region of a target leaf image, the target leaf image containing the leaf to be evaluated and the insect body; A construction module is used to identify insects in the target leaf image and construct an uncertain region characterizing the range of insect occlusion and interference. The determination module is used to determine the missing area of ​​the leaf based on the pre-acquired reference leaf area, the residual leaf area corresponding to the residual leaf region, and the uncertain area corresponding to the uncertain region, and to generate a leaf damage assessment result based on the missing area of ​​the leaf.

Images