A method and device for quantitatively representing characteristics of vegetable pathogen infection based on three-dimensional reconstruction

Abstract

The application discloses a kind of vegetable pathogen infection feature quantitative characterization method and device based on three-dimensional reconstruction, which comprises the following steps: obtaining vegetable pathogen multilayer microscopic image;Based on multilayer microscopic image, corresponding first instance mask of each layer is obtained by instance segmentation network;Based on the first instance mask of each layer, the three-dimensional reconstruction of vegetable pathogen infection structure is carried out, and the three-dimensional structure of pathogen is obtained;Based on the three-dimensional structure of pathogen, the infection feature quantitative characterization result of vegetable pathogen is obtained.The application realizes the efficient reconstruction of pathogen infection structure based on the multilayer microscopic image of vegetable pathogen, and obtains the multidimensional morphological feature quantitative characterization result based on the three-dimensional structure of pathogen, so as to quantify the key phenotype of boundary complexity, morphological regularity, expansion direction and aggregation mode of pathogen infection structure, represent the physiological state change characteristics, improve the accuracy of infection feature characterization result, reduce the dependence on high-end three-dimensional imaging technology, simplify the sample preparation process and reduce the cost.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and apparatus for quantitative characterization of vegetable pathogen infection features based on three-dimensional reconstruction. Background Technology

[0002] Fungal diseases in vegetables are widespread, severely impacting crop yield and quality and posing a persistent threat to agricultural production safety. Among these diseases, the characteristics of the various infection structures (such as appressoria) formed by pathogens during infection—in terms of quantity, morphology, and three-dimensional spatial distribution—are closely related to the pathogen's transmission methods, infectivity, and environmental adaptability, demonstrating a significant structure-function correlation. Therefore, accurate extraction and quantitative characterization of infection structures are crucial for deeply revealing the infection mechanisms and transmission patterns of pathogens.

[0003] In existing technologies, high-precision three-dimensional reconstruction of vegetable pathogen infection structures is achieved through advanced three-dimensional imaging techniques (such as optical coherence tomography (OCT)), providing rich information dimensions for research. However, such equipment is complex in structure, expensive, and has stringent requirements for sample preparation and low imaging throughput, making it difficult to meet the needs of large-sample statistical analysis required for pathogen infection behavior research, resulting in inaccurate infection feature characterization results. Alternatively, image processing methods based on deep learning can automate and perform high-throughput analysis of two-dimensional images captured by bright-field microscopy, thereby reducing dependence on expensive and complex hardware equipment and offering advantages such as low cost and high throughput. While high accuracy is an advantage, pathogen infection is a three-dimensional process. Analyzing it solely based on two-dimensional images cannot fully reconstruct its complex three-dimensional morphological features, neglecting structural information and dynamic changes in the depth direction, leading to inaccurate infection feature representation results. Alternatively, processing two-dimensional microscopic images based on graphic morphology operations, such as edge detection, morphological filtering, and connected component analysis, can calculate basic morphological parameters such as the perimeter, area, and convexity of the infected structure. However, morphological parameters calculated based on two-dimensional images cannot effectively represent the three-dimensional morphological characteristics of the infected structure, resulting in inaccurate infection feature representation results. Summary of the Invention

[0004] The present invention aims to at least partially solve one of the technical problems in the related art.

[0005] Therefore, one objective of this invention is to propose a quantitative characterization method for vegetable pathogen infection features based on three-dimensional reconstruction. This method can obtain the corresponding first instance mask for each layer based on multi-layer microscopic images of vegetable pathogens through an instance segmentation network, and obtain the quantitative characterization results of vegetable pathogen infection features based on the three-dimensional pathogen infection structure corresponding to each layer of the first instance mask. Thus, it can achieve efficient reconstruction of pathogen infection structure based on multi-layer microscopic images of vegetable pathogens, and obtain multi-dimensional morphological feature quantitative characterization results based on the three-dimensional structure of pathogens. This allows for the quantification of key phenotypes such as boundary complexity, morphological regularity, expansion direction, and aggregation pattern of pathogen infection structure, characterizing its physiological state change characteristics, improving the accuracy of infection feature characterization results, reducing dependence on high-end three-dimensional imaging technology, simplifying the sample preparation process, and reducing costs.

[0006] Another objective of this invention is to propose a device for quantitative characterization of vegetable pathogen infection characteristics based on three-dimensional reconstruction.

[0007] To achieve the above objectives, one embodiment of the present invention proposes a method for quantitative characterization of vegetable pathogen infection features based on three-dimensional reconstruction, comprising: Obtain multi-layer microscopic images of vegetable pathogens; Based on the multilayer microscopic images, the corresponding first instance masks for each layer are obtained through an instance segmentation network; Based on the first instance mask of each layer, the three-dimensional reconstruction of the vegetable pathogen infection structure is performed to obtain the three-dimensional structure of the pathogen. The quantitative characterization results of the infection characteristics of the vegetable pathogens were obtained based on the three-dimensional structure of the pathogens.

[0008] The quantitative characterization method for vegetable pathogen infection based on three-dimensional reconstruction in this invention may also have the following additional technical features: Further, in one embodiment of the present invention, the instance segmentation network includes a backbone network and a decoder; the step of obtaining the corresponding first instance mask for each layer through the instance segmentation network based on the multilayer microscopic image includes: The backbone network extracts features from the multilayer microscopic images to obtain the corresponding first feature maps for each layer. The decoder generates a mask based on the first feature map to obtain the corresponding first instance mask for each layer.

[0009] Furthermore, in one embodiment of the present invention, the backbone network includes an input layer, a pooling layer, residual blocks, and a segmented attention mechanism; the backbone network extracts features from the multi-layer microscopic image to obtain corresponding first feature maps for each layer, including: The input layer performs channel and dimension transformations on the multi-layer microscopic image to obtain the corresponding second feature map of each layer. The pooling layer downsamples the second feature map to obtain the corresponding third feature maps for each layer. The residual block performs feature extraction on the third feature map to obtain the corresponding fourth feature maps for each layer; The segmented attention mechanism performs feature optimization on the fourth feature map to obtain the corresponding first feature maps for each layer.

[0010] Furthermore, in one embodiment of the present invention, the segmented attention mechanism includes a spatial attention mechanism, a channel attention mechanism, and a multi-head attention mechanism; the segmented attention mechanism performs feature optimization on the fourth feature map to obtain corresponding first feature maps for each layer, including: The spatial attention mechanism optimizes the spatial features of the fourth feature map to obtain the corresponding fifth feature map. The channel attention mechanism optimizes the channel features of the fourth feature map to obtain the corresponding sixth feature map; The fifth feature map and the sixth feature map are concatenated to obtain the corresponding seventh feature map; The multi-head attention mechanism performs feature optimization on the seventh feature map to obtain the corresponding first feature maps for each layer.

[0011] Furthermore, the three-dimensional reconstruction of the vegetable pathogen infection structure based on the first instance masks of each layer to obtain the three-dimensional structure of the pathogen includes: The geometric features of each first instance mask are calculated based on the first instance mask of each layer, wherein the geometric features include the centroid position and area of ​​the instance; Three-dimensional voxel expansion is performed on each of the first instance masks to obtain each of the second instance masks. Based on the geometric features of each layer, inter-layer matching is performed on the instances in the second instance mask of each layer to obtain the inter-layer path of the vegetable pathogen; The infection structure of the vegetable pathogen was reconstructed in three dimensions based on the interlayer pathways of the pathogen, resulting in a three-dimensional structure of the pathogen.

[0012] Further, in one embodiment of the present invention, the step of performing inter-layer matching on instances in the second instance mask of each layer based on the geometric features of each layer to obtain the inter-layer path of the vegetable pathogen includes: Starting from the initial layer, each instance in the second instance mask is matched and judged layer by layer according to the matching rules and the geometric features to obtain the matching result. If the matching result is successful, the layer index and instance coordinates of the successfully matched layer are added to the inter-layer path until the matching result is unsuccessful or the last layer is reached, thus obtaining the inter-layer path of the vegetable pathogen.

[0013] Furthermore, in one embodiment of the present invention, the step of performing a matching judgment based on the matching rules and the geometric features to obtain a matching result includes: If the Euclidean distance between the centroids of the same type of instances in two layers is less than a set threshold, then the distance threshold condition is satisfied. If the area ratio of similar instances in the two layers is greater than the similarity threshold, then the area similarity threshold condition is satisfied. If instances of the same type in the two layers are within the enhanced 3D connected region, then the connectivity verification is passed. If instances of the same type in the two layers satisfy the distance threshold condition and the area similarity threshold condition, and pass the connectivity verification, then the matching result is determined to be a successful match; otherwise, the matching result is determined to be a failed match.

[0014] Furthermore, in one embodiment of the present invention, the quantitative characterization of infection features includes at least one of the following: Surface area; volume; Sphericity; Aspect ratio; Fractal dimension; Shape index.

[0015] To achieve the above objectives, another embodiment of the present invention proposes a device for quantitative characterization of vegetable pathogen infection characteristics based on three-dimensional reconstruction, the device comprising: The acquisition module is used to acquire multi-layer microscopic images of vegetable pathogens; The segmentation module is used to obtain the corresponding first instance mask for each layer based on the multi-layer microscopic image through an instance segmentation network. The reconstruction module is used to perform three-dimensional reconstruction of the vegetable pathogen infection structure based on the first instance mask of each layer, so as to obtain the three-dimensional structure of the pathogen. The characterization module is used to obtain quantitative characterization results of the infection characteristics of the vegetable pathogen based on the three-dimensional structure of the pathogen.

[0016] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0017] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 A flowchart of a method for quantitative characterization of vegetable pathogen infection features based on three-dimensional reconstruction according to an embodiment of the present invention; Figure 2This is a schematic diagram of the segmented attention mechanism according to an embodiment of the present invention; Figure 3 A flowchart illustrating the process of obtaining a three-dimensional structure of a pathogen according to an embodiment of the present invention; Figure 4 This is a schematic diagram of a device for quantitative characterization of vegetable pathogen infection characteristics based on three-dimensional reconstruction, according to an embodiment of the present invention. Detailed Implementation

[0018] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0019] The following describes, with reference to the accompanying drawings, a method and apparatus for quantitative characterization of vegetable pathogen infection characteristics based on three-dimensional reconstruction, according to an embodiment of the present invention.

[0020] First, the quantitative characterization method for vegetable pathogen infection characteristics based on three-dimensional reconstruction, proposed according to an embodiment of the present invention, will be described with reference to the accompanying drawings.

[0021] Figure 1 This is a flowchart of a method for quantitative characterization of vegetable pathogen infection features based on three-dimensional reconstruction according to an embodiment of the present invention.

[0022] like Figure 1 As shown, the quantitative characterization method for vegetable pathogen infection features based on three-dimensional reconstruction includes the following steps: Step S1: Obtain multi-layer microscopic images of vegetable pathogens.

[0023] In one embodiment of the present invention, a multi-layer microscopic image of the infection structure of vegetable pathogens can be obtained using an optical microscope (Zeiss Axio Imager Z2). In one embodiment of the present invention, the aforementioned multi-layer microscopic image can be a 16-layer microscopic image.

[0024] In one embodiment of the present invention, when acquiring multi-layer microscopic images of vegetable pathogen infection structures using a Zeiss Axio Imager Z2 optical microscope, the microscope settings can be bright field imaging mode, with an objective lens model of "ECPlan-Neofluar 40x / 0.75 M27", a light source of "TL Halogen Lamp", an illumination intensity of 3.00 Volt, an imaging device of "Axiocam 712", an exposure time of 10 ms, a depth of focus of 1.96 μm, and a scanning direction of "Unidirectional". Furthermore, in another embodiment of the present invention, a 200× microscope can be used to initially locate pathogen infection structures, followed by imaging under a 400× microscope.

[0025] Furthermore, in one embodiment of the present invention, the image resolution can be set to 1280×1000, and the pixel scale bar is 1 pixel: 0.259μm.

[0026] It should be noted that, in one embodiment of the present invention, after obtaining a multilayer microscopic image of the vegetable pathogen infection structure through the above steps, the multilayer microscopic image can be cropped to obtain a multilayer microscopic image of a preset size. For example, in one embodiment of the present invention, an algorithm can be used to crop the image into a square along the center to obtain a 16-layer image file of size "1000×1000×3", corresponding to an actual size of "259μm×259μm×60μm".

[0027] Step S2: Based on the multi-layer microscopic image, obtain the first instance mask for each layer through the instance segmentation network.

[0028] In one embodiment of the present invention, after obtaining multi-layer microscopic images through the above steps, the corresponding first instance masks of each layer can be obtained through an instance segmentation network based on the multi-layer microscopic images, so that the vegetable pathogen infection structure can be reconstructed in three dimensions based on the first instance masks of each layer.

[0029] Furthermore, in one embodiment of the present invention, the instance segmentation network may include a backbone network and a decoder. Based on this, in one embodiment of the present invention, the method for obtaining corresponding first instance masks for each layer based on multi-layer microscopic images through an instance segmentation network may include the following steps: Step S21: The backbone network extracts features from the multilayer microscopic images to obtain the first feature map of each layer.

[0030] In one embodiment of the present invention, the backbone network may include an input layer, a pooling layer, residual blocks, and a segmented attention mechanism.

[0031] Furthermore, in one embodiment of the present invention, the method for the backbone network to extract features from multilayer microscopic images to obtain corresponding first feature maps for each layer may include the following steps: Step S211: The input layer performs channel and dimension transformations on the multi-layer microscopic images to obtain the corresponding second feature maps for each layer.

[0032] In one embodiment of the present invention, the input layer can perform channel and dimension transformations on multi-layer microscopic images to obtain corresponding second feature maps for each layer.

[0033] In step S212, the pooling layer downsamples the second feature map to obtain the corresponding third feature maps for each layer.

[0034] In one embodiment of the present invention, the pooling layer can downsample the second feature map to quickly capture global low-frequency features of the image (such as contours and large-scale textures), reduce the computational load of subsequent layers, and provide basic support for fine feature extraction of subsequent residual blocks.

[0035] In step S213, the residual block extracts features from the third feature map to obtain the corresponding fourth feature maps for each layer.

[0036] In one embodiment of the present invention, deep feature extraction of the third feature map is achieved by stacking residual blocks to obtain the corresponding fourth feature maps of each layer.

[0037] Step S214: The segmented attention mechanism optimizes the features of the fourth feature map to obtain the corresponding first feature maps of each layer.

[0038] In one embodiment of the present invention, the segmented attention mechanism may include a spatial attention mechanism, a channel attention mechanism, and a multi-head attention mechanism. Figure 2 This is a schematic diagram of a segmented attention mechanism proposed in an embodiment of the present invention.

[0039] Specifically, such as Figure 2 As shown, in one embodiment of the present invention, the method for optimizing the fourth feature map using the above-mentioned segmented attention mechanism to obtain the corresponding first feature maps of each layer may include the following steps: Step S2141: The spatial attention mechanism optimizes the spatial features of the fourth feature map to obtain the corresponding fifth feature map.

[0040] In one embodiment of the present invention, the spatial attention mechanism can assign weights to different spatial locations of the fourth feature map, that is, by strengthening key spatial regions (such as target locations in an image) and suppressing irrelevant regions (such as background), a fifth feature map that is more focused on task-related spatial features is obtained.

[0041] In one embodiment of the present invention, such as Figure 2 As shown, the fourth feature map is ,in , , These represent the number of channels, height, and width, respectively.

[0042] Furthermore, in one embodiment of the present invention, the method for optimizing the spatial features of the fourth feature map using the above-described spatial attention mechanism to obtain the corresponding fifth feature map may include: optimizing the fourth feature map... Along the channel dimension, using Specify that max pooling is performed, and obtain the first feature result after taking the maximum value along the channel direction. ,in, This represents max pooling with dimensions 1×H×W; for the first feature result Convolution and activation processing are performed to obtain the second feature result. The dimension is 1×H×W. Represents convolution. The fifth feature map is obtained by multiplying the second feature map by the fourth feature map. , where the dimensions are C×H×W.

[0043] In step S2142, the channel attention mechanism optimizes the channel features of the fourth feature map to obtain the corresponding sixth feature map.

[0044] In one embodiment of the present invention, the channel attention mechanism can assign weights to each channel of the fourth feature map to obtain a sixth feature map that focuses on important channel features.

[0045] Specifically, in one embodiment of the present invention, the method for optimizing the channel features of the fourth feature map using the aforementioned channel attention mechanism to obtain the corresponding sixth feature map may include: optimizing the fourth feature map... Simultaneously along (Height) and The third feature is obtained by performing max pooling on the two spatial dimensions (Width). The dimension is C×1×1, and an activation function and a first fully connected network are used. The third feature result is processed to obtain the fourth feature result. The dimension is C×1×1; for the fourth feature map Simultaneously along and The fifth feature is obtained by performing average pooling on both spatial dimensions. The dimension is C×1×1, and an activation function and a second fully connected network are used. The fifth feature result is processed to obtain the sixth feature result. The dimension is C×1×1; the seventh feature result is obtained by adding the fourth and sixth feature results and performing activation processing. Multiply the result of the seventh feature map with the result of the fourth feature map to obtain the sixth feature map. , where the dimensions are C×H×W.

[0046] Step S2143: Concatenate the fifth feature map and the sixth feature map to obtain the corresponding seventh feature map.

[0047] In one embodiment of the present invention, after obtaining the fifth feature map and the sixth feature map through the above steps, the fifth feature map and the sixth feature map can be concatenated to obtain the corresponding seventh feature map. The seventh feature map has dimensions of 2C×H×W.

[0048] Step S2144: The multi-head attention mechanism performs feature optimization on the seventh feature map to obtain the corresponding first feature maps for each layer.

[0049] In one embodiment of the present invention, the multi-head attention mechanism can focus on different parts of the features in the seventh feature map by multiple attention heads to obtain the corresponding first feature maps of each layer, thereby improving the feature representation capability of the model.

[0050] In one embodiment of the present invention, the multi-head attention mechanism uses the seventh feature map. Mapped to queries respectively ,key ,value ,in, , , ,and It is a linear transformation matrix. Dimensions for each attention head.

[0051] Furthermore, in one embodiment of the present invention, the output of each attention head in the multi-head attention mechanism... The weighted result is obtained by weighting. ,in, It is the output transformation matrix.

[0052] Furthermore, in one embodiment of the present invention, the weighted result is residually connected with the seventh feature map to obtain the enhanced first feature map of the corresponding layer. .

[0053] Furthermore, in one embodiment of the present invention, the above-mentioned segmented attention mechanism enhances the ability to capture detailed features of the pathogen infection region and improves the ability to model global relationships of features.

[0054] In step S22, the decoder generates a mask based on the first feature map to obtain the corresponding first instance mask for each layer.

[0055] In one embodiment of the present invention, the decoder obtains the corresponding first instance mask for each layer through a query-based mask generation mechanism.

[0056] In one embodiment of the present invention, the decoder generates masks for different regions or instances using a fixed set of query vectors, where each query vector represents a potential intrusive structure instance or segmentation region. Then, the decoder combines the features output by the encoder with a self-attention mechanism to generate a mask corresponding to each query vector, thus obtaining the corresponding first instance mask for each layer.

[0057] In one embodiment of the present invention, the instance segmentation network is obtained through training. Specifically, in one embodiment, after acquiring a multi-layer microscopic image dataset, instance segmentation and annotation can be performed on the multi-layer microscopic images layer by layer. The Labelme software is used to perform polygonal mask annotation on the infection structures in the images. Then, the annotated vegetable pathogen microscopic images and labels are randomly divided into training, validation, and test sets according to a preset ratio (e.g., 8:1:1), and the data volume is expanded to four times the original size using noise addition, translation, and flipping methods, respectively. Furthermore, in one embodiment of the present invention, the instance segmentation network can be obtained from the expanded training, validation, and test sets.

[0058] Step S3: Based on the first instance mask of each layer, the three-dimensional reconstruction of the vegetable pathogen infection structure is performed to obtain the three-dimensional structure of the pathogen.

[0059] In one embodiment of the present invention, after obtaining the first instance mask of each layer through the above steps, the three-dimensional reconstruction of the vegetable pathogen infection structure can be performed based on the first instance mask of each layer to obtain the three-dimensional structure of the pathogen.

[0060] In one embodiment of the present invention, the method for three-dimensional reconstruction of the vegetable pathogen infection structure based on each layer of first instance masks to obtain the three-dimensional structure of the pathogen may include: calculating the geometric features of each layer of first instance masks based on each layer of first instance masks, wherein the geometric features include the centroid position and area of ​​the instance; performing three-dimensional voxel expansion on each layer of first instance masks to obtain each layer of second instance masks; performing inter-layer matching on the instances in each layer of second instance masks based on the geometric features of each layer to obtain the inter-layer path of the vegetable pathogen; and performing three-dimensional reconstruction of the vegetable pathogen infection structure based on the inter-layer path of the vegetable pathogen to obtain the three-dimensional structure of the pathogen.

[0061] Step S4: Quantitative characterization results of the infection characteristics of vegetable pathogens are obtained based on the three-dimensional structure of the pathogens.

[0062] In one embodiment of the present invention, after obtaining the three-dimensional structure of the pathogen through the above steps, the quantitative characterization results of the infection characteristics of vegetable pathogens can be obtained based on the three-dimensional structure of the pathogen.

[0063] Furthermore, in one embodiment of the present invention, the above-mentioned quantitative characterization results of infection features may include at least one of the following: Surface area; volume; Sphericity; Aspect ratio; Fractal dimension; Shape index.

[0064] Specifically, in one embodiment of the present invention, the surface area can be obtained by a first formula. The first formula is:

[0065] in, , and These are the coordinates of the vertices of the triangles that form the surface of the stained structure.

[0066] Furthermore, in one embodiment of the present invention, the volume can be obtained using a second formula. The second formula is:

[0067] in, , , and These are the vertex coordinates of the tetrahedron that constitutes the invasive structure.

[0068] Furthermore, in one embodiment of the present invention, the sphericity can be obtained through a third formula. The third formula is:

[0069] Furthermore, in one embodiment of the present invention, the aspect ratio can be obtained through a fourth formula. The fourth formula is:

[0070] in, The length of the major axis. This is the length of the minor axis.

[0071] Furthermore, in one embodiment of the present invention, the fractal dimension can be obtained through the fifth formula. The fifth formula is:

[0072] in, It uses the size as The number of boxes required for the box-covering structure.

[0073] Furthermore, in one embodiment of the present invention, the shape index can be obtained through the sixth formula. The sixth formula is:

[0074] Furthermore, in one embodiment of the present invention, the aforementioned surface area... With volume Sphericity indicates the total surface area and volume of the pathogen in contact with the external environment, affecting the pathogen's ability to absorb nutrients and its interaction with the host; Measuring how closely the infection structure resembles an ideal sphere reflects the spread pattern of pathogen infection. A high sphericity value indicates a structure closer to a sphere, suggesting a more compact and uniform infection structure with even diffusion of pathogens between cells or within tissues. Conversely, a lower sphericity value reflects irregularity or the extension of the infection pathway. The aspect ratio of the infection structure... Related to its infection mechanism, elongated structures facilitate penetration of plant cell walls and expansion within plant tissues, while short structures may infect through attachment and toxin secretion; fractal dimension The fractal dimension measures the complexity and roughness of the surface of the smeared structure; a higher fractal dimension indicates a more complex or irregular surface, reflecting the richness of surface detail; shape index The relationship between surface area and volume describes the overall morphological characteristics of a structure, which can be used to compare the similarities and differences between different structures.

[0075] According to the embodiments of the present invention, a method for quantitative characterization of vegetable pathogen infection features based on three-dimensional reconstruction is proposed. This method involves acquiring multi-layer microscopic images of vegetable pathogens; obtaining corresponding first instance masks for each layer using an instance segmentation network based on the multi-layer microscopic images; performing three-dimensional reconstruction of the vegetable pathogen infection structure based on the first instance masks for each layer to obtain the three-dimensional structure of the pathogen; and obtaining quantitative characterization results of the infection features of the vegetable pathogen based on the three-dimensional structure. Therefore, the present invention can achieve efficient reconstruction of the pathogen infection structure based on multi-layer microscopic images of vegetable pathogens, and obtain quantitative characterization results of multi-dimensional morphological features based on the three-dimensional pathogen infection structure. This allows for the quantification of key phenotypes such as boundary complexity, morphological regularity, expansion direction, and aggregation pattern of the pathogen infection structure, characterizing its physiological state changes, improving the accuracy of infection feature characterization results, reducing reliance on high-end three-dimensional imaging technology, simplifying the sample preparation process, and reducing costs.

[0076] Figure 3 This is a flowchart illustrating the process of obtaining a three-dimensional structure of a pathogen according to an embodiment of the present invention.

[0077] like Figure 3 As shown, the method for obtaining the three-dimensional structure of the pathogen includes the following steps: Step S31: Calculate the geometric features of the first instance mask of each layer based on the first instance mask of each layer. The geometric features include the centroid position and area of ​​the instance.

[0078] In one embodiment of the present invention, the method for calculating the geometric features of each first instance mask based on each layer may include: calculating the geometric features in each first instance mask based on each layer using a seventh formula, wherein the seventh formula is:

[0079]

[0080] Where c is the centroid location of the instance, and A is the area of ​​the instance. Represents the vertex coordinates of the polygon mask. The number of vertices in the instance.

[0081] Step S32: Perform three-dimensional voxel expansion on the first instance mask of each layer to obtain the second instance mask of each layer.

[0082] In one embodiment of the present invention, the method of performing three-dimensional voxel expansion on each layer of the first instance mask to obtain each layer of the second instance mask may include: constructing a three-dimensional binary mask for each layer of the first instance mask, embedding the two-dimensional instance into the three-dimensional space, and using morphological dilation operation to expand the instance boundary to obtain each layer of the second instance mask.

[0083] In one embodiment of the present invention, it is possible to utilize Extend instance boundaries, where, For morphological operations, it is a structural element.

[0084] Step S33: Based on the geometric features of each layer, perform inter-layer matching on the instances in the second instance mask of each layer to obtain the inter-layer path of vegetable pathogens.

[0085] In one embodiment of the present invention, after determining the geometric features of each layer through the above steps, inter-layer matching can be performed on the instances in the second instance mask of each layer based on the geometric features of each layer to obtain the inter-layer path of vegetable pathogens.

[0086] Specifically, in one embodiment of the present invention, the method for performing inter-layer matching of instances in the second instance mask of each layer based on the geometric features of each layer to obtain the inter-layer path of vegetable pathogens may include the following steps: Step S331: Starting from the initial layer, the instances in the second instance mask are matched and judged layer by layer according to the matching rules and geometric features to obtain the matching results. In one embodiment of the present invention, the initialization path structure for each instance in the second instance mask can be:

[0087] in, This is a layer index, meaning it corresponds to the layer where the same instance exists. Center of mass, These are the instance coordinates.

[0088] In one embodiment of the present invention, the method for obtaining a matching result based on matching rules and geometric features may include the following steps: Step 1: If the Euclidean distance between the centroids of the same type of instances in two layers is less than a set threshold, then the distance threshold condition is satisfied. In one embodiment of the present invention, if the centroid of the instance in the i-th layer... Centroid corresponding to the same type of instance in layer j Euclidean distance Less than the set threshold That is to say, If so, then it is determined that two instances of the same type satisfy the distance threshold condition.

[0089] Step 2: If the area ratio of similar instances in two layers is greater than the similarity threshold, then the area similarity threshold condition is satisfied. In one embodiment of the present invention, if the instance area in the i-th layer Area of ​​the same type of instance in layer j The ratio of the minimum area to the maximum area is greater than the similarity threshold, that is, If so, then it is determined that two instances of the same type satisfy the area similarity threshold condition.

[0090] Step 3: If instances of the same type in two layers are within the enhanced 3D connected region, then the connectivity verification is passed. In one embodiment of the present invention, if instances of the same type in the i-th layer and the j-th layer are within the enhanced three-dimensional connected region, then it is determined that the two instances of the same type in the i-th layer and the j-th layer pass the connectivity verification.

[0091] Step 4: If instances of the same type in two layers meet the distance threshold and area similarity threshold conditions, and pass the connectivity verification, then the matching result is determined to be a successful match; otherwise, the matching result is determined to be a failed match.

[0092] Step S332: If the matching result is successful, add the successfully matched layer index and instance coordinates to the inter-layer path until the matching result is unsuccessful or the last layer is reached, thus obtaining the inter-layer path of vegetable pathogens.

[0093] In one embodiment of the present invention, if the matching result is successful, the coordinates of the successfully matched instance and the layer index are added to the inter-layer path until the matching fails or the last layer is reached, thus obtaining the inter-layer path of the vegetable pathogen. At this time, the inter-layer path of the vegetable pathogen can be... ,in, This refers to the layer index in the above inter-layer path, that is, the layer corresponding to the layer where the same instance exists. Let be the centroid of each instance in the inter-layer path. These are the coordinates of each instance in the inter-layer path.

[0094] Step S34: Based on the interlayer pathways of vegetable pathogens, the infection structure of vegetable pathogens is reconstructed in three dimensions to obtain the three-dimensional structure of the pathogens.

[0095] In one embodiment of the present invention, after obtaining the interlayer path of vegetable pathogens through the above steps, the infection structure of vegetable pathogens can be reconstructed in three dimensions based on the interlayer path of vegetable pathogens to obtain the three-dimensional structure of the pathogens.

[0096] In one embodiment of the present invention, the method for three-dimensional reconstruction of the infection structure of vegetable pathogens based on the interlayer paths of vegetable pathogens to obtain the three-dimensional structure of the pathogens may include: combining the segmentation masks of each layer based on the interlayer paths of vegetable pathogens to obtain the three-dimensional structure of the pathogens.

[0097] In this embodiment of the invention, the above steps can achieve efficient reconstruction of the spatial structure of pathogens based on multi-layer two-dimensional microscopic images, effectively reducing the dependence on high-end three-dimensional imaging technology, simplifying the sample preparation process, significantly reducing experimental costs, and possessing high-throughput processing capabilities.

[0098] Next, referring to the accompanying drawings, we describe the quantitative characterization device for vegetable pathogen infection characteristics based on three-dimensional reconstruction according to an embodiment of the present invention.

[0099] Figure 4 This is a schematic diagram of a device for quantitative characterization of vegetable pathogen infection characteristics based on three-dimensional reconstruction, according to an embodiment of the present invention.

[0100] like Figure 4 As shown, the quantitative characterization device 10 for vegetable pathogen infection characteristics based on three-dimensional reconstruction includes: an acquisition module 401, a segmentation module 402, a reconstruction module 403, and a characterization module 404, wherein... The acquisition module 401 is used to acquire multi-layer microscopic images of vegetable pathogens; The segmentation module 402 is used to obtain the first instance mask of each layer based on the multi-layer microscopic image through an instance segmentation network. Reconstruction module 403 is used to perform three-dimensional reconstruction of the vegetable pathogen infection structure based on the first instance mask of each layer to obtain the three-dimensional structure of the pathogen. Characterization module 404 is used to obtain quantitative characterization results of the infection characteristics of vegetable pathogens based on the three-dimensional structure of the pathogens.

[0101] Furthermore, the instance segmentation network described above includes a backbone network and a decoder; the segmentation module 402 is specifically used for: The backbone network extracts features from multilayer microscopic images to obtain the first feature map of each layer. The decoder generates a mask based on the first feature map, and obtains the corresponding first instance mask for each layer.

[0102] Furthermore, the aforementioned backbone network includes an input layer, a pooling layer, residual blocks, and a segmented attention mechanism; the aforementioned segmentation module 402 is also used for: The input layer performs channel and dimension transformations on the multi-layer microscopic images to obtain the corresponding second feature maps for each layer. The pooling layer downsamples the second feature map to obtain the corresponding third feature maps for each layer. The residual block extracts features from the third feature map to obtain the corresponding fourth feature maps for each layer. The segmented attention mechanism optimizes the fourth feature map to obtain the corresponding first feature maps for each layer.

[0103] Furthermore, the aforementioned segmented attention mechanism includes spatial attention mechanism, channel attention mechanism, and multi-head attention mechanism; the aforementioned segmentation module 402 is also used for: The spatial attention mechanism optimizes the spatial features of the fourth feature map to obtain the corresponding fifth feature map. The channel attention mechanism optimizes the channel features of the fourth feature map to obtain the corresponding sixth feature map. The fifth and sixth feature maps are concatenated to obtain the corresponding seventh feature map; The multi-head attention mechanism optimizes the seventh feature map to obtain the corresponding first feature maps for each layer.

[0104] Furthermore, the aforementioned reconstruction module 403 is specifically used for: The geometric features of the first instance mask of each layer are calculated based on the first instance mask of each layer. The geometric features include the centroid position and area of ​​the instance. Three-dimensional voxel expansion is performed on the first instance mask of each layer to obtain the second instance mask of each layer; Based on the geometric features of each layer, inter-layer matching is performed on the instances in the second instance mask of each layer to obtain the inter-layer path of vegetable pathogens; The infection structure of vegetable pathogens was reconstructed in three dimensions based on the interlayer pathways of the pathogens, resulting in a three-dimensional structure of the pathogens.

[0105] Furthermore, the aforementioned reconstruction module 403 is also used for: Starting from the initial layer, the instances in the second instance mask are matched and judged layer by layer based on matching rules and geometric features to obtain the matching results. If the matching result is successful, the layer index and instance coordinates of the successful match are added to the inter-layer path until the matching result is unsuccessful or the last layer is reached, thus obtaining the inter-layer path of the vegetable pathogens.

[0106] Furthermore, the aforementioned reconstruction module 403 is also used for: If the Euclidean distance between the centroids of the same type of instances in two layers is less than a set threshold, then the distance threshold condition is satisfied. If the area ratio of similar instances in two layers is greater than the similarity threshold, then the area similarity threshold condition is satisfied. If instances of the same type in two layers are within the enhanced 3D connected region, then the connectivity verification is passed. If instances of the same type in two layers meet the distance threshold and area similarity threshold conditions, and pass the connectivity verification, then the matching result is determined to be a successful match; otherwise, the matching result is determined to be a failed match.

[0107] Furthermore, the quantitative characterization results of the above-mentioned infection features include at least one of the following: Surface area; volume; Sphericity; Aspect ratio; Fractal dimension; Shape index.

[0108] According to an embodiment of the present invention, a quantitative characterization device for vegetable pathogen infection features based on three-dimensional reconstruction is proposed. This device acquires multi-layer microscopic images of vegetable pathogens; based on the multi-layer microscopic images, it obtains corresponding first instance masks for each layer through an instance segmentation network; it performs three-dimensional reconstruction of the vegetable pathogen infection structure based on the first instance masks for each layer, obtaining the three-dimensional structure of the pathogen; and it obtains quantitative characterization results of the infection features of the vegetable pathogen based on the three-dimensional structure. Therefore, the present invention can achieve efficient reconstruction of the pathogen infection structure based on multi-layer microscopic images of vegetable pathogens, and obtain quantitative characterization results of multi-dimensional morphological features based on the three-dimensional pathogen infection structure. This allows for the quantification of key phenotypes such as boundary complexity, morphological regularity, expansion direction, and aggregation pattern of the pathogen infection structure, characterizing its physiological state changes, improving the accuracy of infection feature characterization results, reducing reliance on high-end three-dimensional imaging technology, simplifying the sample preparation process, and reducing costs.

[0109] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0110] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0111] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for quantitative characterization of vegetable pathogen infection features based on three-dimensional reconstruction, characterized in that, The method includes: Obtain multi-layer microscopic images of vegetable pathogens; Based on the multilayer microscopic images, the corresponding first instance masks for each layer are obtained through an instance segmentation network; Based on the first instance mask of each layer, the three-dimensional reconstruction of the vegetable pathogen infection structure is performed to obtain the three-dimensional structure of the pathogen. The quantitative characterization results of the infection characteristics of the vegetable pathogen were obtained based on the three-dimensional structure of the pathogen. The three-dimensional reconstruction of the vegetable pathogen infection structure based on the first instance mask of each layer, to obtain the three-dimensional structure of the pathogen, includes: The geometric features of each first instance mask are calculated based on the first instance mask of each layer, wherein the geometric features include the centroid position and area of ​​the instance; Three-dimensional voxel expansion is performed on each of the first instance masks to obtain each of the second instance masks. Based on the geometric features of each layer, inter-layer matching is performed on the instances in the second instance mask of each layer to obtain the inter-layer path of the vegetable pathogen; The infection structure of the vegetable pathogen was reconstructed in three dimensions based on the interlayer pathways of the pathogen, resulting in a three-dimensional structure of the pathogen.

2. The method according to claim 1, characterized in that, The instance segmentation network includes a backbone network and a decoder; the step of obtaining the corresponding first instance mask for each layer based on the multilayer microscopic image through the instance segmentation network includes: The backbone network extracts features from the multilayer microscopic images to obtain the corresponding first feature maps for each layer. The decoder generates a mask based on the first feature map to obtain the corresponding first instance mask for each layer.

3. The method according to claim 2, characterized in that, The backbone network includes an input layer, pooling layers, residual blocks, and a segmented attention mechanism; the backbone network extracts features from the multi-layer microscopic images to obtain the corresponding first feature maps for each layer, including: The input layer performs channel and dimension transformations on the multi-layer microscopic image to obtain the corresponding second feature map of each layer. The pooling layer downsamples the second feature map to obtain the corresponding third feature maps for each layer. The residual block performs feature extraction on the third feature map to obtain the corresponding fourth feature maps for each layer; The segmented attention mechanism performs feature optimization on the fourth feature map to obtain the corresponding first feature maps for each layer.

4. The method according to claim 3, characterized in that, The segmented attention mechanism includes spatial attention, channel attention, and multi-head attention; the segmented attention mechanism performs feature optimization on the fourth feature map to obtain the corresponding first feature maps for each layer, including: The spatial attention mechanism optimizes the spatial features of the fourth feature map to obtain the corresponding fifth feature map. The channel attention mechanism optimizes the channel features of the fourth feature map to obtain the corresponding sixth feature map; The fifth feature map and the sixth feature map are concatenated to obtain the corresponding seventh feature map; The multi-head attention mechanism performs feature optimization on the seventh feature map to obtain the corresponding first feature maps for each layer.

5. The method according to claim 1, characterized in that, The step of performing inter-layer matching on instances in the second instance mask of each layer based on the geometric features of each layer to obtain the inter-layer path of the vegetable pathogen includes: Starting from the initial layer, each instance in the second instance mask is matched and judged layer by layer according to the matching rules and the geometric features to obtain the matching result. If the matching result is successful, the layer index and instance coordinates of the successfully matched layer are added to the inter-layer path until the matching result is unsuccessful or the last layer is reached, thus obtaining the inter-layer path of the vegetable pathogen.

6. The method according to claim 5, characterized in that, The matching judgment based on the matching rules and the geometric features to obtain the matching result includes: If the Euclidean distance between the centroids of the same type of instances in two layers is less than a set threshold, then the distance threshold condition is satisfied. If the area ratio of similar instances in the two layers is greater than the similarity threshold, then the area similarity threshold condition is satisfied. If instances of the same type in the two layers are within the enhanced 3D connected region, then the connectivity verification is passed. If instances of the same type in the two layers satisfy the distance threshold condition and the area similarity threshold condition, and pass the connectivity verification, then the matching result is determined to be a successful match; otherwise, the matching result is determined to be a failed match.

7. The method according to claim 1, characterized in that, The quantitative characterization results of the infection features include at least one of the following: Surface area; volume; Sphericity; Aspect ratio; Fractal dimension; Shape index.

8. A device for quantitative characterization of vegetable pathogen infection features based on three-dimensional reconstruction, characterized in that, The device includes: The acquisition module is used to acquire multi-layer microscopic images of vegetable pathogens; The segmentation module is used to obtain the corresponding first instance mask for each layer based on the multi-layer microscopic image through an instance segmentation network. The reconstruction module is used to perform three-dimensional reconstruction of the vegetable pathogen infection structure based on the first instance mask of each layer, so as to obtain the three-dimensional structure of the pathogen. The characterization module is used to obtain quantitative characterization results of the infection characteristics of the vegetable pathogen based on the three-dimensional structure of the pathogen; The reconstruction module is also used for: The geometric features of each first instance mask are calculated based on the first instance mask of each layer, wherein the geometric features include the centroid position and area of ​​the instance; Three-dimensional voxel expansion is performed on each of the first instance masks to obtain each of the second instance masks. Based on the geometric features of each layer, inter-layer matching is performed on the instances in the second instance mask of each layer to obtain the inter-layer path of the vegetable pathogen; The infection structure of the vegetable pathogen was reconstructed in three dimensions based on the interlayer pathways of the pathogen, resulting in a three-dimensional structure of the pathogen.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

Images