Needle-leaved wood cross section tracheid cell cavity detection method, device, medium and equipment

Abstract

The invention discloses a coniferous wood cross section tracheid cell cavity detection method, device, medium and equipment, and belongs to the technical field of wood identification. Based on the technical scheme of the active learning method, sparse data sampling is carried out in combination with the generative adversarial network, the output of the target detection model is adopted as an SAM prompt strategy, automatic segmentation of the cross section cell cavity of the coniferous wood is achieved, and quantitative anatomical data is obtained. According to the method, by optimizing the data sampling process and enhancing the adaptability of the model, the precision and efficiency of cell cavity segmentation can be effectively improved, and the method can be suitable for any coniferous material cross section microscopic images. The construction of a low-cost coniferous material microscopic image database and the development of a coniferous material tracheid cell cavity detection, segmentation and measurement model are realized, and the problems of difficulty in manual detection, high acquisition and labeling cost and the like caused by complex structure and sampling difficulty of the tracheid cell cavity of the cross section of the coniferous material are solved; and automatic and rapid detection, segmentation and measurement of the coniferous wood tracheid cell cavity are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of wood identification, and particularly to a method, device, medium and equipment for detecting the cell lumen of the cross-section of softwood tracheids. Background Art

[0002] Forestry plays a crucial role in ecological environmental protection and the development of the green economy. Especially in the scientific management and rational utilization of wood resources, the progress of forestry technology is of great significance for ensuring the sustainable development of wood resources. Softwood, as a major category of forestry resources, has always been a hot topic in wood science research due to its unique biological characteristics and wide application value.

[0003] The cell structure of softwood determines its physiological, physical, chemical and mechanical properties, which have a direct impact on plant physiology, ecology, evolutionary biology, wood processing and utilization, papermaking and fiber industry, climate reconstruction, tree growth research, and tree taxonomy research. In wood science research, accurately labeling and measuring the characteristics of cell lumens is crucial for tree species identification and wood property analysis.

[0004] Traditional quantitative anatomical research methods for cell lumens often rely on microscopic observation and manual measurement. These methods are not only time-consuming and laborious, but also due to the intervention of human factors, it is difficult to guarantee the work efficiency and accuracy of large-scale data measurement. With the development of science and technology, especially the progress of computer image processing technology and machine learning technology, automated and intelligent methods for measuring cell quantitative anatomical data have gradually become a new trend in research.

[0005] In the field of computer images, traditional cell segmentation methods mostly rely on image processing techniques of feature engineering, but they are not ideal when dealing with cells with complex morphology, overlapping or low contrast. Deep learning can adapt to different types of cell segmentation tasks by automatically extracting features from data, and has gradually become a research hotspot.

[0006] Existing automated methods still face many challenges in the processing of quantitative anatomical data of tracheid lumens in the cross-section of softwood. First of all, the tracheid lumens in the cross-section of softwood have complex structures and high variability, which pose great difficulties for automated segmentation and recognition. Secondly, there are subtle morphological differences in the tracheid lumens of the cross-sections of softwoods of different tree species, which puts higher requirements on the universality and discrimination ability of the algorithm. Due to the rich number of tree species in the world, the morphological differences of xylem lumens of each tree species are relatively large, and there are also significant differences in cell characteristics of the same tree species under different growth environments. This diversity of the number of tree species and data not only increases the complexity of data annotation, but also puts higher requirements on the robustness of the algorithm. In addition, differences in staining techniques are also an important factor. The staining process is not only affected by the characteristics of the staining reagent itself, but also by cellular chemical substances, resulting in differences between different stained samples. This diversity of staining further increases the complexity of data annotation and poses challenges to the applicability of automated recognition and modeling algorithms. It is difficult to fully cover all possible cell morphologies and variation situations in actual acquisition, and the amount of sparse data collected is insufficient, resulting in the inability to comprehensively reflect the diversity of tree species and environments, thus affecting the representativeness and accuracy of model training. Summary of the Invention

[0007] To solve the defects of the prior art, the present invention provides a method, device, medium and equipment for detecting tracheid lumens in the cross-section of softwood, which solves the problems of difficult manual detection of tracheid lumens in the cross-section of softwood due to complex structures and sampling difficulties, and high costs of acquisition and annotation, and realizes automatic and rapid detection of tracheid lumens in softwood.

[0008] The technical solutions provided by the present invention are as follows:

[0009] In a first aspect, the present invention provides a method for detecting tracheid lumens in the cross-section of softwood, the method comprising:

[0010] S1: Obtain a microscopic image of the cross-section of softwood;

[0011] S2: Use a trained simulation model of softwood microscopic images to perform sparse data sampling enhancement on the obtained microscopic image of the cross-section of softwood to obtain an image data set;

[0012] S3: Perform tracheid lumen annotation on the image data set based on an active learning framework;

[0013] S4: Construct a lumen detection model, and use the annotated image data set to train the lumen detection model;

[0014] S5: Obtain the microscopic image of the cross-section of the softwood to be detected, and use the trained cell lumen detection model to detect the tracheid cell lumen in the microscopic image of the cross-section of the softwood to be detected, and output the tracheid cell lumen bounding box;

[0015] S6: Construct a cell lumen segmentation model and train it, and use the trained cell lumen segmentation model to segment the cell lumen in the image contained in the tracheid cell lumen bounding box to obtain the tracheid cell lumen segmentation result.

[0016] Further, the softwood microscopic image simulation model is trained by the following method:

[0017] Construct a softwood microscopic image dataset;

[0018] Construct a style-based generative adversarial network as the softwood microscopic image simulation model, and use the softwood microscopic image dataset to train the generative adversarial network;

[0019] Among them, the generative adversarial network includes a parallel generator and discriminator. The generator includes a style mapping network and a generation network, which are used to sample latent features from the latent space for image generation. The discriminator includes multiple convolutional layers, and each convolutional layer downsamples the image through a pooling operation to extract high-level features of the image, and finally outputs a binary discrimination result;

[0020] Construct an inversion network based on a forward convolutional neural network to perform inversion training on the aforementioned generative adversarial network;

[0021] Among them, the forward inversion network is a forward structure, including multiple convolutional layers and a fully connected layer. The fully connected layer outputs latent features, compares them with the latent features of the generative adversarial network that generates the corresponding image, and constructs a loss function for optimization.

[0022] Further, the S2 includes:

[0023] Based on the generative adversarial network and the forward inversion network, perform latent feature inversion on the microscopic image of the cross-section of the softwood, and realize sparse data sampling enhancement by linearly adjusting the latent features and generating sparse data through the generation network.

[0024] Further, the S3 includes:

[0025] Construct a basic annotation model, construct a pre-training dataset containing a small number of tracheid cell lumen position annotations, and use the pre-training dataset to train the basic annotation model;

[0026] Use the basic annotation model to obtain the convolutional features of each layer of all images in the image dataset, calculate the feature similarity of all images by integrating the convolutional features of each layer, and construct a similarity matrix;

[0027] Obtain representative images from the image dataset according to the similarity matrix for manual annotation, and optimize the basic annotation model;

[0028] Train multiple uncertainty annotation models based on the basic annotation model using the labeled data, calculate the standard deviation of the bounding box confidence levels of the multiple uncertainty annotation models, normalize the standard deviation, perform uncertainty sampling with a set degree of uncertainty, obtain model uncertainty annotations for manual annotation, and optimize the basic annotation model.

[0029] Further, S6 includes:

[0030] Construct and train a SAM general segmentation model improved based on the cell lumen detection model, and replace the original prompt encoder of the SAM general segmentation model with the output of the cell lumen detection model;

[0031] Perform cell lumen segmentation on the image contained in the tracheid cell lumen prompt box output by the cell lumen detection model through the SAM general segmentation model to obtain the tracheid cell lumen segmentation result.

[0032] Further, the method further includes:

[0033] S7: Post-process the tracheid cell lumen segmentation result, separate the cell lumen regions of each cell, and extract the quantitative anatomical data of each cell lumen.

[0034] In a second aspect, the present invention provides a device for detecting tracheid cell lumens in a cross-section of softwood, the device including:

[0035] A data acquisition module for acquiring microscopic images of the cross-section of softwood;

[0036] A sparse enhancement module for performing sparse data sampling enhancement on the acquired microscopic images of the cross-section of softwood using a trained microscopic image simulation model of softwood to obtain an image dataset;

[0037] An active learning module for performing tracheid cell lumen annotation on the image dataset based on an active learning framework;

[0038] A cell lumen detection model construction module for constructing a cell lumen detection model and training the cell lumen detection model using the labeled image dataset;

[0039] The cell cavity detection module is used to obtain the microscopic image of the cross-section of the softwood to be detected, and use the trained cell cavity detection model to detect the tracheid cell cavity in the microscopic image of the cross-section of the softwood to be detected, and output the tracheid cell cavity hint box;

[0040] The cell cavity segmentation module is used to construct and train a cell cavity segmentation model, and use the trained cell cavity segmentation model to segment the cell cavity in the image contained in the tracheid cell cavity hint box to obtain the tracheid cell cavity segmentation result.

[0041] Further, the softwood microscopic image simulation model is trained through the following process:

[0042] Construct a softwood microscopic image dataset;

[0043] Construct a style-based generative adversarial network as the softwood microscopic image simulation model, and use the softwood microscopic image dataset to train the generative adversarial network;

[0044] Among them, the generative adversarial network includes a parallel generator and discriminator. The generator includes a style mapping network and a generation network, which are used to sample latent features from the latent space for image generation. The discriminator includes multiple convolutional layers, and each convolutional layer downsamples the image through a pooling operation to extract high-level features of the image, and finally outputs a binary discrimination result;

[0045] Construct an inversion network based on a forward convolutional neural network to perform inversion training on the aforementioned generative adversarial network;

[0046] Among them, the forward inversion network is a forward structure, including multiple convolutional layers and a fully connected layer. The fully connected layer outputs latent features, compares them with the latent features of the generative adversarial network that generates the corresponding image, and constructs a loss function for optimization.

[0047] Further, the sparse enhancement module is used for:

[0048] Based on the generative adversarial network and the forward inversion network, perform latent feature inversion on the microscopic image of the cross-section of the softwood, linearly adjust the latent features, and generate sparse data through the generation network to achieve sparse data sampling enhancement.

[0049] Further, the active learning module includes the following process:

[0050] Construct a basic annotation model, construct a pre-training dataset containing a small number of tracheid cell cavity position annotations, and use the pre-training dataset to train the basic annotation model;

[0051] Use the basic annotation model to obtain the convolutional features of each layer of all images in the image dataset, calculate the feature similarity of all images by integrating the convolutional features of each layer, and construct a similarity matrix;

[0052] Obtain representative images from the image dataset according to the similarity matrix for manual annotation, and optimize the basic annotation model;

[0053] Train multiple uncertainty annotation models based on the basic annotation model using the labeled data, calculate the standard deviation of the bounding box confidence levels of the multiple uncertainty annotation models, normalize the standard deviation, perform uncertainty sampling with a set degree of uncertainty, obtain model uncertainty annotations for manual annotation, and optimize the basic annotation model.

[0054] Further, the cell lumen segmentation module includes the following processes:

[0055] Construct and train a SAM general segmentation model improved based on a cell lumen detection model, and replace the original prompt encoder of the SAM general segmentation model with the output of the cell lumen detection model;

[0056] Perform cell lumen segmentation on the image contained in the tracheid cell lumen prompt box output by the cell lumen detection model through the SAM general segmentation model to obtain the tracheid cell lumen segmentation result.

[0057] Further, the device further includes:

[0058] A post-processing module for post-processing the tracheid cell lumen segmentation result, separating the cell lumen regions of each cell, and extracting the quantitative anatomical data of each cell lumen.

[0059] In a third aspect, the present invention provides a computer-readable storage medium for tracheid cell lumen detection of a coniferous wood cross-section, including a memory for storing processor-executable instructions, and when the instructions are executed by the processor, the steps of the tracheid cell lumen detection method described in the first aspect are implemented.

[0060] In a fourth aspect, the present invention provides a device for tracheid cell lumen detection of a coniferous wood cross-section, characterized in that it includes at least one processor and a memory for storing computer-executable instructions, and when the processor executes the instructions, the steps of the tracheid cell lumen detection method described in the first aspect are implemented.

[0061] The present invention has the following beneficial effects:

[0062] The present invention realizes the construction of a low-cost microscopic image database of softwood by using an active learning strategy, as well as the development of a detection, segmentation, and measurement model for the lumen of softwood tracheids, solving the problems of difficult manual detection of the lumen of softwood tracheids in cross-sections due to complex structures and sampling difficulties, and high costs for collection and annotation. It can address challenges such as the amount of annotation, data diversity, staining differences, and the adaptability of existing models, and realizes the automatic, rapid detection, segmentation, and measurement of the lumen of softwood tracheids. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 is a flowchart of the method for detecting the lumen of softwood tracheids in cross-sections according to the present invention;

[0064] Figure 2 is a diagram of the active learning strategy and model architecture according to the present invention;

[0065] Figure 3 is a sparse data sampling diagram of the linear latent space after inversion based on StyleGAN3 according to the present invention;

[0066] Figure 4 is a similarity comparison diagram based on convolutional feature similarity calculation under the active learning framework according to the present invention;

[0067] Figure 5 is an uncertainty sampling diagram under the active learning framework according to the present invention;

[0068] Figure 6 is the original microscopic image of Pinus elliottii in the embodiment of the present invention;

[0069] Figure 7 is Figure 6 a diagram of the lumen detection of the original microscopic image of Pinus elliottii shown;

[0070] Figure 8 is Figure 6 a diagram of the lumen segmentation of the original microscopic image of Pinus elliottii shown;

[0071] Figure 9 is a schematic diagram of the device for detecting the lumen of softwood tracheids in cross-sections according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0072] To make the technical problems, technical solutions, and advantages to be solved by the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Usually, the components of the embodiments of the present invention described and shown in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the present invention to be protected, but only represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present invention.

[0073] An embodiment of the present invention provides a method for detecting the lumen of tracheids in the cross-section of softwood, which is used to segment the lumen contour of tracheids in the cross-section of softwood and quickly obtain quantitative anatomical data. As Figure 1 shown, the method includes:

[0074] S1: Obtain a microscopic image of the cross-section of softwood.

[0075] In the present invention, a push-type microtome can be used to obtain wood sections, and an optical microscope can be used to obtain picture data.

[0076] S2: Use the trained simulation model of softwood microscopic images to perform sparse data sampling enhancement on the obtained microscopic image of the cross-section of softwood to obtain an image dataset.

[0077] This step is used to generate sparse microscopic image data of the cross-section of softwood to cover the information of species complex variability. First, relying on the large-scale specimen data of the Wood Specimen Museum of the Chinese Academy of Forestry, train a simulation model of softwood microscopic images based on the generative adversarial network, perform potential space inversion on the forward network designed for the simulation model of softwood microscopic images, and sample the sparse sampling area.

[0078] In one example, the simulation model of softwood microscopic images can be trained by the following method:

[0079] S21: Construct a dataset of softwood microscopic images.

[0080] Relying on the Wood Specimen Resource Library of the Chinese Academy of Forestry, construct a high-quality dataset of softwood microscopic images. Randomly select 40 tree species, each tree species is magnified 100 times microscopically, and 20 high-precision images with a resolution of 2048×2048 are taken. To meet the training requirements, on the basis of lossless magnification, perform random cropping for each tree species, extract 10 images of 1024×1024 with different central positions, and a total of 8000 images are obtained.

[0081] S22: Construct a style-based generative adversarial network (StyleGAN3) as a simulation model for coniferous wood microscopic images, and use the coniferous wood microscopic image dataset to train the generative adversarial network.

[0082] Among them, the generative adversarial network includes a parallel generator and discriminator. The generator includes a style mapping network and a generation network, which are used to sample latent features from the latent space for image generation. The discriminator includes multiple (e.g., 8) convolutional layers. Each convolutional layer downsamples the image through a pooling operation to extract high-level features of the image, and finally outputs a binary discrimination result.

[0083] S23: Construct an inversion network based on a forward convolutional neural network to perform inversion training on the aforementioned generative adversarial network.

[0084] Among them, the forward inversion network is a simple forward structure, including multiple (e.g., 6) convolutional layers and a fully connected layer. The fully connected layer outputs latent features, which are compared with the latent features of the generative adversarial network that generates the corresponding image, and a loss function is constructed for optimization.

[0085] After training, based on the aforementioned generative adversarial network and forward inversion network, potential feature inversion can be performed on the coniferous wood cross-section microscopic image to obtain a potential feature vector. By linearly adjusting the potential features and generating sparse data through the generation network, sparse data sampling enhancement can be realized to cover species complex variability information, such as Figure 3 shown.

[0086] During the execution process, the coniferous wood microscopic image simulation model performs potential space inversion through a forward network-based method, and uses deep learning to optimize the latent space (z space) and the intermediate representation layer (w space) to achieve precise control of the sparse sampling area. Specifically, the inversion network first receives the target microscopic image as input and processes it through a series of deep network layers to identify and extract the key features that make up the image. Subsequently, these features are mapped into the w feature space to generate a preliminary w feature vector. During this process, the loss function that defines the optimization process is shown in formula (1):

[0087] #timg# （1）

[0088] Taking the sparse coniferous wood microscopic image I and the pre-trained coniferous wood microscopic image simulation model G as inputs, a loss function Loss is established. The loss function Loss consists of a perceptual space loss and Pixel-wise Loss. Initialize a random feature space vector , represents the distance in the perceptual space, which represents the distance between the sparse coniferous wood microscopic image I and the one based on The simulated microscopic image after being generated by the generator G in the feature space The distance in the perceptual space between them. The distance in the perceptual space can compare the differences between two images in terms of high-level semantics. Represents the Pixel-wise Loss, which can capture the differences between two images at the pixel level. The combination of the two losses can comprehensively consider and The semantic differences between the low-level and high-level. By optimizing the Loss, the w feature space closest to the sparse simulated microscopic image can be obtained. During the simulation process, by adjusting the sampling points in the w feature space, sparse sampling is achieved. Specifically, by the truncation coefficient the range of activities of the sampling points in the latent space is restricted to control the amplitude of style changes, as shown in Formulas (2) and (3):

[0089] #timg# （2）

[0090] #timg# （3）

[0091] where represents the original feature space, randomly sampled by the generator network and used as the original output feature control vector, represents the average w space; is the truncation coefficient used to regulate the degree of feature change in the space towards the feature space; represents the regulated feature space. Therefore, by controlling the image latent generation vector w can be changed, as Figure 3 shown.

[0092] S3: Conduct tracheid cell lumen annotation on the image dataset based on the active learning framework.

[0093] This step is used for representative cell annotation based on the active learning framework. By combining the active learning framework with deep learning models and human-computer interaction annotation, the annotation efficiency and accuracy are improved, as Figure 2 shown.

[0094] In one example, this step includes:

[0095] S31: Build a basic annotation model. Construct a pre-training dataset containing the annotation of the positions of a small number of tracheid cell lumens, and use the pre-training dataset to train the basic annotation model.

[0096] Specifically, this step can perform transfer learning on the YOLOv11 model on the COCO dataset based on the pre-training dataset with a small amount of manually annotated data to train the basic annotation model.

[0097] S32: Obtain the convolutional features of each layer of all images in the image dataset using the basic annotation model, calculate the feature similarity of all images by integrating the convolutional features of each layer, and construct a similarity matrix.

[0098] Specifically, use the basic annotation model to extract convolutional features from the microscopic images of the cross-section of slash pine, including an input layer, 13 convolutional layers, 5 pooling layers, and 3 fully connected layers. Obtain multi-layer feature maps, and define the convolutional feature of each layer as , where H, W, and C represent the height, width, and number of channels of the feature map respectively. For the convolutional features of two images and calculate the cosine similarity of each layer, as shown in formula (4):

[0099] #timg# （4）

[0100] Among them, the numerator represents the dot product, and the denominator represents the Euclidean norm. Subsequently, average the similarities of all layers to obtain the comprehensive feature similarity between images, as shown in formula 5:

[0101] #timg# （5）

[0102] Among them, n represents the number of layers of convolutional features. Organize the feature similarity results of all images into a matrix: , where represents the similarity between image i and image j. Screen the representative data according to the similarity matrix with a threshold of 0.65, and at the same time retain the low-similarity images to cover the diversity of data distribution, as Figure 4 shown.

[0103] S33: Obtain the representative images of the image dataset according to the similarity matrix for manual annotation, and optimize the basic annotation model.

[0104] This step is used to annotate the cell cavity positions of representative microscopic section images and further train the basic annotation model.

[0105] S34: Based on the labeled data, train multiple uncertainty annotation models based on the basic annotation model, calculate the standard deviation of the confidence degrees of the annotation boxes of the multiple uncertainty annotation models, normalize the standard deviation, and perform uncertainty sampling with a set degree of uncertainty (for example, 0.8) to obtain model uncertainty annotations for manual annotation, and further optimize the basic annotation model.

[0106] Based on the transfer learning method, after multiple trainings of the representative data annotation, a set of basic annotation models { } is generated. For each image in the slash pine image set { } Use the models sequentially to make predictions and obtain a set of detection boxes { }. For each set of detection boxes extract their spatial coordinates ( ) and confidence levels . Stack the confidence matrices of all models into a three-dimensional tensor: , where has dimensions (H, W, n) and represents the pixel confidence of the image . As shown in formulas (6) and (7), calculate the standard deviation of the tensor along the last dimension (model dimension) to obtain the uncertainty matrix :

[0107] #timg# （6） #timg# （7）

[0108] where is the average confidence of all models at pixel (x, y). After normalizing the confidence standard deviation, sample with an uncertainty of 0.8 to finally obtain the uncertainty heatmap, as Figure 5 shown. Obtain model uncertainty annotations based on the uncertainty heatmap, perform manual annotations, and further optimize the basic annotation model.

[0109] S35: Based on the aforementioned generative adversarial network and the aforementioned forward inversion network, perform latent feature inversion for sparse real microscopic images. After linear adjustment of the latent features and sparse data generation through the generative network to achieve sparse data sampling, model sparse annotations can be obtained for manual annotation, and the basic annotation model can be further optimized.

[0110] The present invention performs representative sampling and uncertainty sampling on the image dataset generated by the aforementioned coniferous wood microscopic image simulation model. This process uses two sampling strategies to ensure the selection of the most representative and uncertain sample data for subsequent processing: representative sampling aims to select samples that can comprehensively reflect the entire data distribution, while uncertainty sampling focuses on selecting samples with higher model uncertainty. The combination of the two helps to better train the model, enabling the model to learn as much valuable information as possible in the case of scarce labeled data. After sampling, the selected samples are handed over to humans for annotation to ensure data accuracy and annotation quality. These manually annotated data will be used to train the initial cell lumen segmentation model and perform cyclic training on the basis of the pre-trained annotation model to gradually improve the model performance.

[0111] S4: Construct a cell lumen detection model and use the labeled image dataset to train the cell lumen detection model.

[0112] After annotating the tracheid cell lumens of a large number of coniferous wood microscopic image data using the active learning method based on the aforementioned S3, after generating a large amount of image data. A cell lumen detection model is constructed based on YOLOv11, and the cell lumen detection model is trained based on the aforementioned tracheid cell lumen annotation dataset.

[0113] S5: Obtain the microscopic image of the cross-section of the coniferous wood to be detected, and use the trained cell lumen detection model to detect the tracheid cell lumens in the microscopic image of the cross-section of the coniferous wood to be detected, and output the tracheid cell lumen bounding box.

[0114] Exemplarily, the microscopic image of the cross-section of the coniferous wood to be detected is as Figure 6 shown, which is the original microscopic image of Pinus elliottii.

[0115] S6: Construct and train a cell lumen segmentation model, and use the trained cell lumen segmentation model to segment the cell lumens in the image contained in the tracheid cell lumen bounding box, and obtain the tracheid cell lumen segmentation result.

[0116] In one example, the S6 includes:

[0117] S61: Construct and train a SAM general segmentation model improved based on the cell lumen detection model, and replace the original prompt encoder of the SAM general segmentation model with the output of the cell lumen detection model.

[0118] S62: Segment the cell lumens in the image contained in the tracheid cell lumen bounding box output by the cell lumen detection model through the SAM general segmentation model, and obtain the tracheid cell lumen segmentation result.

[0119] The present invention provides an optimized SAM model prompt generation strategy, which makes full use of the prediction box generated by the YOLO model (i.e., the tracheid cell lumen bounding box) as a prompt generator and integrates it into the Prompt input of the mask encoder of the SAM model. The tracheid cell lumen bounding box is used as a prompt input to SAM to generate a cell lumen segmentation mask.

[0120] Exemplarily, taking the Figure 6 original microscopic image of Pinus elliottii as the detection object, after the cell lumen detection model is trained, it can automatically detect the cell lumen area in the microscopic image of the cross-section of the xylem of Pinus elliottii, as Figure 7 shown. By using the cell lumen detection result as prompt information and inputting it into the prompt encoder of the SAM model, the segmentation ability of the model is further optimized. This process is continuously iterated through active learning, gradually improving the accuracy and generalization ability of the model, and finally achieving the precise and independent segmentation of each cell lumen in the microscopic image of the cross-section of the xylem of Pinus elliottii, as Figure 8 shown.

[0121] As an improvement, the method of the present invention further includes:

[0122] S7: Post-process the segmentation results of the tracheid cell lumen, separate the cell lumen regions of each cell, and extract the quantitative anatomical data of each cell lumen in pixels.

[0123] The described quantitative anatomical data includes the tangential diameter of the cell lumen, the radial diameter of the cell lumen, the cell lumen area, the cell lumen perimeter, the cell lumen circularity, etc.

[0124] Specifically, use the cell lumen mask as the calculation object, where 1 represents the cell lumen region and 0 represents other regions. The cell lumen area is the number of pixel points with a value of 1 in the cell lumen mask, which can be obtained by counting the pixel points of 1 in the mask image: . The cell lumen perimeter is the length of the boundary of the cell lumen region, which can be calculated by contour detection. The cell lumen circularity is used to measure the degree of circularity of the shape: . Draw a vertical line and a parallel line, intersecting at the center point of the cell lumen mask. Define the length of the vertical line inside the cell lumen mask as the radial diameter of the cell lumen, and define the length of the parallel line inside the cell lumen mask as the tangential diameter of the cell lumen.

[0125] As shown in Table 1, for the Figure 8 segmented mask of each cell lumen, perform quantitative anatomical feature measurements, including cell lumen area, cell lumen perimeter, cell lumen tangential diameter, cell lumen radial diameter, and cell lumen circularity.

[0126] Table 1: Example of quantitative anatomical data of cell lumen (unit: pixel)

[0127]

[0128] Based on the technical solution of the active learning method, the present invention combines a generative adversarial network for sparse data sampling, and uses the output of the object detection model as the SAM prompting strategy, aiming to achieve automatic segmentation of the cell lumen of the cross-section of softwood with strong generalization ability and quickly obtain quantitative anatomical data. By optimizing the data sampling process and enhancing the adaptability of the model, this method can effectively improve the accuracy and efficiency of cell lumen segmentation and can be applied to any cross-sectional microscopic image of softwood.

[0129] The present invention uses an active learning strategy to achieve the construction of a low-cost softwood microscopic image database and the development of a detection, segmentation, and measurement model for softwood tracheid cell lumens, solving the problems of difficult manual detection of tracheid cell lumens in the cross-section of softwood due to complex structures and sampling difficulties, and high costs for acquisition and annotation. It can address challenges such as the amount of annotation, data diversity, staining differences, and the adaptability of existing models, and realizes automatic, rapid detection, segmentation, and measurement of softwood tracheid cell lumens.

[0130] The specific beneficial effects are described as follows:

[0131] 1. The present invention integrates an active learning framework, a StyleGAN3 simulation model, a SAM segmentation model, and cell lumen quantitative data extraction, providing a method for obtaining quantitative anatomical data of tracheid cell lumens in the cross-section of coniferous wood with strong sparse data generalization ability, low annotation cost, and higher automation level.

[0132] 2. Based on the active learning framework, the present invention successfully solves the problem of high annotation cost for cell lumen segmentation of xylem cells in the special field of cell detection.

[0133] 3. Based on the StyleGAN3 model, the present invention designs a forward network and introduces latent space inversion technology to perform data augmentation on sparse data regions caused by special sampling difficulties in coniferous wood.

[0134] 4. Based on the SAM model, the present invention introduces a target detection model to improve the prompt generator, improves the prompt level and mask quality of the SAM model, and improves the cell lumen segmentation speed and quality.

[0135] An embodiment of the present invention also provides a device for detecting tracheid cell lumens in the cross-section of coniferous wood, as Figure 9 shown. The device includes:

[0136] A data acquisition module 1 for acquiring microscopic images of the cross-section of coniferous wood.

[0137] A sparse enhancement module 2 for performing sparse data sampling enhancement on the acquired microscopic images of the cross-section of coniferous wood using a trained microscopic image simulation model of coniferous wood to obtain an image dataset.

[0138] An active learning module 3 for annotating tracheid cell lumens based on the active learning framework for the image dataset.

[0139] A cell lumen detection model construction module 4 for constructing a cell lumen detection model and training the cell lumen detection model using the annotated image dataset.

[0140] A cell lumen detection module 5 for acquiring microscopic images of the cross-section of coniferous wood to be detected and using the trained cell lumen detection model to detect tracheid cell lumens in the microscopic images of the cross-section of coniferous wood to be detected, and outputting a tracheid cell lumen prompt box.

[0141] A cell lumen segmentation module 6 for constructing and training a cell lumen segmentation model, and using the trained cell lumen segmentation model to perform cell lumen segmentation on the image included in the tracheid cell lumen prompt box to obtain a tracheid cell lumen segmentation result.

[0142] The above-mentioned coniferous wood microscopic image simulation model is trained through the following process:

[0143] Construct a coniferous wood microscopic image dataset.

[0144] Construct a style-based generative adversarial network as the coniferous wood microscopic image simulation model, and use the coniferous wood microscopic image dataset to train the generative adversarial network.

[0145] Among them, the generative adversarial network includes a parallel generator and discriminator. The generator includes a style mapping network and a generation network, which are used to sample latent features from the latent space for image generation. The discriminator includes multiple convolutional layers, and each convolutional layer downsamples the image through a pooling operation to extract high-level features of the image, and finally outputs a binary discrimination result.

[0146] Construct an inversion network based on a forward convolutional neural network to perform inversion training on the above-mentioned generative adversarial network;

[0147] Among them, the forward inversion network is a forward structure, including multiple convolutional layers and a fully connected layer. The fully connected layer outputs latent features, compares them with the latent features of the generative adversarial network that generates the corresponding image, and constructs a loss function for optimization.

[0148] Based on the above-mentioned trained coniferous wood microscopic image simulation model, the sparse enhancement module is used for:

[0149] Based on the generative adversarial network and the forward inversion network, perform latent feature inversion on the coniferous wood cross-section microscopic image, linearly adjust the latent features, and generate sparse data through the generation network to achieve sparse data sampling enhancement.

[0150] The active learning module in the present invention includes the following processing procedures:

[0151] Construct a basic annotation model, construct a pre-training dataset containing a small number of tracheid cell cavity position annotations, and use the pre-training dataset to train the basic annotation model.

[0152] Use the basic annotation model to obtain the convolutional features of each layer of all images in the image dataset, calculate the feature similarity of all images by integrating the convolutional features of each layer, and construct a similarity matrix.

[0153] Obtain representative images of the image dataset according to the similarity matrix for manual annotation to optimize the basic annotation model.

[0154] Train multiple uncertainty annotation models based on a basic annotation model using the labeled data, calculate the standard deviation of the annotation box confidence levels of the multiple uncertainty annotation models, normalize the standard deviation, perform uncertainty sampling at a set degree of uncertainty, obtain model uncertainty annotations for manual annotation, and optimize the basic annotation model.

[0155] Further, the aforementioned cell lumen segmentation module includes the following processing procedures:

[0156] Construct and train a SAM general segmentation model improved based on a cell lumen detection model, and replace the original prompt encoder of the SAM general segmentation model with the output of the cell lumen detection model;

[0157] Perform cell lumen segmentation on the image contained in the tracheid cell lumen prompt box output by the cell lumen detection model through the SAM general segmentation model to obtain the tracheid cell lumen segmentation result.

[0158] As an improvement, the device of the present invention further includes:

[0159] A post-processing module for post-processing the tracheid cell lumen segmentation result, separating the cell lumen regions of each cell, and extracting the quantitative anatomical data of each cell lumen.

[0160] The device provided in the above embodiments, its implementation principle and the technical effects produced correspond one by one to the embodiments of the foregoing method. For the sake of brief description, for the parts not mentioned in the embodiments of the device, reference may be made to the corresponding content in the embodiments of the foregoing method. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the modules and units described in the device can all refer to the corresponding processes in the embodiments of the foregoing method, and will not be repeated here.

[0161] The method for detecting tracheid cell lumens in the cross-section of coniferous wood described in the above embodiments provided by the present invention can implement the business logic through a computer program and record it on a storage medium. The storage medium can be read and executed by a computer to achieve the effects of the solution described in the method embodiments of this specification. Therefore, the embodiments of the present invention also provide a computer-readable storage medium for detecting tracheid cell lumens in the cross-section of coniferous wood, including a memory for storing processor-executable instructions, and when the instructions are executed by the processor, they implement the steps of the method for detecting tracheid cell lumens in the cross-section of coniferous wood including the foregoing embodiments.

[0162] The storage medium may include a physical device for storing information, typically digitizing the information and then storing it in a medium utilizing electrical, magnetic, or optical means. Examples of such storage media include: devices that use electrical energy to store information, such as various types of memory, such as RAM and ROM; devices that use magnetic energy to store information, such as hard disks, floppy disks, magnetic tapes, magnetic core memories, bubble memories, and USB flash drives; and devices that use optical means to store information, such as CDs or DVDs. Of course, there are also other types of readable storage media, such as quantum memories and graphene memories.

[0163] The storage medium described above may also include other implementation methods according to the description of the method embodiment. The implementation principle and technical effects produced by this embodiment are the same as those of the aforementioned method embodiment. For details, please refer to the description of the relevant method embodiment, and no further details will be given here.

[0164] Embodiments of the present invention also provide a device for detecting tracheid cavities in coniferous material cross sections. The device may be a standalone computer or may include an operating device that utilizes one or more of the methods or one or more of the apparatuses described in this specification. The device may include at least one processor and a memory storing computer-executable instructions. When the processor executes the instructions, it implements any one or more of the steps described above in the method for detecting tracheid cavities in coniferous material cross sections.

[0165] The device described above may also include other implementation methods according to the description of the method embodiment. The implementation principle and technical effects produced by this embodiment are the same as those of the aforementioned method embodiment. For details, please refer to the description of the relevant method embodiment, and no further description will be given here.

[0166] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, intended to illustrate the technical solutions of the present invention, rather than to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can, within the technical scope disclosed by the present invention, modify or readily conceive of variations to the technical solutions described in the above-described embodiments, or substitute equivalently for some of the technical features thereof. Such modifications, variations, or substitutions do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. They should all be encompassed within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for detecting the cell lumen of tracheids in the cross-section of softwood, characterized in that, The method includes: S1: Obtain the microscopic image of the cross-section of softwood; S2: Use the trained microscopic image simulation model of softwood to perform sparse data sampling enhancement on the obtained microscopic image of the cross-section of softwood to obtain an image dataset; S3: Perform tracheid cell lumen annotation on the image dataset based on the active learning framework; S4: Construct a cell lumen detection model, and use the annotated image dataset to train the cell lumen detection model; S5: Obtain the microscopic image of the cross-section of softwood to be detected, and use the trained cell lumen detection model to detect the tracheid cell lumen in the microscopic image of the cross-section of softwood to be detected, and output a tracheid cell lumen prompt box; S6: Construct and train a cell lumen segmentation model, and use the trained cell lumen segmentation model to segment the cell lumen in the image included in the tracheid cell lumen prompt box to obtain a tracheid cell lumen segmentation result.

2. The method for detecting the cell lumen of tracheids in the cross-section of softwood according to claim 1, wherein The microscopic image simulation model of softwood is trained by the following method: Construct a microscopic image dataset of softwood; Construct a style-based generative adversarial network as the microscopic image simulation model of softwood, and use the microscopic image dataset of softwood to train the generative adversarial network; Among them, the generative adversarial network includes a parallel generator and discriminator. The generator includes a style mapping network and a generation network, which are used to sample latent features from the latent space for image generation. The discriminator includes multiple convolutional layers, and each convolutional layer downsamples the image through a pooling operation to extract high-level features of the image, and finally outputs a binary discrimination result; Construct an inversion network based on a forward convolutional neural network to perform inversion training on the aforementioned generative adversarial network; Among them, the forward inversion network is a forward structure, including multiple convolutional layers and a fully connected layer. The fully connected layer outputs latent features, compares them with the latent features of the generative adversarial network that generates the corresponding image, and constructs a loss function for optimization.

3. The method for detecting the lumen of tracheid cells in the cross-section of softwood according to claim 2, characterized in that, S2 includes: Based on the generative adversarial network and the forward inversion network, perform latent feature inversion on the microscopic image of the cross-section of softwood, realize sparse data sampling enhancement by linearly adjusting the latent features and generating sparse data through the generation network.

4. The method for detecting the lumen of tracheid cells in the cross-section of softwood according to claim 3, wherein, S3 includes: Construct a basic annotation model, construct a pre-training dataset containing a small number of tracheid cell lumen position annotations, and use the pre-training dataset to train the basic annotation model; Use the basic annotation model to obtain the convolutional features of each layer of all images in the image dataset, calculate the feature similarity of all images by integrating the convolutional features of each layer, and construct a similarity matrix; Obtain representative images of the image dataset according to the similarity matrix for manual annotation, and optimize the basic annotation model; Train multiple uncertainty annotation models based on the basic annotation model based on the annotated data, calculate the standard deviation of the annotation box confidence of the multiple uncertainty annotation models, normalize the standard deviation, perform uncertainty sampling with a set degree of uncertainty, obtain model uncertainty annotations for manual annotation, and optimize the basic annotation model.

5. The method for detecting the lumen of tracheids in the cross-section of softwood according to claim 4, characterized in that, S6 includes: Construct a general SAM segmentation model improved based on the cell lumen detection model and train it. Replace the original prompt encoder of the general SAM segmentation model with the output of the cell lumen detection model. Perform cell lumen segmentation on the image contained in the tracheid cell lumen prompt box output by the cell lumen detection model through the general SAM segmentation model to obtain the tracheid cell lumen segmentation result.

6. The tracheid lumen detection method for the cross-section of softwood according to any one of claims 1-5, characterized in that, The method further includes: S7: Post-process the tracheid cell lumen segmentation result, separate the cell lumen regions of each cell, and extract the quantitative anatomical data of each cell lumen.

7. A detection device for the cell lumen of tracheids in the cross-section of softwood, characterized in that, The device includes: A data acquisition module for acquiring microscopic images of the cross-section of softwood. A sparse enhancement module for performing sparse data sampling enhancement on the acquired microscopic images of the cross-section of softwood using the trained microscopic image simulation model of softwood to obtain an image dataset. An active learning module for performing tracheid cell lumen annotation on the image dataset based on the active learning framework. A cell lumen detection model construction module for constructing a cell lumen detection model and training the cell lumen detection model using the annotated image dataset. A cell lumen detection module for acquiring the microscopic image of the cross-section of softwood to be detected and performing tracheid cell lumen detection on the microscopic image of the cross-section of softwood to be detected using the trained cell lumen detection model, and outputting a tracheid cell lumen prompt box. A cell lumen segmentation module for constructing and training a cell lumen segmentation model, and performing cell lumen segmentation on the image contained in the tracheid cell lumen prompt box using the trained cell lumen segmentation model to obtain the tracheid cell lumen segmentation result.

8. A computer-readable storage medium for detecting the cell lumen of tracheid cells in the cross-section of softwood, characterized in that, It includes a memory for storing processor-executable instructions, and when the instructions are executed by the processor, the steps of the tracheid cell lumen detection method according to any one of claims 1-6 are implemented.

9. An apparatus for detecting the lumen of tracheid cells in the cross-section of softwood, characterized in that, It includes at least one processor and a memory for storing computer-executable instructions, and when the processor executes the instructions, the steps of the tracheid cell lumen detection method according to any one of claims 1-6 are implemented.

Images