Cell microtubule array uncertainty quantification segmentation method fusing large visual models

Abstract

The present application belongs to the technical field of image processing, and relates to a cell microtubule array uncertainty quantification segmentation method fusing a large visual model, which utilizes a microtubule segmentation large visual model to process a cell microtubule microscopic image to obtain a pixel-level microtubule likelihood map; topological analysis is performed on the microtubule likelihood map to obtain a plurality of manifold structures, and a union set of the manifold structures constitutes a microtubule topological skeleton; the uncertainty interval of each manifold structure in the topological skeleton is calculated through a manifold uncertainty interval quantification network; prior knowledge is introduced to construct a 'pixel-topology-prior' three-evidence-source fusion framework based on D-S evidence theory, and the internal uncertainty of the evidence sources is explicitly modeled, so that a microtubule accurate segmentation result is obtained. The present application improves the accuracy and reliability of the final microtubule segmentation result by respectively performing uncertainty modeling and fusion analysis on the microtubule morphology extraction results obtained at the pixel level, the topological structure level and the prior knowledge level.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method for uncertain quantization segmentation of cell microtubule arrays that integrates a large visual model. Background Technology

[0002] Microtubules, as important filamentous structures in the cytoskeleton, are highly dynamic and complex supramolecular systems in eukaryotes, playing an irreplaceable role in key life activities such as maintaining cell morphology, mediating intracellular transport, and regulating cell division. The biological functions of microtubules are closely related to their array structures established through dynamic behaviors such as growth, contraction, and fragmentation. Since different compounds can exert selective toxicity on diseased cells by inhibiting the topological evolution of microtubule arrays, a deep understanding of microtubule array morphology and its changing patterns is crucial for developing targeted therapeutic drugs. Existing microscopic imaging techniques can capture microtubule array images with good spatiotemporal resolution, making it possible to observe microtubule structures. However, accurately segmenting microtubule structures and extracting fine microtubule morphological features from microscopic images remains a challenge. Traditional thresholding methods, active contour methods, and spline curve fitting methods are only effective for sparse microtubules; for dense microtubule arrays with inherent geometric overlap and topological complexity, these methods all exhibit significant limitations.

[0003] In recent years, deep learning has become the preferred solution for processing complex microscopic images of microtubes due to its high robustness to high-noise, low-contrast, and dense microscopic images, and its convenient end-to-end learning paradigm. However, deep learning methods suffer from a severe reliance on the scale and quality of training data. The outer diameter of a microtube cross-section is only about 25 nanometers, while according to Abbe's diffraction law, microscopes based on optical imaging principles have a resolution limit of about 200 nanometers. This observation limit is much larger than the size of microtubes, so traditional optical microscopes cannot directly observe clear microtube structures. Although super-resolution microscopy and electron microscopy can obtain clear structures, they are both limited by extremely high costs. Therefore, research on morphological extraction and segmentation of cellular microscopic images suffers from data scarcity. In addition, the pixel-level classification strategy used by deep learning methods has inherent limitations. It processes each pixel independently, relying solely on the predicted probability of a single pixel to determine whether it belongs to a microtube region, ignoring the spatial correlation between pixels, which makes the segmentation results prone to microtube branch breakage problems. Summary of the Invention

[0004] To address the aforementioned problems, the present invention aims to propose a method for uncertainty quantification segmentation of cellular microtubule arrays that integrates a large visual model. First, it constructs a large visual model for microtubule segmentation with few-sample learning capabilities, alleviating the dependence of deep learning methods on data scale. Second, it utilizes Discrete Morse theory to construct the topological skeleton of the microtubules and designs a manifold uncertainty interval quantification network to analyze the uncertainty intervals of each manifold structure within the topological skeleton. Then, it designs a set of prior knowledge and constructs it as a metric function. Finally, based on Dempster-Shafer evidence theory, it constructs a "pixel-topology-prior" three-evidence source fusion framework.

[0005] This invention is achieved through the following technical solution. A method for uncertainty quantification and segmentation of cellular microtubule arrays integrating a large visual model, comprising the following steps:

[0006] Step 1: Collect microscopic images of cell microtubules and label microtubule morphology information to construct a microtubule dataset;

[0007] Step 2: Combine deep trainable cue layers with low-rank adaptive methods to perform structural optimization and domain fine-tuning on the base large vision model, thereby constructing a micro-tube segmentation large vision model;

[0008] Step 3: Use the microtubule segmentation large visual model to process the cell microscopic image of microtubules to obtain the microtubule likelihood map at the pixel level;

[0009] Step 4: Perform topological analysis on the likelihood diagram of the microtube to obtain several manifold structures. The union of the manifold structures constitutes the topological framework of the microtube.

[0010] Step 5: Quantize the uncertainty intervals of each manifold structure in the topological skeleton of the network using manifold uncertainty intervals;

[0011] Step 6: Develop a set of prior knowledge based on microtubule biological characteristics and heuristic rules, and design the prior knowledge into a computable metric function;

[0012] Step 7: Construct a "pixel-topology-prior" three-source evidence fusion framework based on DS evidence theory. After explicitly modeling the uncertainty within the evidence sources, the complementary or conflicting information of each evidence source is effectively integrated to obtain the accurate microtube segmentation result.

[0013] Further preferably, the microtubule segmentation large vision model consists of an image embedding module, an image encoder, and a mask decoder. The image embedding module divides the input cell microtubule microscopic image into image blocks, and then embeds each image block into a vector. The position encoding of the image block is then incorporated into the embedding vector to obtain the initial features of the cell microtubule microscopic image. Subsequently, the initial features of the cell microtubule microscopic image are processed by the image encoder and the mask decoder to finally obtain the microtubule likelihood map.

[0014] In a further preferred embodiment, the image encoder, based on the image encoder of the base large vision model, removes the cue word encoding module, and inserts a deep trainable cue layer before each level of the image encoder. The deep trainable cue layer is mainly composed of convolution, regularization and activation operations.

[0015] Further optimization involves inserting low-rank adaptive modules into each level of the image encoder of the base large vision model using a low-rank adaptive method.

[0016] Further optimization involves the following step: In each layer of the image encoder of the microtubule segmentation large vision model, the deep trainable cue layer first processes the vector input to that layer. The resulting feature vector is then copied, and the two feature vectors are processed by the frozen pedestal large vision model layer and the low-rank adaptive module, respectively. The feature vector is converted into a low-rank matrix through a lower projection matrix, and the low-rank matrix is ​​then converted into a feature vector through an upper projection matrix. The feature vectors output by the frozen pedestal large vision model layer and the low-rank adaptive module are fused according to certain weights. The resulting fused vector is then processed by another part of the frozen pedestal large vision model layer, and the resulting feature vector is then passed into the next layer and the neck module, respectively.

[0017] Further optimization involves adding random Gaussian noise to the likelihood map of the microtube obtained from the large visual model of microtube segmentation to perturb the likelihood map. Then, the perturbed likelihood map is analyzed using discrete Morse theory to obtain several manifold structures. The union of the manifold structures constitutes the microtube topological skeleton.

[0018] Further preferably, the manifold uncertainty interval quantization network takes the manifold structure as input and has three branches;

[0019] Branch 1 treats manifolds as nodes to construct a graph structure. If there is an overlap between two manifolds, it is assumed that there is an edge between the two nodes. Then, a graph convolutional network is used to process the graph structure to obtain a spatial feature map that captures the adjacency relationships between manifolds.

[0020] Branch 2 embeds all manifold structures as vectors to construct manifold features. The manifold features are transformed to the frequency domain after fast Fourier transform and processed by multiple Fourier operators in the frequency domain. Then, they are restored from the frequency domain by inverse Fourier transform to obtain the frequency domain feature map.

[0021] Branch 3 statistically analyzes four manifold scalar values ​​for manifold structure: mean manifold size, mean manifold likelihood, persistence of manifold structure calculated by persistence cohomology method, and fundamental noise.

[0022] The spatial feature map, frequency feature map, and manifold scalar value obtained from the last three branches will be merged to jointly calculate the uncertainty value and logarithmic variance of each manifold structure.

[0023] In a further optimization, in step five, Monte Carlo sampling is used and a certain proportion of the internal parameters of the manifold uncertainty interval quantization network are randomly discarded, thereby perturbing the manifold uncertainty interval quantization network.

[0024] Further optimization involves step six, where a set of prior knowledge includes two points: one is spatial continuity prior, and the other is edge false positive prior.

[0025] The present invention provides a computer-readable storage medium storing computer instructions thereon, which, when executed by a processor, implement the aforementioned cell microtubule array uncertainty quantization segmentation method.

[0026] This invention combines a domain-fine-tuned large-view model, topological skeleton construction and uncertainty analysis methods, and prior knowledge of microtube morphology with DS evidence theory for handling multi-source uncertain information fusion, and has the following advantages:

[0027] 1. By applying a large vision model to the microtubule segmentation task, and through efficient parameter fine-tuning and a few-sample learning strategy, the dependence of traditional deep learning methods on large-scale labeled data is reduced, providing a feasible solution for microtubule morphology analysis in scenarios with scarce samples.

[0028] 2. A large-scale vision model for microtubule segmentation was constructed. This model utilizes a low-rank adaptive method to fine-tune the base large-scale vision model within its domain, enabling it to adapt to microtubule segmentation tasks while maintaining general representation capabilities. Furthermore, a depth cueing layer was designed, and the internal structure of the base large-scale vision model was optimized, thereby achieving a paradigm shift from manual interaction to automatic recognition in segmentation.

[0029] 3. A microtube topology-level skeleton structure was constructed, and a manifold uncertainty interval quantization network was designed to achieve accurate calculation of the uncertainty of topological branches. Furthermore, considering both accidental and cognitive uncertainties, a topological uncertainty interval estimation method was proposed, transforming uncertainty assessment from a single scalar value into a probability interval, thereby improving the reliability of topological analysis results.

[0030] 4. A three-source evidence fusion framework based on DS evidence theory, namely “pixel-topology-prior”, is proposed. After explicitly modeling the internal uncertainties of the three evidence sources, the complementary or conflicting information of each evidence source is effectively integrated through basic probability assignment and Dempster combination rules, so as to obtain more accurate microtubule segmentation results. Attached Figure Description

[0031] Figure 1 This is a flowchart of an uncertainty quantification segmentation method for cell microtubule arrays that integrates a large visual model, provided by the present invention.

[0032] Figure 2 This is a basic framework diagram of the microtubule segmentation large-scale visual model;

[0033] Figure 3 This is a schematic diagram of the topological skeleton and its uncertainty interval construction method;

[0034] Figure 4 It is a microscopic image of a microtube used for testing;

[0035] Figure 5 This is a diagram showing the segmentation effect of the present invention in a comparative experiment;

[0036] Figure 6 This is a comparison of the segmentation results of the Unet model in the experiment;

[0037] Figure 7 This is a comparison of the segmentation results of the Mask R-CNN model in the experiment;

[0038] Figure 8 This is a segmentation result of the pre-trained DPT model with a size of 224 in the comparison experiment;

[0039] Figure 9 This is a segmentation result of the pre-trained DPT model with a size of 384 in the comparison experiment; Detailed Implementation

[0040] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0041] Reference Figure 1 An uncertainty quantification segmentation method for cell microtubule arrays integrating a large visual model, comprising the following steps:

[0042] Step 1: Collect microscopic images of cell microtubules and label microtubule morphology information to construct a microtubule dataset;

[0043] Step 2: Combine deep trainable cue layers with low-rank adaptive methods to perform structural optimization and domain fine-tuning on the base large vision model, thereby constructing a micro-tube segmentation large vision model;

[0044] Step 3: Use the microtubule segmentation large visual model to process the cell microscopic image of microtubules to obtain the microtubule likelihood map at the pixel level;

[0045] Step 4: Perform topological analysis on the likelihood diagram of the microtube to obtain several manifold structures. The union of the manifold structures constitutes the topological framework of the microtube.

[0046] Step 5: Quantize the uncertainty intervals of each manifold structure in the topological skeleton of the network using manifold uncertainty intervals;

[0047] Step 6: Develop a set of prior knowledge based on microtubule biological characteristics and heuristic rules, and design the prior knowledge into a computable metric function;

[0048] Step 7: Construct a "pixel-topology-prior" three-source evidence fusion framework based on DS evidence theory. After explicitly modeling the uncertainty within the evidence sources, the complementary or conflicting information of each evidence source is effectively integrated to obtain the accurate microtube segmentation result.

[0049] Figure 2 This paper demonstrates the basic framework of the large-scale visual model for microtubule segmentation constructed in this invention. The framework mainly consists of an image embedding module, an image encoder, and a mask decoder. The image embedding module divides the input microscopic image of cell microtubules into image blocks, then embeds each image block into a vector. The positional encoding of the image blocks is then incorporated into the embedding vectors to obtain the initial features of the microscopic image of cell microtubules. These initial features are then processed by the image encoder and mask decoder to finally obtain the microtubule likelihood map.

[0050] Existing large-scale visual models require users to specify external cues. While this design ensures zero-sample prediction capability across different scenarios, it also leads to problems such as the model's inability to automatically identify foreground objects, necessitating manual prompts, resulting in excessive human effort and low interaction efficiency. For microtubule segmentation tasks, although dye colors and imaging sharpness vary, all microtubules in cell microscopic images appear as fluorescent highlights, forming a clear contrast with the black background. Furthermore, cell microscopic images only contain microtubules, making it a single-target segmentation task, eliminating the need to handle complex multi-object segmentation problems. Based on these two reasons, this invention optimizes the structure of the large-scale visual model by removing the cues encoding module and eliminating four forms of external cues: cue points, cue boxes, cue text, and cue masks, thus eliminating the need for cue features.

[0051] To automatically extract cue information from microscopic images of cell microtubules, this invention inserts a deep trainable cue layer before each layer (the original layers of the base large vision model) of the image encoder, thereby converting the previously externally specified hard cue into an adaptive soft cue. The deep trainable cue layer learns the implicit representation of the microtubule foreground through end-to-end fine-tuning, thus realizing the transformation from explicit instructions given by the user to implicit feature encoding, achieving fully automatic prediction without additional manual processing. The deep trainable cue layer mainly consists of convolution, regularization, and activation operations. To alleviate the gradient vanishing or exploding problem, residual connections are used. Its data processing can be represented by formula (1):

[0052] (1);

[0053] Where num represents the layer number of the image encoder in the base large vision model, x num For the input data of the deep trainable cue layer, y num This represents the output data of a deep trainable cue layer, where Conv represents the convolution operation and Norm represents the regularization operation. β is the activation function, and β is the residual coefficient.

[0054] To address the issue that the large-scale visual model of the pedestal tends to output blocky regions and cannot identify filamentous structures during segmentation tasks, this invention employs a low-rank adaptive method to insert low-rank adaptive modules into each level of the image encoder of the large-scale visual model of the pedestal, followed by neighborhood fine-tuning using a small number of cellular microscopic images. This process can be represented by formula (2):

[0055] (2);

[0056] in, This represents the input vector of the num-th layer. This represents the output vector of the num-th layer. These are the pre-trained weights for the base large vision model in the num layer. Let be the upward projection matrix. Let be the lower projection matrix. and Both represent the data field, d num The input and output feature vectors represent their dimensions, r is the rank of the low-rank adaptive module, and α represents the fusion weights.

[0057] In each layer of the image encoder of the microtubule segmentation large vision model, the deep trainable cue layer first processes the input vector of that layer. The resulting feature vector is then copied, and the two feature vectors are processed by the frozen pedestal large vision model layer and the low-rank adaptive module, respectively. The feature vectors are converted into low-rank matrices through a down-projection matrix, and the low-rank matrices are then converted back into feature vectors through an up-projection matrix. The feature vectors output by the frozen pedestal large vision model layer and the low-rank adaptive module are fused according to certain weights. The resulting fused vector is then processed by another part of the frozen pedestal large vision model layer, and the resulting feature vectors are then fed into the next layer and the neck module, respectively. The image encoder stacks N layers.

[0058] Microtubule segmentation large-scale vision models use pixels as the basic unit. Since there are no explicit microtubule framework connectivity constraints between discrete pixels, filamentary structure breakage is unavoidable. To alleviate this problem, this invention constructs a microtubule framework through topological analysis. For example... Figure 3 As shown, the present invention first adds random Gaussian noise to the likelihood map of the microtube obtained by the microtube segmentation large vision model to perturb the likelihood map of the microtube. Then, the perturbed likelihood map of the microtube is analyzed by discrete Morse theory to obtain several manifold structures (e.g., N). The union of the manifold structures constitutes the microtube topological skeleton.

[0059] To further calculate the uncertainty of each microtube branch in the topological skeleton, this invention constructs a manifold uncertainty interval quantization network. The manifold uncertainty interval quantization network takes the manifold structure as input and has three branches. Branch one treats the manifold structure as nodes to construct a graph structure. If there is an overlap between two manifold structures, an edge is considered to exist between the two nodes. A graph convolutional network is then used to process the graph structure, thereby obtaining a spatial feature map that captures the adjacency relationships between manifold structures. Branch two embeds all manifold structures as vectors to construct manifold features. These features are transformed to the frequency domain by a Fast Fourier Transform (FFT) and processed by multiple Fourier operators in the frequency domain. Then, an Inverse Fourier Transform (IFT) is performed to restore the frequency domain, thus obtaining a frequency domain feature map that captures a large number of global features. Branch three statistically analyzes four manifold scalar values: average manifold size, average manifold likelihood, the persistence of the manifold structure calculated by the persistence cohomology method, and fundamental noise. The spatial feature map, frequency feature map, and manifold scalar value obtained from the last three branches will be merged to jointly calculate the uncertainty value and logarithmic variance of each manifold structure.

[0060] Branch 2 employs Fast Fourier Transform, which converts convolution in the spatial domain into element-wise multiplication in the frequency domain, significantly reducing computational overhead and avoiding the unbearable computational costs and time consumption caused by the large number of manifold structures in the microtube likelihood diagram. The data processing process of Branch 2 can be represented by the following formulas: Formula (3) reflects the computational process of manifold features changing from the spatial domain to the frequency domain; Formula (4) is the element-wise multiplication operation in the frequency domain; Formula (5) is the inverse Fourier transform, restoring the processing results in the frequency domain.

[0061] (3);

[0062] (4);

[0063] (5);

[0064] Where S is the spatial feature size, n and m are the row and column coordinates of the spatial domain, x(n,m) represents the spatial signal, k1 and k2 are the horizontal and vertical components in the frequency domain, X(k1,k2) is the frequency domain signal, z is the imaginary number that the bisection of z is equal to -1, exp(·) is the complex exponential function, W(k1,k2) is the frequency domain convolution kernel, ⊙ is the Hadamard product, and Y(k1,k2) is the frequency domain convolution result.

[0065] Uncertainty generally includes two types: random uncertainty and cognitive uncertainty. The former depends on the inherent uncertainty present in the samples. This invention makes it possible to assess random uncertainty by introducing random Gaussian noise into the microtube likelihood plot; its value can be directly quantized by the output of the manifold uncertainty interval quantization network. However, cognitive uncertainty reflects the degree of doubt the manifold uncertainty interval quantization network has about its own output. It is determined by the internal parameters of the manifold uncertainty interval quantization network and cannot be obtained during its training. Therefore, this invention uses Monte Carlo sampling and randomly discards a certain proportion of the internal parameters to perturb the network, hoping to determine the network's cognitive uncertainty based on the magnitude of the change in the network's output.

[0066] After H independent samplings, the manifold uncertainty interval quantization network will output a set of predicted uncertainty values ​​for the manifold structure. Where i is the sampling number, and the sample variance of this set is the cognitive uncertainty of the manifold uncertainty interval quantization network. The manifold uncertainty interval quantization network will also output a set of log-variances. The value of random uncertainty can be obtained by calculating the arithmetic mean of the set and exponentializing it. The sum of the two uncertainties is the total uncertainty. its square root That is, the standard deviation, which is the sum of the values ​​of the set of uncertain predicted values. The uncertainty interval that together constitutes the manifold structure .

[0067] This invention uses DS evidence theory to combine pixel-level segmentation results obtained from a large-visual model of microtube segmentation with topological segmentation results obtained from a manifold uncertainty interval quantization network, and formulates a set of prior knowledge to assist in optimizing segmentation performance. This invention first identifies the evidence source as three parts: a microtube likelihood map, a microtube topological skeleton composed of a set of manifolds and the uncertainty intervals of each manifold, and a set of prior knowledge. The set of prior knowledge specifically includes two points: first, a spatial continuity prior, because microtubes have a continuous filamentary morphology, randomly distributed isolated points may be background noise; second, an edge false positive prior, according to heuristic rules, false positive results are more likely to occur at image edges.

[0068] Next, this invention designs a set of prior knowledge into a computable metric function. For the spatial continuity prior, this invention creates a circle with a radius of R pixels as a local window centered on the target pixel, and calculates the proportion of pixels within the window that belong to the microtube region. The higher the proportion, the more likely the current target pixel is to be located on a continuous microtube structure, rather than an isolated noise point. Therefore, the design of the metric function for whether the target pixel belongs to a microtube for the spatial continuity prior is as shown in formula (6). For the edge false positive prior, this invention measures the Euclidean distance from each pixel to the nearest edge of the image and maps the distance to a confidence level. Therefore, the edge false positive prior metric function can be expressed by formula (7).

[0069] (6);

[0070] (7);

[0071] Wherein, cont represents the degree of spatial continuity prior to the certainty that the target pixel belongs to a microtube. Represents the x and y coordinates of the target pixel. Let F represent the x and y coordinates of each pixel within the local window, F represent the number of pixels within the local window identified as microtubules, M represent the binary mask derived from the pixel-topological segmentation results, edge represent the degree of certainty that the edge false positive prior indicates the target pixel belongs to a microtubule, and exp(·) is an exponential function. This represents the Euclidean distance from the target pixel to the nearest edge. These are the weight parameters.

[0072] Subsequently, a binary discriminant framework is constructed, whose proposition space corresponds to the set C equal to... Where A represents the microtubule region, and ¬A represents the non-microtubule region. The power set of set C (the power set is all possible subsets of set C, including the empty set and set C itself; the new set constructed from these subsets is denoted as 2) C It can be defined as { , {A}, { ¬A}, θ}, where Belongs to the empty set, representing an impossible event; θ is the set C itself, satisfying θ = C = {A, ¬A}. Since θ exists for both A representing the microtubule region and ¬A representing the non-microtubule region, θ represents a completely uncertain state (because C belongs to the proposition space of the binary discriminant frame, and θ represents a completely uncertain state, even if the two sets are equal, their meanings are different, hence two different characters are used to represent them). The domain is the power set of set C (2 C The fundamental probability function (BPA) on a given surface is denoted by f, and f is a mapping. , must meet and To measure the degree of contradiction among three different sources of evidence, this invention sets a conflict factor K. If the value of K is close to 1, it indicates that there is a high degree of contradiction between the evidence; if its value is close to 0, it indicates that there is a high degree of consistency between the evidence. It can be calculated by formula (8).

[0073] (8)

[0074] Where o and p are the numbers of the evidence sources, The conflict factor between type o evidence and type p evidence. Let be the basic probability function for the o-th type of evidence. Let f1(A) be the basic probability function for the p-th type of evidence. f1(A) represents the basic probability allocation function for the microtube likelihood diagram as a source of evidence, which measures the degree of support of the microtube likelihood diagram as a source of evidence for proposition A. f1(¬A) represents the degree of support of the microtube likelihood diagram as a source of evidence for proposition ¬A. f2(A) represents the basic probability allocation function for the uncertainty intervals of the microtube topological framework and each manifold as a source of evidence, which measures the degree of support of this source of evidence for proposition A. f2(¬A) represents the degree of support of this source of evidence for proposition ¬A. f3(A) represents the basic probability allocation function for the set of prior knowledge as a source of evidence, which measures the degree of support of this set of prior knowledge as a source of evidence for proposition A. f3(¬A) represents the degree of support of this set of prior knowledge as a source of evidence for proposition ¬A.

[0075] Next, this invention designs the Dempster combination rule to handle conflicts and integrate evidence from different sources. This rule can be expressed by formula (9), which quantifies the comprehensive support for proposition A after comprehensively considering two sources of evidence, o and p.

[0076] (9);

[0077] in, This indicates the overall support for proposition A after integrating and analyzing evidence sources o and p. E and F belong to the power set of set C (2). C The elements in the power set 2 will form a propositional pair, E and F. During the calculation, the power set 2 will be traversed. C Then select all elements in the set and only choose the propositional pairs that satisfy the union of A. Let o be the initial degree of support for proposition E from the source of evidence. Let p be the initial degree of support for proposition F from evidence source p.

[0078] This invention employs a hierarchical approach to fuse three levels of evidence sources: pixel, topology, and prior. Since the topological evidence source only labels the microtubule skeleton, which has an extremely small width, morphological dilation is first used to expand the skeleton to match the width of the microtubules in the likelihood map, resolving the scale mismatch between the abstract nature of topological representation and the concreteness of visual features. Then, Dempster's combination rule is used to perform a preliminary fusion of the likelihood map and the topological manifold uncertainty map, and the consistency between visual features and topological structure is dynamically evaluated through real-time calculated conflict factors. Subsequently, two constraints—spatial continuity prior and edge false positive prior—are introduced for directional optimization, using a knowledge-driven approach to calibrate the preliminary fusion results obtained from data-driven methods.

[0079] To better measure the few-shot learning ability of the present invention, in this embodiment of the invention, 10 samples were randomly selected from the microtube dataset as training data, and 3 types of out-of-distribution (OOD) data that differed from the training data in terms of microtube density, staining method, and imaging device clarity were selected as test data.

[0080] In this embodiment, a pre-trained SAM2 model is used as the base large-view model. Segmentation performance is measured using seven metrics: Accuracy (Acc), Preciousness (Precious), Recall (Recall), Specificity (Specificity), Intersection over Union (IoU), Dice (Dice), Bookmaker Informedness (BM), and Matthews Correlation Coefficient (MCC). The values ​​of Informedness and Matthews Correlation Coefficient are both in the range [-1, 1]. BM will be negative when the model's prediction performance is worse than random guessing, and MCC will be negative when the model's predictions are negatively correlated with the true labels.

[0081] To demonstrate the advantages of this invention in low-sample scenarios, a comparative experiment was designed. The method of this invention was comprehensively compared with four other segmentation models on three types of out-of-distribution data using eight evaluation metrics. The Unet and Mask R-CNN models are de novo trained models, initially initialized with random weights, and then learned the features of microtube microscopic images on 10 training images. Mask R-CNN is a two-stage model; the first stage identifies candidate bounding boxes for the object to be segmented, and the second stage performs object detection and instance segmentation. To better adapt Mask R-CNN to microtube segmentation tasks, this invention removed its first stage of processing. Since microtubes typically occupy the entire microscopic image region, the complete image is used as the candidate bounding box. In the second stage, the object detection-related modules are removed, retaining only the mask output module. The DPT model is a model based on the Visual Transformer (ViT) architecture for dense task prediction, with a larger number of parameters compared to the previous two models. Therefore, this invention uses the weights pre-trained on the ImageNet dataset for the DPT model, and then freezes the image encoder of the pre-trained DPT model before efficiently fine-tuning its parameters on 10 microtube microscopic images. Considering that the input image size required for pre-trained weights may affect the efficiency of parameter fine-tuning, this invention specifically selected pre-trained DPT model weights of 224 and 384 sizes, and scaled the microtube image to the corresponding size through linear interpolation before efficient parameter fine-tuning. The few-shot segmentation performance of each model is shown in Table 1.

[0082] Table 1. Comparison of segmentation performance of various models in few-shot learning

[0083]

[0084] As can be seen from the data in Table 1, the method of this invention demonstrates excellent performance on the three out-of-distribution data that differ from the training samples, and its performance is significantly higher than the other four segmentation models in all metrics except recall. To illustrate the specific segmentation results of each model, this invention selects a test sample, such as... Figure 4 As shown, from Figure 4 As can be seen, this microtube image is relatively dense and contains some background noise. The segmentation result of this invention's method for this sample is as follows: Figure 5 As shown, it can be seen that the method of the present invention can effectively process microscopic images of microtube arrays with a certain degree of complexity.

[0085] The other four models showed low scores on all seven segmentation metrics, but their recall scores were exceptionally high, generally above 0.95, with some even reaching 1. This indicates that these segmentation models lack the ability to identify the filamentary structures within microtubule arrays; they can only label blurry regions of the microtubule array, which is clearly ineffective segmentation. The Unet model... Figure 4 The segmentation results of the microtubule image shown are as follows: Figure 6 As shown; the segmentation result of Mask RCNN is as follows. Figure 7 As shown; the segmentation result of the pre-trained DPT model with size 224 is as follows. Figure 8 As shown; the segmentation result of the pre-trained DPT model with a size of 384 is as follows. Figure 9 As shown. From Figure 9 As can be seen, when the size of the training samples is limited, none of the four segmentation models can accurately segment the microtube array. They all tend to predict large blocky regions rather than filamentous regions.

[0086] This embodiment provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the aforementioned method for uncertain quantization segmentation of cellular microtubule arrays based on a fusion of large visual models.

[0087] The above-described invention merely illustrates implementation methods of the present invention and should not be construed as limiting the scope of the invention patent, nor as imposing any form of limitation on the structure of the embodiments of the present invention. It should be noted that those skilled in the art can make various changes and improvements without departing from the concept of the embodiments of the present invention, and these all fall within the protection scope of the embodiments of the present invention.

Claims

1. A method for uncertainty quantification and segmentation of cellular microtubule arrays integrating a large visual model, characterized in that, The steps are as follows: Step 1: Collect microscopic images of cell microtubules and label microtubule morphology information to construct a microtubule dataset; Step 2: Combine the deep trainable cue layer with the low-rank adaptive method to optimize the structure and fine-tune the domain of the base large vision model, thereby constructing a microtubule segmentation large vision model. The microtubule segmentation large vision model consists of an image embedding module, an image encoder, and a mask decoder. The image embedding module divides the input cell microtubule microscopic image into blocks to obtain image blocks, and then embeds the image blocks into vectors. The position encoding of the image blocks is then incorporated into the embedding vectors to obtain the initial features of the cell microtubule microscopic image. Subsequently, the initial features of the microscopic images of cell microtubules are processed by an image encoder and a mask decoder to finally obtain a microtubule likelihood map. The image encoder, based on the image encoder of the base large vision model, removes the cue word encoding module. A deep trainable cue layer is inserted before each level of the image encoder. The deep trainable cue layer is mainly composed of convolution, regularization, and activation operations. A low-rank adaptive method is used to insert low-rank adaptive modules into each level of the image encoder of the base large vision model. Step 3: Use the microtubule segmentation large visual model to process the cell microscopic image of microtubules to obtain the microtubule likelihood map at the pixel level; Step 4: Perform topological analysis on the likelihood diagram of the microtube to obtain several manifold structures. The union of the manifold structures constitutes the topological framework of the microtube. Step 5: Quantize the uncertainty intervals of each manifold structure in the topological skeleton of the network using manifold uncertainty intervals; Step 6: Develop a set of prior knowledge based on microtubule biological characteristics and heuristic rules, and design the prior knowledge into a computable metric function; Step 7: Construct a "pixel-topology-prior" three-source evidence fusion framework based on DS evidence theory. After explicitly modeling the uncertainty within the evidence sources, the complementary or conflicting information of each evidence source is effectively integrated to obtain the accurate microtube segmentation result.

2. The cell microtubule array uncertainty quantification segmentation method according to claim 1, characterized in that, The sources of evidence are identified as three parts: the microtube likelihood diagram, the microtube topological framework consisting of a set of manifolds and the uncertainty interval of each manifold, and a set of prior knowledge.

3. The cell microtubule array uncertainty quantification segmentation method according to claim 2, characterized in that, A set of prior knowledge includes two points: one is spatial continuity prior, and the other is marginal false positive prior.

4. The cell microtubule array uncertainty quantification segmentation method according to claim 3, characterized in that, For the spatial continuity prior, a circle with a radius of R pixels is created as a local window centered on the target pixel, and the proportion of pixels within the window that belong to the microtube region is calculated. For edge false positive priors, measure the Euclidean distance from each pixel to the nearest edge of the image and map the Euclidean distance to a confidence level.

5. The cell microtubule array uncertainty quantification segmentation method according to claim 4, characterized in that, In each layer of the image encoder of the microtubule segmentation large vision model, the deep trainable cue layer first processes the vector input to that layer. The resulting feature vector is then copied, and the two feature vectors are processed by the frozen pedestal large vision model layer and the low-rank adaptive module, respectively. The feature vector is converted into a low-rank matrix through a lower projection matrix, and the low-rank matrix is ​​then converted into a feature vector through an upper projection matrix. The feature vectors output by the frozen pedestal large vision model layer and the low-rank adaptive module are fused according to certain weights. The resulting fused vector is then processed by another part of the frozen pedestal large vision model layer, and the resulting feature vector is then passed into the next layer and the neck module, respectively.

6. The cell microtubule array uncertainty quantification segmentation method according to claim 1, characterized in that, In step four, random Gaussian noise is first added to the likelihood map of the microtube obtained from the large visual model of microtube segmentation to perturb the likelihood map. Then, the perturbed likelihood map of the microtube is analyzed using discrete Morse theory to obtain several manifold structures. The union of the manifold structures constitutes the microtube topological skeleton.

7. The cell microtubule array uncertainty quantification segmentation method according to claim 1, characterized in that, The manifold uncertainty interval quantization network takes the manifold structure as input and has three branches; Branch 1 treats manifolds as nodes to construct a graph structure. If there is an overlap between two manifolds, it is assumed that there is an edge between the two nodes. Then, a graph convolutional network is used to process the graph structure to obtain a spatial feature map that captures the adjacency relationships between manifolds. Branch 2 embeds all manifold structures as vectors to construct manifold features. The manifold features are transformed to the frequency domain after fast Fourier transform and processed by multiple Fourier operators in the frequency domain. Then, they are restored from the frequency domain by inverse Fourier transform to obtain the frequency domain feature map. Branch 3 statistically analyzes four manifold scalar values ​​for manifold structure: mean manifold size, mean manifold likelihood, persistence of manifold structure calculated by persistence cohomology method, and fundamental noise. The spatial feature map, frequency feature map, and manifold scalar value obtained from the last three branches will be merged to jointly calculate the uncertainty value and logarithmic variance of each manifold structure.

8. The cell microtubule array uncertainty quantification segmentation method according to claim 7, characterized in that, In step five, Monte Carlo sampling is used and a certain proportion of the internal parameters of the manifold uncertainty interval quantization network are randomly discarded to perturb the manifold uncertainty interval quantization network.

9. The cell microtubule array uncertainty quantification segmentation method according to claim 1, characterized in that, The Dempster combination rule was designed to handle conflicts and integrate evidence from different sources.

10. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed by the processor, they implement the cell microtubule array uncertainty quantization segmentation method according to any one of claims 1-9.

Images