Intelligent fine segmentation method for fluorescence images

Abstract

The application discloses a kind of fluorescent image intelligent fine segmentation method, comprising: obtaining fluorescent microscopic image, input instruction is classified probability calculation, if mitochondrion fluorescent microscopic image, decompose into multiple image blocks, each image block is mapped to two feature spaces as input, capture intragranular sparse area feature and perinuclear dense area feature, intragranular sparse area feature and perinuclear dense area feature linear fusion obtains segmentation result;If it is cell, cell nucleus and cell membrane fluorescent microscopic image, using SAM is segmented, add local segmentation, inflation and corrosion, fill empty hole and the like post-processing step, unifiedly remove edge non-complete instance, obtain fine semantic segmentation mask and instance segmentation mask.The mitochondrion image segmentation model of the application accurately extracts real mitochondrion morphology, and using the subcellular structure segmentation model improved on the basis of SAM technology can improve the segmentation capability on cell, cell nucleus and cell membrane and other cells and subcellular targets.

Description

Technical Field

[0001] This invention relates to a method for intelligent fine segmentation of fluorescence images. Background Technology

[0002] Fluorescence microscopy is an important technique for observing cellular and subcellular structures in biomedical research. With the development of high-throughput imaging technology, the amount of microscopic image data is growing rapidly, and relying on manual cell counting, morphological measurement, or tissue identification of a large number of images is difficult to meet the requirements of efficiency and consistency.

[0003] In various biomedical applications, the accuracy of image segmentation directly determines the accuracy of subsequent quantitative analysis. For example, in tumor pathology, accurate segmentation of the nuclear-cytoplasmic boundary can be used to calculate the nuclear-cytoplasmic ratio, which is one of the important indicators to help determine the benign or malignant nature of tumors. In neuroscience, continuous segmentation of neuronal membrane structures can be used to track the morphological orientation of axons and dendrites, thereby assisting in the analysis of Alzheimer's disease-related neuronal degeneration and abnormal circuit connectivity. In the study of mitochondrial-related neurodegenerative diseases, fine segmentation of mitochondria can be used to quantify the mitochondrial fragmentation index (MFI), providing a reference for the study of early pathological changes in Parkinson's disease.

[0004] Fluorescence microscopy is widely used for live cell dynamic imaging and large-scale sample observation due to its fast imaging speed, large field of view, relatively simple system structure, and low phototoxicity. However, in practical applications, fluorescence imaging is easily affected by the optical diffraction limit and the superposition of out-of-focus fluorescence, resulting in problems such as strong background interference, low contrast, blurred boundaries, obscured details, and signal adhesion between adjacent structures in the image.

[0005] Existing methods for microscopic image segmentation and structural analysis are as follows:

[0006] 1. Based on traditional image processing methods such as threshold segmentation, mathematical morphology, watershed, and skeletonization, it can usually be implemented using platforms such as ImageJ / Fiji (an open-source software platform widely used for biological microscopic image processing and analysis, where Fiji is an enhanced version of ImageJ with a large number of plugins).

[0007] 2. A multi-scale structure enhancement and preprocessing method based on Hessian matrix (a mathematical matrix used to characterize local second-order curvature changes in an image) or Frangi filter (a filtering method for multi-scale enhancement of vascular-like structures, fibrous-like structures, or tubular structures) is applicable to slender or tubular targets such as mitochondria and membrane structures.

[0008] 3. A supervised deep learning segmentation method based on U-Net (U-shaped Convolutional Network, a typical encoder-decoder type convolutional neural network segmentation architecture) and its improved structure, which achieves end-to-end pixel-level segmentation through the encoder-decoder structure.

[0009] 4. General-purpose deep learning cell segmentation tools for microscopic images, such as Cellpose (tool name, a general segmentation model for multiple cell types) and StarDist (tool name, a method that uses star-convex polygons to model the target morphology and improve the segmentation effect of dense targets).

[0010] 5. Segmentation methods based on basic models, such as SAM (Segment Anything Model, a general image segmentation basic model that can segment any target) and its extended μSAM (Segment Anything for Microscopy, i.e., a version for microscopic images) for microscopic scenes.

[0011] 6. Specialized analysis tools for subcellular structures such as mitochondria, such as Mitometer (tool name, used for mitochondrial segmentation and tracking), MoDL (tool name, used for mitochondrial segmentation), and Nellie (tool name, used for organelle segmentation, tracking, and hierarchical feature extraction), are used for segmentation, tracking, and morphological feature extraction.

[0012] While the methods described above improve the efficiency of microscopic image segmentation to some extent, they still have significant limitations in fluorescence microscopy applications. Fluorescence microscopy simultaneously receives fluorescence signals both within and outside the focal plane. Affected by point spread function diffusion and background superposition, images often exhibit strong background interference, reduced contrast, blurred boundaries, and signal adhesion between adjacent structures. For subcellular structures such as mitochondria and cell membranes, which are small, elongated, and topologically complex, these imaging characteristics further exacerbate discontinuities in delicate structures, missed detection of small targets, and misconnection of adjacent targets, thus increasing the difficulty of precise segmentation and subsequent quantitative analysis.

[0013] The core reason why the model used in the above method exhibits the aforementioned defects under a microscope is:

[0014] 1. Anisotropic diffusion of the point spread function (PSF): Microscopes collect a large number of photons outside the focal plane. After PSF convolution, the signal of real cell structures diffuses across the image. Under an optical microscope, due to the diffraction limit, dense areas of mitochondria are often not objects with sharp edges, but rather clusters of blurry point spread function spots. Furthermore, the background haze formed by out-of-focus light reduces the signal-to-noise ratio of the image.

[0015] 2. Traditional image processing methods based on threshold segmentation, mathematical morphology, watersheds, and skeletonization mainly rely on gray-level distribution, local gradients, and manually set parameters to identify targets. When the fluorescent image background is uneven, the signal-to-noise ratio is low, and the boundary transition is unclear, it is easy to misidentify the blurred background as the target or to fragment the continuous mitochondrial network into multiple fragments. As a result, the segmentation results are more sensitive to parameter changes and lack stability.

[0016] 3. While multi-scale structure enhancement methods based on Hessian matrix or Frangi filter can enhance tubular or fibrous structures within a certain range, their effectiveness relies on the target having a relatively clear geometric prior. However, mitochondria in fluorescence images often appear simultaneously as elongated, granular, or clustered shapes, and are more significantly affected by diffraction blurring and out-of-focus hazy backgrounds in high-density regions such as the perinuclear area. Therefore, these methods often struggle to accurately distinguish targets in densely distributed, weakly boundaryed, and complexly overlapping regions.

[0017] 4. Supervised deep learning methods based on U-Net and its improved structures, general deep learning segmentation tools for microscopic images, and basic model segmentation methods based on SAM and μSAM can improve overall segmentation accuracy to some extent. However, their effectiveness usually depends on the coverage of the training data and the consistency between the data distribution and the actual scene. Because the boundaries of subcellular structures in fluorescence images are inherently blurred and manual annotation is difficult, the training data is prone to boundary uncertainties. Furthermore, significant differences exist between different microscopic systems, staining conditions, and signal-to-noise ratio levels, resulting in shortcomings in high-resolution input adaptation, fine-grained boundary recovery, subcellular hierarchical structure recognition, and stability under complex imaging conditions. Summary of the Invention

[0018] The purpose of this invention is to provide a method for intelligent fine segmentation of fluorescence images that can improve the precision of image segmentation, the efficiency of image analysis and the reliability of quantitative results, and enhance the automation and ease of use of image segmentation.

[0019] The objective of this invention is achieved through the following technical measures: a method for intelligent fine segmentation of fluorescence images, characterized by comprising the following steps:

[0020] S1. Acquire fluorescence microscopic images. The user inputs a command expressed in natural language. The command generates word vectors through word frequency statistics and word segmentation. Then, the word vectors are classified and the probability is calculated. If the classification result is a mitochondrial fluorescence microscopic image, proceed to step S2; if the classification result is a cell, cell nucleus, and cell membrane fluorescence microscopic image, proceed to step S3.

[0021] S2. First, the mitochondrial fluorescence microscopy image is decomposed into multiple image patches of a preset resolution. Then, each image patch is used as input and mapped to two independent feature spaces to capture features of sparse intraplasmic regions and dense perinuclear regions, respectively. The capture of sparse intraplasmic region features is achieved by using a large-parameter Frangi filter to capture mitochondrial structures with clear contrast. The capture of dense perinuclear region features... dense The process involves first denoising and restoring the image to improve resolution, then using a small-parameter Frangi filter for micro-tubular confinement, and finally using a dark area segmentation algorithm to extract the optimal segmentation line.

[0022] The segmentation result is obtained by linearly fusing the features of sparse regions within the matrix and the features of dense regions around the kernel, that is, obtaining a fine semantic segmentation mask and an instance segmentation mask, and then proceeding to step S4.

[0023] S3. First, the SAM encoder and decoder are used to segment the fluorescence microscopy images of cells, cell nuclei, and cell membranes respectively. For the fluorescence microscopy images of cell nuclei, after segmentation, the best result is extracted based on the default erosion kernel. For the fluorescence microscopy images of cells, after segmentation, small targets are removed first, then the bright areas in the image are locally segmented, and then the edge gaps and internal holes of each cell mask are filled. For the fluorescence microscopy images of cell membranes, after segmentation, small targets are removed first, then the edge gaps and internal holes of each cell mask are filled, and then the outer cell membrane is obtained through erosion operation. Finally, incomplete instances at the edges are uniformly removed to obtain a refined semantic segmentation mask and an instance segmentation mask, and then proceed to step S4.

[0024] S4, Output Results.

[0025] This invention employs a mitochondrial image segmentation model formed by decoupling fusion and dark area segmentation techniques, thereby improving its ability to segment mitochondrial images.

[0026] The ability to identify and separate fine, variable, and closely adjacent mitochondrial structures in densely packed perinuclear regions can effectively separate complex mitochondrial morphologies from the background in images. In particular, it successfully disentangles physically adjacent but not fused mitochondria, enabling accurate extraction of the true mitochondrial morphology based on this high-precision mask. Furthermore, this invention employs a subcellular structure segmentation model improved upon SAM technology, incorporating local segmentation, dilation and erosion, and cavity filling processes tailored to fluorescence microscopy images, thus enhancing its segmentation capabilities for cellular and subcellular targets such as cells, nuclei, and cell membranes. Additionally, this invention utilizes natural language processing technology, allowing users to describe the segmentation task using natural language, automatically completing model selection, parameter configuration, and segmentation processing, thereby improving the automation and ease of use of microscopic image segmentation.

[0027] This invention analyzes the instance segmentation mask in the segmentation results of mitochondrial fluorescence microscopy images, extracts multidimensional morphological features, and constructs a comprehensive mitochondrial fragmentation index (MFI) calculation model:

[0028]

[0029] MFI stands for Comprehensive Mitochondrial Fragmentation Index; This represents the density of mitochondrial instances per unit area; N is the total number of mitochondrial instances. This represents the total number of pixels across all mitochondrial instances. The average ratio of major to minor axis; Average firmness; , , The weighting coefficients preset for the system.

[0030] The formula for calculating the major-minor axis ratio in this invention is as follows:

[0031]

[0032] in, Let be the ratio of the major axis to the minor axis of the i-th mitochondrial instance; Let be the length of the major axis of the i-th mitochondrial instance; Let be the length of the minor axis of the i-th mitochondrial instance;

[0033] The formula for calculating firmness is as follows:

[0034]

[0035] The target region area refers to the total number of actual pixels contained in a single mitochondrial instance in the mask image; the convex hull area of ​​the target region refers to the area contained in the smallest convex polygon that can completely enclose the mitochondrial instance.

[0036] This invention analyzes the instance segmentation mask in the segmentation results of fluorescence microscopy images of cell nuclei, cells, and cell membranes, and extracts multidimensional morphological features. It can be used for many other quantitative or qualitative biological applications, such as cell classification, cell mitosis cycle determination, differentiation between cancer cells and normal cells, differentiation between live cells and dead cells, and drug toxicity studies.

[0037] This invention uses the semantic segmentation mask from the fluorescence microscopy image segmentation results of cell nuclei, cells, and cell membranes as the mask for calculating quantitative FRET, achieving FRET efficiency E. d The calculation formula is as follows:

[0038] ; ;

[0039] Where Fc is the receptor-sensitized fluorescence intensity, a, b, c, and d are spectral crosstalk factors, and G is the system parameter.

[0040] In step S3 of this invention, the average brightness of the bright area is more than 3 times the average brightness of the entire image.

[0041] In step S3 of this invention, after the cell, cell nucleus and cell membrane fluorescence microscopy images are segmented, the size of the cell nucleus, the size of the cell and the thickness of the cell membrane can be generated by controlling the expansion and erosion operations according to user instructions.

[0042] Compared with the prior art, the present invention has the following significant effects:

[0043] (1) The fluorescence microscopy image mitochondrial decoupling and fusion segmentation model of the present invention, combined with the optical point spread function (PSF) denoising and deconvolution method, "penetrates" physical blur at the algorithm level to achieve sub-pixel level edge recovery. This solves the problem that existing deep learning models such as DCNN are purely data-driven and are easily affected by microscope out-of-focus background light and diffraction limit interference.

[0044] (2) The fluorescence microscopy image mitochondrial decoupling and fusion segmentation model of the present invention adopts decoupling and fusion and dark area segmentation technology to ensure accurate segmentation of "complex network topology" while being compatible with segmentation of "small spherical fragments", thus solving the problem that existing models such as Nellie rely on tubular filtering and are not sensitive to small spherical circles.

[0045] This invention combines high-precision segmentation with MFI calculation and proposes a quantitative assessment model of MFI based on multi-feature weighting. Due to the improvement in segmentation accuracy, the calculation of MFI is transformed from "fuzzy estimation" to "precise quantification", thereby significantly improving the sensitivity of early PD diagnosis and the accuracy of drug screening. Existing MFI assessment is still in the stage of qualitative analysis.

[0046] This invention employs a subcellular structure segmentation model improved upon SAM technology, and incorporates local segmentation, dilation and erosion, and void filling processes tailored to fluorescence microscopy images.

[0047] Enhance its segmentation capabilities on cellular and subcellular targets such as cells, nuclei, and cell membranes.

[0048] This invention employs algorithms such as local post-processing, dilation and erosion, introduction of deconvolution, and dark area segmentation to achieve more refined semantic and instance segmentation of cells, cell nuclei, cell membranes, and mitochondria.

[0049] This invention employs natural language processing technology, enabling users to describe segmentation tasks using natural language.

[0050] It automatically completes model selection, parameter configuration, and segmentation processing to generate the most expected segmentation results, improving the automation and ease of use of microscopic image segmentation. It aims to overcome the industry pain points of imprecise segmentation, low automation, and weak cross-sample generalization of complex and tiny organelles under fluorescence microscopy. Attached Figure Description

[0051] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0052] Figure 1 This is one of the flowcharts of the present invention;

[0053] Figure 2 This is the second flowchart of the present invention;

[0054] Figure 3 This is the third flowchart of the present invention;

[0055] Figure 4 These are segmentation results using the present invention and the Nellie method (from left to right: original mitochondrial image, segmentation result using the Nellie method, and segmentation result using the present invention).

[0056] Figure 5 These are fine-grained localization maps of the FRET occurrence area using the present invention and existing methods (from left to right: RAW map, Ed map using existing methods, and Ed map using the present invention). Detailed Implementation

[0057] The present invention will be further described below through specific embodiments, but this is not a limitation of the present invention. Those skilled in the art can make various modifications or improvements based on the basic idea of ​​the present invention, but as long as they do not depart from the basic idea of ​​the present invention, they are all within the protection scope of the present invention.

[0058] like Figures 1-3 As shown, the present invention provides a method for intelligent fine segmentation of fluorescence images, comprising the following steps:

[0059] S1. Obtain a fluorescence microscopic image. The user inputs a command expressed in natural language. This command generates word vectors through TF-IDF word frequency statistics and Jieba word segmentation. Then, SVM support vector machine is used as a text decoder to calculate the classification probability of the word vectors. If the classification result is a fluorescence microscopic image of mitochondria, proceed to step S2.

[0060] If the classification result is a fluorescence micrograph of a cell, cell nucleus, and cell membrane, proceed to step S3;

[0061] S2. First, decompose the fluorescence microscopy image (e.g., 2048×2048 pixels) into multiple image blocks of a preset resolution (e.g., 1024×1024 pixels). This reduces the memory consumption of the subsequent denoising deconvolution model and preserves high-resolution local details. Then, use the mitochondrial decoupling and fusion segmentation model to generate a binarization mask.

[0062] Based on the differences in mitochondrial distribution at different locations, each image patch is input as I raw Mapped to two independent feature spaces, each capturing the intramolecular sparse region features S. sparse Features of densely packed perinuclear regions S dense ;

[0063] Among them, the feature S of the sparse region within the mass is captured. sparse This is achieved by using a large-parameter Frangi filter (setting a low sensitivity parameter, i.e., setting a large value for β, such as 0.5) to capture mitochondrial structures with clear contrast.

[0064]

[0065] Among them, the multi-scale tubular enhancement operator This represents the multi-scale Frangi filter of the present invention; β is the sensitivity parameter. This operator is used to enhance the tubular structure of mitochondria.

[0066] Perinuclear dense region characteristics S dense The process involves first denoising and restoring the image to improve resolution, then using a small-parameter Frangi filter for micro-tubular confinement, and finally employing a dark area segmentation algorithm to extract the optimal segmentation line.

[0067]

[0068] Among them, the deconstruction operator This represents the denoising extraction and restoration mapping based on the ZS-DeconvNet pre-trained model; dark area segmentation operator. It is a segmentation algorithm specifically designed for clustered regions:

[0069]

[0070] in, Represents the final selected optimal dividing line; minimum value comparison operator. This indicates selecting the element from the candidate set that minimizes the function value; This indicates a candidate line that is currently being evaluated; This refers to the total number of pixels contained in the candidate line L, i.e., the length of the line; This is for normalization, to prevent short candidate lines from scoring low due to fewer pixels, and to ensure that the comparison is made on "average darkness" rather than "total darkness"; Represents each pixel on the candidate line; This represents the brightness value of the corresponding pixel in the original fluorescence image.

[0071] Segmentation result I seg Characteristic S of sparse regions within the mass sparse Features of densely packed perinuclear regions S dense The linear fusion yields:

[0072]

[0073] This process yields a high-precision mitochondrial semantic segmentation mask and an instance segmentation mask. This step successfully separates the complex mitochondrial morphology from the background in fluorescence microscopy images, particularly successfully untying physically adjacent but not fused mitochondria. Based on this high-precision mask, accurate extraction of the true mitochondrial morphology becomes possible.

[0074] This invention provides a weighted mathematical model for calculating the comprehensive mitochondrial fragmentation index (MFI) based on segmentation results. Specifically, it involves analyzing the instance segmentation mask in the mitochondrial fluorescence microscopy image segmentation results to identify each independent mitochondrial instance within the cell. For each independent instance, multidimensional morphological features are extracted, including but not limited to: area, long axis, short axis, aspect ratio, and firmness, transforming pixel-level image data into a structured numerical matrix. Since a single morphological feature cannot fully describe the complex degradation process of mitochondria from a "network-like" structure to "fragmentation," this data is used to construct a comprehensive mitochondrial fragmentation index (MFI) calculation model. Because pathological or drug-induced fragmentation is characterized by reduced mitochondrial monomer size, a tendency towards spherical shape (decreased aspect ratio, increased firmness), and broken network branches, the calculation formula is as follows:

[0075]

[0076] MFI stands for Comprehensive Mitochondrial Fragmentation Index; This represents the density of mitochondrial instances per unit area; N is the total number of mitochondrial instances. This represents the total number of pixels across all mitochondrial instances. The average ratio of major to minor axis; Average firmness; , , The weight coefficients preset for the system need to be obtained through statistical regression training on healthy cell lines and fragmented cell lines after pathological or drug treatment.

[0077] The formula for calculating the ratio of the major axis to the minor axis is as follows:

[0078]

[0079] in, Let be the ratio of the major axis to the minor axis of the i-th mitochondrial instance. Let be the length of the major axis of the i-th mitochondrial instance. Let be the minor axis length of the i-th mitochondrial instance. The formula for calculating firmness is as follows:

[0080]

[0081] The target region area refers to the total number of actual pixels contained in a single mitochondrial instance in the mask image, while the convex hull area of ​​the target region refers to the area contained in the smallest convex polygon that can completely enclose the mitochondrial instance. By combining high-precision segmentation with quantitative analysis, the calculation of MFI has been transformed from "fuzzy estimation" to "precise quantification." The calculated MFI is a precise quantitative indicator that can be subsequently applied to clinical auxiliary diagnosis or high-throughput drug screening.

[0082] Proceed to step S4;

[0083] S3. To achieve more precise segmentation of cells, nuclei, and cell membranes, a SAM encoder and decoder are used to segment the fluorescence microscopy images and perform post-processing:

[0084] ① For fluorescence micrographs of cell nuclei, after segmentation, the best result is extracted based on the default erosion kernel (default erosion algorithm with a kernel size of 3 and smoothed boundaries). The size of the generated kernel can be controlled by dilation and erosion operations according to the user's natural language (instructions).

[0085] ② For the fluorescence microscopy image of cells, after segmentation, small targets are removed first, and then local threshold segmentation is performed on the bright areas in the image (the average brightness of the area is more than 3 times the average brightness of the whole image). Then, the edge gaps and internal cavities of each cell mask are filled, and the size of the generated cells can be controlled by dilation and erosion operations according to the user's natural language.

[0086] Local thresholding segmentation uses local Otsu thresholding to assist in specular contour capture, thereby accurately identifying the contour of each specular instance. This prevents the model from failing to locate specular contours while fully segmenting dark contours, thus improving the accuracy of the segmentation model. For images Arbitrary split instance Its local average brightness is defined as:

[0087]

[0088] in, This represents the total number of pixels in the instance. If the instance is determined to be a highlight region, the following inequality must be satisfied:

[0089]

[0090] in, This represents the global average brightness, and r is the highlight ratio.

[0091] ③ For fluorescence micrographs of cell membranes, after segmentation, small targets are removed first, then the edge gaps and internal cavities of each cell mask are filled, and then the outer cell membrane is obtained through erosion. The thickness of the generated cell membrane can be controlled by dilation and erosion operations according to the user's natural language.

[0092] The formal expression of the outer cell membrane is obtained as follows:

[0093]

[0094]

[0095] The initial binary cell mask is defined as follows: Remove small target operator Area opening operations are typically used, while morphological closing operations are applied to fill gaps at the mask edges; that is, expansion followed by erosion. Indicates expansion. The structuring element representing erosion and the closing operation is: Then use Operators process internal voids to obtain refined cell masks. The structural element of the erosion operation is defined as follows: Its scale (i.e., the size of the corrosion nucleus) is defined as Finally, by using the combined difference between the fine mask and the erosion mask, the thickness of the outer cell membrane, determined by the size of the erosion nucleus, was calculated. .

[0096] Finally, incomplete instances at the edges are uniformly removed to eliminate the interference of incomplete instances at the image edges, resulting in a refined semantic segmentation mask and instance segmentation mask.

[0097] High-precision mask-based connected component analysis and multidimensional morphological feature extraction: Instance segmentation masks in fluorescence microscopy images of cell nuclei, cells, and cell membranes are analyzed to identify each independent target instance (cell / nucleus / cell membrane). For each independent instance, multidimensional morphological features are extracted, including but not limited to: area, major axis, minor axis, aspect ratio, and firmness, transforming pixel-level image data into a structured numerical matrix.

[0098] This invention provides a method for locating FRET occurrence regions based on segmentation results and enhancing the interpretability of quantitative FRET analysis results. Specifically, it uses semantic segmentation masks from fluorescence microscopy image segmentation results of cell nuclei, cells, and cell membranes as masks when calculating quantitative FRET, thereby improving FRET efficiency E. d The calculation formula is as follows:

[0099] ; ;

[0100] Where Fc is the receptor-sensitized fluorescence intensity, a, b, c, and d are spectral crosstalk factors, and G is the system parameter.

[0101] Combining high-precision segmentation with quantitative FRET analysis makes the location of FRET occurrence areas more accurate and improves the accuracy of quantitative FRET analysis.

[0102] S4. Output results, including segmentation results, multidimensional mitochondrial morphological features, and comprehensive mitochondrial fragmentation.

[0103] MFI index, multidimensional morphological characteristics of cells, nuclei and cell membranes, and FRET efficiency (E) d .

[0104] The segmentation models of this invention include a mitochondrial decoupling and fusion segmentation model for fluorescent microscopy images (used for segmenting mitochondrial fluorescent microscopy images) and a subcellular target segmentation model for fluorescent microscopy images based on SAM (used for segmenting cells, cell nuclei, and cell membranes in fluorescent microscopy images).

[0105] The segmentation model of this invention is now compared with existing advanced segmentation models. The Dice coefficient is used to evaluate the semantic segmentation effect and the F1 score is used to evaluate the instance segmentation result.

[0106] Dice coefficient: Primarily used to evaluate the similarity between a predicted region and the ground truth, it is widely used in the field of medical imaging. Its formula is defined as:

[0107]

[0108] Where A refers to the model prediction region, B refers to the ground truth label region, |A∩B| represents the number of pixels at the intersection of the prediction and ground truth regions, and |A|+|B| represents the total number of pixels in the prediction and ground truth regions.

[0109] Instance precision and recall: These are specifically used to evaluate instance segmentation models. In instance segmentation, the criterion for determining whether a predicted mask is "correct" typically relies on an IoU threshold (commonly 0.5). According to the Mask R-CNN research, a prediction is considered a true instance (TruePositive, TP) only when the IoU between the predicted mask and the ground truth mask is ≥threshold.

[0110] Instance precision measures the proportion of true positive instances out of all targets predicted as positive by the model. It reflects the "reliability" or "purity" of the model's predictions. Its formula is:

[0111]

[0112] FP (false positives) represents non-target objects or background that the model incorrectly identifies.

[0113] Instance recall: Recall measures the proportion of real-world objects in an image that are correctly predicted by the model. It reflects the model's "capture ability" or "recall rate" of the target. Its formula is expressed as:

[0114]

[0115] Here, FN (false negatives) represents the true targets that the model misses.

[0116] The F1 score, the harmonic mean of precision and recall, is an important tool for comprehensively evaluating the performance of instance segmentation models. Its formula is as follows:

[0117]

[0118] The segmentation model of this invention was compared with the Dice coefficient and F1 score of existing advanced models at home and abroad. The dataset included 80 MCF-7 cells expressing CY protein, 146 BPAE cell nuclei stained with DAPI, 199 293T cell membranes expressing CY protein, and 1049 BPAE bovine pulmonary artery mitochondria stained with Mito-tracker. Table 1 compares the segmentation performance of the method of this invention with that of the Nellie method. As can be seen from Table 1, the model of this invention improved the Dice coefficient and F1 score by 5.6% and 41.7% respectively compared with the Nellie model. The comparison of the segmentation effect of the model of this invention with the Nellie model is as follows. Figure 4 As shown.

[0119]

[0120] (Table 1)

[0121] Table 2 compares the segmentation performance of the proposed method with the SAM3 method. As can be seen from Table 2, the proposed model has significant advantages in cell segmentation, cell membrane segmentation, and cell nucleus segmentation, and is significantly stronger than general image segmentation methods.

[0122] Cut model SAM3.

[0123]

[0124] (Table 2)

[0125] Example:

[0126] Microscopic image acquisition: Image data were acquired using the FRETSCOPE live-cell molecular interaction instrument, a product developed in our laboratory. Observation was performed using an Olympus 60X oil immersion microscope, and images were captured using a Hamamatsu C13440-20CU camera. The mitochondrial images are mitochondria from BPAE bovine pulmonary arteries stained with Mito-tracker. The cell images are C5V co-transfected samples of MCF-7 cells transfected with the CV standard plasmid.

[0127] 1. Segmenting the mitochondrial image:

[0128] ①Pretreatment

[0129] To reduce memory consumption, the 2048*2048 pixel mitochondrial fluorescence microscopy image was segmented into 1024*1024 pixel slices (image blocks).

[0130] ② Decoupling process

[0131] The mitochondrial fluorescence microscopy images were then divided into sparse intraplasmic components (features of sparse intraplasmic regions) and dense perinuclear components (features of dense perinuclear regions) according to their distribution location differences, and were processed separately.

[0132] ③ Noise reduction deconvolution

[0133] For the dense component region around the nucleus, the ZS-Convnet model is used to perform noise reduction and restoration processing on the image, resulting in an image with noise and out-of-focus fluorescence removed.

[0134] ④ Multi-scale Frangi filter enhancement

[0135] Multi-scale Fragi filtering enhancement was performed on the sparse component region within the mass and the dense component region around the nucleus to obtain tubular and circular enhanced component images.

[0136] ⑤ Dark area segmentation

[0137] For the enhanced perinuclear dense component region, the dark area segmentation algorithm is used to segment larger individual instances and remove adhesion.

[0138] ⑥ The fused components are used to obtain a segmentation mask.

[0139] The processed intraplasmic sparse components are fused with the perinuclear dense components to obtain the semantic segmentation mask and instance segmentation mask for the mitochondrial fluorescence image. If the user only wants the mitochondria in the center region of the image, they can continue to input the natural language command "segment the mitochondria in the center region of the image," and the model will extract the mitochondrial mask of the center region, generating semantic and instance segmentation masks containing only the mitochondria in the center region.

[0140] ⑦ Calculate multiple feature dimensions

[0141] Based on the instance analysis results in the above steps, high-throughput analysis was performed to obtain features such as the number of instances, the major axis, minor axis, major-to-minor axis ratio, area statistics, and firmness in the image. Ultimately, 269 mitochondria were obtained.

[0142] ⑧ Calculate MFI

[0143] Based on data from 269 mitochondrial instances within the aforementioned images, the mitochondrial index was calculated. Total mitochondrial area: = 81609 pixels. Mitochondrial density per unit area: (The value is relatively small, reflecting a low degree of mitochondrial fragmentation.) The aspect ratio of all individual instances was calculated to obtain the average aspect ratio. = 2.42 (A relatively high value, indicating a higher proportion of slender structures). Average firmness = 0.82 (The value is not very high, reflecting that the shape is not very smooth).

[0144] Substituting into the comprehensive mitochondrial fragmentation index (MFI) calculation model: assuming the preset weighting coefficients are... = 100, =1, = 1, then .

[0145] 2. Segment the cell image:

[0146] Based on the segmentation of the SAM encoder and decoder, the cell regions in the cell image are initially segmented.

[0147] Instance segmentation yields a candidate cell instance mask.

[0148] ② Perform connected component analysis on the obtained initial instance segmentation results, count the pixel area of ​​each instance region, and remove small targets with an area smaller than a preset threshold.

[0149] ③ For each retained candidate cell instance, calculate the average brightness within its instance region and compare it with the average brightness of the entire image to identify abnormally bright cell regions. Instances with an average brightness greater than three times the average brightness of the entire image are considered bright regions.

[0150] ④ For the identified bright region instances, perform local threshold segmentation within the local area of ​​the corresponding instance to refine the instance mask.

[0151] ⑤ Perform mask repair on the updated cell instance mask to fill edge gaps and internal voids, improving the integrity of the cell outline.

[0152] ⑥ To meet the need for adjusting the cell mask area in different usage scenarios, this embodiment also supports users to controllably adjust the cell size through natural language input. Based on the obtained semantic parsing results, corresponding morphological operations are performed on the cell instance mask to adjust the final cell region size.

[0153] ⑦ Based on the instance analysis results in the above steps, perform high-throughput analysis to obtain features such as the number of instances, the major axis, minor axis, major-minor axis ratio, area statistics, and solidity in the image.

[0154] ⑧ Based on the semantic segmentation mask of cell instances in the three-channel image, the FRET efficiency is calculated. After pixel-by-pixel calculation using the segmentation mask, the spectral crosstalk factors are obtained as a = 0.2071, b = 0.0356, c = 0.0094, d = 0.4390, and the system parameters are G = 2.3695 and k = 1.4785. Therefore, the FRET efficiency E... d =0.435, which is only 1.3% different from the literature value of 0.43 (see Table 3). It can be seen that the present invention is superior to the hand loop method and the empirical method with a 3 times threshold.

[0155]

[0156] (Table 3)

[0157] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A method for intelligent fine segmentation of fluorescence images, characterized in that... Includes the following steps: S1. Acquire fluorescence microscopic images. The user inputs a command expressed in natural language. The command generates word vectors through word frequency statistics and word segmentation. Then, the word vectors are classified and the probability is calculated. If the classification result is a mitochondrial fluorescence microscopic image, proceed to step S2; if the classification result is a cell, cell nucleus, and cell membrane fluorescence microscopic image, proceed to step S3. S2. First, the mitochondrial fluorescence microscopy image is decomposed into multiple image patches of preset resolution. Then, each image patch is used as input and mapped to two independent feature spaces to capture the features of sparse intraplasmic regions and dense perinuclear regions, respectively. Among them, the sparse intraplasmic region features are captured by using a large-parameter Frangi filter to capture mitochondrial structures with clear contrast. The dense perinuclear region features are first denoising and restoring to improve image resolution, then using a small-parameter Frangi filter for micro-tubular confinement, and finally using a dark area segmentation algorithm to extract the optimal segmentation line. The segmentation result is obtained by linearly fusing the sparse intraplasmic region features and the dense perinuclear region features, that is, obtaining a fine semantic segmentation mask and an instance segmentation mask, and then proceeding to step S4. S3. First, the SAM encoder and decoder are used to segment the fluorescence microscopy images of cells, cell nuclei, and cell membranes respectively. For the fluorescence microscopy images of cell nuclei, after segmentation, the best result is extracted based on the default erosion kernel. For the fluorescence microscopy images of cells, after segmentation, small targets are removed first, then the bright areas in the image are locally segmented, and then the edge gaps and internal holes of each cell mask are filled. For the fluorescence microscopy images of cell membranes, after segmentation, small targets are removed first, then the edge gaps and internal holes of each cell mask are filled, and then the outer cell membrane is obtained through erosion operation. Finally, incomplete instances at the edges are uniformly removed to obtain a refined semantic segmentation mask and an instance segmentation mask, and then proceed to step S4. S4, Output Results.

2. The intelligent fine segmentation method for fluorescence images according to claim 1, characterized in that: After completing step S2, the instance segmentation mask in the mitochondrial fluorescence microscopy image segmentation results is analyzed, multidimensional morphological features are extracted, and a comprehensive mitochondrial fragmentation index (MFI) calculation model is constructed: MFI stands for Comprehensive Mitochondrial Fragmentation Index. This represents the density of mitochondrial instances per unit area; N is the total number of mitochondrial instances. This represents the total number of pixels across all mitochondrial instances. The average ratio of major to minor axis; Average firmness; , , The weighting coefficients preset by the system; The calculated comprehensive mitochondrial fragmentation index (MFI) is output as the result.

3. The intelligent fine segmentation method for fluorescence images according to claim 2, characterized in that: The formula for calculating the ratio of the major axis to the minor axis is as follows: ;in, Let be the ratio of the major axis to the minor axis of the i-th mitochondrial instance; Let be the length of the major axis of the i-th mitochondrial instance; Let be the length of the minor axis of the i-th mitochondrial instance; The formula for calculating firmness is as follows: The target region area refers to the total number of actual pixels contained in a single mitochondrial instance in the mask image; the convex hull area of ​​the target region refers to the area contained in the smallest convex polygon that can completely enclose the mitochondrial instance.

4. The intelligent fine segmentation method for fluorescence images according to claim 3, characterized in that: After completing step S3, the instance segmentation mask in the fluorescence microscopy image segmentation results of cell nucleus, cell and cell membrane is analyzed to extract multidimensional morphological features, which are then output as the result.

5. The intelligent fine segmentation method for fluorescence images according to claim 4, characterized in that: After completing step S3, the semantic segmentation mask from the fluorescence microscopy image segmentation results of the cell nucleus, cell, and cell membrane is used as the mask when calculating the quantitative FRET, and the FRET efficiency E d The calculation formula is as follows: ; Where Fc is the receptor-sensitized fluorescence intensity, a, b, c, and d are spectral crosstalk factors, and G is a system parameter; The calculated FRET efficiency E d As the result, output it.

6. The intelligent fine segmentation method for fluorescence images according to claim 5, characterized in that: In step S3, after the cell, cell nucleus and cell membrane fluorescence microscopy images are segmented, the size of the cell nucleus, the size of the cell and the thickness of the cell membrane can be generated by controlling the expansion and erosion operations according to the user's instructions.

7. The intelligent fine segmentation method for fluorescence images according to claim 6, characterized in that: In step S3, the average brightness of the bright areas is more than 3 times the average brightness of the entire image.

Images