Subcellular structure image segmentation and extraction method based on double exponential decay function

Abstract

The present application relates to a kind of subcellular structure image segmentation extraction method based on double exponential decay function.The present application constructs three double exponential decay standard curves, and the gray value time series data of each pixel point is extracted in turn, and double exponential decay function fitting is carried out after being normalized according to maximum value, error comparison is carried out to the fitting result obtained with three standard curves, and preliminary segmentation result is obtained.Combined with image morphological operation, segmentation result is optimized.High-quality subcellular structure microscopic image-segmentation result data set is constructed, and deep learning model is trained;Finally, the efficient subcellular structure segmentation extraction model for single frame microscopic image is obtained.The present application overcomes the problem of insufficient segmentation accuracy of traditional image segmentation method in complex background, solves the problem that deep learning technology cannot be used to realize efficient subcellular structure segmentation due to the difficulty in obtaining high-quality training data set, and provides efficient and accurate technical means for cell structure analysis and multi-target segmentation in complex background.

Description

Technical Field

[0001] This invention relates to the fields of microscopic image processing and biomedical image analysis, specifically a method for subcellular structure image segmentation and extraction based on a double exponential decay model of fluorescence signals. This method aims to improve the efficiency and robustness of image segmentation and extraction in biomedical research, and is suitable for cutting-edge research in cell biology and molecular biology. Background Technology

[0002] With the rapid development of molecular biology and cell biology, studying the dynamic distribution and interactions of subcellular structures has become a core topic for understanding the nature of life activities and disease mechanisms. Subcellular structures (such as the nucleus, mitochondria, Golgi apparatus, and endoplasmic reticulum) play important roles in maintaining cell function, regulating metabolic activities, and responding to external stimuli. Accurate localization and analysis of these subcellular structures not only helps in exploring basic cellular functions but also provides important evidence for disease diagnosis and drug development.

[0003] Fluorescence microscopy, as an important tool for studying the internal structure of cells, is widely used in biomedical research due to its high sensitivity and non-invasiveness. Based on this technology, massive, high-quality datasets of subcellular structure microscopic images can now be acquired. Automatically and efficiently segmenting and extracting regions of interest from microscopic image data is a crucial step in supporting downstream quantitative analysis tasks. Traditional object segmentation algorithms mainly divide images into different regions based on features such as color, texture, and shape, resulting in differences between regions and similarities within regions. Classic traditional image segmentation methods include threshold-based segmentation, edge-based segmentation, region-based segmentation, and graph-based segmentation.

[0004] While traditional image segmentation algorithms have been widely applied in various fields, they also have many limitations. First, traditional image segmentation algorithms often lack sufficient representational power, making it difficult to capture complex structures and details in images, which may lead to inaccurate segmentation results. Furthermore, traditional algorithms typically require long computation times and high computational resources when processing large-scale images, limiting their use in real-time applications. Finally, traditional algorithms are often designed for specific application scenarios and therefore have limited applicability, lacking versatility. Therefore, in practical applications, users often need to spend considerable time and effort experimenting and adjusting to select the appropriate algorithm.

[0005] In recent years, the application of deep learning technology in biomedical image analysis has developed rapidly. Deep learning models, represented by U-Net and its improved versions (such as U-Net++ and Attention U-Net), have been widely used in tasks such as cell segmentation and subcellular structure detection. These models are trained using large amounts of labeled data and can accurately identify structural features in microscopic images. High-quality training datasets are crucial for the success of deep learning models. However, in the field of subcellular structure segmentation, manually labeled training datasets are scarce, data labeling is costly and time-consuming, and often results in insufficient segmentation accuracy due to data imbalance within the dataset. Therefore, how to efficiently generate high-quality deep learning training sets has become an important research direction in this field.

[0006] Fluorescence signals exhibit unique bleaching fluctuation characteristics in continuously acquired microscopic images, which are significantly different from background signals (e.g., Figure 1 (As shown). This invention proposes a subcellular structure image segmentation and extraction method based on a double exponential decay function. It utilizes the unique decay rate of fluorescent molecules over time, distinct from the background signal, and combines a mathematical model with image processing algorithms to achieve accurate subcellular structure segmentation. The segmented images are then optimized to obtain a microscopic image-segmentation result dataset. Finally, a fast and efficient subcellular structure segmentation and extraction model is trained using a U-Net network. Using this trained deep learning model, high-quality and rapid segmentation and extraction of subcellular structures from a single frame of microscopic image can be achieved. Summary of the Invention

[0007] The purpose of this invention is to provide a subcellular structure image segmentation and extraction method based on a double exponential decay function.

[0008] Specifically, the present invention is as follows:

[0009] First, continuous time-series images of fluorescently labeled subcellular structures in dead cells are acquired to obtain rich grayscale fluctuation information. N frames of continuous time-series images are acquired for each imaging field. In the first frame of the N-frame time-series image set for each imaging field, a set of pixels representing subcellular structures, a set of pixels representing intracellular background, and a set of pixels representing extracellular background are selected to construct a standard curve. Each set includes M different pixels.

[0010] Then, after normalizing the grayscale data of each group of M pixels in the N-frame time sequence image, the standard curve is fitted by the double exponential decay function to obtain the subcellular structure standard decay curve, the intracellular background standard decay curve, and the extracellular background standard curve.

[0011] Then, all pixels in each imaging field of view are classified. The grayscale data of each pixel in N frames of time-series images are read, and all grayscale data of each pixel are normalized before curve fitting. The fitted curve is compared with three standard curves to achieve pixel classification. After traversing all pixels in each imaging field of view, a preliminary segmentation map is output based on the classification results.

[0012] Then, the skeletonization algorithm is used to extract the subcellular structure skeleton map from the first frame of the microscopic image of each imaging field. Combined with image morphology operations, the preliminary segmentation map is filled and shrunken to optimize the preliminary segmentation map and obtain a segmentation extraction result map with coherent structure and high accuracy.

[0013] Finally, following the above process, data from F different imaging fields are processed to obtain F segmentation and extraction result images. For each imaging field, its first frame of microscopic image and its corresponding segmentation and extraction result image are taken to construct a microscopic image-segmentation and extraction result image data pair. Using these F data pairs, a U-Net network is trained to obtain a high-quality subcellular structure segmentation and extraction model. Any single frame of acquired fluorescence microscopic subcellular structure image is input into the trained subcellular structure segmentation and extraction model to efficiently and accurately obtain the segmented and optimized subcellular structure image.

[0014] To ensure the accuracy of model training, the N... 100, F 50.

[0015] Furthermore, the aforementioned double exponential decay function is: Where a, b, c, d are the parameters to be fitted, and x is the time series variable. These represent the image grayscale values ​​at the corresponding time points. During the standard curve fitting process, a nonlinear least squares method is used to optimize the fitting parameters a, b, c, and d to minimize the mean square error between the fitted curve and the actual data.

[0016] Furthermore, the classification process for each pixel is as follows:

[0017] The grayscale value of this pixel in each frame of the image Divide by its maximum gray value throughout the entire time series. The normalized grayscale value is obtained as follows: ,in This represents the grayscale value of the pixel in the t-th frame of the image.

[0018] Calculate the mean square error between the normalized gray value of this pixel and the three standard curves in each frame of the image. Where k=1 represents the standard curve on the subcellular structure, k=2 represents the standard curve of the intracellular background, and k=3 represents the standard curve of the extracellular background. Let be the normalized grayscale value of the pixel to be classified at time t. Let N be the value of the standard curve for category k at time point t, and N be the number of image frames acquired.

[0019] The above formula is used to calculate the mean square error of the normalized gray value sequence of each pixel relative to the three standard curves. The minimum mean square error value is obtained. ;

[0020] The calculated minimum mean square error value Compare with the set background threshold and target threshold (background threshold > target threshold):

[0021] like If the value is greater than the background threshold, the pixel is classified as background.

[0022] like The value is less than the background threshold, and the calculated value is... If the k value is not 1, the pixel is classified as background.

[0023] like The value is less than or equal to the background threshold, and the calculated value is... The value of k is 1, for Further comparison of the values: If If the value is less than the target threshold, the pixel is classified as a subcellular structure; if If the value is greater than or equal to the target threshold, the pixel is classified as background.

[0024] Furthermore, the optimization process of the preliminary segmentation map is as follows:

[0025] The Guo-Hall thinning algorithm was used to process the first frame of the microscopic image, iteratively deleting pixels until the skeleton converged to a single-pixel-wide centerline. In each iteration, the algorithm checked each pixel to see if it met the deletion criteria. If so, the pixel was marked for deletion. The subcellular skeleton was extracted as a reference for the basic structure. The extracted skeleton image was combined with the preliminary segmentation image, prioritizing the preservation of structural details to ensure overall coherence.

[0026] Using the skeleton image as a mask, the continuity and detail integrity of the cell structure in the segmentation image are ensured. Broken parts at the skeleton connection points in the segmentation image are repaired to maintain the integrity of the cell structure.

[0027] Fill in the highlighted areas of the original image to repair holes or breaks in the segmentation image caused by errors. Extract the highlighted areas from the original image, use logical operations to fill holes in the segmentation image, perform a bitwise AND operation between the segmentation image and the highlighted areas to enhance the foreground pixels of the highlighted areas, and use a small-area filling algorithm to repair small holes. For overly inflated areas in the segmentation image, apply local erosion to better align them with the original image. The erosion operation reduces the highlighted areas, eliminates misclassified background pixels, and improves the accuracy of the segmentation image.

[0028] Furthermore, for the aforementioned microscopic image-segmentation result image data pairs, random image cropping and random reversal operations are combined to augment the dataset. The augmented dataset is then used to train the U-Net network, thereby improving the accuracy of model training.

[0029] This invention leverages the prior knowledge that fluorescence signals exhibit unique characteristics over time, distinct from background signals, and constructs a standard decay function using a double exponential decay function. By iterating through the decay curve of each pixel in the fitted image and comparing it with the standard curve, pixel-level classification is achieved, yielding preliminary segmentation results. Subsequently, image morphology processing is combined to achieve precise segmentation of cells and their background, constructing a high-quality subcellular structure segmentation dataset. A deep learning model is then used to train the subcellular structure segmentation model, ultimately achieving efficient and stable segmentation of subcellular structures in single-frame fluorescence microscopy images. Addressing the characteristics of fluorescence signal decay over time, the double exponential decay model and the dynamic changes in pixel grayscale values ​​enable precise segmentation of cellular substructures, providing an accurate and effective training dataset for the deep learning model. The subsequent integration of a deep learning model enables high-quality subcellular structure segmentation using single-frame fluorescence microscopy images. This effectively solves the problems of insufficient accuracy and robustness in subcellular structure segmentation of microscopic images using existing image segmentation techniques; it maintains high segmentation accuracy even under complex backgrounds and noise interference, providing efficient and reliable technical support for biomedical image analysis, and is particularly suitable for cell structure analysis and research in biomedical images. Attached Figure Description

[0030] Figure 1 An example diagram showing the different fluctuation characteristics of pixels in different regions over time;

[0031] Figure 2 This is a flowchart illustrating the overall technical process of the present invention.

[0032] Figure 3 A schematic diagram of the fitting process for the double exponential decay function;

[0033] Figure 4 A flowchart illustrating the pixel classification function;

[0034] Figure 5A flowchart illustrating the process of skeleton extraction and segmentation map optimization;

[0035] Figure 6 This is a schematic diagram illustrating the segmentation effects of cell images under different processing steps.

[0036] Figure 7 This is a schematic diagram illustrating the training of a deep learning segmentation model. Detailed Implementation

[0037] The invention will be further described below with reference to the accompanying drawings. For ease of description, in this embodiment, 50 fields of view F are selected, and 100 frames of continuous time-series images are acquired, with the number of image frames N=100; each group of pixels in the standard curve includes 10 pixels, M=10. In practical applications, to ensure the accuracy of model training, F is set... 50. N 100, M 10.

[0038] like Figure 2 As shown, a subcellular structure image segmentation and extraction method based on a double exponential decay function specifically includes the following steps:

[0039] Step 1: Acquire time-series microscopic images rich in fluorescence signal fluctuation information:

[0040] First, continuous time-series images of fluorescently labeled subcellular structures in dead cells were acquired using a fluorescence microscope. 100 frames of time-series images were continuously acquired for each imaging field to obtain rich information on grayscale value fluctuations. For subsequent training of the deep learning model, time-series grayscale image data from at least 50 fields were acquired.

[0041] Step 2: Construct the standard curve of the double exponential decay function:

[0042] like Figure 3 As shown, for each imaging field of view, the first frame of the microscopic image is taken, and a set of subcellular structure pixels, a set of intracellular background pixels, and a set of extracellular background pixels are selected to construct a standard curve. Each set includes 10 different pixels.

[0043] After normalizing the grayscale values ​​of each group of 10 pixels in the N-frame time sequence image to the maximum value (the normalization method is the same as the normalization method used in the pixel classification process, see below for details), the standard curve is fitted by a double exponential decay function to obtain the subcellular structure standard decay curve, the intracellular background standard decay curve, and the extracellular background standard curve.

[0044] The double exponential decay function is Where a, b, c, d are the parameters to be fitted, and x is the time series variable. The values ​​represent the image grayscale values ​​at the corresponding time points. The standard curve fitting process uses a nonlinear least squares method to optimize the fitting parameters a, b, c, and d, minimizing the mean square error between the fitted curve and the actual data.

[0045] Step 3: Classify all pixels in each imaging field of view:

[0046] like Figure 4 As shown, the grayscale data of each pixel in 100 frames of time-series images are read, all grayscale data of each pixel are normalized and then curve fitting is performed. The fitted curve is compared with three standard curves to achieve pixel classification.

[0047] The specific method is as follows: The grayscale value of the pixel at each time point... Divide by its maximum gray value throughout the entire time series. The normalized grayscale value sequence is obtained as follows: .

[0048] The mean square error between the normalized grayscale value sequence of this pixel and three standard curves was calculated. (Mean Squared Error, MSE) classifies each pixel.

[0049] Mean square error The calculation formula is: ,in, This represents the mean square error between the pixel and the k-class standard curves. Here, k=1 represents the subcellular structure standard attenuation curve, k=2 represents the intracellular background standard curve, and k=3 represents the extracellular background standard curve. Let be the normalized gray value of the pixel to be classified at time t, be the value of the standard curve of category k at time t, and N be the total length of the time series.

[0050] The above formula is used to calculate the mean square error of the normalized gray value sequence of each pixel relative to the three standard curves. The minimum mean square error value is obtained. ;

[0051] The calculated minimum mean square error value Compare with the set background threshold and target threshold (background threshold > target threshold):

[0052] like The value is greater than the background threshold, regardless of the calculated value. The value of k determines whether the pixel is classified as background.

[0053] like The value is less than the background threshold, and the calculated value is... If the k value is not 1, the pixel is classified as background.

[0054] like The value is less than or equal to the background threshold, and the calculated value is... The value of k is 1, for Further comparison of the values: If If the value is less than the target threshold, the pixel is classified as a subcellular structure; if If the value is greater than or equal to the target threshold, the pixel is classified as background.

[0055] After traversing all pixels in each imaging field of view, a preliminary segmentation map is output based on the classification results.

[0056] Step 4: Optimization of the initial segmentation image:

[0057] like Figure 5 As shown, the Guo-Hall thinning algorithm is used to process the first frame of the microscopic image. Pixels are iteratively deleted until the skeleton converges to a single-pixel-wide centerline. In each iteration, the algorithm checks whether each pixel meets the deletion criteria. If it does, the pixel is marked for deletion. The skeleton of the subcellular structure is extracted as a reference for the basic structure.

[0058] The extracted skeleton image is combined with the preliminary segmentation image, prioritizing the preservation of the skeleton's structural details to ensure overall coherence. Specific operations include using the skeleton image as a mask to ensure the coherence and detail integrity of the cellular structures in the segmentation image, and repairing any broken sections at the skeleton's connections in the segmentation image to maintain the integrity of the cellular structures.

[0059] Filling is performed within the highlighted areas of the original image to repair holes or breaks in the segmentation image caused by errors. Highlighted areas are extracted from the original image, and logical operations are used to fill holes in the segmentation image. A bitwise AND operation is performed between the segmentation image and the highlighted areas to enhance the foreground pixels in the highlighted areas. A small-area filling algorithm is used to repair small holes. For overly inflated areas in the segmentation image, a local erosion operation is applied to better align them with the original image. The erosion operation reduces the highlighted areas, eliminates misclassified background pixels, and improves the accuracy of the segmentation image. A comparison of the segmentation results before and after optimization is shown below. Figure 6 As shown.

[0060] Step 5: Train the deep learning segmentation model:

[0061] like Figure 7As shown, for each imaging field, the first frame of a time-series microscopic image and its corresponding segmentation and extraction result image are selected to form a microscopic image-segmentation and extraction result image data pair. Data augmentation is achieved by combining operations such as random image cropping and random reversal. The augmented dataset is then used to train a U-Net network, resulting in a deep learning model capable of subcellular structure segmentation and extraction from a single frame of microscopic image data. In subsequent applications, only a single frame of fluorescence microscopic subcellular structure image needs to be acquired; the trained model can then be used to obtain an efficient and accurate subcellular structure image with optimized segmentation.

[0062] The method described in this invention has the following beneficial effects: 1. It constructs a high-quality subcellular structure segmentation dataset. This method utilizes the double exponential decay characteristic of fluorescence signals over time, which differs from the background signal, to construct three standard curves for subcellular structures, intracellular background, and extracellular background. It then performs pixel-level classification on fluorescence microscopy subcellular structure images to obtain high-precision segmentation results, providing a high-quality training dataset for realizing a deep learning model for single-frame subcellular structure segmentation. 2. It achieves efficient single-frame subcellular structure segmentation. Using the subcellular structure segmentation training dataset provided by this invention, and taking single-frame fluorescence microscopy image data as input, the deep learning model trained using the double exponential decay segmentation results as the gold standard can achieve efficient single-frame subcellular structure segmentation and extraction in later application stages. 3. It reduces hardware dependence. This method takes multiple frames of microscopic images as input, mainly focusing on the time decay characteristics of fluorescence signals, and is applicable to any fluorescence microscopy imaging system. 4. It has strong ability to handle complex backgrounds. For complex background problems in fluorescence microscopy imaging, this method performs pixel-level classification by fitting the time decay curve of each pixel, avoiding missegmentation caused by background interference in traditional segmentation methods.

Claims

1. A subcellular structure image segmentation and extraction method based on a double exponential decay function, characterized in that: Specifically: First, continuous time-series images of fluorescently labeled subcellular structures in dead cells are acquired to obtain rich grayscale fluctuation information. N frames of continuous time-series images need to be acquired for each imaging field of view. In the first frame of the N frames of time-series images for each imaging field of view, a set of subcellular structure pixels, a set of intracellular background pixels, and a set of extracellular background pixels are selected to construct a standard curve. Each set includes M different pixels. Then, after normalizing the grayscale data of each group of M pixels in the N-frame time sequence image, the standard curve is fitted by the double exponential decay function to obtain the subcellular structure standard decay curve, the intracellular background standard decay curve, and the extracellular background standard curve. Then, all pixels in each imaging field of view are classified. The gray value data of each pixel in the N frames of time-series images are read, and the gray value data of each pixel are normalized and then curve fitting is performed. The fitted curve is compared with three standard curves to achieve pixel classification. After traversing all pixels in each imaging field of view, a preliminary segmentation map is output based on the classification results. Then, the skeletonization algorithm is used to extract the subcellular structure skeleton map in the first frame of the microscopic image of each imaging field, and the preliminary segmentation map is filled and shrunken by combining image morphology operations to optimize the preliminary segmentation map and obtain a segmentation result map with coherent structure and high accuracy. Finally, the data from F different imaging fields are processed according to the above process to obtain F segmentation result images; For each imaging field of view, the first frame of the microscopic image and its corresponding segmentation and extraction result image are taken to construct a microscopic image-segmentation and extraction result image data pair; using these F data pairs, the U-Net network is trained to obtain a high-quality subcellular structure segmentation and extraction model; any single frame of fluorescence microscopic subcellular structure image is input into the trained subcellular structure segmentation and extraction model to efficiently and accurately obtain the segmented and optimized subcellular structure image.

2. The subcellular structure image segmentation and extraction method based on a double exponential decay function as described in claim 1, characterized in that: The N 100, F 50.

3. The subcellular structure image segmentation and extraction method based on a double exponential decay function as described in claim 1, characterized in that: The aforementioned double exponential decay function is Where a, b, c, d are the parameters to be fitted, and x is the time series variable. The corresponding image grayscale value is denoted as d. During the standard curve fitting process, the nonlinear least squares method is used to optimize the fitting parameters a, b, c, d in order to minimize the mean square error between the fitted curve and the actual data.

4. The subcellular structure image segmentation and extraction method based on a double exponential decay function as described in claim 1, characterized in that: The classification process for each pixel is as follows: The grayscale value of this pixel in each frame of the image Divide by its maximum gray value throughout the entire time series. The normalized grayscale value is obtained as follows: ,in This represents the grayscale value of the pixel in the t-th frame of the image; Calculate the mean square error between the normalized gray value of this pixel and the three standard curves in each frame of the image. Where k=1 represents the standard curve for subcellular structures, k=2 represents the standard curve for intracellular background, and k=3 represents the standard curve for extracellular background; Let be the normalized grayscale value of the pixel to be classified at time t. Let N be the value of the standard curve for category k at time point t, and N be the number of image frames acquired. The above formula is used to calculate the mean square error of the normalized gray value sequence of each pixel relative to the three standard curves. The minimum mean square error value is obtained. ; The calculated minimum mean square error value Compare with the set background threshold and target threshold: like If the value is greater than the background threshold, the pixel is classified as background. like The value is less than the background threshold, and the calculated value is... If the k value is not 1, the pixel is classified as background; like The value is less than or equal to the background threshold, and the calculated value is... The value of k is 1, for Further comparison of the values: If If the value is less than the target threshold, the pixel is classified as a subcellular structure; if If the value is greater than or equal to the target threshold, the pixel is classified as background.

5. The subcellular structure image segmentation and extraction method based on a double exponential decay function as described in claim 1, characterized in that: The optimization process of the preliminary segmentation map is as follows: The Guo-Hall thinning algorithm was used to process the first frame of the microscopic image. Pixels were deleted iteratively until the skeleton converged to a single-pixel-wide centerline. In each iteration, the algorithm checked whether each pixel met the deletion criteria. If it did, the pixel was marked for deletion. The skeleton of the subcellular structure was extracted as a reference for the basic structure. The extracted skeleton image was combined with the preliminary segmentation image, prioritizing the preservation of the skeleton's structural details to ensure overall coherence. Using the skeleton image as a mask, the continuity and detail integrity of the cell structure in the segmentation image are ensured, and broken parts at the skeleton connection points in the segmentation image are repaired to maintain the integrity of the cell structure. Fill the highlighted areas in the original image to repair holes or breaks caused by errors in the segmentation image; extract the highlighted areas in the original image, use logical operations to fill the holes in the segmentation image, perform a bitwise AND operation between the segmentation image and the highlighted areas to enhance the foreground pixels of the highlighted areas, and use a small area filling algorithm to repair small holes; for over-expanded areas in the segmentation image, apply local erosion to better align them with the original image. The erosion operation improves the accuracy of the segmentation map by reducing the highlighted areas and eliminating misclassified background pixels.

6. The subcellular structure image segmentation and extraction method based on a double exponential decay function as described in claim 1, characterized in that: For the aforementioned microscopic image-segmentation result image data pairs, random image cropping and random reversal operations are combined to augment the dataset, and the augmented dataset is used to train the U-Net network.

Images