Deep learning-based system and method for analyzing immunofluorescence images of autophagy

Abstract

The application discloses a kind of cell autophagy immunofluorescence image analysis system and method based on deep learning, including cell segmentation module, GFP-LC3 positive bright spot segmentation module, quantitative analysis module.Cell segmentation module carries out segmentation to the cell region in cell autophagy immunofluorescence image;GFP-LC3 positive bright spot segmentation module carries out segmentation to GFP-LC3 positive bright spot in cell autophagy immunofluorescence image;Quantitative analysis module carries out the statistics and calculation of quantity, area and fluorescence intensity to cell region and GFP-LC3 positive bright spot region in cell autophagy immunofluorescence image respectively, and judges whether cell occurs autophagy.The application is simple to operate, and has accurate deep learning algorithm and quantitative analysis function.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of digital cell autophagy fluorescence image analysis, and particularly relates to a cell autophagy immunofluorescence image analysis system and method based on deep learning. BACKGROUND

[0002] Cell autophagy is a mechanical self-degradation process in eukaryotic cells, which can help cells to remove misfolded proteins and damaged organelles inside the cells. Cell autophagy plays a key role in maintaining the homeostasis of cells, and is of great significance in the research of tumor occurrence, embryonic development, cell apoptosis and immune response. Cell autophagy can be monitored by observing the production of GFP-LC3 fusion protein under a fluorescence microscope. When autophagy occurs in cells with GFP as a probe, it is manifested as an increase in GFP-LC3 bright spots. Although counting immunofluorescence spots cannot reflect the size of autophagy flux, counting autophagic cells in fluorescence microscopic images or quantitatively analyzing the number and fluorescence intensity of GFP-LC3 bright spots in a single cell can still quantify the autophagy activity in cells, and is one of the standards for cell autophagy detection.

[0003] Currently, researchers in the field of biomedicine mostly adopt the method of selecting 2-3 immunofluorescence images for each group of experiments, and then selecting several autophagic cells in the images, manually counting the GFP-LC3 positive bright spots in the cells and taking the average value, or observing whether the GFP-LC3 positive bright spots have obvious increase or decrease by naked eye. Both of the two manual methods are time-consuming and laborious, and the analysis results are subjective. SUMMARY

[0004] The present application aims at the deficiencies of the prior art, and provides a cell autophagy immunofluorescence image analysis system and method based on deep learning, which solves the problem that the current cell autophagy immunofluorescence image analysis system cannot simultaneously consider the simplicity of the operation process, the accuracy of the segmentation result and the quantitative analysis function.

[0005] The present application is realized by the following technical scheme: a cell autophagy immunofluorescence image analysis system based on deep learning, comprising:

[0006] A cell segmentation module is used to segment the cell region in the cell autophagy immunofluorescence image;

[0007] A GFP-LC3 positive bright spot segmentation module is used to segment the GFP-LC3 positive bright spot region in the cell autophagy immunofluorescence image;

[0008] The quantitative analysis module is used for statistical calculation of the number, area and fluorescence intensity of the segmented cell region and the segmented GFP-LC3 positive bright spot region respectively; and the segmentation result of the cell region and the segmentation result of the GFP-LC3 positive bright spot region are combined to determine whether autophagy occurs in the cell.

[0009] Further, the cell segmentation module realizes segmentation of the cell region in the cell autophagy immunofluorescence image, and the specific process is as follows: first, a target segmentation channel in the cell autophagy immunofluorescence image is selected, and the cell autophagy immunofluorescence image is converted into a two-channel image, and the two-channel content is the nucleus and the cytoplasm; then, the cell autophagy immunofluorescence image converted into the two-channel image is sequentially subjected to image enhancement processing and data normalization processing; then, the size of the cell region in the two-channel image is selected, and the size of the two-channel image subjected to the image enhancement processing and the data normalization processing is scaled; finally, the scaled two-channel image is input into the deep learning model to obtain the segmentation result of the cell region.

[0010] Further, before using the deep learning model, the deep learning model is trained and tested, specifically: a training set and a data set are separated from the cell autophagy immunofluorescence image; the cell regions in the training set and the test set are pixel-level labeled, so that different cell regions have different label values; the labeled training set is used for training the deep learning model, and the labeled test set is used for testing the deep learning model to obtain the deep learning model.

[0011] Further, the formula for scaling the size of the image is:

[0012]

[0013] wherein, size output is the size of the scaled image, diam is the average diameter of the cell region in the selected two-channel image, size input is the size of the image before scaling.

[0014] Further, the image enhancement processing has the following specific process: first, backup the two-channel image, and extract the cytoplasm channel of the backup two-channel image; then, use the IQR outlier detection method to calculate the threshold value of the cytoplasm channel, take the upper whisker as the threshold value, threshold the pixels in the cytoplasm channel greater than the upper whisker, and remove the GFP-LC3 positive bright spot; finally, use the adaptive histogram equalization method to adaptively enhance the brightness and contrast of the cytoplasm channel.

[0015] Further, the GFP-LC3 positive bright spot segmentation module realizes segmentation of the GFP-LC3 positive bright spot region, and the specific process is as follows: first, input the cell autophagy immunofluorescence image converted into a two-channel image; then, select a thresholding method and / or a method of manually selecting a threshold to segment the GFP-LC3 positive bright spot, to obtain a preliminary GFP-LC3 positive bright spot region segmentation result; then, adjust the bright spot area threshold to screen the preliminary GFP-LC3 positive bright spot region segmentation result, to obtain a further GFP-LC3 positive bright spot region segmentation result; then, seed extraction is performed on the further GFP-LC3 positive bright spot region segmentation result; then, it is inquired whether the segmentation result is accurate, and if not, the bright spot center point is manually marked; finally, a watershed algorithm based on seeds is used to complete the GFP-LC3 positive bright spot region segmentation; the thresholding method is maximum entropy or Renyi entropy.

[0016] Further, the quantitative analysis module includes a bright spot quantitative analysis module and a cell quantitative analysis module.

[0017] The bright spot quantitative analysis module is used to calculate the number, fluorescence intensity and area of the GFP-LC3 positive bright spot region in the cell autophagy immunofluorescence image.

[0018] The cell quantitative analysis module is used to calculate the number, fluorescence intensity and area of the cell region in the cell autophagy immunofluorescence image, and to determine whether the cell has autophagy; if the cell region contains a bright spot region, the cell has autophagy; otherwise, the cell does not have autophagy.

[0019] A deep learning-based cell autophagy immunofluorescence image analysis method based on the above system, comprising the following steps:

[0020] (1) Obtain a cell autophagy immunofluorescence image; and convert the cell autophagy immunofluorescence image into a two-channel image;

[0021] (2) Segment the cell region of the cell autophagy immunofluorescence image converted into a two-channel image; including the following sub-steps:

[0022] (2.1) Perform image enhancement processing on the cell autophagy immunofluorescence image converted into a two-channel image;

[0023] (2.2) Perform data normalization processing on the cell autophagy immunofluorescence image processed in step (2.1);

[0024] (2.3) Perform scaling processing on the size of the cell autophagy immunofluorescence image processed in step (2.2);

[0025] (2.4) input the autophagy immunofluorescence image of the cells treated in step (2.3) into a deep learning model to obtain a segmentation result of the cell region;

[0026] (3) segment the GFP-LC3 positive bright spot region of the autophagy immunofluorescence image converted into a two-channel image; including the following sub-steps:

[0027] (3.1) extract the cytoplasm channel of the two-channel image;

[0028] (3.2) select a thresholding method and / or a method of manually selecting a bright spot threshold to segment the GFP-LC3 positive bright spot, to obtain a preliminary GFP-LC3 positive bright spot region segmentation result; the thresholding method is maximum entropy or Renyi entropy;

[0029] (3.3) adjust the bright spot area threshold for further segmentation, to obtain a further GFP-LC3 positive bright spot region segmentation result;

[0030] (3.4) seed extraction is performed on the further GFP-LC3 positive bright spot region segmentation result;

[0031] (3.5) inquire whether the segmentation result is accurate, if not, manually mark the bright spot center point;

[0032] (3.6) use a seed-based watershed algorithm to complete the GFP-LC3 positive bright spot region segmentation, to obtain a final GFP-LC3 positive bright spot region segmentation result;

[0033] (4) count, calculate the fluorescence intensity and calculate the area of the segmented cell region and the finally segmented bright spot region, respectively;

[0034] (5) determine whether the cell contains a bright spot according to the coordinates of the cell region and the bright spot region, if yes, the cell autophagy occurs; if not, the cell does not autophagy.

[0035] Further, the image enhancement processing in step (2.1) includes the following steps:

[0036] (1) backup the two-channel image, and extract the cytoplasm channel of the backup two-channel image;

[0037] (2) calculate the cytoplasm channel using the IQR outlier detection method, take the upper whisker as the threshold, and threshold the pixel in the cytoplasm channel greater than the upper whisker to remove the GFP-LC3 positive bright spot;

[0038] (3) use the adaptive histogram equalization method to adaptively enhance the brightness and contrast of the cytoplasm channel.

[0039] Further, the normalization processing of the data in the step (2.2) is realized by using the maximum-minimum normalization method for different channels of the two-channel image respectively, and in order to eliminate the influence of abnormal values, the max(x) and min(x) in the maximum-minimum normalization method are respectively the 99% percentile and the 1% percentile of the pixels in the channel, and the calculation formula of the normalization processing is:

[0040]

[0041] Wherein, x represents the pixel value of a pixel in the image, and x' is the pixel value of the pixel after the normalization processing.

[0042] The seed extraction in the step (3.4) is realized by continuously performing two times of morphological erosion operation with a kernel size of 5 on the further GFP-LC3 positive bright spot region segmentation result obtained in the step (3.3).

[0043] The beneficial effects of the present application are:

[0044] (1) The accurate segmentation of the cell region and the GFP-LC3 positive bright spot region in the cell autophagy fluorescence image analysis system is realized;

[0045] (2) The modular design of the segmentation task of the cell region, the segmentation task of the GFP-LC3 positive bright spot region and the quantitative analysis task of the cell region and the GFP-LC3 positive bright spot region in the cell autophagy fluorescence image analysis system is realized;

[0046] (3) The simplicity of the operation process, the accuracy of the segmentation result and the quantitative analysis function are taken into account, and the automatic analysis function of the cell autophagy fluorescence image is realized. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced, and obviously, the drawings in the following description can only be some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0048] Figure 1 The overall flowchart of the analysis system of the present application is shown in the figure.

[0049] Figure 2 The figure for enhancing the cytoplasm channel by the adaptive histogram equalization method of the present application is shown in the figure, wherein (a) is the original image of the cytoplasm channel, and (b) is the image of the cytoplasm channel after the adaptive histogram equalization processing.

[0050] Figure 3 The GFP-LC3 positive bright spot segmentation flowchart of the present application is shown in the figure. Detailed Implementation

[0051] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0052] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0053] It should be understood that although the terms first, second, third, etc., may be used in this invention to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of this invention, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0054] The present invention will now be described in detail with reference to the accompanying drawings. Unless otherwise specified, the features of the following embodiments and implementations can be combined with each other.

[0055] This invention provides a deep learning-based immunofluorescence image analysis system for autophagy, such as... Figure 1 As shown, it includes a cell segmentation module, a GFP-LC3 positive bright spot segmentation module, and a quantitative analysis module.

[0056] The cell segmentation module is used for segmenting the cell region in the autophagy immunofluorescence image. The specific process is as follows: first, a target segmentation channel in the autophagy immunofluorescence image is selected, and the autophagy immunofluorescence image is converted into a two-channel image, and the two-channel content is the nucleus and the cytoplasm; then, the autophagy immunofluorescence image converted into the two-channel image is sequentially subjected to image enhancement processing and data normalization processing; then, the size of the cell region in the two-channel image is selected, and the size of the two-channel image subjected to the image enhancement processing and the data normalization processing is scaled; finally, the scaled two-channel image is input into a deep learning model to obtain a segmentation result of the cell region. The autophagy immunofluorescence image converted into the two-channel image can be backed up, and the backed-up image can be applied to the cell segmentation module or the GFP-LC3 positive bright spot segmentation module.

[0057] Before using the deep learning model, the deep learning model is trained and tested. Specifically, a training set and a data set are separated from the two-channel image; the cell regions in the training set and the test set are pixel-level labeled, so that different cell regions have different label values; the labeled training set is used for training the deep learning model, and the labeled test set is used for testing the deep learning model, to obtain the deep learning model.

[0058] The deep learning model has four layers, uses U-net as a backbone network, uses Swin Transformer as an encoder, and uses ResNet as a decoder. The Swin Transformer is used to down-sample the input autophagy immunofluorescence image, extract feature information of the image that is helpful for segmenting the cell region, such as edge information, texture information, and position information, increase the size of the perception domain, and ensure the robustness of the model. In each down-sampling operation, the feature map size extracted by the encoder is continuously reduced, and finally the abstract semantic information is obtained; the ResNet is used to restore and decode the abstract features extracted by the encoder, and outputs a cell region segmentation result with the same size as the input image; the encoder and the decoder adopt a cross-layer connection operation (the feature information extracted by the encoder is directly input into the decoder), which improves the accuracy of the cell region segmentation.

[0059] The image enhancement processing includes bright spot removal and adaptive histogram equalization operation, and is used for enhancing the visual effect of the image. The specific process is as follows: the cytoplasm channel of the autophagy immunofluorescence image converted into the two-channel image is extracted; then, the IQR outlier detection method is used to calculate the threshold value of the cytoplasm channel, the upper whisker is taken as the threshold value, the pixel greater than the upper whisker in the cytoplasm channel is thresholded, and the GFP-LC3 positive bright spot is removed; finally, the adaptive histogram equalization method is used to adaptively enhance the brightness and contrast of the cytoplasm channel.

[0060] The threshold calculation is specifically calculating the upper quartile and the lower quartile of the pixels, and the difference between the upper quartile and the lower quartile is the IQR value, and the upper whisker is the upper quartile plus 1.5 times the IQR value, and the threshold is the upper whisker.

[0061] The data normalization processing is used for normalizing the pixel values of different channels of the input image, and in order to eliminate the influence of abnormal values, the max(x) and min(x) in the max-min normalization method are respectively the 99th percentile and the 1st percentile of the channel pixels, and the calculation formula of the normalization processing is:

[0062]

[0063] Wherein, x represents the pixel value of a pixel in the image, and x' is the pixel value of the pixel after the normalization processing;

[0064] The image scaling processing is used for scaling the size of the input image, and the formula of the size scaling is:

[0065]

[0066] Wherein, size output is the size of the scaled image, diam is the average diameter size of the cell region in the selected two-channel image, and size input is the size of the image before scaling.

[0067] The GFP-LC3 positive bright spot segmentation module is used for segmenting the GFP-LC3 positive bright spot region in the cell autophagy immunofluorescence image;

[0068] The GFP-LC3 positive bright spot segmentation module realizes the segmentation of the GFP-LC3 positive bright spot region, such as Figure 3As shown, the specific process is as follows: first, input the autophagy immunofluorescence image converted into a two-channel image; then, select a thresholding method and / or a method of manually selecting a threshold to segment the GFP-LC3 positive bright spots, to obtain a preliminary GFP-LC3 positive bright spot region segmentation result; then, adjust the bright spot area threshold to further segment, to obtain a further GFP-LC3 positive bright spot region segmentation result; then, seed extraction is performed on the further GFP-LC3 positive bright spot region segmentation result; then, it is inquired whether the segmentation result is accurate, if not, the bright spot center point is manually marked; finally, a watershed algorithm based on seeds is used to complete the GFP-LC3 positive bright spot region segmentation, to obtain a final GFP-LC3 positive bright spot region segmentation result; wherein, the process of manually marking after inquiring whether it is inaccurate is a man-machine loop strategy. The thresholding method is MaxEntropy or RenyiEntropy.

[0069] The quantitative analysis module is used to statistically and calculate the number, area and fluorescence intensity of the cell regions and the GFP-LC3 positive bright spot regions in the autophagy immunofluorescence image respectively; and to determine whether the cells have autophagy in combination with the segmentation result of the cell regions and the segmentation result of the GFP-LC3 positive bright spot regions.

[0070] The quantitative analysis module includes a bright spot quantitative analysis module and a cell quantitative analysis module.

[0071] The bright spot quantitative analysis module is used to calculate the number, fluorescence intensity and area of the GFP-LC3 positive bright spot regions after final segmentation.

[0072] The cell quantitative analysis module is used to calculate the number, fluorescence intensity and area of the segmented cell regions, and to determine whether the cells have autophagy. If the cell regions contain bright spot regions, the cells have autophagy; otherwise, the cells do not have autophagy.

[0073] The cell autophagy immunofluorescence image analysis method based on deep learning provided by the application includes segmentation of cell regions in an image, segmentation of GFP-LC3 positive bright spot regions in the image and quantitative analysis.

[0074] First, obtain the cell autophagy immunofluorescence image; and convert the cell autophagy immunofluorescence image into a two-channel image

[0075] The obtained cell autophagy immunofluorescence image can come from a cell autophagy immunofluorescence image collected by a confocal fluorescence microscope. In this embodiment, mammalian cell autophagy is analyzed, wherein the cells can be Hela cells (or other cells capable of autophagy, without limitation), and the mammalian cells can be induced to autophagy before the cell autophagy immunofluorescence image is obtained.

[0076] Segmentation of the cell region in the image, the steps are as follows:

[0077] (1) The cell autophagy immunofluorescence image converted into a two-channel image is subjected to image enhancement processing to obtain an enhanced image.

[0078] The image enhancement processing has the following specific process: first, the cytoplasm channel of the cell autophagy immunofluorescence image converted into a two-channel image is extracted; then, the IQR outlier detection method is used to calculate the threshold value of the cytoplasm channel, the upper whisker is taken as the threshold value, and the pixels greater than the upper whisker in the cytoplasm channel are thresholded to remove the GFP-LC3 positive bright spots; finally, the adaptive histogram equalization method is used to adaptively enhance the brightness and contrast of the cytoplasm channel.

[0079] The threshold calculation is specifically calculating the upper quartile and the lower quartile of the pixels, the difference between the upper quartile and the lower quartile is the IQR value, and the upper whisker is the upper quartile plus 1.5 times the IQR value, which is taken as the threshold value. In this embodiment, the upper whisker is 17.

[0080] The adaptive histogram equalization method is specifically an adaptive enhancement method for brightness and contrast of the cytoplasm channel by adaptively clipping pixels (the clipping range can be [0%, 100%]), and then stretching the remaining pixels to enhance the visual effect of the image. In this embodiment, the image is 8-bit, and taking 1% as an example, the clipping rule is to clip 0.5% of pixels at both ends of the gray scale histogram, and then stretch the remaining pixels to 0 to 255 to enhance the visual effect of the image. The enhanced image is as shown in Figure 2 .

[0081] (2) The enhanced image is subjected to data normalization processing.

[0082] The data normalization processing of the image can ensure that the deep learning model can quickly converge during training and improve the calculation efficiency. Specifically, the maximum and minimum normalization method is used for different channels of the two-channel image, and in order to eliminate the influence of abnormal values, the max(x) and min(x) in the maximum and minimum normalization method are respectively the 99th percentile and the 1st percentile of the pixels in the channel. The calculation formula of the normalization processing is:

[0083]

[0084] Wherein, x represents the pixel value of a pixel in the image, and x' represents the pixel value of the pixel after normalization processing.

[0085] (3) The size of the image subjected to normalization processing is scaled.

[0086] The formula for scaling the size of the image is:

[0087]

[0088] wherein size output is the size of the output image of the cell diameter selection module, diam is the input cell diameter in the cell diameter selection module, and size input is the size of the input image of the cell diameter selection module. In this embodiment, diam is 320, size input is 2048, and size output is 224.

[0089] (4) input the scaled image into the deep learning model to obtain the segmentation result of the cell region.

[0090] In order to ensure the accuracy of the cell segmentation result, the deep learning model needs to be trained and tested before step (4). Specifically, a training set and a test set are divided from the cell autophagy immunofluorescence image converted into a two-channel image in a ratio of 8:2; the cell regions in the training set and the test set are pixel-level labeled so that different cell regions have different label values; the labeled training set is used for training the deep learning model, and the labeled test set is used for testing the deep learning model to obtain the deep learning model.

[0091] The deep learning model has 4 layers, uses U-net as the backbone network, uses Swin Transformer as the encoder, and uses ResNet as the decoder. Swin Transformer is used to downsample the input cell autophagy immunofluorescence image, extract feature information of the image that is helpful for segmentation of the cell region, such as edge information, texture information, and position information, increase the size of the perception domain, and ensure the robustness of the model. In each downsampling operation, the feature map size extracted by the encoder is continuously reduced, and finally the abstract semantic information is obtained. ResNet is used to restore and decode the abstract features extracted by the encoder, and outputs the cell region segmentation result with the same size as the input image. The encoder and the decoder adopt a cross-layer connection operation (the feature information extracted by the encoder is directly input into the decoder), which improves the accuracy of the cell region segmentation.

[0092] The neural network training process is further illustrated by an example:

[0093] The method in the application can be used for a physical device, and the computing device of the physical device can be a GPU or a CPU. In the embodiment, the number of pictures in each batch for training is 8. After image enhancement, data normalization, and size scaling, the cell autophagy immunofluorescence image is input into the model, and the image output by the model is a cell region segmentation result image with the same size as the input image (in the embodiment, the size of the input image is 224*224). Each cell region in the cell region segmentation result image has a different label value.

[0094] The segmentation of the GFP-LC3 positive bright spot region in the image is as follows:

[0095] (1) Read the cell autophagy immunofluorescence image converted into a two-channel image, and extract the cytoplasm channel of the two-channel image.

[0096] (2) Select a thresholding method and / or a method of manually selecting a bright spot threshold to segment the GFP-LC3 positive bright spot, to obtain a preliminary GFP-LC3 positive bright spot segmentation result. The thresholding method is maximum entropy or Renyi entropy.

[0097] The preliminary GFP-LC3 positive bright spot region segmentation result can be obtained by first performing bright spot segmentation by the maximum entropy or Renyi entropy thresholding method, and then manually selecting a bright spot threshold (in the embodiment, the threshold is 56), or directly manually selecting a bright spot threshold (in the embodiment, the threshold is 56), or directly performing bright spot segmentation by the maximum entropy or Renyi entropy thresholding method.

[0098] (3) Adjust the bright spot area threshold to further segment, to obtain a further GFP-LC3 positive bright spot region segmentation result.

[0099] The GFP-LC3 bright spot is selected by adjusting the bright spot area threshold (in the embodiment, the area threshold is 20). Because part of the GFP-LC3 positive bright spot can be cut off in the imaging process, the number of GFP-LC3 positive bright spots can have a certain deviation from the actual value, and the bright spot area needs to be set to a threshold to screen out the GFP-LC3 positive bright spots with a small area after being cut off. Finally, the further GFP-LC3 positive bright spot region segmentation result is obtained.

[0100] (4) Seed extraction is performed on the further GFP-LC3 positive bright spot region segmentation result.

[0101] The seed extraction specifically includes: performing a morphological erosion operation with a kernel size of 5 on the further GFP-LC3 positive bright spot region segmentation result twice in succession.

[0102] (5) Inquire whether the segmentation result is accurate, if not, manually mark the center point of the bright spot;

[0103] The standard for the segmentation result being accurate is that there is no missed detection of the GFP-LC3 positive bright spot observed by naked eyes.

[0104] (6) The seed-based watershed algorithm is used to complete the segmentation of the GFP-LC3 positive bright spot region, and the final segmentation result of the GFP-LC3 positive bright spot region is obtained.

[0105] The watershed algorithm for completing the segmentation of the GFP-LC3 positive bright spot is realized by the cv2.watershed function in opencv.

[0106] The quantitative analysis is specifically: the number, area size and fluorescence intensity of the segmented GFP-LC3 positive bright spot region and the segmented cell region are calculated; then it is judged whether the cell contains the bright spot according to the cell region and the bright spot region coordinates, if yes, the cell autophagy occurs; if not, the cell autophagy does not occur.

[0107] The fluorescence intensity calculation method of the cell region or the GFP-LC3 positive bright spot is the ratio of the total pixel value of the region to the area of the region.

[0108] The above only describes the preferred embodiments of the present application and is not intended to limit the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0109] The above embodiments are only used to illustrate the design ideas and characteristics of the present application, and the purpose is to enable those skilled in the art to understand the content of the present application and implement it, and the protection scope of the present application is not limited to the above embodiments. Therefore, any equivalent changes or modifications made according to the principles and design ideas disclosed by the present application are within the protection scope of the present application.

[0110] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. The application is intended to cover any variations, uses or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice in the art to which the application pertains. The specification and examples are to be construed as merely illustrative of the present application.

[0111] It should be understood that the application is not limited to the precise construction that has been described above and shown in the accompanying drawings and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application.

Claims

1. A deep learning-based method for analyzing autophagy-based immunofluorescence images, characterized in that, Includes the following steps: (1) Obtain immunofluorescence images of cell autophagy; The autophagy immunofluorescence images were then converted into two-channel images. (2) Segment the cell regions in the autophagy immunofluorescence images converted into two-channel images; (3) The GFP-LC3 positive bright spot region was segmented from the cell autophagy immunofluorescence image converted into a two-channel image; Includes the following sub-steps: (3.1) Extract the cytoplasmic channel from the two-channel image; (3.2) Select a thresholding method and / or manually select a bright spot threshold to segment GFP-LC3 positive bright spots to obtain preliminary GFP-LC3 positive bright spot region segmentation results; the thresholding method is maximum entropy or Raney entropy; (3.3) Adjust the bright spot area threshold for further segmentation to obtain further GFP-LC3 positive bright spot region segmentation results; (3.4) Seed extraction was performed on the further segmentation results of GFP-LC3 positive bright spot regions; (3.5) Inquire whether the segmentation result is accurate. If it is not accurate, manually mark the center point of the bright spot. (3.6) The seed-based watershed algorithm is used to complete the segmentation of the GFP-LC3 positive bright spot region and obtain the final GFP-LC3 positive bright spot region segmentation result. (4) Count, calculate fluorescence intensity and area of ​​the segmented cell regions and the final segmented bright spot regions, respectively; (5) Determine whether the cell contains a bright spot based on the coordinates of the cell region and the bright spot region. If it does, the cell has undergone autophagy; if it does not, the cell has not undergone autophagy.

2. The method for analyzing cell autophagy immunofluorescence images based on deep learning according to claim 1, characterized in that, Cell regions were segmented from the autophagy immunofluorescence images converted into two-channel images. Includes the following sub-steps: (2.1) The autophagy immunofluorescence images converted into two-channel images were subjected to image enhancement processing; (2.2) Normalize the data of the autophagy immunofluorescence images processed in step (2.1); (2.3) The size of the cell autophagy immunofluorescence image processed in step (2.2) is scaled; (2.4) Input the cell autophagy immunofluorescence image processed in step (2.3) into the deep learning model to obtain the cell region segmentation results.

3. The method for analyzing autophagy-based immunofluorescence images according to claim 1, characterized in that, The image enhancement process in step (2.1) includes the following steps: (1) Backup the two-channel image and extract the cytoplasmic channel from the backup two-channel image; (2) The IQR outlier detection method was used to calculate the cytoplasmic channel, and the upper whisker was taken as the threshold. The pixels in the cytoplasmic channel larger than the upper whisker were thresholded to remove GFP-LC3 positive bright spots. (3) Adaptive histogram equalization method is used to adaptively enhance the brightness and contrast of cytoplasmic channels.

4. The method for analyzing cell autophagy immunofluorescence images based on deep learning according to claim 1, characterized in that, In step (2.2), the data normalization process is implemented using the max-min normalization method for different channels of the two-channel image. To eliminate the influence of outliers, the max(x) and min(x) in the max-min normalization method are calculated using the 99th percentile and 1st percentile of the pixels in that channel, respectively. The normalization calculation formula is as follows: Where x represents the pixel value of a certain pixel in the image, and x' is the pixel value of that pixel after normalization. The seed extraction in step (3.4) is achieved by performing two consecutive morphological etching operations with a nuclear size of 5 on the GFP-LC3 positive bright spot region segmentation results obtained in step (3.3).

5. A deep learning-based immunofluorescence image analysis system for autophagy, characterized in that, A method for implementing a deep learning-based immunofluorescence image analysis method for autophagy as described in any one of claims 1-4, comprising: The cell segmentation module is used to segment cell regions in autophagy immunofluorescence images; The GFP-LC3 positive bright spot segmentation module is used to segment the GFP-LC3 positive bright spot region in the autophagy immunofluorescence image; The quantitative analysis module is used to statistically analyze and calculate the number, area, and fluorescence intensity of the segmented cell regions and the segmented GFP-LC3 positive bright spot regions, respectively; and to determine whether autophagy has occurred by combining the segmentation results of the cell regions and the segmentation results of the GFP-LC3 positive bright spot regions.

6. The deep learning-based cell autophagy immunofluorescence image analysis system according to claim 5, characterized in that, The cell segmentation module segments cell regions in autophagy immunofluorescence images. The specific process is as follows: First, the target segmentation channel in the autophagy immunofluorescence image is selected, and the image is converted into a two-channel image, with the cell nucleus and cytoplasm as the two channels, respectively. Then, image enhancement and data normalization are performed sequentially on the converted two-channel image. Finally, the diameter of the cell region in the two-channel image is selected, and the size of the two-channel image after image enhancement and data normalization is scaled. Finally, the scaled two-channel image is input into the deep learning model to obtain the segmentation results of the cell region.

7. The deep learning-based cell autophagy immunofluorescence image analysis system according to claim 6, characterized in that, Before using the deep learning model, the deep learning model is trained and tested. Specifically, the training set and the test set are separated from the autophagy immunofluorescence images. The cell regions in the training set and the test set are labeled at the pixel level so that different cell regions have different label values. The labeled training set is used to train the deep learning model, and the labeled test set is used to test the deep learning model, thus obtaining the deep learning model.

8. The deep learning-based cell autophagy immunofluorescence image analysis system according to claim 6, characterized in that, The formula for scaling an image is: Where, size output The size is the scaled image size, where `diam` is the average diameter of the cell region in the selected two-channel image, and `size` is the image size after scaling. input This refers to the size of the image before scaling.

9. The deep learning-based cell autophagy immunofluorescence image analysis system according to claim 6, characterized in that, The image enhancement process is as follows: First, back up the two-channel image and extract the cytoplasmic channel from the backed-up two-channel image; then, use the IQR outlier detection method to calculate the threshold for the cytoplasmic channel, taking the upper whisker as the threshold, and thresholding the pixels in the cytoplasmic channel that are larger than the upper whisker to remove GFP-LC3 positive bright spots; finally, use the adaptive histogram equalization method to adaptively enhance the brightness and contrast of the cytoplasmic channel.

10. The deep learning-based cell autophagy immunofluorescence image analysis system according to claim 5, characterized in that, The GFP-LC3 positive bright spot segmentation module segmentes the GFP-LC3 positive bright spot region. The specific process is as follows: First, an autophagy immunofluorescence image converted to a two-channel image is input; then, a thresholding method and / or manual threshold selection are used to segment the GFP-LC3 positive bright spot, obtaining preliminary GFP-LC3 positive bright spot region segmentation results; next, the bright spot area threshold is adjusted to filter the preliminary GFP-LC3 positive bright spot region segmentation results, obtaining further GFP-LC3 positive bright spot region segmentation results; then, seed extraction is performed on the further GFP-LC3 positive bright spot region segmentation results; next, the accuracy of the segmentation results is checked, and if inaccurate, the center point of the bright spot is manually marked; finally, a seed-based watershed algorithm is used to complete the GFP-LC3 positive bright spot region segmentation; the thresholding method is maximum entropy or Raney entropy.

11. The deep learning-based cell autophagy immunofluorescence image analysis system according to claim 5, characterized in that, The quantitative analysis module includes a bright spot quantitative analysis module and a cell quantitative analysis module; The bright spot quantitative analysis module is used to calculate the number, fluorescence intensity, and area of ​​GFP-LC3 positive bright spot regions in autophagy immunofluorescence images; The cell quantitative analysis module is used to calculate the number, fluorescence intensity, and area of ​​cell regions in the autophagy immunofluorescence image; and to determine whether autophagy has occurred; if the cell region contains bright spots, then autophagy has occurred; otherwise, autophagy has not occurred.

Images