Cell structure segmentation method and system based on multi-round fluorescence in-situ hybridization images

Abstract

The invention relates to a cellular structure segmentation method and system based on multi-round fluorescence in-situ hybridization images. The invention provides a method for automatically segmenting a cell structure in a fluorescence image. The method comprises the following steps: acquiring a fluorescence image to be analyzed as image data to be processed; training data are selected from the to-be-processed image, a cell truth value label data set is made through image labeling software, and the to-be-processed image is preprocessed; training a multi-task learning-based convolutional neural network based on the marked data set, and outputting a nearest neighbor affinity image between pixels and a probability image that the pixels belong to a boundary; on the basis of a network output result, a distance transformation watershed algorithm and a graph division algorithm are adopted to generate a cell instance segmentation result; and automatically identifying a segmentation error region in the segmentation result, artificially and selectively correcting the segmentation result, and obtaining a corrected cell boundary result through edge extraction.

Description

Technical Field

[0001] The present invention belongs to the technical field of biological and neural image segmentation, and particularly relates to a method and system for automatically segmenting cell structures in multiple rounds of fluorescence in situ hybridization images. Background Art

[0002] Spatial transcriptome sequencing is a new technology that has emerged in recent years, offering new insights into neuroscience research. Based on multiple rounds of fluorescence in situ hybridization (FISH) imaging, spatial transcriptome sequencing can provide single-cell resolution, which is crucial for understanding the spatial location of cells, tissue properties, and function.

[0003] To obtain cell and gene expression matrices, upstream analysis of FISH images typically requires two key tasks: cell segmentation and transcriptome detection. The goal of cell segmentation is to determine the boundaries of each cell in the image and assign transcriptomes to each cell, enabling neuroscientists to further analyze the image data downstream.

[0004] Existing cell recognition methods generally include two types of methods: one is a top-down method based on target detection, which regards each cell as a detection target and then uses a deep network to achieve instance segmentation of the cell structure; the other is a bottom-up method based on target segmentation, which uses a deep network to predict the probability that each pixel in the image belongs to a cell, and then uses a watershed algorithm to distinguish adjacent cell areas.

[0005] However, due to the limitations of sample tissue type and fluorescence microscope resolution, cells in FISH images are closely arranged together, and cells present an adhesion morphology. It is difficult for the above methods to accurately identify the boundaries of each cell, and it is difficult to obtain satisfactory segmentation results. Specifically, the former method is not suitable for images with dense cell areas, and the latter method is prone to produce under-segmentation results of cells. Therefore, this field is in urgent need of an automated and high-precision segmentation method that is suitable for dense cell areas, so as to achieve accurate and efficient spatial transcriptome sequencing analysis tasks. Summary of the Invention

[0006] The present invention relates to the field of image segmentation technology, and more specifically, to a method for automatically segmenting cellular structures in fluorescent images. In some embodiments, the method of the present invention addresses the problem that traditional cell segmentation methods cannot guarantee universality and high accuracy for densely populated cell areas.

[0007] In some aspects, provided herein are inventions described by one or more of the following.

[0008] 1. A method for automatically segmenting cell structures in a fluorescence image, comprising:

[0009] Step S10: Acquire a fluorescence image to be analyzed as image data to be processed, wherein the fluorescence image may be any appropriate cell staining image, such as a cell nucleus and / or cell membrane staining image, such as a fluorescence in situ hybridization image, especially a multi-round fluorescence in situ hybridization image;

[0010] Step S20: selecting training data for the above-mentioned image to be processed, creating a cell true value label data set through image annotation software, and preprocessing the image to be processed;

[0011] Step S30: Based on the above-mentioned labeled data set, a convolutional neural network based on multi-task learning is trained to output the nearest neighbor affinity image between pixels and the probability image of the pixel belonging to the boundary, such as a segmentation model based on multi-task learning, and output the semantic segmentation result of the cell, wherein the segmentation model is constructed based on a convolutional neural network and supervised learning is performed through manually labeled training samples. The cell semantic segmentation result is composed of the nearest neighbor affinity image between pixels and the probability image of the pixel belonging to the cell boundary.

[0012] Step S40: Based on the above network output results, a distance transform watershed algorithm and a graph partitioning algorithm are used to generate a cell instance segmentation result, for example, by further optimizing the network output results to be processed through a graph optimization model to obtain a cell instance segmentation result, wherein the graph optimization model is constructed based on the cell nucleus superpixel region and the Multicut model, and is solved by a greedy additive edge contraction algorithm.

[0013] Step S50: automatically identifying the segmentation error areas in the above segmentation results, manually and selectively correcting the segmentation results, and obtaining the boundary results of the corrected cells through edge extraction.

[0014] 2. The method described in item 1, wherein step S10 includes acquiring a cell nucleus staining image of a fluorescence in situ hybridization (FISH) image, such as a DAPI channel image in the FISH image.

[0015] 3. The method of item 1 or 2, wherein step S20 comprises:

[0016] Step S21: select images from the images to be processed as training data, and manually annotate all cell regions in the training set images using image annotation software.

[0017] Step S22: normalize the original grayscale image in the above selected training set and use it as the input image for the subsequent convolutional neural network.

[0018] Step S23: pre-process the labeled training set data as the input label image for the subsequent convolutional neural network. For each manually annotated image in the training data, an affinity image and a cell edge binary image are generated by using the nearest neighbor affinity operator and edge detection algorithm.

[0019] Step S24: The training data set finally obtained through the above steps consists of grayscale images and corresponding label images.

[0020] 4. The method according to any one of items 1 to 3, wherein step S30 comprises:

[0021] Step S31, defining a convolutional neural network based on a multi-task learning framework,

[0022] In step S32, the convolutional neural network in step S31 is trained based on the data set obtained in step S24, and the difference between the true value label in the training set and the network prediction result is measured by the loss function. When the loss function value of the model is stable, the network training process is stopped.

[0023] Step S33: Based on the trained network model, the remaining image data to be processed is predicted, and a nearest neighbor affinity image and a boundary probability image are generated for each image to be processed.

[0024] 5. The method according to any one of items 1 to 4, wherein step S40 comprises:

[0025] Step S41: add and average the affinity images in the network output results, and then binarize them using a predefined threshold to obtain a mask of the cell area.

[0026] Step S42: Binarize the boundary probability image using a predefined threshold, and generate an over-segmented cell region result based on the distance transform watershed algorithm.

[0027] Step S43, using the graph partitioning model Multicut to merge the cell over-segmentation fragments in step S42 to obtain the cell instance segmentation result,

[0028] In step S44, based on the cell instance segmentation result obtained in step S43 and the cell region mask obtained in step S41, the intersection-over-union ratio is calculated, and only the segmentation results belonging to the cell structure are retained.

[0029] 6. The method of item 5, wherein step S43 comprises:

[0030] Step S431: construct a region adjacency graph based on the superpixel results.

[0031] Step S432: assign an edge weight to each edge. The edge weight reflects the connection strength of the edge.

[0032] Step S433: construct a graph partitioning model based on Multicut and superpixel results, and solve the superpixel partitioning result through the greedy additive edge contraction algorithm. The partitioning result is the instance segmentation result of the cell.

[0033] 7. The method according to any one of items 1 to 6, wherein step S50 comprises:

[0034] Step S51, automatically identifying the cell areas that may be segmented incorrectly in the above automatic segmentation results through the boundary probability value,

[0035] Step S52: Input the segmentation error region positions obtained in step S51 and the corresponding original grayscale image and cell segmentation results into the image verification software, and manually select and correct the cell segmentation results of these regions.

[0036] In step S53, the cell segmentation result obtained in step S52 is used as the final segmentation result, and the cell contour is extracted by an edge extraction operator as the final cell boundary result.

[0037] 8. A cell structure segmentation system based on fluorescence images, comprising the following modules:

[0038] an image acquisition module, configured to acquire a fluorescence image to be analyzed as image data to be processed;

[0039] a data set preprocessing module configured to select labeled samples for the image to be processed, manually annotate the cell structure of the labeled samples, perform image normalization on the image to be processed, and preprocess the labeled data to obtain a training sample set;

[0040] A multi-task neural network module is configured as a segmentation model based on multi-task learning, outputting semantic segmentation results of cells; wherein the segmentation model is built based on a convolutional neural network and uses supervised learning with manually labeled training samples. The cell semantic segmentation results are composed of nearest neighbor affinity images between pixels and probability images of pixels belonging to cell boundaries.

[0041] A graph optimization post-processing module is configured to further optimize the network output result to be processed through a graph optimization model to obtain a cell instance segmentation result, wherein the graph optimization model is constructed based on superpixels and a multicut model and is solved by a greedy additive edge contraction algorithm.

[0042] The segmentation result correction module is configured to automatically detect the segmentation error areas in the cell instance segmentation results through the recognition algorithm, manually correct the segmentation results, and extract the segmentation contours to obtain the final cell segmentation results and boundary results.

[0043] 9. A computer system comprising:

[0044] processor;

[0045] A memory having instructions stored thereon, which, when executed by the processor, cause the processor to perform the method according to any one of items 1-7.

[0046] 10. A computer-readable storage medium storing instructions which, when executed by a processor, cause the processor to perform the method according to any one of items 1-7.

[0047] In some embodiments, the method of the present invention includes the following steps: obtaining a DAPI channel image in the FISH data to be analyzed as the image data to be processed, wherein the FISH image is collected by an optical imaging device; selecting training data for the above-mentioned image to be processed, creating a cell true value label data set through image annotation software, and preprocessing the image to be processed; training a convolutional neural network based on multi-task learning based on the above-mentioned annotated data set, outputting a nearest neighbor affinity image between pixels and a probability image of the pixel belonging to the boundary; based on the output result of the above-mentioned network, using the distance transform watershed algorithm and the graph partitioning algorithm to generate a cell instance segmentation result; automatically identifying the segmentation error area in the above-mentioned segmentation result, manually selectively correcting the segmentation result, and obtaining the boundary result of the corrected cell by edge extraction. The present invention can be used for the automatic recognition and segmentation of cell structures in light microscopy images, with high efficiency and good accuracy, and has high practicality and a wide range of applications. The method of the present invention solves the problem that traditional cell segmentation methods cannot guarantee high accuracy and universality.

[0048] In some embodiments, the present invention provides a method for automatically segmenting cell structures in multiple rounds of fluorescence in situ hybridization images. In some embodiments, the present invention provides a two-stage cell segmentation method based on multi-task learning and graph optimization model, the method comprising the following steps:

[0049] Step S10: acquiring a DAPI channel image in the FISH data to be analyzed as image data to be processed, wherein the FISH image is acquired by an optical imaging device;

[0050] Step S20: selecting training data for the above-mentioned image to be processed, creating a cell true value label data set through image annotation software, and preprocessing the image to be processed;

[0051] Step S30: training a convolutional neural network based on multi-task learning based on the labeled dataset, and outputting a nearest neighbor affinity image between pixels and a probability image of pixels belonging to boundaries;

[0052] Step S40: Based on the above network output results, a distance transform watershed algorithm and a superpixel map partitioning algorithm are used to generate a cell instance segmentation result;

[0053] Step S50: automatically identifying the segmentation error areas in the above segmentation results, manually and selectively correcting the segmentation results, and obtaining the boundary results of the corrected cells by extracting the edges.

[0054] Another aspect of the present invention provides a cell structure segmentation system based on multiple rounds of fluorescence in situ hybridization images, the system comprising the following modules:

[0055] an image acquisition module, configured to acquire DAPI channel data in the FISH image to be analyzed as image data to be processed;

[0056] a data set preprocessing module configured to select labeled samples for the image to be processed, manually annotate the cell structure of the labeled samples, perform image normalization on the image to be processed, and preprocess the labeled data to obtain a training sample set;

[0057] A multi-task neural network module, configured as a segmentation model based on multi-task learning, outputs the semantic segmentation results of cells;

[0058] The segmentation model is constructed based on a convolutional neural network and supervised learning is performed through manually labeled training samples. The cell semantic segmentation result is composed of the nearest neighbor affinity image between pixels and the probability image of the pixel belonging to the cell boundary.

[0059] A graph optimization post-processing module, configured to further optimize the to-be-processed network output result through a graph optimization model to obtain a cell instance segmentation result;

[0060] The graph optimization model is constructed based on superpixels and Multicut models, and is solved by a greedy additive edge contraction algorithm.

[0061] The segmentation result correction module is configured to automatically detect the segmentation error areas in the cell instance segmentation results through the recognition algorithm, manually correct the segmentation results, and extract the segmentation contours to obtain the final cell segmentation results and boundary results.

[0062] The beneficial effects of the present invention are in one or more of the following aspects:

[0063] (1) The present invention is based on a cell structure segmentation method based on multiple rounds of fluorescence in situ hybridization images. Considering that cells in tissues are closely arranged together, combining cell edge information helps to distinguish the adhesion cell morphology, thereby designing a multi-task learning segmentation network, which can significantly reduce under-segmentation results;

[0064] (2) The present invention designs a two-stage cell segmentation process based on a cell structure segmentation method based on multiple rounds of fluorescence in situ hybridization images, combined with a graph optimization model, which can optimize the inaccurate boundary results of fuzzy areas, thereby further improving the accuracy of cell segmentation;

[0065] (3) The present invention is based on a cell structure segmentation method based on multiple rounds of fluorescence in situ hybridization images. Experimental results on different tissue samples show that the comprehensive cell segmentation accuracy can reach about 95%, which accelerates the time for analyzing gene expression relationships in spatial transcriptome sequencing.

[0066] In some embodiments, advantageous features of the present invention include, inter alia, one or more of the following:

[0067] (1) The present invention adopts a multi-task learning network framework for cell segmentation. The network multi-task learning framework includes two outputs, one of which is the affinity relationship map of the image in the xy direction, and the other is the binary image of the cell edge. By combining these two outputs, the cell region result and the boundary result are generated;

[0068] (2) The present invention adopts an over-segmentation aggregation strategy, uses a distance transform watershed algorithm on the boundary image to generate super-pixel results of cells, thereby reducing under-segmentation of cells, and constructs a Multicut graph optimization model based on super-pixels to reduce over-segmentation results of cells;

[0069] (3) A simple and efficient segmentation error finding algorithm was invented. The bounding box of the segmentation error superpixel was determined by locating the fuzzy cell boundary weights, and the non-maximum suppression algorithm was used to optimize the error area positioning results. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 This is a flow chart of a method for segmenting cell structures in fluorescence microscopy images proposed by the present invention.

[0071] Figure 2 This is a schematic diagram of the workflow of the graph optimization post-processing algorithm proposed in the present invention.

[0072] Figure 3 Schematic diagram of the workflow of the segmentation error finding algorithm proposed in the present invention.

[0073] Figure 4 is a DAPI image to be processed.

[0074] Figure 5 It is the cell affinity image output by the above DAPI image through the convolutional segmentation network.

[0075] Figure 6 It is the edge probability image output by the convolutional segmentation network of the above DAPI image.

[0076] Figure 7 It is the cell segmentation image output by the graph optimization model of the above DAPI image.

[0077] Figure 8 This is a comparison of the results of the present invention and other segmentation methods. DETAILED DESCRIPTION

[0078] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0079] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0080] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0081] The cell structure segmentation method based on multiple rounds of fluorescence in situ hybridization images of the present invention comprises:

[0082] Step S10: acquiring a DAPI channel image in the FISH data to be analyzed as image data to be processed, wherein the FISH image is acquired by an optical imaging device;

[0083] Step S20: selecting training data for the above-mentioned image to be processed, creating a cell true value label data set through image annotation software, and preprocessing the image to be processed;

[0084] Step S30: training a convolutional neural network based on multi-task learning based on the labeled dataset, and outputting a nearest neighbor affinity image between pixels and a probability image of pixels belonging to boundaries;

[0085] Step S40: Based on the above network output results, a watershed algorithm and a graph partitioning algorithm are used to generate a cell instance segmentation result;

[0086] Step S50: automatically identifying the segmentation error areas in the above segmentation results, manually and selectively correcting the segmentation results, and obtaining the boundary results of the corrected cells through edge extraction.

[0087] In some preferred embodiments, step S20 includes:

[0088] Step S21: From the image to be processed (see Figure 2 ) selects several images as training data X={x i}, all cell regions in the training set images were manually annotated using image annotation software, requiring that the label value of each cell structure be different, i.e., for X = {x i} has a corresponding true value label image Y = {y i The annotation software can be the TrakEM2 plug-in of the open source ImageJ software, or other open source tools can be specified for data annotation.

[0089] Step S22: normalize the original grayscale images in the above-selected training set and use them as input images for the subsequent convolutional neural network. Specifically, for each original DAPI image x in the training data X, i , adjust the pixel intensity by grayscale stretching and perform pixel normalization so that the pixel t∈x i The value range is [0, 1].

[0090] Step S23: pre-process the labeled training set data as the input label image for the subsequent convolutional neural network. Specifically, for each manually annotated image y in the training data Y i , through the nearest neighbor affinity operator and edge detection algorithm, generate affinity image A={a 1 i , a 2 i}, and the cell edge binary image B={b i}, where a 1 i , a 2 i Represents the affinity in the x and y directions respectively. The nearest neighbor affinity relationship indicates that a pixel and its nearest neighbor pixels belong to the same cell. The affinity operator K is defined as:

[0091]

[0092] where y i m represents the label value corresponding to the current pixel m, and m+1 is the nearest neighbor of pixel m. The edge detection algorithm is based on the Sobel operator, calculating the gradient changes in the x and y directions. Areas where the sum of the absolute values ​​of the x and y gradients is greater than 0 are identified as boundaries. Finally, the boundaries are expanded by 3 pixels using morphological image processing.

[0093] Step S24: Through the above steps, the training data set finally obtained consists of the grayscale image X and the label images A and B.

[0094] In some preferred embodiments, step S30 includes:

[0095] Step S31, based on the multi-task learning framework, defines a convolutional neural network. The goal of the neural network is [A, B] = f(X; θ), where θ is the model parameter to be learned and f is the network structure. Specifically, the network structure is based on the U-Net network, in which the feature extraction module is combined with the ResNet residual module to improve the learning ability of the model; in addition, feature maps of different levels are fused through element-wise addition operations, replacing the fusion operation of feature map channel stacking in the original U-Net structure. The input image of the network model is 1-channel data, and the dimension of the output image is 3-channel. The output image includes the nearest neighbor affinity relationship image. and the probability image of the pixel belonging to the boundary The loss function of the model is defined as a binary cross entropy function, which can also replace the commonly used loss functions in other segmentation models.

[0096] In step S32, the convolutional neural network in step S31 is trained based on the data set obtained in step S24. The difference between the true value label in the training set and the network prediction result is measured by the loss function. The model parameter θ is updated by the back propagation algorithm. When the loss function value of the model is stable, the network training process is stopped.

[0097] Step S33: Based on the trained network model, the remaining image data to be processed is predicted, and the nearest neighbor affinity image is generated for each image to be processed. (See Figure 3 ) and boundary probability image (See Figure 4 ).

[0098] In some preferred embodiments, step S40 includes (see flowchart for details): Figure 2 ):

[0099] Step S41: Output the affinity relationship image in the network output The sum is added and averaged, and then binarized using a predefined threshold T1 to obtain the mask M of the cell area. The calculation formula of the mask M is

[0100] Step S42, using the boundary probability image Generate over-segmented cell region results. Specifically, the boundary probability image in the network output result The initial cell boundary result is obtained by binarization using a pre-defined threshold T2; the initial boundary result is converted into a grayscale energy map through Euclidean distance transformation, where the pixel value is lower the closer to the boundary; then, the above grayscale energy map is smoothed by a Gaussian function, where the Gaussian function is used to adjust the degree of over-segmentation of cell fragments; the local minimum value in the energy map after Gaussian smoothing is used as the seed point, and the watershed algorithm is used based on the seed point to convert the smoothed energy map into a superpixel result, where the superpixel is the over-segmented cell area.

[0101] Step S43: Use the graph segmentation model Multicut to merge the cell segments in step S42, reduce the over-segmentation results, and obtain the cell instance segmentation results. The specific steps are as follows:

[0102] Step S431: construct a regional adjacency graph G = (V, E) based on the above superpixel results, where V is the node set composed of superpixels, represents the set of edges connecting any adjacent nodes;

[0103] Step S432: assign an edge weight w to each edge e∈E e , edge weight w e Reflects the connection strength of the edge, which is represented by the similarity probability p e It is calculated by the negative log-likelihood formula, where the similarity probability p e The negative log-likelihood formula is obtained by the average affinity of the two superpixels connected by edge e:

[0104]

[0105] Where β controls the degree of over-segmentation and the default value is 0.5.

[0106] Step S433: Based on Multicut, a superpixel graph partitioning model is constructed, and the superpixel partitioning result is solved by the greedy additive edge contraction algorithm. The partitioning result is the instance segmentation result of the cell. The Multicut model is defined as follows:

[0107]

[0108]

[0109] where x e A binary indicator variable representing edge e, which takes the values ​​0 or 1, with 0 indicating a connection and 1 indicating a merge. cycles(G) represents the set of all cycles in the graph G = (V, E). The solution to the inequality constraint model is a valid partition of the graph G. The greedy additive edge contraction algorithm is a heuristic greedy algorithm that merges the two nodes connected by the largest positive edge at each iteration. This algorithm produces a fixed solution for each given optimization problem.

[0110] In step S44, the IoU is calculated based on the cell instance segmentation result obtained in step S43 and the cell region mask obtained in step S41, and only the segmentation results belonging to the cell structure are retained. Specifically, the IoU is calculated for each label region in the segmentation result and the cell mask M. A maximum IoU threshold T3 is specified, and labels in the segmentation result that exceed the threshold T3 are deleted.

[0111] In some preferred embodiments, step S50 includes:

[0112] Step S51, flow chart see Figure 3 , through the boundary probability value, automatically identify the cell area that may be segmented incorrectly in the above automatic segmentation results. Specifically, the probability threshold T is pre-set low and T high , find the similarity probability p e <T low and p e >T high The superpixel area connected by the edges that meet the above conditions is regarded as a potential segmentation error area; the position of the error area is located by extracting the maximum bounding rectangle of the superpixel; at the same time, considering that there may be many areas with segmentation errors, the intersection-over-union ratio of the located maximum bounding rectangle is calculated, and the non-maximum suppression algorithm is used to remove the similar bounding boxes to obtain the final error area position.

[0113] In step S52, the locations of the segmentation error regions obtained in step S51, along with the corresponding original grayscale image and cell segmentation results, are input into image verification software, and the cell segmentation results of these regions are selectively corrected manually. The image verification software can be the TrakEM2 plug-in for the open-source ImageJ software, or other open-source tools can be used for result verification.

[0114] Step S53: The cell segmentation result obtained in step S52 is used as the final segmentation result, and the cell outline is extracted by the edge extraction operator as the final cell boundary result (see Figure 5 ).

[0115] In another aspect of the present invention, a cell structure segmentation system based on multiple rounds of fluorescence in situ hybridization images is proposed, which includes the following modules:

[0116] an image acquisition module, configured to acquire DAPI channel data in the FISH image to be analyzed as image data to be processed;

[0117] a data set preprocessing module configured to select labeled samples for the image to be processed, manually annotate the cell structure of the labeled samples, perform image normalization on the image to be processed, and preprocess the labeled data to obtain a training sample set;

[0118] A multi-task neural network module, configured as a segmentation model based on multi-task learning, outputs the semantic segmentation results of cells;

[0119] The segmentation model is constructed based on a convolutional neural network and supervised learning is performed through manually labeled training samples. The cell semantic segmentation result is composed of the nearest neighbor affinity image between pixels and the probability image of the pixel belonging to the cell boundary.

[0120] A graph optimization post-processing module, configured to further optimize the to-be-processed network output result through a graph optimization model to obtain a cell instance segmentation result;

[0121] The graph optimization model is constructed based on superpixels and Multicut models, and is solved by a greedy additive edge contraction algorithm.

[0122] The post-processing module is configured to automatically identify the segmentation error areas in the cell instance segmentation, manually correct the segmentation results, and extract the segmentation contours to obtain the final cell segmentation results and boundary results.

Claims

1. A method for automatically segmenting cell structures in a fluorescent image, comprising: Step S10: Acquire the fluorescence image to be analyzed as image data to be processed; Step S20: selecting training data for the above-mentioned image to be processed, creating a cell true value label data set through image annotation software, and preprocessing the image to be processed; Step S30: training a convolutional neural network based on multi-task learning based on the labeled data set, and outputting a nearest neighbor affinity image between pixels and a probability image of pixels belonging to the boundary; Step S40: Based on the network output results, a distance transform watershed algorithm and a graph partitioning algorithm are used to generate a cell instance segmentation result; Step S50: automatically identifying the segmentation error area in the segmentation result, manually and selectively correcting the segmentation result, and obtaining the boundary result of the corrected cell by edge extraction. 2 . The method of claim 1 , wherein step S10 comprises acquiring a cell nucleus staining image of a fluorescence in situ hybridization (FISH) image.

3. The method of claim 1 or 2, wherein step S20 comprises: Step S21, select an image from the images to be processed as training data, and manually annotate all cell regions in the training set images using image annotation software. Step S22, normalize the original grayscale image in the above selected training set as the input image of the subsequent convolutional neural network. Step S23, preprocessing the labeled training set label data as the input label image of the subsequent convolutional neural network, wherein for each manually annotated image in the training data, an affinity image and a cell edge binary image are generated through the nearest neighbor affinity operator and edge detection algorithm. Step S24: The training data set finally obtained through the above steps consists of grayscale images and corresponding label images.

4. The method according to any one of claims 1 to 3, wherein step S30 comprises: Step S31, defining a convolutional neural network based on a multi-task learning framework, Step S32, based on the data set obtained in step S24, the convolutional neural network in step S31 is trained, and the difference between the true value label in the training set and the network prediction result is measured by the loss function. When the loss function value of the model is stable, the network training process is stopped. Step S33: Based on the trained network model, the remaining image data to be processed is predicted, and a nearest neighbor affinity image and a boundary probability image are generated for each image to be processed.

5. The method according to any one of claims 1 to 4, wherein step S40 comprises: Step S41, the affinity images in the network output results are added and averaged, and then binarized using a predefined threshold to obtain a mask of the cell area. Step S42, generating an over-segmented cell region result through a boundary probability image, Step S43, using the graph partitioning model Multicut to merge the cell segments in step S42, reducing the over-segmentation results, and obtaining the cell instance segmentation results. Step S44, based on the cell instance segmentation result obtained in step S43 and the cell region mask obtained in step S41, the intersection-over-union ratio is calculated, and only the segmentation results belonging to the cell structure are retained.

6. The method of claim 5, wherein step S43 comprises: Step S431, constructing a region adjacency graph based on the superpixel results, Step S432: assign an edge weight to each edge, where the edge weight reflects the connection strength of the edge. Step S433, constructing a superpixel graph partitioning model based on Multicut, and solving the superpixel partitioning result based on the greedy additive edge contraction algorithm, the partitioning result is the instance segmentation result of the cell.

7. The method according to any one of claims 1 to 6, wherein step S50 comprises: Step S51, automatically identifying the cell regions that may be segmented incorrectly in the above automatic segmentation results through the boundary probability value, Step S52, inputting the segmentation error region positions obtained in step S51 and the corresponding original grayscale image and cell segmentation results into the image verification software, and manually selectively correcting the cell segmentation results of these regions. Step S53, taking the cell segmentation result obtained in step S52 as the final segmentation result, and extracting the cell contour by an edge extraction operator as the final cell boundary result.

8. A cell structure segmentation system based on fluorescence images, the system comprising the following modules: An image acquisition module, configured to acquire a fluorescence image to be analyzed as image data to be processed; A data set preprocessing module is configured to select labeled samples for the image to be processed, manually annotate the cell structure of the labeled samples, perform image normalization processing on the image to be processed, preprocess the labeled data, and obtain a training sample set; The multi-task neural network module is configured as a segmentation model based on multi-task learning and outputs the semantic segmentation results of cells; The segmentation model is built based on a convolutional neural network and supervised learning is performed through training samples with manual labels. The cell semantic segmentation result is composed of the nearest neighbor affinity image between pixels and the probability image of the pixel belonging to the cell boundary. A graph optimization post-processing module is configured to further optimize the network output result to be processed through a graph optimization model to obtain a cell instance segmentation result, wherein the graph optimization model is constructed based on superpixels and a Multicut model and is solved through a greedy additive edge contraction algorithm, The segmentation result correction module is configured to automatically identify the segmentation error area in the cell instance segmentation, manually correct the segmentation result, and extract the segmentation contour to obtain the final cell segmentation result and boundary result.

9. A computer system comprising: processor; A memory having instructions stored thereon, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing instructions which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 7.

Images