Methods and apparatus for bright-field cell image segmentation in fluorescence microscopy

Abstract

This invention relates to a method and apparatus for bright-field cell image segmentation in fluorescence microscopy. Combining an improved two-dimensional OTSU threshold segmentation algorithm, it filters out noise points in single-cell images during image preprocessing and further refines cell segmentation using binary image mathematical morphology. Then, it segments adhered cells using a label-controlled watershed segmentation algorithm based on cell nucleus images. This invention improves segmentation results through image enhancement and refines them using various segmentation methods, effectively addressing issues such as weak edges, poor contrast, irregular cell shapes, and cell adhesion in bright-field cell images. Therefore, the overall visual segmentation effect is quite good.

Description

Technical Field

[0001] This invention belongs to the field of image processing, and particularly relates to a method and apparatus for bright-field cell image segmentation in fluorescence microscopy. Background Technology

[0002] Image segmentation is the technique and process of dividing an image into several specific regions with unique properties and extracting targets of interest. Image segmentation is based on two main properties: the similarity and discontinuity of gray values. For example, thresholding segmentation segments targets based on the discontinuity of gray values, while region growing segments targets based on the similarity of image values.

[0003] Because cell imaging is transparent and irregularly shaped, and the intensity distribution is not significantly different from the background, it is difficult to visualize unstained cells in this way.

[0004] Bright-field imaging is used to locate cells before fluorescence imaging, minimizing the degradation of fluorescence signals by photobleaching. However, bright-field cell images suffer from weak edges, poor contrast, cell adhesion, and inconsistent cell shapes, making direct image segmentation difficult. Common image segmentation methods for bright-field cell images include threshold-based segmentation, active contour-based segmentation, K-means clustering-based segmentation, and deep learning-based segmentation. Threshold-based segmentation methods are suitable when the grayscale values ​​of target and background pixels are easily distinguishable. For example, the classic Otsu thresholding method uses the maximum inter-class variance between the target and background to find the optimal threshold, leading to the development of two-dimensional Otsu thresholding methods, which improve robustness to noise. Active contour-based segmentation methods use energy functionals for modeling, such as parametric active contour models and geometric active contour models. These methods are sensitive to the choice of initial points. K-means clustering-based segmentation methods segment the target based on the similarity of pixels within k clusters. This method requires manually setting the parameter k, which is time-consuming. Reference (Ali R, Gooding M, Christlieb M, et al. Phase-based segmentation of cells from brightfield microscopy[C] / / 20074th IEEE International Symposium on Biomedical Imaging:From Nano to Macro.IEEE,2007:57-60) proposes a method combining Gaussian smoothing filtering and level set segmentation for brightfield cell image segmentation. Reference (Long Jianwu, Shen Xuanjing, Zang Hui, Chen Haipeng, Adaptive threshold segmentation algorithm for estimating background in Gaussian scale space[J]. Acta Automatica Sinica, 2014, 40(08):1773-1782) proposes an adaptive threshold segmentation algorithm for estimating background in Gaussian scale space to effectively process non-uniformly illuminated images. The literature (Yin Sumin, Zhu Jinping, Wang Zusheng, Jiang Yu, Lu Wenjun. Research on image segmentation method based on top-hat transform and maximum inter-class variance method [J]. Science Technology and Engineering, 2014, 14(07): 60-64) proposes an image segmentation method based on top-hat transform and maximum inter-class variance method (OTSU) to overcome the shortcomings of the image segmentation method that directly uses the maximum inter-class variance method (OTSU) in accurately segmenting images with uneven brightness backgrounds.The paper (Zhang K, Zhang L, Song H, et al. Active contours with selective local or global segmentation: a new formulation and level set method[J]. Image and Vision computing, 2010, 28(4): 668-67) proposes a new region-based active contour model, which is implemented using the Selective Binary and Gaussian Filtered Regularized Level Set (SBGFRLS) method. This method first selectively penalizes the level set function to binary, and then regularizes it using a Gaussian smoothing kernel.

[0005] The image segmentation methods described above are only applicable to images under specific conditions. Image preprocessing is required for the cell images to be segmented to improve the segmentation results. Furthermore, accuracy, speed, noise resistance, and versatility have always been research challenges in improving segmentation algorithms. Summary of the Invention

[0006] The purpose of this invention is to provide a method and apparatus for segmenting bright-field cell images in fluorescence microscopy, which can quickly and accurately segment similar bright-field cell images or similar images. It can effectively handle the problems of weak edges, irregular cell shapes and cell adhesion in bright-field cell images, thereby successfully and accurately segmenting the cell outline, laying the foundation for subsequent research on precise fluorescence dot counting inside biological cells.

[0007] This invention discloses a method for bright-field cell image segmentation in fluorescence microscopy, comprising the following steps:

[0008] Step 1: Input bright-field cell images and corresponding cell nucleus images;

[0009] Step 2: Segment the input cell nucleus image to obtain a binarized cell nucleus image. Determine the centroid of each cell based on the binarized cell nucleus image. Map the centroids onto the bright field cell image and then crop the bright field cell image according to single cells to obtain multiple single cell images.

[0010] Step 3: Perform image enhancement operations on the single-cell images respectively, including background interference removal, cell edge enhancement, and noise removal.

[0011] Step 4: Combine the single-cell images that have undergone cell edge enhancement and noise removal processing into a two-dimensional histogram, and process it using the improved two-dimensional OTSU threshold segmentation method to obtain preliminary segmentation results.

[0012] Step 5: Further optimize the segmentation results by performing binary image mathematical morphology operations on the single-cell image after two-dimensional OTSU thresholding.

[0013] Step 6: Using the binary image of the cell nucleus as marker points, merge all the single-cell images processed in Step 5. Then, further segment the merged cell images using a marker-controlled watershed segmentation algorithm to complete the cell image segmentation.

[0014] Step 2 specifically involves: performing OTSU threshold segmentation on the input cell nucleus image to obtain preliminary cell nucleus segmentation results; then using binary image mathematical morphology processing to remove stray points; finally, using the centroid formula to obtain the centroid of each cell nucleus; mapping the centroid onto the bright field cell image; and then cropping a rectangular frame containing the cell with the centroid as the center, thereby cropping the bright field cell image according to single cells to obtain multiple single cell images.

[0015] The background interference removal process involves applying a double Gaussian filter to the single-cell image to remove background interference, where the Gaussian function formula is:

[0016]

[0017] Where x, y are the coordinates on the single-cell image, σ i Let be the standard deviation of the Gaussian function, and be the number of the Gaussian filter function. Its radius follows the 3σ criterion. By selecting different values ​​of σ and performing interpolation, the image after double Gaussian filtering is obtained as f. dg :

[0018] f dg =f*(G2-G1) (4)

[0019] Where f is the input single-cell grayscale image, * denotes convolution operation, G2 is a large Gaussian filter, G1 is a small Gaussian filter, and f dg This is the image after double Gaussian filtering. The standard deviation σ1 of the Gaussian function of the small Gaussian filter is similar to the cell size, and the value of σ2 is obtained through quantitative analysis.

[0020] The process of highlighting cell edges involves using a top-hat transform on a single-cell image that has undergone double Gaussian filtering to highlight cell edges against a dark background. The formula for this top-hat transform is as follows:

[0021]

[0022] h = gg o (6)

[0023] Where g represents the single-cell image to be processed, S is the structural unit, and g orepresents the single-cell image after the opening operation, h represents the single-cell image after the top-hat transformation, and the structural unit is a disk-shaped structural element.

[0024] The noise removal process involves applying median filtering to the single-cell image after top-hat transformation to eliminate noise interference. The formula for this median filtering is as follows:

[0025]

[0026] Where f(x,y) is the input single-cell image, taking the κ×κ neighboring pixels around each pixel, and m(x,y) is the single-cell image after median filtering.

[0027] The single-cell image obtained after top-hat transformation is used as one dimension, and the single-cell image after median filtering is used as another dimension. The two form a two-dimensional histogram, which is then processed using an improved two-dimensional OTSU thresholding method to obtain preliminary segmentation results.

[0028] Assuming the input image is of size M×N, denoted as f(x,y), with L gray levels, the image after double Gaussian filtering and top-hat transform is f dgt The image after median filtering is f(x,y). dgtm (x,y), graph f dgt and image f dgtm Let the gray values ​​at pixel (x,y) be i and j, respectively, and F ij This represents the frequency at the gray value (i,j), and the joint probability p ij Represented as:

[0029]

[0030] Where i,j=0,1,...,L-1 and

[0031] Suppose a two-dimensional histogram is divided into two categories: background C0 and foreground C1. The background and foreground correspond to the dark and bright areas in the image, respectively. s and t represent the optimal segmentation thresholds of the two-dimensional histogram in the two dimensions, respectively. The probabilities ω0 of C0 and ω1 of C1 are expressed as follows:

[0032]

[0033]

[0034] Meanwhile, the class mean vector is represented as:

[0035]

[0036]

[0037] And the total mean vector of the two-dimensional histogram is represented as:

[0038]

[0039] The between-class variance matrix is ​​defined as:

[0040]

[0041] Use rank to measure between-class variance:

[0042]

[0043] Among them, u ki and u kj As the class mean, equations (9) and (10) show its between-class mean vector, u ti and u Tj For the total mean, equation (11) shows its total mean vector;

[0044] If the region far from the diagonal in a two-dimensional histogram is ignored:

[0045] ω0+ω1≈1,u T =ω0u0+ω1u1 (16)

[0046] The rank of the inter-class variance then simplifies to:

[0047]

[0048] in:

[0049]

[0050] By making trσ b Maximizing the value of (s,t) yields the optimal segmentation threshold vector (s). * ,t * )for:

[0051]

[0052] The optimal segmentation threshold is found by using a one-dimensional search method for each dimension of the two-dimensional histogram. The process of finding the optimal segmentation threshold in two dimensions is transformed into cell image segmentation by solving for the optimal segmentation threshold in each dimension of the two-dimensional histogram. Then, formula (19) can be expressed as:

[0053]

[0054] Where i and j are the binary tuples (i,j) on the projection plane of the two-dimensional histogram composed of the top-hat transformed image and the median filtered image, and the range of i and j is the same as the range of gray levels of the original image.

[0055] The binary image mathematical morphology operations in step 5 include closing, filling holes, opening, and area constraints. Both closing and opening operations select disk-shaped structural elements, and the radius of the disk-shaped structural elements for the closing and opening operations is selected according to the actual situation.

[0056] The watershed segmentation algorithm in step 6 is based on the theory of mathematical morphology, simulating a three-dimensional terrain surface. It uses the binary cell nucleus image obtained in step 2 as the marker point of the watershed, and further segments the adherent cells by the marker-controlled watershed segmentation algorithm on the merged cell image.

[0057] A bright-field cell image segmentation device for fluorescence microscopy includes an input module, a single-cell local image segmentation module, an image preprocessing module, and an image segmentation module connected in sequence.

[0058] Input module: used to read bright-field cell images and corresponding cell nucleus images;

[0059] Single-cell local image segmentation module: used to determine the centroid of each cell based on the cell nucleus image, and after mapping the centroid to the bright field cell image, the bright field cell image is cropped according to single cells to obtain multiple single-cell images and output to the image preprocessing module;

[0060] Image preprocessing module: includes a background interference removal processing module, a cell edge enhancement processing module, and a noise removal processing module connected in sequence. It is used to perform image enhancement operations on the single-cell image output by the single-cell image segmentation module, and output the single-cell image after background interference removal, cell edge enhancement, and noise removal to the image segmentation module.

[0061] Image segmentation module: This module includes an improved two-dimensional OTSU threshold segmentation module, a mathematical morphology module, and a watershed segmentation module connected in sequence. It uses each of the single-cell images, after edge highlighting and noise removal, as a dimension to form a two-dimensional histogram. The improved two-dimensional OTSU threshold segmentation method is used to process the histogram to obtain a preliminary segmentation result. Then, binary image mathematical morphology operations are used to further refine the preliminary segmentation result. Using cell nucleus images as markers, all single-cell images are merged, and a marker-controlled watershed segmentation algorithm is used to further segment adherent cells, resulting in a cell segmentation image.

[0062] This background interference removal module is used to perform double Gaussian filtering on single-cell images to remove background interference;

[0063] This cell edge enhancement module is used to highlight cell edges in a dark background using a top-hat transform on a single-cell image that has undergone double Gaussian filtering.

[0064] This noise removal module is used to perform median filtering on the single-cell image after top-hat transformation to eliminate noise interference;

[0065] The improved two-dimensional OTSU threshold segmentation module takes the single-cell image obtained after top-hat transformation as one dimension and the single-cell image after median filtering as another dimension. The two together form a two-dimensional histogram. The improved two-dimensional OTSU threshold segmentation method is used to process the two-dimensional histogram to obtain preliminary segmentation results and output them to the mathematical morphology module.

[0066] This mathematical morphology module is used to further refine the initial segmentation results by applying binary image mathematical morphology operations to single-cell images that have undergone two-dimensional OTSU thresholding.

[0067] This watershed segmentation module uses cell nucleus images as markers. After merging all single-cell images output by the mathematical morphology module, it further segments the adherent cells using a marker-controlled watershed segmentation algorithm to obtain cell segmentation images.

[0068] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement any of the above-described bright-field cell image segmentation methods in fluorescence microscopy.

[0069] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described bright-field cell image segmentation methods in fluorescence microscopy.

[0070] The technical solution of this invention has the following beneficial effects:

[0071] (1) By determining the cell centroid from the cell nucleus image, mapping the centroid onto the bright field cell image, cropping out the rectangular box containing the cell, and processing the cell within the rectangular box, the cell outline can be accurately obtained, overcoming the shortcomings of inconsistent cell shapes and poor segmentation accuracy, thereby effectively improving the cell image segmentation accuracy.

[0072] (2) By performing image preprocessing on single-cell images, including double Gaussian filtering and top-hat transformation, the effect of eliminating image background and highlighting cell edges can be achieved, overcoming the shortcomings of weak edges and poor contrast in bright-field cell image segmentation, thereby obtaining accurate cell contours.

[0073] (3) Since merging the segmentation results of single-cell images into one image results in cell adhesion, the present invention uses cell nucleus-labeled images and adopts a watershed segmentation algorithm based on label control to effectively handle the segmentation problem of adhered cells and improve the performance of cell segmentation.

[0074] Furthermore, compared with other segmentation algorithms, this invention has higher precision, recall, and F1 score, and faster computation speed. The overall performance of this invention is significantly better than other segmentation algorithms. Attached Figure Description

[0075] Figure 1 This is a flowchart of the bright-field cell image segmentation method in this invention;

[0076] Figure 2 This is a functional block diagram of the bright-field cell image segmentation method in this invention;

[0077] Figure 3 The present invention uses σ² value and f dg A curve obtained from the standard deviation of the image;

[0078] Figure 4 This is a projection of the two-dimensional histogram obtained by the two-dimensional threshold segmentation method of the present invention;

[0079] Figure 5 This invention provides schematic diagrams of various segmentation results for bright-field cell images.

[0080] The technical solutions in the embodiments of the present invention will be further described clearly and completely below with reference to the accompanying drawings. Detailed Implementation

[0081] like Figure 1 and 2 As shown, this invention provides a method for bright-field cell image segmentation in fluorescence microscopy, comprising the following steps:

[0082] Step 1: Input bright-field cell images and corresponding cell nucleus images;

[0083] Step 2: Segment the input cell nucleus image to obtain a binarized cell nucleus image. Determine the centroid of each cell based on this binarized cell nucleus image. Map the centroids onto the bright-field cell image, and then crop the bright-field cell image according to single cells to obtain multiple single-cell images (it is possible to have only one single-cell image). Specifically:

[0084] After performing OTSU thresholding on the input cell nucleus image to obtain preliminary cell nucleus segmentation results, binary image mathematical morphology processing is used to remove stray points. Finally, the centroid of each cell nucleus is calculated using the centroid formula. After mapping the centroids onto the bright-field cell image, a 256×256 rectangle containing the cell is cropped around the centroid, thus cropping the bright-field cell image as a single cell to obtain multiple single-cell images. The centroid along the x-axis is calculated using the centroid formula as follows:

[0085]

[0086] The centroid along the y-axis is:

[0087]

[0088] Where x and y represent the coordinates on the x-axis and y-axis, respectively, and f(x,y) represents the gray value of the image at the (x,y) coordinate position. The centroid of the image along the x-axis. The centroid of the image along the y-axis;

[0089] Step 3: Perform image enhancement operations on the single-cell images, including background interference removal, cell edge enhancement, and noise removal.

[0090] The background interference removal process, in this embodiment of the invention, involves performing a double Gaussian filter on the single-cell image to remove background interference, wherein the Gaussian function formula is:

[0091]

[0092] Where x, y are the coordinates on the single-cell image, σ i Let be the standard deviation of the Gaussian function, and i be the index of the Gaussian filter function whose radius follows the 3σ criterion. i The larger the value of σ, the less high-frequency information is retained after Gaussian filtering, and the blurrier the image becomes. i The smaller the value of σ, the more high-frequency information is retained after Gaussian filtering, resulting in a clearer image. Performing a difference operation on the two values ​​filters out low-frequency information while retaining high-frequency information. By selecting different values ​​of σ and performing the difference operation, the image after double Gaussian filtering is obtained as f. dg :

[0093] f dg =f*(G2-G1) (4)

[0094] Where f is the input single-cell grayscale image, * denotes convolution operation, G2 is a large Gaussian filter, G1 is a small Gaussian filter, and f dgThis is the image after double Gaussian filtering. The standard deviation σ1 of the Gaussian function of the small Gaussian filter is similar to the cell size. To fully preserve the detailed information in the cell image, σ1 is set to 1 in this invention. For σ2, if the value is too small, it will affect the extraction of low-frequency information, indirectly affecting the extraction of high-frequency information; if the value is too large, the improvement to the segmentation result is not significant. To select a suitable σ2 value, quantitative analysis is required. The contrast of the image can be reflected by the standard deviation of the image; the larger the standard deviation, the more obvious the contrast between the cells and the background in the image. Figure 3 As shown, through the σ² value and f dg The standard deviation of the images was used to obtain a curve. Five cells were selected for statistical analysis. Figure 3 As can be seen from this, when the value of σ² is greater than 20, f dg The change in standard deviation is not significant; in this embodiment of the invention, σ² is selected as 20.

[0095] The process of highlighting cell edges, in this embodiment of the invention, involves using a top-hat transform on a single-cell image that has undergone double Gaussian filtering to highlight cell edges against a dark background. The formula for this top-hat transform is:

[0096]

[0097] h = gg o (6)

[0098] Where g represents the single-cell image to be processed, S is the structural unit, go represents the single-cell image after the opening operation, h represents the single-cell image after the top-hat transformation, and the structural unit is a disk-shaped structural element with a radius of 6.

[0099] The noise removal process, in this embodiment of the invention, involves performing median filtering on the single-cell image after top-hat transformation to eliminate noise interference. The formula for this median filtering is:

[0100]

[0101] Where f(x,y) is the input single-cell image, taking the κ×κ neighboring pixels around each pixel, and m(x,y) is the single-cell image after median filtering. In this embodiment of the invention, κ is set to 5.

[0102] Step 4: Combine the single-cell images after cell edge enhancement and noise removal into two dimensions to form a two-dimensional histogram. Then, process this histogram using an improved two-dimensional OTSU thresholding method to obtain preliminary segmentation results. In this embodiment, the single-cell image obtained after top-hat transformation is used as one dimension, and the single-cell image after median filtering is used as another dimension. These two dimensions form a two-dimensional histogram, which is then processed using an improved two-dimensional OTSU thresholding method to obtain preliminary segmentation results.

[0103] Assuming the input image is of size M×N, denoted as f(x,y), with L gray levels, the image after double Gaussian filtering and top-hat transform is f dgt The image after median filtering is f(x,y). dgtm (x,y), graph f dgt and image f dgtm Let the gray values ​​at pixel (x,y) be i and j, respectively, and F ij This represents the frequency at the gray value (i,j), and the joint probability p ij Represented as:

[0104]

[0105] Where i,j=0,1,...,L-1 and

[0106] An image is divided into two categories: background (C0) and foreground (C1). The background and foreground correspond to the dark and light areas of the image, respectively. Figure 4 As shown, the background corresponds to region 4 in the two-dimensional histogram, where s and t represent the optimal segmentation thresholds in the two dimensions of the two-dimensional histogram composed of the image, respectively. The foreground corresponds to region 1 in the two-dimensional histogram, and the probabilities ω0 and ω1 of C0 and C1 are expressed as follows:

[0107]

[0108]

[0109] Meanwhile, the class mean vector is represented as:

[0110]

[0111]

[0112] And the total mean vector of the two-dimensional histogram is represented as:

[0113]

[0114] The between-class variance matrix is ​​defined as:

[0115]

[0116] Use rank to measure between-class variance:

[0117]

[0118] Where, uki and ukj are class means, and equations (11) and (12) show their inter-class mean vectors, and uti and uTj are total means, and equation (13) shows their total mean vectors.

[0119] In general, the region far from the diagonal in a two-dimensional histogram can be ignored.

[0120] ω0+ω1≈1,u T =ω0u0+ω1u1 (16)

[0121] The rank of the inter-class variance can be simplified to:

[0122]

[0123] in:

[0124]

[0125] By making trσ b Maximizing the value of (s,t) yields the optimal segmentation threshold vector (s). * ,t * )for:

[0126]

[0127] The optimal segmentation threshold is found by using a one-dimensional search method for each dimension of the two-dimensional histogram. The process of finding the optimal segmentation threshold in two dimensions is transformed into cell image segmentation by solving for the optimal segmentation threshold in each dimension of the two-dimensional histogram. Then, formula (19) can be expressed as:

[0128]

[0129] Where i and j are the binary tuples (i,j) on the projection plane of the two-dimensional histogram composed of the top-hat transformed image and the median filtered image, and the range of i and j is the same as the range of gray levels of the original image.

[0130] Step 5: Apply binary image mathematical morphology operations to the single-cell image after two-dimensional OTSU threshold segmentation to further optimize the segmentation result. The binary image mathematical morphology operations include closing operation, hole filling, opening operation, and area constraint. Both closing and opening operations select disk-shaped structural elements. The radius of the disk-shaped structural elements for closing and opening operations is selected according to the actual situation. In this embodiment, the closing operation selects a disk-shaped structural element with a radius of 8, and the opening operation selects a disk-shaped structural element with a radius of 5. The area size in the area constraint is set to 1400.

[0131] Step 6: Using the binary image of the cell nucleus as marker points, merge all single-cell images processed in Step 5. Then, apply a marker-controlled watershed segmentation algorithm to the merged cell images to further improve segmentation accuracy, completing the cell image segmentation.

[0132] The watershed segmentation algorithm is based on the theory of mathematical morphology and simulates a three-dimensional terrain surface. It is a region-based image segmentation method. The binary image of the cell nucleus obtained in step (2) is used as the marker point of the watershed. The merged cell image is further segmented by the marker-controlled watershed segmentation algorithm.

[0133] like Figure 5 The image shows the results of segmenting cell images using different methods: (a) the input cell image; (b) the traditional OTSU threshold segmentation algorithm; (c) the K-means algorithm; (d) the active contour method; and (e) the image segmentation algorithm of this invention. All segmentation algorithms perform segmentation after image preprocessing.

[0134] Traditional OTSU thresholding segmentation algorithms can segment prominent cell edges, but they are sensitive to noise, prone to oversegmentation, and cannot handle broken cell edges. The K-means algorithm, due to the lack of prominent cell outlines in cell images, tends to classify cells as background and treat the background as the target, resulting in a white bounding box. Active contouring methods are crucial for initial point selection; because the cell image outlines are not prominent enough, the initial points cannot smoothly evolve to the cell outlines, resulting in a white segmentation image. This invention combines an improved two-dimensional OTSU thresholding segmentation algorithm, filtering out noise points in the image preprocessing stage, and further employing binary image mathematical morphology methods to improve cell segmentation. Then, for adhered cells, a label-controlled watershed segmentation algorithm is used in conjunction with the cell nucleus image for further segmentation. This means that this invention uses image enhancement operations to improve the segmentation results and utilizes multiple segmentation methods to refine the results, effectively solving the problems of weak edges, poor contrast, inconsistent cell shapes, and cell adhesion in bright-field cell images. Therefore, the overall visual segmentation effect is the best.

[0135] like Figure 2 The bright-field cell image segmentation device for fluorescence microscopy of the present invention comprises, in sequence, an input module, a single-cell local image segmentation module, an image preprocessing module, and an image segmentation module.

[0136] Input module: used to read bright-field cell images and corresponding cell nucleus images;

[0137] Single-cell local image segmentation module: used to determine the centroid of each cell based on the cell nucleus image, and after mapping the centroid to the bright field cell image, the bright field cell image is cropped according to single cells to obtain multiple single-cell images and output to the image preprocessing module;

[0138] Image preprocessing module: Includes a background interference removal module, a cell edge enhancement module, and a noise removal module connected in sequence. It performs image enhancement operations on the single-cell images output by the single-cell image segmentation module, and outputs the single-cell images with background interference removed, cell edges enhanced, and noise removed to the image segmentation module.

[0139] This background interference removal module is used to perform double Gaussian filtering on single-cell images to remove background interference;

[0140] This cell edge enhancement module is used to highlight cell edges in a dark background using a top-hat transform on a single-cell image that has undergone double Gaussian filtering.

[0141] This noise removal module is used to perform median filtering on the single-cell image after top-hat transformation to eliminate noise interference;

[0142] Image segmentation module: This module includes an improved 2D OTSU thresholding segmentation module, a mathematical morphology module, and a watershed segmentation module connected in sequence. The improved 2D OTSU thresholding method is used to process the resulting 2D histogram to obtain preliminary segmentation results. Then, binary image mathematical morphology operations are used to further refine the preliminary segmentation results. Using cell nucleus images as markers, all single-cell images are merged, and a marker-controlled watershed segmentation algorithm is used to further segment adherent cells, resulting in a cell segmentation image.

[0143] The improved two-dimensional OTSU threshold segmentation module takes the single-cell image obtained after top-hat transformation as one dimension and the single-cell image after median filtering as another dimension. The two form a two-dimensional histogram, which is then processed by the improved two-dimensional OTSU threshold segmentation method to obtain preliminary segmentation results and output them to the mathematical morphology module.

[0144] This mathematical morphology module is used to further refine the initial segmentation results by applying binary image mathematical morphology operations to single-cell images that have undergone two-dimensional OTSU thresholding.

[0145] The watershed segmentation module uses cell nucleus images as markers. After merging all single-cell images output by the mathematical morphology module, it further segments the adherent cells using a marker-controlled watershed segmentation algorithm to obtain cell segmentation images.

[0146] Each module in the aforementioned bright-field cell image segmentation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0147] The above-described bright-field cell image segmentation apparatus can be used to execute the bright-field cell image segmentation method provided in any of the above embodiments, and has the corresponding functions and beneficial effects.

[0148] In one embodiment, a computer device is provided, which may be a terminal. The computer device includes a processor, memory, a network interface, a display screen, and an input device connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a bright-field cell image segmentation method. The display screen of the computer device may be a liquid crystal display (LCD) or an e-ink display. The input device of the computer device may be a touch layer covering the display screen, or buttons, a trackball, or a touchpad located on the casing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0149] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the bright-field cell image segmentation method provided in any of the above embodiments, and has the corresponding functions and beneficial effects.

[0150] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the bright-field cell image segmentation method provided in any of the above embodiments, and has the corresponding functions and beneficial effects.

[0151] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0152] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0153] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A brightfield cell image segmentation method in fluorescence microscopy imaging, characterized in that It comprises the following steps: Step 1, input bright field cell image and corresponding cell nucleus image; Step 2, after OTSU threshold segmentation of the input cell nucleus image, the preliminary cell nucleus segmentation result is obtained, then the binary image mathematical morphology processing is used to remove stray points, and finally the centroid formula is used to obtain the centroid of each cell nucleus, the centroid is mapped to the bright field cell image respectively, and a rectangular frame containing the cell is cropped with the centroid as the center, so that the bright field cell image is cropped according to single cell to obtain multiple single cell images; Step 3, the single cell image is subjected to image enhancement operation, including background interference removal processing, cell edge highlighting processing and noise removal processing in turn; The background interference removal processing is double Gaussian filtering processing of the single cell image to remove background interference, the cell edge highlighting processing is top hat transformation of the single cell image subjected to double Gaussian filtering processing to highlight the cell edge in dark background, and the noise removal processing is median filtering processing of the single cell image subjected to top hat transformation to eliminate noise interference; Step 4, the single cell images subjected to cell edge highlighting processing and noise removal processing are each taken as a dimension to form a two-dimensional histogram, and an improved two-dimensional OTSU threshold segmentation method is used to process the two-dimensional histogram to obtain a preliminary segmentation result; Step 5, the single cell image subjected to two-dimensional OTSU threshold segmentation is subjected to binary image mathematical morphology operation to further optimize the segmentation result, and the binary image mathematical morphology operation includes closing operation, hole filling, opening operation and area constraint condition; Step 6, the single cell images subjected to step 5 are merged with the cell nucleus binary image as a marker point, and the merged cell image is further segmented by using a watershed segmentation algorithm based on marker control to complete cell image segmentation, and the watershed segmentation algorithm is established on the basis of mathematical morphology theory, simulates a three-dimensional terrain surface, and uses the binary cell nucleus image obtained in step 2 as a marker point of the watershed to further segment the merged cell image.

2. The brightfield cell image segmentation method in fluorescence microscopy imaging according to claim 1, wherein, In step 3, the Gaussian function formula is: (3) wherein, is a coordinate on the single-cell image, is a standard deviation of a Gaussian function, is a number of Gaussian filter functions, whose radii obey the 3σ criterion, by selecting different sizes of After the difference operation, the image after double Gaussian filtering is obtained as : (4) wherein, is the input single-cell grayscale image, * denotes convolution operation, is a large Gaussian filter, is a small Gaussian filter, is the image after double Gaussian filtering, the standard deviation of the Gaussian function of the small Gaussian filter has the same value as the cell size, the value is obtained by quantitative analysis; The top hat transformation formula is: (5) (6) Wherein, g represents the single-cell image to be processed, S is a structural unit, represents the single-cell image after the opening operation, h represents the single-cell image after the top-hat transformation, and the structural unit is a disc-shaped structural element; The median filtering formula is: (7) wherein, is the input single-cell image, take each pixel around neighbor pixel points, is the median filtered single-cell image.

3. The brightfield cell image segmentation method in fluorescence microscopy imaging according to claim 2, characterized in that The single cell image obtained after top hat transformation is taken as a dimension, and the single cell image subjected to median filtering processing is taken as another dimension, and the two dimensions form a two-dimensional histogram, and an improved two-dimensional OTSU threshold segmentation method is used to process the two-dimensional histogram to obtain a preliminary segmentation result: Assume that the input image size is , denoted as , the gray level is L, the image after double Gaussian filtering and top hat transformation is , the image after median filtering is , the image and the image The gray value at pixel is and , represents the frequency size at the gray value , and the joint probability is represented as: (8) wherein and ; Let the two-dimensional histogram be divided into background and foreground two classes, the background and foreground corresponding to dark and bright regions in the image, respectively, the optimal threshold values in the two dimensions of the two-dimensional histogram, respectively, the probability and the probability are represented as: (9) (10) At the same time, the class mean vector is expressed as: (11) (12) And the total mean vector of the two-dimensional histogram is expressed as: (13) The inter-class variance matrix is defined as: (14) The rank is used to measure the inter-class variance: (15) wherein is the class mean, and equations (9) and (10) show the class mean vector, is the overall mean, and equation (11) shows the overall mean vector; If the area far from the diagonal line in the two-dimensional histogram is ignored: (16) Then the rank of the inter-class variance is simplified as: (17) Wherein: (18) The optimal segmentation threshold vector is obtained by maximizing the value of is:​ (19) A one-dimensional search method is used to find the best segmentation threshold for each dimension of the two-dimensional histogram, the process of finding the best segmentation threshold in two dimensions is converted to finding the best segmentation threshold of each dimension of the two-dimensional histogram to realize cell image segmentation, and formula (19) is expressed as: wherein is a two-tuple on the projection plane of the two-dimensional histogram of the top-hat transformed image and the median filtered image , range is the same as the range of the original image gray levels.

4. The brightfield cell image segmentation method in fluorescence microscopy according to claim 1, wherein, In step 5, both the closing operation and the opening operation select a disc-shaped structural element, and the radius of the disc-shaped structural element of the closing operation and the opening operation is selected according to actual conditions.

5. A brightfield cell image segmentation apparatus in fluorescence microscopy imaging using the method as claimed in claim 1, characterized in that, The input module, the single-cell local image segmentation module, the image preprocessing module and the image segmentation module are sequentially connected. The input module is used for reading the bright-field cell image and the corresponding cell nucleus image. The single-cell local image segmentation module is used for determining the center of each cell according to the cell nucleus image, mapping the center to the bright-field cell image respectively, cutting the bright-field cell image according to the single cell, obtaining a plurality of single-cell images and outputting the single-cell images to the image preprocessing module. The image preprocessing module comprises a background interference elimination processing module, a cell edge highlighting processing module and a noise removal processing module which are sequentially connected, and is used for performing image enhancement operation on the single-cell images output by the single-cell image segmentation module, outputting the single-cell images after the background interference elimination, the cell edge highlighting and the noise removal to the image segmentation module. The image segmentation module comprises an improved two-dimensional Otsu threshold segmentation module, a mathematical morphology module and a watershed segmentation module which are sequentially connected, takes each single-cell image after the cell edge highlighting processing and the noise removal processing as a dimension to form a two-dimensional histogram, processes the two-dimensional histogram by using the improved two-dimensional Otsu threshold segmentation method, obtains a preliminary segmentation result, further refines the preliminary segmentation result by using the binary image mathematical morphology operation, takes the cell nucleus image as a marker point, combines all the single-cell images, further segments the adherent cells by using the marker-controlled watershed segmentation algorithm to obtain a cell segmentation image.

6. The bright-field cell image segmentation device in fluorescence microscopic imaging according to claim 5, characterized in that: The background interference elimination processing module is used for performing double Gaussian filtering on the single-cell image to remove the background interference. The cell edge highlighting processing module is used for highlighting the cell edge in the dark background by using top-hat transformation on the single-cell image after the double Gaussian filtering. The noise removal processing module is used for performing median filtering on the single-cell image after the top-hat transformation to eliminate the noise interference. The improved two-dimensional Otsu threshold segmentation module takes the single-cell image after the top-hat transformation as one dimension and takes the single-cell image after the median filtering as another dimension to form a two-dimensional histogram, processes the two-dimensional histogram by using the improved two-dimensional Otsu threshold segmentation method, obtains a preliminary segmentation result and outputs the preliminary segmentation result to the mathematical morphology module. The mathematical morphology module is used for further refining the preliminary segmentation result by using the binary image mathematical morphology operation on the single-cell image after the two-dimensional Otsu threshold segmentation. The watershed segmentation module takes the cell nucleus image as a marker point, combines all the single-cell images output by the mathematical morphology module, further segments the adherent cells by using the marker-controlled watershed segmentation algorithm to obtain a cell segmentation image. 7.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-6 when the computer program is executed by the processor. The processor executes the computer program to implement the bright-field cell image segmentation method in fluorescence microscopic imaging according to any one of claims 1 to 4.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by a processor to implement the bright field cell image segmentation method in fluorescence microscopic imaging according to any one of claims 1 to 4.

Images