Machine learning-based cell segmentation and typing method, device, instrument, and medium

Abstract

The present disclosure provides a machine learning based cell segmentation and typing method, device, instrument and medium, which includes: obtaining a metabolic image of at least one cell of a target object; performing single-cell image segmentation on the metabolic image of at least one cell by a machine learning segmentation model to obtain metabolic images of multiple single cells; extracting single-cell features for each metabolic image of the multiple single cells, and obtaining a single-cell image feature spectrum corresponding to the metabolic image of the single cell; combining the single-cell image feature spectrum corresponding to each metabolic image of the multiple single cells to obtain an image feature spectrum of a target object; and typing the cell by clustering the image feature spectrum of the target object.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to a Chinese patent application filed on December 29, 2021, with application number 202111628829.2, the entire contents of which are incorporated herein by reference. [Technical Field]

[0002] The present disclosure relates to the field of image segmentation and typing, and more particularly to machine learning-based cell segmentation and typing methods, devices, apparatus, and media. [Background technology]

[0003] Currently, in clinical practice, cytopathological examinations (e.g., H&E or RAP staining) are performed on exfoliated cells to determine information about cancer cells (e.g., gastric cancer cells, lung cancer cells), the number of cancer cells, etc. However, these examination methods are significantly influenced by the subjective factors of the pathologist, and the agreement between different cases and different pathologists is relatively low, and the sensitivity is relatively low (generally less than 60%).

[0004] Furthermore, cell pathology testing differs from histopathology in that cell pathology testing lacks tissue positioning information. Furthermore, cells may be subjected to processes such as staining, smearing, and fixation, which can damage the cell morphology, making it difficult to accurately quantify single cells. This is very detrimental to subsequent assessment of the degree of lesion in the target object.

[0005] Therefore, a new method is needed to solve the above problems. Summary of the Invention

[0006] To address the above-mentioned problems, the present disclosure provides a machine learning-based cell segmentation and typing method, which can determine the degree of lesion in a target object by avoiding damage to cell morphology and achieving accurate cell typing without being influenced by the subjective factors of the pathologist.

[0007] An embodiment of the present disclosure provides a machine learning-based cell segmentation and typing method, which includes: obtaining a metabolic image of at least one cell of a target object; performing single-cell image segmentation on the metabolic image of the at least one cell using a machine learning segmentation model to obtain metabolic images of multiple single cells; performing single-cell feature extraction on each metabolic image of the multiple single cells to obtain an image feature spectrum of the single cell corresponding to the metabolic image of the single cell, wherein the image feature spectrum of the single cell includes at least a metabolic feature of the cell; combining the image feature spectrum of the single cell corresponding to each metabolic image of the single cell among the multiple single-cell metabolic images to obtain an image feature spectrum of the target object; and typing the cell by clustering the image feature spectrum of the target object, wherein the typing indicates the cell type to which the cell belongs.

[0008] According to an embodiment of the present disclosure, typing the cells by clustering the image feature spectrum of the target object includes clustering the image feature spectrum of the target object to obtain a number of different types of cells, and typing the cells based on the number of different types of cells.

[0009] According to an embodiment of the present disclosure, the image feature spectrum of the target object is clustered using at least one of a k-means clustering method, a hierarchical clustering method, a clustering method using a self-organizing map (SOM), and a fuzzy clustering technique to obtain the number of different types of cells.

[0010] According to an embodiment of the present disclosure, the cells are typed based on the number of the different types of cells using at least one classifier selected from the group consisting of a Support Vector Machine (SVM), a Linear Discriminant Classifier, a k-nearest neighbor classifier (k-NN), a Logistic Regression Classifier, a Random Forest decision tree classifier, an Artificial Neural Network (ANN), and a Deep Learning Convolutional Neural Network (CNN or ConvNet) classifier.

[0011] According to an embodiment of the present disclosure, the cell segmentation and typing method further includes: analyzing principal components of the image feature spectrum of the target object to obtain principal component information corresponding to the image feature spectrum of each single cell, where the principal component information of different types of cells is different; obtaining metabolic characteristic targets of the same type of cells based on the principal component information; and determining the degree of lesion of the target object based on the metabolic characteristic targets.

[0012] According to an embodiment of the present disclosure, determining the lesion degree of the target object based on metabolic feature targets includes inputting the number of different types of cells and the metabolic feature targets of the same type of cells into a pre-trained machine learning classification model to determine the lesion degree of the target object.

[0013] According to an embodiment of the present disclosure, performing single-cell image segmentation on the metabolic image of the at least one cell using a machine learning segmentation model to obtain metabolic images of a plurality of single cells includes performing single-cell image segmentation on the metabolic image of the at least one cell using a neural network based on transfer learning to obtain metabolic images of the plurality of single cells.

[0014] According to an embodiment of the present disclosure, performing single-cell image segmentation on the metabolic image of the at least one cell using a machine learning segmentation model to obtain metabolic images of multiple single cells includes performing a first single-cell image segmentation on the metabolic image of the at least one cell using a neural network based on transfer learning, and performing a second segmentation on the image after the first single-cell segmentation using a watershed segmentation method or a flood-fill segmentation method to obtain metabolic images of the multiple single cells.

[0015] According to an embodiment of the present disclosure, the image feature spectrum of the single cell further includes morphological features of the cell.

[0016] According to an embodiment of the present disclosure, the morphological characteristics of the cell include at least one of the cell area, the sphericity of the cell shape, the circularity of the cell boundary, the cell center, the cell center eccentricity, the equivalent diameter, the cell circumference, the length of the major axis, the length of the minor axis, the ratio of the major axis to the minor axis, and the rotation angle of the major axis / minor axis.

[0017] According to an embodiment of the present disclosure, the metabolic characteristics of the cell include at least one of the following: lipid content, lipid concentration, protein content, protein concentration, deoxyribonucleic acid concentration, lipid / protein content ratio, lipid / protein concentration ratio, lipid / deoxyribonucleic acid concentration ratio, number of lipid droplets (lipid droplets, LD), lipid droplet area, ratio of lipid droplet area to total cell area, lipid / protein concentration ratio within the lipid droplet range, lipid component / protein component area ratio, lipid component / deoxyribonucleic acid component area ratio, lipid component ratio to total cell area, protein component ratio to total cell area, and lipid / protein concentration ratio within the lipid component range.

[0018] According to an embodiment of the present disclosure, combining image feature spectra of single cells corresponding to the metabolic images of each single cell among the metabolic images of the plurality of single cells to obtain an image feature spectrum of the target object includes arranging image feature spectra of single cells corresponding to the metabolic images of each single cell among the metabolic images of the plurality of single cells in a predetermined order to obtain an image feature spectrum of the target object.

[0019] According to an embodiment of the present disclosure, the metabolic image of the cell is an image based on Raman imaging.

[0020] According to an embodiment of the present disclosure, the cell types include cancer cells, immune cells, lymphocytes, mesothelial cells, epithelial cells, blood cells, or granulocytes.

[0021] An embodiment of the present disclosure provides a machine learning-based cell segmentation and typing device, which includes: an acquisition module configured to acquire a metabolic image of at least one cell of a target object; a segmentation module configured to perform single-cell image segmentation on the metabolic image of the at least one cell using a machine learning segmentation model to acquire metabolic images of multiple single cells; a feature extraction module configured to extract single-cell features for each metabolic image of the multiple single cells and acquire a single-cell image feature spectrum corresponding to the metabolic image of the single cell, where the single-cell image feature spectrum includes at least a metabolic feature of the cell; a spectrum combination module configured to combine the single-cell image feature spectrum corresponding to each metabolic image of the multiple single cells among the metabolic images of the multiple single cells to acquire an image feature spectrum of the target object; and a typing module configured to type the cell by clustering the image feature spectrum of the target object, where the typing indicates the cell type to which the cell belongs.

[0022] According to an embodiment of the present disclosure, the typing module includes: clustering an image feature spectrum of the target object to obtain a number of different types of cells; and typing the cells based on the number of different types of cells.

[0023] According to an embodiment of the present disclosure, the image feature spectrum of the target object is clustered using at least one of the following methods: k-means clustering, hierarchical clustering, self-organizing feature map clustering, and fuzzy clustering, to obtain the number of different types of cells.

[0024] According to an embodiment of the present disclosure, the cells are typed based on the number of the different types of cells using at least one classifier from the group consisting of a support vector machine classifier, a linear discriminant classifier, a K-nearest neighbor classifier, a logistic regression classifier, a random forest decision tree classifier, an artificial neural network classifier, and a deep learning convolutional neural network classifier.

[0025] According to an embodiment of the present disclosure, the cell segmentation and typing device further includes: a principal component analysis module configured to analyze principal components of the image feature spectrum of the target object to obtain principal component information corresponding to the image feature spectrum of each single cell, where the principal component information of different types of cells is different; a target acquisition module configured to acquire metabolic feature targets of the same type of cells based on the principal component information; and a lesion determination module configured to determine the degree of lesion of the target object based on the metabolic feature targets.

[0026] According to an embodiment of the present disclosure, the lesion determination module further inputs the number of different types of cells and the metabolic feature target of the same type of cells into a pre-trained machine learning classification model to determine the lesion degree of the target object.

[0027] According to an embodiment of the present disclosure, the segmentation module includes performing single-cell image segmentation on the metabolic image of the at least one cell using a neural network based on transfer learning to obtain metabolic images of the plurality of single cells.

[0028] According to an embodiment of the present disclosure, the segmentation module includes: a first segmentation module configured to perform a first single-cell image segmentation on the metabolic image of the at least one cell using a neural network based on transfer learning; and a second segmentation module configured to perform a second segmentation on the image after the first single-cell segmentation using a watershed segmentation method or a flooding segmentation method, thereby obtaining metabolic images of the plurality of single cells.

[0029] According to an embodiment of the present disclosure, the image feature spectrum of the single cell further includes morphological features of the cell.

[0030] According to an embodiment of the present disclosure, the morphological characteristics of the cell include at least one of the following: cell area, sphericity of cell shape, circularity of cell boundary, cell center, cell center eccentricity, equivalent diameter, cell circumference, major axis length, minor axis length, major axis / minor axis ratio, and major axis / minor axis rotation angle.

[0031] According to an embodiment of the present disclosure, the metabolic characteristics of the cell include at least one of the following: lipid content, lipid concentration, protein content, protein concentration, deoxyribonucleic acid concentration, lipid / protein content ratio, lipid / protein concentration ratio, lipid / deoxyribonucleic acid concentration ratio, number of lipid droplets, lipid droplet area, ratio of lipid droplet area to total cell area, lipid / protein concentration ratio within the lipid droplet range, lipid component / protein component area ratio, lipid component / deoxyribonucleic acid component area ratio, ratio of lipid components to total cell area, ratio of protein components to total cell area, and lipid / protein concentration ratio within the lipid component range.

[0032] According to an embodiment of the present disclosure, the spectral combination module includes arranging image feature spectra of single cells corresponding to the metabolic images of each single cell among the plurality of single-cell metabolic images in a predetermined order, and obtaining an image feature spectrum of the target object.

[0033] According to an embodiment of the present disclosure, the metabolic image of the cell is an image based on Raman imaging.

[0034] According to an embodiment of the present disclosure, the cell types include cancer cells, immune cells, lymphocytes, mesothelial cells, epithelial cells, blood cells, or granulocytes.

[0035] An embodiment of the present disclosure provides a machine learning based cell segmentation and typing apparatus, including a processor and a memory storing computer executable instructions that, when executed by the processor, cause the processor to perform the method.

[0036] An embodiment of the present disclosure provides a computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a processor, cause the processor to perform the above-described method.

[0037] An embodiment of the present disclosure provides an apparatus for performing image segmentation using a multi-layer neural network model, the apparatus including a processor and a memory having stored thereon computer-executable instructions that, when executed by the processor, cause the processor to perform the above-described method.

[0038] An embodiment of the present disclosure provides a computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a processor, cause the processor to perform the above-described method.

[0039]

[0009] The present disclosure provides a method, apparatus, device, and medium for cell segmentation and typing based on machine learning. Based on the machine learning-based cell segmentation and typing method of the present disclosure, cells can be accurately segmented using a machine learning segmentation model, and accurate cell typing can be achieved by clustering the cell's feature spectrum, and the degree of lesion in the target object can be accurately determined. The method of the present disclosure effectively avoids the influence of the pathologist's subjective factors and does not require damage to cell morphology. [Brief explanation of the drawings]

[0040] In order to more clearly explain the technical solutions of the embodiments of the present disclosure, the following briefly describes the accompanying drawings that need to be used in the description of the embodiments. It is obvious that the accompanying drawings in the following description are only some exemplary embodiments of the present disclosure, and those skilled in the art can obtain other accompanying drawings based on these accompanying drawings without any creative efforts. [Figure 1] 1 shows a flowchart of a machine learning based cell segmentation and typing method according to an embodiment of the present disclosure. [Figure 2] 1 shows an effect diagram of single cell segmentation on a cellular metabolic image based on a machine learning segmentation model according to the present disclosure. [Figure 3A] 10 shows a distribution map of lipid content in the image feature spectrum of a single cell corresponding to the acquired metabolic image of the single cell. [Figure 3B] 1 shows the characteristic spectrum obtained for the single case. [Figure 3C] A schematic diagram of cancer cells and normal cells is shown. [Figure 4] 1 shows an exemplary diagram of a cell segmentation and typing method according to an embodiment of the present disclosure. [Figure 5] Figure 1 shows the effect of typing single cells from positive and negative cell lines. [Figure 6]1 shows a schematic diagram of the values ​​of metabolic feature significance parameters (p-values). [Figure 7] A schematic diagram showing the sensitivity and specificity of single-cell metabolic imaging for diagnosing peritoneal metastasis of gastric cancer is shown. [Figure 8] 8 shows a block diagram of a machine learning based cell segmentation and typing device 800 according to an embodiment of the present disclosure. [Figure 9] 9 shows a structural diagram of a machine learning based cell segmentation and typing device 900 according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0041] In order to make the objectives, technical solutions, and advantages of the present disclosure clearer, exemplary embodiments according to the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, it should be understood that the described embodiments are only some embodiments of the present disclosure, but not all embodiments of the present disclosure, and the present disclosure is not limited to the exemplary embodiments described herein.

[0042] In this specification and the accompanying drawings, substantially the same or similar steps and elements are represented by the same or similar reference numerals, and repeated descriptions of those steps and elements are omitted. At the same time, in the description of this disclosure, terms such as "first", "second", etc. are used only to distinguish between descriptions, and should not be understood as indicating or implying relative importance or ordering.

[0043] Currently, clinical methods for determining cancer cell-related information are heavily influenced by the pathologist's subjective factors, have low consistency, and may even damage cell morphology, which is very detrimental to the subsequent judgment of the lesion extent of the target object.

[0044] To solve the above problems, the present disclosure provides a cell segmentation and typing method based on machine learning. Based on the method disclosed herein, cells can be accurately segmented using a machine learning segmentation model, and accurate cell typing can be achieved by clustering the cell's feature spectrum, and accurate determination of the degree of lesion in the target object can be achieved. The method disclosed herein effectively avoids the influence of the pathologist's subjective factors and does not damage cell morphology.

[0045] The machine learning-based cell segmentation and typing method according to the present disclosure will be described in detail below with reference to the accompanying drawings.

[0046] FIG. 1 shows a flowchart of a machine learning based cell segmentation and typing method according to an embodiment of the present disclosure.

[0047] The method disclosed herein can perform cell segmentation and typing based on label-free stimulated Raman imaging (e.g., stimulated Raman scattering (SRS)-based imaging). The stimulated Raman technique utilizes the wavelength difference between two lasers to excite molecular vibrations of specific chemical bonds in the C-H region.

[0048] Referring to FIG. 1, in step S110, a metabolic image of at least one cell of the target object can be acquired.

[0049] For example, the target object may be a human body organ or tissue, such as a stomach, a lung, etc. The target object may also be exfoliated cells obtained from a human body organ or tissue, such as exfoliated cells obtained from the stomach to determine the status of cancer cells in the stomach.

[0050] By way of example, the metabolic image of the cell may be an image based on Raman imaging.

[0051] As an example, a metabolic image of a cell may be acquired through one channel, such as a protein channel, a lipid channel, or a DNA channel.

[0052] As another example, a metabolic image of a cell may be acquired through multiple channels, including three channels: a protein channel, a lipid channel, and a DNA channel.

[0053] For example, for one case (eg, a stomach cancer case or a lung cancer case), a metabolic image of the at least one cell can be acquired by one or more channels.

[0054] In step S120, single-cell image segmentation can be performed on the metabolic image of the at least one cell using a machine learning segmentation model to obtain a plurality of single-cell metabolic images.

[0055] According to an embodiment of the present disclosure, performing single-cell image segmentation on the metabolic image of the at least one cell using a machine learning segmentation model to obtain metabolic images of a plurality of single cells may include performing single-cell image segmentation on the metabolic image of the at least one cell using a neural network based on transfer learning to obtain metabolic images of the plurality of single cells.

[0056] For example, by using an existing single-cell segmentation database and neural network segmentation model (e.g., a database and neural network segmentation model for single-cell segmentation in fluorescence images) and a small amount of stimulated Raman cell images and artificial marking data to train the above content using transfer learning, the required machine learning segmentation model can be obtained, thereby achieving high-precision single-cell metabolic image segmentation.

[0057] Unlike the conventional method of using neural network algorithms to achieve image segmentation, the image segmentation method disclosed herein uses the machine learning segmentation model obtained based on transfer learning, thereby avoiding the need to collect large amounts of clinical data and the need for large amounts of artificial marking (e.g., by pathologist experts), which significantly shortens the development cycle of related learning models and greatly popularizes the clinical application of single-cell metabolic imaging technology.

[0058] According to an embodiment of the present disclosure, performing single-cell image segmentation on the metabolic image of the at least one cell using a machine learning segmentation model to obtain metabolic images of multiple single cells may include performing a first single-cell image segmentation on the metabolic image of the at least one cell using a neural network based on transfer learning, and performing a second segmentation on the image after the first single-cell segmentation using a watershed segmentation method or a flooding segmentation method to obtain metabolic images of the multiple single cells.

[0059] For example, by using an existing single-cell segmentation database and neural network segmentation model (e.g., a database and neural network segmentation model for single-cell segmentation in fluorescence images) and a small amount of stimulated Raman cell images and artificial marking data to train the above content using transfer learning, the necessary transfer learning-based neural network can be obtained, thereby performing a first single-cell image segmentation on the metabolic image of the cells. After the first single-cell image segmentation, the metabolic image of the cells may have problems such as cells being too close together, resulting in insufficient segmentation. To further improve the accuracy of the single-cell segmentation, a second segmentation is performed on the image that underwent the first single-cell segmentation using a watershed segmentation method (e.g., a WaterShed segmentation algorithm) or a flooding segmentation method (e.g., a flood-fill algorithm), thereby obtaining metabolic images of the multiple single cells.

[0060] Compared with conventional single-cell segmentation methods using artificial rings, the segmentation method disclosed herein exhibits superior values ​​for relevant parameters used to evaluate single-cell segmentation effectiveness, such as an F1 score parameter value of 95% and a DICE parameter value of 89%. Figure 2 shows an effect diagram of single-cell segmentation performed on a metabolic image of a cell based on the machine learning segmentation model disclosed herein. (a) in Figure 2 shows a conceptual diagram of the machine learning segmentation model disclosed herein, with convolution kernel sizes of 64x64, 128x128, 256x256, and 512x512, respectively. (b) in Figure 2 shows an effect diagram after single-cell image segmentation is performed on metabolic images of two cells, where metabolic images of multiple single cells are segmented. As can be seen from the effect diagram, the machine learning segmentation model of the present disclosure can perform image segmentation of single cells very accurately for metabolic images of cells. Note that the machine learning segmentation model of the present disclosure is not limited to performing image segmentation of single cells for metabolic images of cells. In fact, the machine learning segmentation model of the present disclosure can perform image segmentation of single cells for any image, for example, for fluorescently imaged images and immunohistochemically captured images.

[0061] Continuing to refer to FIG. 1, in step S130, single-cell features are extracted for each single-cell metabolic image among the plurality of single-cell metabolic images, and a single-cell image feature spectrum corresponding to the single-cell metabolic image can be obtained, and the single-cell image feature spectrum includes at least the metabolic features of the cell.

[0062] For example, single-cell features can be extracted by any known method, such as measurement, calculation, etc. For each single-cell metabolic image, multiple single-cell features can be extracted, and the multiple single-cell features can be combined (e.g., sequenced) to obtain an image feature spectrum of the single cell.

[0063] For example, the metabolic characteristics of a cell may include at least one of the following: lipid content (Lipid Intensity), lipid concentration, protein content, protein concentration, deoxyribonucleic acid (DNA) concentration, lipid / protein content ratio (Lipid / Protein Intensity), lipid / protein concentration ratio, lipid / DNA concentration ratio, number of lipid droplets, lipid droplet area, ratio of lipid droplet area to total cell area, lipid / protein concentration ratio within the lipid droplet area, lipid component / protein component area ratio, lipid component / DNA component area ratio, lipid component fraction to total cell area (Lipid Area Fraction), ratio of protein components to total cell area, and lipid / protein concentration ratio within the lipid component area.

[0064] FIG. 3A shows a distribution map of lipid content in the image feature spectrum of a single cell corresponding to the acquired metabolic image of the single cell. As can be seen from the lipid content distribution map shown in FIG. 3A, the metabolic image of the single cell shows a large number of cells with lipid content between 1.0 and 3.0, and the color of the cells becomes darker as the lipid content increases. As shown in FIG. 3A, when the lipid content is relatively low (e.g., between 0.0 and 1.0), the color of the cells is relatively light. As shown in FIG. 3A (a), when the lipid content ratio is relatively high (e.g., between 1.0 and 3.0), the color of the cells is relatively dark. As shown in FIG. 3A (b), when the lipid content is high (e.g., between 3.0 and 4.0), the color of the cells is darkest, as shown in FIG. 3A (c).

[0065] According to an embodiment of the present disclosure, the image feature spectrum of the single cell may further include morphological features of the cell.

[0066] For example, the morphological characteristics of the cell include at least one of the cell area (Area), the sphericity of the cell shape (Round), the circularity of the cell boundary (Circularity), the cell center (Center), the cell center eccentricity (Eccentricity), the equivalent diameter (Equivalent Diameter), the cell perimeter (Perimeter), the length of the major axis (Max Axis Length), the length of the minor axis (Max Axis Length), the ratio of the major axis / minor axis, and the rotation angle of the major axis / minor axis (Orientation).

[0067] In step S140, the image feature spectrum of the target object can be obtained by combining the image feature spectrum of the single cells corresponding to each metabolic image of the single cells among the plurality of metabolic images of the single cells.

[0068] According to an embodiment of the present disclosure, combining image feature spectra of single cells corresponding to the metabolic images of each single cell among the metabolic images of the plurality of single cells to obtain an image feature spectrum of the target object may include arranging image feature spectra of single cells corresponding to the metabolic images of each single cell among the metabolic images of the plurality of single cells in a predetermined order to obtain an image feature spectrum of the target object.

[0069] For example, by sequentially combining the single-cell feature spectra corresponding to the metabolic images of each single cell, a feature spectrum for a target object can be obtained, such as a feature spectrum for cells exfoliated from the stomach. In other words, by arranging and combining the single-cell feature spectra corresponding to the metabolic images of each single cell, a feature spectrum for a single case (e.g., stomach cancer) can be obtained. As shown in Figure 3B, the feature spectra obtained for the single case are shown. The horizontal axis represents each single cell, taking values ​​from the first single cell to the last single cell, and the vertical axis represents the features. Seven features are selected from the above morphological and metabolic features as examples. The different shades of color shown in the area surrounded by the horizontal and vertical axes indicate the high and low feature values, with the higher the feature value, the darker the color.

[0070] Continuing to refer to FIG. 1, in step S150, the cells can be typed by clustering the image feature spectrum of the target object, and the typing indicates the type of cell to which the cell belongs.

[0071] According to an embodiment of the present disclosure, typing the cells by clustering the image feature spectrum of the target object may include clustering the image feature spectrum of the target object to obtain a number of different types of cells, and typing the cells based on the number of different types of cells.

[0072] For example, the image feature spectrum of the target object can be clustered using at least one of the following clustering methods: k-means clustering, hierarchical clustering, self-organizing map (SOM) clustering, and fuzzy c-means (FCM) clustering, to obtain the number of different types of cells.

[0073] By way of example, the cell types may include cancer cells, immune cells (e.g., neutrophils, eosinophilic granulocytes), lymphocytes, mesothelial cells, epithelial cells, blood cells, or granulocytes.

[0074] For example, the cells can be typed based on the number of the different types of cells using at least one classifier including a support vector machine learning (SVM) classifier, a linear discriminant classifier, a K nearest neighbor (KNN) classifier, a logistic regression (LR) classifier, a random forest (RF) decision tree classifier, an artificial neural network (ANN) classifier, and a deep learning convolutional neural network (e.g., AlexNet, ResNet, Inception, NASnet, VGG, etc.) classifier.

[0075] The above clustering method can be used to collect the same or similar features and obtain the number of different types of cells, and then determine the type of the cells based on the average value of all the feature values ​​of the same type after clustering. For example, the number of cells of the first type is 2000, the number of cells of the second type is 1000, and the number of cells of the third type is 10000. The average value obtained for all the feature values ​​of the first type is, for example, 1.3, the average value obtained for all the feature values ​​of the second type is, for example, 0.8, and the average value obtained for all the feature values ​​of the third type is, for example, 2.2. For example, based on a previous clinical trial, cells with an average value of 1 or less are designated as epithelial cells, those with an average value between 1 and 2 are designated as lymphocytes, and those with an average value between 2 and 3 are designated as cancer cells. As can be seen from the above results, the first type of cells are lymphocytes, the second type of cells are epithelial cells, and the third type of cells are cancer cells. The above is merely an illustrative example, and those skilled in the art can flexibly set appropriate values ​​according to the actual situation.

[0076] According to an embodiment of the present disclosure, the cell segmentation and typing method (not shown in FIG. 1 ) of the present disclosure may further include: analyzing principal components of the image feature spectrum of the target object to obtain principal component information corresponding to the image feature spectrum of each single cell, where the principal component information of different types of cells is different; obtaining metabolic feature targets of the same type of cells based on the principal component information; and determining the degree of lesion of the target object based on the metabolic feature targets.

[0077] For example, Principal Component Analysis (PCA) on the image feature spectrum of a target object can help reduce the dimension of the acquired cell features, thereby facilitating the quantification of each feature.

[0078] For example, the principal component information of different types of cells is different, and the metabolic feature targets (also called significant feature points) of the same type of cells can be obtained based on the principal component information, which can help determine the central position of the metabolic features of the same type of exfoliated cells. For example, after principal component analysis, all the features of one single cell are dimensionally reduced to three features, and then all the dimensionally reduced features of all the single cells of the same type are analyzed together, so that the central position of the metabolic features of the cells of that type, i.e., the metabolic feature targets, can be determined.

[0079] For example, the principal components of the metabolic signatures of cancer cells are significantly different from those of other types of exfoliated cells. After achieving single-cell segmentation and typing through the above-mentioned principal component analysis and unsupervised learning clustering algorithm, we obtained the schematic diagram of cancer cells and normal cells shown in Figure 3C, where POS represents positive cases and NEG represents negative cases. PC1 and PC2 are the axes of principal component analysis and are orthogonal to each other. In Figure 3C, the large circle indicates the area of ​​normal cells, and the small circle indicates the area of ​​tumor cells. Each point represents a cell. A, B, and C in Figure 3C indicate the location of tumor cells in the cellular metabolic image.

[0080] As can be seen from the single-cell typing achieved by the unsupervised learning clustering algorithm, the method disclosed herein can achieve high-accuracy single-cell typing without the need for a large number of SRS images or artificial markings, further shortening the development time for related models. Furthermore, the quantification of heterogeneous shed cell counts and metabolic feature targets using the principal component analysis-associated clustering method has higher accuracy than machine learning models that use raw image feature training as feature input for training a gastric cancer peritoneal diagnosis model, for example.

[0081] According to an embodiment of the present disclosure, determining the lesion degree of the target object based on metabolic feature targets may include inputting the number of different types of cells and the metabolic feature targets of the same type of cells into a pre-trained machine learning classification model to determine the lesion degree of the target object.

[0082] For example, taking gastric cancer as an example, by inputting the number of cancer cells, the number of epithelial cells, the number of immune cells, the number of blood cells, the metabolic signature target of cancer cells, the metabolic signature target of epithelial cells, the metabolic signature target of immune cells, the metabolic signature target of blood cells, and the corresponding actual test results (e.g., early stage cancer, mid-stage cancer, mid-late stage cancer, and late stage cancer) into a machine learning classification model for training, a pre-trained machine learning classification model can be obtained.

[0083] The number of different types of cells and the metabolic feature target of the same type of cells can be input into a pre-trained machine learning classification model to quickly and accurately determine the lesion level of the target object, thereby quickly and accurately diagnosing the target object. For example, determining the lesion level of the target object as peritoneal metastasis positive can help quickly and accurately diagnose the result as "late-stage cancer," thereby helping doctors provide appropriate treatment.

[0084] As can be seen from the machine learning-based cell segmentation and typing method disclosed herein, combining Figures 1 to 3C, the method disclosed herein uses transfer learning to segment single cells and unsupervised clustering to achieve high-precision single-cell segmentation without the need for large numbers of SRS images or artificial markings, significantly shortening the model development period. Furthermore, the use of principal component analysis-associated clustering to quantify the number of heterogeneous shed cells and metabolic signature targets results in a trained machine learning model with higher accuracy. As can be seen from the above, the method disclosed herein avoids damage to cell morphology and achieves accurate cell typing without being influenced by pathologist subjectivity, thereby greatly promoting the clinical application of single-cell metabolic imaging technology.

[0085] To make the above machine learning based cell segmentation and typing method according to the present disclosure clearer, the above method according to the present disclosure will now be described as an example.

[0086] FIG. 4 shows an exemplary diagram of a cell segmentation and typing method according to an embodiment of the present disclosure.

[0087] Referring to Figure 4, in step a, based on Raman imaging technology, two SRS single-cell metabolic images were obtained using two channels, a protein channel and a lipid channel, for exfoliated cells obtained from the stomach of a gastric cancer patient.

[0088] In step b, single-cell segmentation is performed on each of the two SRS single-cell metabolic images using the machine learning segmentation model, and multiple single-cell metabolic images are obtained as shown in the figure.

[0089] In step c, single-cell features are extracted for each segmented single-cell metabolic image from the two SRS single-cell metabolic images. The extracted features include eight features: lipid content, lipid / protein content ratio, lipid component ratio to total cell area, protein component ratio to total cell area, lipid / protein concentration ratio within the lipid component range, cell shape sphericity, cell central eccentricity, and major / minor axis ratio. Combining the single-cell features extracted for each single-cell metabolic image described above allows for the acquisition of a single-cell image feature spectrum corresponding to the single-cell metabolic image (the lipid content distribution diagram is shown in the figure). Combining the single-cell image feature spectra corresponding to each single-cell metabolic image from the two SRS single-cell metabolic images in sequence allows for the acquisition of a feature spectrum for gastric cancer, as shown in the figure.

[0090] In step d, clustering (i.e., using unsupervised learning methods) and principal component analysis are performed on the feature spectrum of the gastric cancer case to obtain the number of different cell types and metabolic feature targets of the same cell types. The figure shows a schematic diagram of the three components PC1, PC2, and PC3 of the principal component analysis for each single cell of the case, as well as a diagram of the effect of single-cell classification.

[0091] In step e, the number of different types of cells obtained in step d and the metabolic feature targets of the same types of cells are input into a pre-trained machine learning classification model (i.e., a model that employs supervised learning), and the gastric cancer case is determined to be peritoneal metastasis positive, explaining that the cancer has reached the late stage at that point.

[0092] As can be seen from the cell segmentation and typing method of the present disclosure, which is described in detail in the form of an example in conjunction with the above-mentioned Figure 4, the method of the present disclosure successfully combines supervised learning and unsupervised learning, improves the accuracy of cell segmentation, typing and determination of the degree of pathology of objects, avoids the influence of the pathologist's subjective factors, and does not damage cell morphology.

[0093] As shown in FIGS. 5 to 7, the method according to the present disclosure has excellent effects in detecting exfoliated cells from lung cancer and stomach cancer.

[0094] Figure 5 shows the effectiveness of typing single cells from positive and negative cell lines. In Figure 5, abnormal lipid metabolism in gastric cancer cells was examined using cell lines. Two cell lines (a positive cell line (e.g., SNU) and a negative cell line (e.g., GES)) were selected and tested. Seven cellular metabolic features were selected and extracted: lipid / protein content ratio (Lipid / PRO Int), lipid area fraction (Lipid area fraction), protein content (Protein Int), lipid content (Lipid Int), cell area (Area), cell shape sphericity (Round), and cell boundary circularity (Circle). 5, (a) shows a multi-channel SRS image, (b) shows the image after principal component analysis and clustering, (c) shows the autocorrelation coefficient of the metabolic features of cells, and (d) shows the cross-validated sensitivity and specificity of statistical single-cell typing, which shows a true positive accuracy rate of 98.36%, a false positive error rate of 1.64%, a false negative error rate of 0%, and a true negative accuracy rate of 100%. As can be seen from the figure, the accuracy rate of cell typing achieved based on the method disclosed herein is 98.36%, and the specificity is 100%.

[0095] Figure 6 shows a schematic diagram of the metabolic feature significance parameter (p-value). Referring to Figure 6, the metabolic features of peritoneal lavage cells from gastric cancer cases with and without peritoneal metastasis were statistically compared. Comparing the peritoneal metastasis and non-metastasis cases, the metabolic features of single cells were significantly different, and the p-values ​​of the metabolic features were smaller than those of the morphological features. This further demonstrates that the metabolic and morphological information provided by single-cell metabolic imaging techniques for diagnosing peritoneal metastasis is more accurate than conventional testing methods that only provide morphological information. In other words, as can be seen from the results shown in Figure 6, the image feature spectrum of single cells, including the metabolic features of cells extracted according to the method disclosed herein, is more accurate in determining the extent of gastric cancer lesions than conventional methods that extract only morphological features of cells.

[0096] Figure 7 shows a schematic diagram of the sensitivity and specificity of single-cell metabolic imaging for diagnosing peritoneal metastasis of gastric cancer. The metabolic signatures of exfoliated cells from 34 cases were initially studied, and the number of exfoliated cells and metabolic signature targets for each case were extracted. A machine learning algorithm was then used to predict whether gastric cancer had peritoneal metastasis or not. The sensitivity and specificity of the statistical machine learning diagnostic model were cross-validated, using intraoperative laparoscopic metastatic lesion detection as the gold standard.

[0097] Referring to Figure 7, (a) shows a comparison of the principal components of metabolic features of cancer cells (e.g., atypical lymphocytes) in 34 positive (POS) cases and negative (NEG) cases, (b) shows the results of cross-validation, and (c) shows the area under the curve (AUC) calculated by creating sensitivity and specificity curves. As can be seen from Figure 7, the sensitivity and specificity achieved based on the method disclosed herein are 84.625% and 85.71%, respectively, with an area under the curve (AUC) of 0.879.

[0098] As can be seen from the detailed experimental data in Figures 5 to 7, the accuracy of single-cell segmentation based on the method disclosed herein is about 95%, the accuracy of single-cell typing verification by positive / negative cell systems reaches more than 98%, there is a significant difference between the single-cell metabolic features of negative / positive gastric cancer cases, with a p-value much smaller than 0.05, the sensitivity and specificity of single-cell metabolic imaging for predicting peritoneal metastasis reaches 85%, and a relationship diagram between sensitivity and (1-specificity) is created to generate an ROC curve and calculate the area under the curve (AUC=0.89).

[0099] In addition, experiments were conducted using pancreatic cancer exfoliation cell testing as an example.

[0100] Specifically, a small amount of pancreatic cancer cells are extracted from the pancreatic tissue of a pancreatic cancer case, and the specimen is smeared. Each sample is then processed in the same manner as above, imaged, and the imaging data analyzed, after which normal tissue, incisal edge tissue, and cancer tissue are distinguished, enabling intraoperative incisal edge examination.

[0101] The experimental data obtained are as follows: There was a significant difference between the metabolic characteristics of single cells in normal and cancerous tissues, with p-values ​​significantly less than 0.05. The sensitivity and specificity of single-cell metabolic imaging for exfoliative cell typing reached 70% and 85%, respectively. A relationship between sensitivity and (1-specificity) was plotted to generate an ROC curve, and the area under the curve (AUC=0.8) was calculated. The sensitivity and specificity of single-cell metabolic imaging for tissue margin testing reached 98% and 98%, respectively. A relationship between sensitivity and (1-specificity) was plotted to generate an ROC curve, and the area under the curve (AUC=0.98) was calculated.

[0102] As can be seen from the various experiments above, the cell segmentation and typing method disclosed herein has high sensitivity and specificity, is well applicable to clinical practice, and provides an effective, rapid, and accurate new method for cancer treatment.

[0103] In addition to the above-mentioned machine learning-based cell segmentation and typing method, the present disclosure further provides a machine learning-based cell segmentation and typing device, which will be described below in conjunction with FIG.

[0104] 8 shows a block diagram of a machine learning-based cell segmentation and typing apparatus 800 according to an embodiment of the present disclosure. The above description of the machine learning-based cell segmentation and typing method equally applies to apparatus 800, unless expressly stated otherwise.

[0105] Referring to FIG. 8, the apparatus 800 may include an acquisition module 810, a segmentation module 820, a feature extraction module 830, a spectrum combination module 840, and a typing module 850.

[0106] According to an embodiment of the present disclosure, the acquisition module 810 may be configured to acquire a metabolic image of at least one cell of the target object.

[0107] For example, the target object may be a human body organ or tissue, such as a stomach, a lung, etc. The target object may also be exfoliated cells obtained from a human body organ or tissue, such as exfoliated cells obtained from the stomach to determine the status of cancer cells in the stomach.

[0108] As an example, the metabolic image of the cell may be an image based on Raman imaging.

[0109] As an example, a metabolic image of a cell may be acquired through one channel, such as a protein channel, a lipid channel, or a DNA channel.

[0110] As another example, a metabolic image of a cell may be acquired through multiple channels, including three channels: a protein channel, a lipid channel, and a DNA channel.

[0111] For example, for one case (eg, a stomach cancer case or a lung cancer case), a metabolic image of the at least one cell can be acquired by one or more channels.

[0112] According to an embodiment of the present disclosure, the segmentation module 820 may be configured to perform single-cell image segmentation on the metabolic image of the at least one cell using a machine learning segmentation model to obtain a plurality of single-cell metabolic images.

[0113] For example, the segmentation module 820 may include performing single-cell image segmentation on the metabolic image of the at least one cell using a neural network based on transfer learning to obtain metabolic images of the plurality of single cells.

[0114] By using existing single-cell segmentation databases and neural network segmentation models (e.g., databases and neural network segmentation models for single-cell segmentation in fluorescence images) and a small amount of stimulated Raman cell images and artificial marking data, and training the above content using transfer learning, the required machine learning segmentation model can be obtained, thereby achieving high-precision single-cell metabolic image segmentation.

[0115] Unlike the conventional method of using neural network algorithms to achieve image segmentation, the image segmentation method disclosed herein uses the machine learning segmentation model obtained based on transfer learning, thereby avoiding the need to collect large amounts of clinical data and perform large amounts of artificial marking (e.g., by pathologist experts), which significantly shortens the development cycle of related learning models and greatly popularizes the clinical application of single-cell metabolic imaging technology.

[0116] As another example, the segmentation module 820 may include a first segmentation module configured to perform a first single-cell image segmentation on the metabolic image of the at least one cell using a neural network based on transfer learning, and a second segmentation module configured to perform a second segmentation on the image after the first single-cell segmentation using a watershed segmentation method or a flooding segmentation method to obtain metabolic images of the plurality of single cells.

[0117] According to an embodiment of the present disclosure, the feature extraction module 830 may be configured to extract single-cell features for each single-cell metabolic image among the plurality of single-cell metabolic images, and obtain a single-cell image feature spectrum corresponding to the single-cell metabolic image, wherein the single-cell image feature spectrum includes at least the metabolic features of the cell.

[0118] For example, single-cell features can be extracted by any known method, such as measurement, calculation, etc. For each single-cell metabolic image, multiple single-cell features can be extracted, and the multiple single-cell features can be combined (e.g., aligned) to obtain an image feature spectrum of the single cell.

[0119] For example, the metabolic characteristics of a cell may include at least one of the following: lipid content (Lipid Intensity), lipid concentration, protein content, protein concentration, deoxyribonucleic acid (DNA) concentration, lipid / protein content ratio (Lipid / Protein Intensity), lipid / protein concentration ratio, lipid / DNA concentration ratio, number of lipid droplets, lipid droplet area, ratio of lipid droplet area to total cell area, lipid / protein concentration ratio within the lipid droplet area, lipid component / protein component area ratio, lipid component / DNA component area ratio, lipid component fraction to total cell area (Lipid Area Fraction), ratio of protein components to total cell area, and lipid / protein concentration ratio within the lipid component area.

[0120] According to an embodiment of the present disclosure, the image feature spectrum of the single cell may further include morphological features of the cell.

[0121] For example, the morphological characteristics of the cell include at least one of the cell area (Area), the sphericity of the cell shape (Round), the circularity of the cell boundary (Circularity), the cell center (Center), the cell center eccentricity (Eccentricity), the equivalent diameter (Equivalent Diameter), the cell perimeter (Perimeter), the length of the major axis (Max Axis Length), the length of the minor axis (Max Axis Length), the ratio of the major axis / minor axis, and the rotation angle of the major axis / minor axis (Orientation).

[0122] According to an embodiment of the present disclosure, the spectral combination module 840 may be configured to combine image feature spectra of single cells corresponding to the metabolic images of each single cell among the plurality of single-cell metabolic images to obtain an image feature spectrum of the target object.

[0123] For example, the spectrum combination module 840 may include arranging the image feature spectra of single cells corresponding to the metabolic images of each single cell among the plurality of metabolic images of single cells in a predetermined order to obtain the image feature spectrum of the target object. For example, the feature spectra of the target object may be obtained by sequentially combining the single cell feature spectra corresponding to the metabolic images of each single cell to obtain a feature spectrum for cells exfoliated from the stomach. In other words, by arranging and combining the single cell feature spectra corresponding to the metabolic images of each single cell, a feature spectrum for a single case (e.g., stomach cancer) can be obtained.

[0124] According to an embodiment of the present disclosure, the typing module 850 may be configured to type the cell by clustering image feature spectra of the target object, the typing indicating the cell type to which the cell belongs.

[0125] For example, the typing module 850 may include clustering the image feature spectrum of the target object to obtain a number of different types of cells, and typing the cells based on the number of different types of cells.

[0126] For example, the image feature spectrum of the target object can be clustered using at least one of the following clustering methods: k-means clustering, hierarchical clustering, self-organizing map (SOM) clustering, and fuzzy c-means (FCM) clustering, to obtain the number of different types of cells.

[0127] By way of example, the cell types may include cancer cells, immune cells (e.g., neutrophilic granulocytes, acidophilic granulocytes), lymphocytes, mesothelial cells, epithelial cells, blood cells, or granulocytes.

[0128] For example, the cells can be typed based on the number of the different types of cells using at least one classifier including a support vector machine learning (SVM) classifier, a linear discriminant classifier, a K nearest neighbor (KNN) classifier, a logistic regression (LR) classifier, a random forest (RF) decision tree classifier, an artificial neural network (ANN) classifier, and a deep learning convolutional neural network (e.g., AlexNet, ResNet, Inception, NASnet, VGG, etc.) classifier.

[0129] The above clustering method can be used to collect the same or similar features and obtain the number of different types of cells, and then determine the type of the cells based on the average value of all the feature values ​​of the same type after clustering. For example, the number of cells of the first type is 2000, the number of cells of the second type is 1000, and the number of cells of the third type is 10000. The average value obtained for all the feature values ​​of the first type is, for example, 1.3, the average value obtained for all the feature values ​​of the second type is, for example, 0.8, and the average value obtained for all the feature values ​​of the third type is, for example, 2.2. For example, based on a previous clinical trial, cells with an average value of 1 or less are designated as epithelial cells, those with an average value between 1 and 2 are designated as lymphocytes, and those with an average value between 2 and 3 are designated as cancer cells. As can be seen from the above results, the first type of cells are lymphocytes, the second type of cells are epithelial cells, and the third type of cells are cancer cells. The above is merely an illustrative example, and those skilled in the art can flexibly set appropriate values ​​according to the actual situation.

[0130] According to an embodiment of the present disclosure, the cell segmentation and typing device of the present disclosure may further include a principal component analysis module, a target acquisition module, and a lesion determination module (not shown in FIG. 8 ), where the principal component analysis module may be configured to analyze principal components of the image feature spectrum of the target object to obtain principal component information corresponding to the image feature spectrum of each single cell, where the principal component information of different types of cells is different, the target acquisition module may be configured to obtain metabolic feature targets of the same types of cells based on the principal component information, and the lesion determination module may be configured to determine the degree of lesion of the target object based on the metabolic feature targets.

[0131] For example, Principal Component Analysis (PCA) on the image feature spectrum of a target object can help reduce the dimension of the acquired cell features, thereby facilitating the quantification of each feature.

[0132] According to an embodiment of the present disclosure, the lesion determination module further inputs the number of different types of cells and the metabolic feature target of the same type of cells into a pre-trained machine learning classification model to determine the lesion degree of the target object.

[0133] For example, taking gastric cancer as an example, for example, the number of cancer cells, the number of epithelial cells, the number of immune cells, the number of blood cells, the metabolic feature targets of cancer cells, the metabolic feature targets of epithelial cells, the metabolic feature targets of immune cells, the metabolic feature targets of blood cells, and the corresponding actual test results (e.g., early cancer, mid-stage cancer, mid-late cancer, and late cancer) can be input into a machine learning classification model for training, thereby obtaining a pre-trained machine learning classification model.

[0134] The number of different types of cells and the metabolic feature target of the same type of cells can be input into a pre-trained machine learning classification model to quickly and accurately determine the lesion level of the target object, thereby quickly and accurately diagnosing the target object. For example, determining the lesion level of the target object as peritoneal metastasis positive can help quickly and accurately diagnose the result as "late-stage cancer," thereby helping doctors provide appropriate treatment.

[0135] The above describes the details of the operations involved in the machine learning-based cell segmentation and typing method of the present disclosure, and for the sake of brevity, we will not go into detail here. However, for relevant details, please refer to the descriptions above with respect to Figures 1 to 7.

[0136] The machine learning-based cell segmentation and typing method and apparatus according to the disclosed embodiments have been described above with reference to Figures 1-8. However, it should be understood that each module in the apparatus shown in Figure 8 may be configured as software, hardware, firmware, or any combination thereof that performs a specific function. For example, the modules may correspond to dedicated integrated circuits, pure software code, or a module that combines software and hardware.

[0137] Although the machine learning-based cell segmentation and typing device 800 has been described above as being divided into modules for performing each process, it will be apparent to those skilled in the art that the processes performed by each module can be performed even if the machine learning-based cell segmentation and typing device does not perform any specific module division or if there are no clear boundaries between the modules. Furthermore, the device described above with reference to FIG. 8 is not limited to including the modules described above, and other modules (e.g., a storage module, a data processing module, etc.) may be added as needed, or the modules may be combined.

[0138] The machine learning-based cell segmentation and typing method according to the present disclosure can be recorded on a computer-readable recording medium. Specifically, the present disclosure provides a computer-readable recording medium storing computer-executable instructions that, when executed by a processor, can prompt the processor to perform the above-described machine learning-based cell segmentation and typing method. Examples of computer-readable recording media include magnetic media (e.g., hard disks, floppy disks, magnetic tapes), optical media (e.g., CD-ROMs, DVDs, etc.), magneto-optical media (e.g., optical disks), and hardware devices specially prepared for storing and executing program instructions (e.g., read-only memory (ROM), random access memory (RAM), flash memory, etc.).

[0139] It should be noted that the present disclosure further provides a machine learning based cell segmentation and typing device, which will now be described in conjunction with FIG. 9.

[0140] 9 shows a structural diagram of a machine learning-based cell segmentation and typing device 900 according to an embodiment of the present disclosure. The above description of the machine learning-based cell segmentation and typing method equally applies to device 900 unless expressly stated otherwise.

[0141] 9, a device 900 may include a processor 901 and a memory 902. The processor 901 and the memory 902 may both be connected by a bus 903.

[0142] The processor 901 can perform various operations and processes based on programs stored in the memory 902. Specifically, the processor 901 may be an integrated circuit chip and has signal processing capabilities. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or an individual hardware assembly. Each method, step, and logic block diagram disclosed in the embodiments of the present application can be realized or executed. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor, such as an X86 architecture or an ARM architecture.

[0143] The memory 902 stores computer-executable instructions that, when executed by the processor 901, implement a machine learning-based cell segmentation and typing method. The memory 902 may be volatile or non-volatile, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM) used as an external cache. By way of example and not limitation, many forms of RAM may be used, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), extended synchronous dynamic random access memory (ESDRAM), synchronously connected dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM). It should be understood that the memory of the methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.

[0144] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functions, and operations that can be implemented in accordance with various exemplary systems, methods, and computer program products of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, program segment, or portion of code, which includes at least one executable instruction for implementing a given logical function. In alternative implementations, the functions displayed in the blocks may occur in a different order than that displayed in the accompanying drawings. For example, two blocks shown in succession may actually be executed substantially in parallel or may be executed in the reverse order depending on the functionality involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs a given function or operation, or by a combination of dedicated hardware and computer instructions.

[0145] In general, various exemplary embodiments of the present disclosure may be implemented in hardware or special purpose circuits, software, firmware, logic, or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. When various aspects of embodiments of the present disclosure are shown or described as block diagrams, flowcharts, or some other graphical representation, it will be understood that the blocks, apparatus, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller, or other computing device, or some combination thereof, as non-limiting examples.

[0146] The exemplary embodiments of the present disclosure described in detail above are merely illustrative and not limiting, and those skilled in the art should understand that various modifications and combinations of these embodiments or their features can be made without departing from the principle and spirit of the present disclosure, and such modifications should fall within the scope of the present disclosure.

Claims

1. acquiring a metabolic image of at least one cell of the target object; performing single-cell image segmentation on the metabolic image of the at least one cell using a machine learning segmentation model to obtain a plurality of single-cell metabolic images; extracting single-cell features for each of the plurality of single-cell metabolic images and obtaining a single-cell image feature spectrum corresponding to the single-cell metabolic image, wherein the single-cell image feature spectrum includes at least a metabolic feature of the cell; combining image feature spectra of the single cells corresponding to each of the plurality of single-cell metabolic images to obtain an image feature spectrum of the target object; A machine learning based cell segmentation and typing method comprising: typing the cells by clustering image feature spectra of the target object, the typing indicating the cell type to which the cells belong.

2. Typing the cells by clustering image feature spectra of the target object includes: Clustering the image feature spectrum of the target object to obtain the number of different types of cells; and typing the cells based on the number of different types of cells.

3. 3. The cell segmentation and typing method of claim 2, wherein the image feature spectrum of the target object is clustered by at least one of the following methods: k-means clustering, hierarchical clustering, self-organizing map clustering, and fuzzy clustering to obtain the number of different types of cells.

4. 3. The method of claim 2, wherein the cells are typed based on the number of the different types of cells with at least one of the following classifiers: a support vector machine classifier, a linear discriminant classifier, a K-nearest neighbor classifier, a logistic regression classifier, a random forest decision tree classifier, an artificial neural network classifier, and a deep learning convolutional neural network classifier.

5. The cell segmentation and typing method comprises: Analyzing the principal components of the image feature spectrum of the target object to obtain principal component information corresponding to the image feature spectrum of each single cell, where the principal component information of different types of cells is different; Obtaining metabolic signature targets of cells of the same type based on the principal component information; The cell segmentation and typing method of claim 2 , further comprising: determining a disease extent of the target object based on metabolic feature targets.

6. Determining the disease extent of the target object based on the metabolic feature target includes: The cell segmentation and typing method of claim 5, further comprising inputting the number of different types of cells and the metabolic feature targets of the same type of cells into a pre-trained machine learning classification model to determine the degree of pathology of the target object.

7. Performing single-cell image segmentation on the metabolic image of the at least one cell using the machine learning segmentation model to obtain a plurality of metabolic images of the single cells includes:

2. The cell segmentation and typing method of claim 1, further comprising performing single-cell image segmentation on the metabolic image of the at least one cell using a neural network based on transfer learning to obtain metabolic images of the plurality of single cells.

8. Performing single-cell image segmentation on the metabolic image of the at least one cell using the machine learning segmentation model to obtain a plurality of metabolic images of the single cells includes: performing a first single-cell image segmentation on the metabolic image of the at least one cell by a neural network based on transfer learning; 2. The cell segmentation and typing method of claim 1, further comprising: performing a second segmentation on the image obtained by the first single-cell segmentation using a watershed segmentation method or a flooding segmentation method, thereby obtaining metabolic images of the plurality of single cells.

9. The cell segmentation and typing method of claim 1 , wherein the image feature spectrum of the single cell further includes morphological features of the cell.

10. 10. The cell segmentation and typing method of claim 9, wherein the morphological features of the cells include at least one of a cell area, a sphericity of a cell shape, a circularity of a cell boundary, a cell center, a cell center eccentricity, an equivalent diameter, a cell circumference, a major axis length, a minor axis length, a major axis / minor axis ratio, and a major axis / minor axis rotation angle.

11. The cell segmentation and typing method of claim 9, wherein the metabolic features of the cell include at least one of the following: lipid content, lipid concentration, protein content, protein concentration, deoxyribonucleic acid concentration, lipid / protein content ratio, lipid / protein concentration ratio, lipid / deoxyribonucleic acid concentration ratio, number of lipid droplets, lipid droplet area, ratio of lipid droplet area to total cell area, lipid / protein concentration ratio within the lipid droplet range, lipid component / protein component area ratio, lipid component / deoxyribonucleic acid component area ratio, ratio of lipid components to total cell area, ratio of protein components to total cell area, and lipid / protein concentration ratio within the lipid component range.

12. Combining the image feature spectra of the single cells corresponding to each of the plurality of metabolic images of the single cells to obtain the image feature spectrum of the target object includes: The cell segmentation and typing method of claim 1 , further comprising arranging image feature spectra of single cells corresponding to each of the plurality of single-cell metabolic images in a predetermined order to obtain the image feature spectrum of the target object.

13. The cell segmentation and typing method according to any one of claims 1 to 12, wherein the metabolic image of the cell is an image based on Raman imaging.

14. The cell segmentation and typing method according to any one of claims 1 to 12, wherein the cell types comprise cancer cells, immune cells, lymphocytes, mesothelial cells, epithelial cells, blood cells or granulocytes.

15. an acquisition module for acquiring a metabolic image of at least one cell of the target object; a segmentation module that performs single-cell image segmentation on the metabolic image of the at least one cell using a machine learning segmentation model to obtain a plurality of metabolic images of the single cells; a feature extraction module that extracts single-cell features from each of the plurality of single-cell metabolic images and obtains a single-cell image feature spectrum corresponding to the single-cell metabolic image, the single-cell image feature spectrum including at least a metabolic feature of the cell; a spectrum combination module that combines image feature spectra of single cells corresponding to each of the plurality of single-cell metabolic images to obtain an image feature spectrum of the target object; a typing module that types the cells by clustering image feature spectra of the target object, the typing indicating the cell type to which the cells belong.

16. The typing module includes: Clustering the image feature spectrum of the target object to obtain the number of different types of cells; and typing the cells based on the number of different types of cells.

17. The cell segmentation and typing device of claim 16, wherein the image feature spectrum of the target object is clustered by at least one of a k-means clustering method, a hierarchical clustering method, a self-organizing feature map clustering method, and a fuzzy clustering method to obtain the number of different types of cells.

18. 17. The cell segmentation and typing apparatus of claim 16, wherein the cells are typed based on the number of the different types of cells with at least one of the following classifiers: a support vector machine classifier, a linear discriminant classifier, a K-nearest neighbor classifier, a logistic regression classifier, a random forest decision tree classifier, an artificial neural network classifier, and a deep learning convolutional neural network classifier.

19. The cell segmentation and typing device comprises: a principal component analysis module that analyzes principal components of the image feature spectrum of the target object to obtain principal component information corresponding to the image feature spectrum of each single cell, where the principal component information of different types of cells is different; a target acquisition module that acquires metabolic signature targets of the same type of cells according to the principal component information; The cell segmentation and typing apparatus of claim 16 , further comprising: a lesion determination module for determining a lesion degree of the target object based on metabolic feature targets.

20. 20. The cell segmentation and typing device of claim 19, wherein the lesion determination module further inputs the number of different types of cells and the metabolic feature targets of the same type of cells into a pre-trained machine learning classification model to determine the lesion degree of the target object.

21. 16. The cell segmentation and typing apparatus of claim 15, wherein the segmentation module further performs single-cell image segmentation on the metabolic image of the at least one cell by a neural network based on transfer learning to obtain metabolic images of the plurality of single cells.

22. The segmentation module a first segmentation module that performs a first single-cell image segmentation on the metabolic image of the at least one cell using a neural network based on transfer learning; and a second segmentation module that performs a second segmentation on an image obtained by a first single-cell segmentation using a watershed segmentation method or a flooding segmentation method, thereby obtaining metabolic images of the plurality of single cells.

23. The cell segmentation and typing apparatus of claim 15 , wherein the image feature spectrum of the single cell further comprises morphological features of the cell.

24. 24. The cell segmentation and typing apparatus of claim 23, wherein the morphological features of the cells include at least one of cell area, sphericity of cell shape, circularity of cell boundary, cell center, cell center eccentricity, equivalent diameter, cell circumference, major axis length, minor axis length, major axis / minor axis ratio, and major axis / minor axis rotation angle.

25. The cell segmentation and typing device of claim 23, wherein the metabolic features of the cell include at least one of the following: lipid content, lipid concentration, protein content, protein concentration, deoxyribonucleic acid concentration, lipid / protein content ratio, lipid / protein concentration ratio, lipid / deoxyribonucleic acid concentration ratio, number of lipid droplets, lipid droplet area, ratio of lipid droplet area to total cell area, lipid / protein concentration ratio within the lipid droplet range, lipid component / protein component area ratio, lipid component / deoxyribonucleic acid component area ratio, ratio of lipid components to total cell area, ratio of protein components to total cell area, and lipid / protein concentration ratio within the lipid component range.

26. The cell segmentation and typing device of claim 15, wherein the spectral combination module further arranges the image feature spectra of single cells corresponding to each of the metabolic images of the plurality of single cells in a predetermined order to obtain the image feature spectrum of the target object.

27. The cell segmentation and typing apparatus according to any one of claims 15 to 26, wherein the metabolic image of the cell is an image based on Raman imaging.

28. The cell segmentation and typing apparatus of any one of claims 15 to 26, wherein the cell types include cancer cells, immune cells, lymphocytes, mesothelial cells, epithelial cells, blood cells or granulocytes.

29. a processor; a memory in which computer-executable instructions are stored; A machine learning based cell segmentation and typing instrument, wherein the computer executable instructions, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 12.

30. A computer-readable recording medium having computer-executable instructions stored thereon, A computer-readable medium which, when executed by a processor, causes the processor to perform the method of any one of claims 1 to 12.

Images