Image-based cell analysis method and device, computer equipment and storage medium

Abstract

The invention relates to an image-based cell analysis method and device, computer equipment and a storage medium, and the method comprises the steps: determining a plurality of target images based on an original microscopic image and a preset analysis region; performing cell segmentation and annotation on each target image, and determining all cells in each target image and cell types of all cells; determining an initial interaction relationship type between a first cell type and a second cell type in each target image based on all the cells in each target image and the cell types of all the cells; and determining a final interaction relationship between the first cell type and the second cell type according to the first number, the second number and the number of the target images. The quantitative data of the interaction between tissue cells can be accurately determined, so that the types of the cell interaction can be more accurately distinguished.

Description

Technical Field

[0001] The present application relates to the field of image analysis, and in particular to an image-based cell analysis method, apparatus, computer device and storage medium. Background Art

[0002] In the field of cell image analysis, when analyzing cells based on microscope images, the correlation between cells and the proportion of cell types are very important analysis parameters.

[0003] For clinical sample analysis, traditional IHC wastes samples, and IHC analysis is a qualitative analysis, and its judgment mostly relies on experience, and its analysis results will be different and inaccurate. Although multiple immunofluorescence histochemistry (mIHC) technology can well make up for the shortcomings of IHC, how to more accurately distinguish the interaction between cells is an important issue. In current technology, when using images to judge the changes of cells at different stages, the detection of PD-1 and PD-L1 cannot fully analyze the cell microenvironment and cannot accurately judge the interaction relationship between cells. Summary of the invention

[0004] Based on this, it is necessary to provide an image-based cell analysis method, device, computer equipment and storage medium to address the above technical problems.

[0005] In a first aspect, the present application provides an image-based cell analysis method, the method comprising: Determine multiple target images based on the original microscopic image and the preset analysis area; Performing cell segmentation and annotation on each of the target images to determine all cells and cell types of all cells in each of the target images; Using an imaging mass spectrometry cell analysis method, based on all cells and cell types of all cells in each of the target images, determining the initial interaction relationship type between a first cell type and a second cell type in each of the target images; the first cell type and the second cell type are any two cell types among all cell types; the interaction relationship type includes an interaction relationship and an avoidance relationship; Determining a first number of interaction relationships between the first cell type and the second cell type according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images; Determining a second number of avoidance relationships between the first cell type and the second cell type according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images; The final interaction relationship type between the first cell type and the second cell type is determined according to the first number, the second number, and the number of target images.

[0006] In one embodiment, the method further comprises: Based on all cells and cell types of all cells in each of the target images, determining the proportion of each cell type in each of the target images; Neighborhood clustering is performed on all cells and cell types of all cells in each of the target images to obtain neighborhood clustering results.

[0007] In one embodiment, determining a plurality of target images based on the original microscopic image and the preset analysis area comprises: Based on the original microscopic image and the preset analysis purpose, a plurality of preset analysis areas are determined; Based on the plurality of preset analysis areas and the original microscopic image, a plurality of target images are determined.

[0008] In one embodiment, performing cell segmentation and annotation on each of the target images to determine all cells and cell types of all cells in each of the target images comprises: Performing cell segmentation on the target image based on a deepcell-based cell segmentation algorithm to determine all cells; Based on the target image and all cells, determining the fluorescence intensity values ​​of all proteins in each cell; Cell annotation was performed based on the fluorescence intensity values ​​of all proteins in each cell and the annotation table to determine the cell type of all cells.

[0009] In one embodiment, determining the proportion of each cell type in each target image based on all cells and cell types of all cells in each target image comprises: Based on all cells and cell types of all cells in each of the target images, determining the proportion of each cell type in each of the target images; Group all target images; A t-test calculation is performed according to the proportion of each cell type in each target image to determine the significant difference p value of each cell type in different groups.

[0010] In one embodiment, performing neighborhood clustering on all cells and cell types of all cells in each of the target images to obtain neighborhood clustering results includes: Determine the spatial information of each cell in the target image according to all cells and cell types of all cells in each of the target images; Based on the kmeans algorithm and the spatial information of each cell, all cells are clustered to obtain the neighborhood clustering results.

[0011] In one embodiment, determining the spatial information of each cell in the target image according to all cells and cell types of all cells in each target image comprises: Obtaining the Euclidean distance between any two cells in the target image; According to the Euclidean distance between two cells, determine the preset number of cells closest to each cell and the corresponding cell type; The preset number of cells and the corresponding cell types are used as the spatial information of the corresponding cells.

[0012] In one embodiment, the method includes: staining the sample cells based on functional protein markers and obtaining the original microscopic image; the functional protein markers include at least one of CD45, CD45RA, CD45RO, CD3, CD4, CD8, CD25, FoxP3, CCR7, CD127, PD-1, LAG3, Tim3, ICOS, CD69, CD103, Granzyme B, CD79A, CD79B, CD19, CD20, CD21, CD138, CXCR5, CD27, CD38, IGD, IGM, CD56, CD161, CD57, and CD94.

[0013] In a second aspect, the present application also provides an image-based cell analysis device, the device comprising: A frame selection module is used to determine multiple target images based on the original microscopic image and the preset analysis area; A segmentation and annotation module, used to perform cell segmentation and annotation on each of the target images, and determine all cells and cell types of all cells in each of the target images; An acquisition module, for determining the initial interaction relationship type between a first cell type and a second cell type in each of the target images based on all cells and cell types of all cells in each of the target images by using an imaging mass spectrometry cell analysis method; the first cell type and the second cell type are any two cell types among all cell types; the interaction relationship type includes an interaction relationship and an avoidance relationship; A first calculation module, configured to determine a first number of interaction relationships between the first cell type and the second cell type according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images; A second calculation module, for determining a second number of avoidance relationships between the first cell type and the second cell type according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images; The determination module is used to determine the final interaction relationship type between the first cell type and the second cell type according to the first number, the second number and the number of target images.

[0014] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements an image-based cell analysis method when executing the computer program: Determine multiple target images based on the original microscopic image and the preset analysis area; Performing cell segmentation and annotation on each of the target images to determine all cells and cell types of all cells in each of the target images; Using an imaging mass spectrometry cell analysis method, based on all cells and cell types of all cells in each of the target images, determining the initial interaction relationship type between a first cell type and a second cell type in each of the target images; the first cell type and the second cell type are any two cell types among all cell types; the interaction relationship type includes an interaction relationship and an avoidance relationship; Determining a first number of interaction relationships between the first cell type and the second cell type according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images; Determining a second number of avoidance relationships between the first cell type and the second cell type according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images; The final interaction relationship type between the first cell type and the second cell type is determined according to the first number, the second number, and the number of target images.

[0015] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, an image-based cell analysis method is implemented: Determine multiple target images based on the original microscopic image and the preset analysis area; Performing cell segmentation and annotation on each of the target images to determine all cells and cell types of all cells in each of the target images; Using an imaging mass spectrometry cell analysis method, based on all cells and cell types of all cells in each of the target images, determining the initial interaction relationship type between a first cell type and a second cell type in each of the target images; the first cell type and the second cell type are any two cell types among all cell types; the interaction relationship type includes an interaction relationship and an avoidance relationship; Determining a first number of interaction relationships between the first cell type and the second cell type according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images; Determining a second number of avoidance relationships between the first cell type and the second cell type according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images; The final interaction relationship type between the first cell type and the second cell type is determined according to the first number, the second number, and the number of target images.

[0016] The above-mentioned image-based cell analysis method, device, computer equipment and storage medium determine multiple target images based on the original microscopic image and the preset analysis area; perform cell segmentation and annotation on each of the target images to determine all cells and the cell types of all cells in each of the target images; use the imaging mass spectrometry cell analysis method to determine the initial interaction relationship type between the first cell type and the second cell type in each of the target images based on all cells and the cell types of all cells in each of the target images; the first cell type and the second cell type are any two cell types among all cell types; the interaction relationship type includes an interaction relationship and an avoidance relationship; according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images, determine the first number of interaction relationships between the first cell type and the second cell type; according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images, determine the second number of avoidance relationships between the first cell type and the second cell type; according to the first number, the second number and the number of target images, determine the final interaction relationship type between the first cell type and the second cell type. The above-mentioned image-based cell analysis method can accurately determine the quantitative data of the interaction between tissue cells, and further distinguish the types of cell interactions more precisely. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the conventional technology, the drawings required for use in the embodiments or the conventional technology descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 is a diagram of an application environment of an image-based cell analysis method in one embodiment; Figure 2 A schematic diagram of a process of an image-based cell analysis method according to an embodiment of the present invention; Figure 3 A schematic diagram of a cell interaction heat map in a target image according to an embodiment of the present invention; Figure 4 A schematic diagram of a heat map of average number of cell interactions in a target image according to an embodiment of the present invention; Figure 5 A schematic diagram of a heat map of cell interaction arrays in a target image according to an embodiment of the present invention; Figure 6 A box plot showing the proportion of all cell types in a target image according to an embodiment of the present invention; Figure 7 A bar graph showing the proportion of all cell types in a target image according to an embodiment of the present invention; Figure 8 A bar graph showing the proportion of all cell types in target images in different groups according to an embodiment of the present invention; Fig. 9 A box plot showing the significance of each cell type grouping in the target image in different groups according to an embodiment of the present invention; Fig.10 A schematic diagram of a target image under different channels according to an embodiment of the present invention; Fig.11 A schematic diagram of a cell annotation image according to an embodiment of the present invention; Fig.12 A schematic diagram of a heat map of a cell group, Cluster, and scale according to an embodiment of the present invention; Fig.13 This is a heat map of cell enrichment of different CNs according to an embodiment of the present invention; Fig.14 A CN voronoi schematic diagram of a target image according to an embodiment of the present invention; Fig.15 This is a schematic diagram showing significant differences between groups AB of CN15 according to an embodiment of the present invention; Fig.16 A schematic diagram of an image of cells after staining according to an embodiment of the present invention; Fig.17 is a structural block diagram of an image-based cell analysis device in one embodiment of the present invention; Fig.18 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0020] Cell neighborhood analysis, cell type proportion analysis, and cell interaction relationship analysis play important roles in biological and medical research, each with unique functions and benefits.

[0021] 1. Cell Neighborhood Analysis Role: Revealing cell-cell interactions: Cell neighborhood analysis helps reveal how cells work together on tissue function and pathological processes by studying the proximity and interactions of cells in space. Understanding disease mechanisms: In disease states, cell-cell interactions may change, affecting disease progression and treatment response. Therefore, studying cell neighborhoods helps reveal disease mechanisms. Guiding treatment strategies: Understanding the spatial distribution and interactions of cells is important for developing therapies targeting specific cell types or specific locations. It helps to discover new therapeutic targets and provide new ideas and methods for disease treatment.

[0022] 2. Analysis of cell type proportion Function: Understand tissue composition: Through cell type proportion analysis, we can understand the distribution and proportion of different cell types in the tissue, thereby revealing the composition characteristics of the tissue. Evaluate disease status: In the disease state, the proportion of certain cell types may change. Therefore, cell type proportion analysis can be used to evaluate the disease state and provide a basis for disease diagnosis. Guide treatment decisions: Understanding the proportion of different cell types helps to develop more accurate treatment plans and improve treatment effects. It helps to discover new disease markers and provide possibilities for early diagnosis of diseases.

[0023] 3. Analysis of Cell Interactions Role: Revealing the signal transduction mechanism: Cell interaction relationship analysis can determine the interaction between cell membrane receptors and ligands, which is an important signal transduction pathway. By analyzing the interaction between cells, the signal transduction mechanism between cells can be revealed. Understanding biological processes: Cell interaction is an important part of maintaining normal physiological state. Through the analysis of cell interaction relationships, we can better understand the role and function of cells in biological processes. Guiding drug development: Cell interaction relationship analysis is of great significance in drug development and drug treatment. By analyzing the interaction between cells, new drug targets can be discovered, providing new directions for drug development. It helps to discover new drug targets and accelerate the process of drug development.

[0024] In summary, cell neighborhood analysis, cell type ratio analysis, and cell interaction relationship analysis play an important role and have benefits in biological and medical research. They not only help to reveal the interaction and signal transduction mechanism between cells, but also help to understand biological processes and disease mechanisms.

[0025] The image-based cell analysis method provided in the embodiments of the present application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 through a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers. The user performs the current behavior on the terminal 102, and the terminal 102 transmits the current behavior data to the server 104. The server 104 determines multiple target images based on the original microscopic image and the preset analysis area; performs cell segmentation and annotation on each of the target images to determine all cells and cell types of all cells in each of the target images; uses the imaging mass spectrometry cell analysis method to determine the initial interaction relationship type between the first cell type and the second cell type in each of the target images based on all cells and cell types of all cells in each of the target images; the first cell type and the second cell type are any two cell types among all cell types; the interaction relationship type includes an interaction relationship and an avoidance relationship; according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images, determine the first number of cells in an interaction relationship with the second cell type; according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images, determine the second number of cells in an avoidance relationship with the first cell type and the second cell type; according to the first number, the second number and the number of target images, determine the final interaction relationship type between the first cell type and the second cell type. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0026] In one embodiment, Figure 2 As shown, an image-based cell analysis method is provided. This embodiment uses the method applied to a terminal as an example. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps: Step S201, determining a plurality of target images based on an original microscopic image and a preset analysis area.

[0027] Specifically, the original microscopic image refers to the original slice image of the cell after staining with a protein marker, and the preset analysis area refers to the analysis area of ​​interest to the user, such as a B cell enrichment area, a T cell enrichment area, etc., which are not listed here one by one, and the preset analysis area can be determined according to the actual analysis purpose; the target image is the cell image that needs to be analyzed intensively, and the target image is included in the preset analysis area. A preset analysis area may include one or more target images. In some embodiments, after acquiring multiple target images, the multiple target images can be grouped based on the relevant features of the target images.

[0028] Step S202: perform cell segmentation and annotation on each of the target images to determine all cells and cell types of all cells in each of the target images.

[0029] Specifically, a cell segmentation algorithm is used to locate the position of each cell nucleus based on the fluorescence expression level fed back from the image of a specific channel, and the cell boundary is found using the cell nucleus as the cell center. Ultimately, a complete cell is identified, and all cells are identified in turn to achieve cell segmentation.

[0030] Specifically, because different types of cells contain different types of proteins, the fluorescence expression levels of different proteins are also different. Therefore, different cell types can be reflected based on different fluorescence expression levels, thereby realizing the judgment of cell types and annotating the cell types based on the judged cell types.

[0031] For example, after the cell segmentation is completed, due to the high background signal intensity of the multiple immunohistochemical fluorescence scheme, mIHC, and the background signal intensity varies to varying degrees between different protein markers and even different samples, it is difficult to determine an overall threshold to remove the background signal intensity. Therefore, the preset cell annotation table can be used to accurately annotate the cell type through the CELESTA (v0.0.0.9000) algorithm.

[0032] Step S203, using an imaging mass spectrometry cell analysis method, based on all cells in each of the target images and the cell types of all cells, determine the initial interaction relationship type between the first cell type and the second cell type in each of the target images; the first cell type and the second cell type are any two cell types among all cell types; the interaction relationship type includes an interaction relationship and an avoidance relationship.

[0033] Specifically, the imaging mass spectrometry cell analysis method includes the testInteractions method in imcRtools (v1.6.5), which can preliminarily determine the initial interaction relationship type between the first cell type and the second cell type in each of the target images. The first cell type and the second cell type are any two cell types among all cell types.

[0034] Step S204: determining a first number of interaction relationships between the first cell type and the second cell type according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images.

[0035] Specifically, because a large number of target images are obtained, the first number is the number of target images of the type of interaction relationship between the first cell type and the second cell type preliminarily determined based on the imaging mass spectrometry cell analysis method.

[0036] Step S205 : determining a second number of avoidance relationships between the first cell type and the second cell type according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images.

[0037] Specifically, because a large number of target images are obtained, the second number is the number of target images of the avoidance relationship type between the first cell type and the second cell type preliminarily determined based on the imaging mass spectrometry cell analysis method.

[0038] Step S206: determining the final interaction relationship type between the first cell type and the second cell type according to the first number, the second number, and the number of target images.

[0039] Specifically, a first ratio of the number of target images with interaction relationships to the total number of target images is calculated based on the first number; a second ratio of the number of target images with avoidance relationships to the total number of target images is calculated based on the second number; and based on the first ratio and the second ratio, the type of interaction relationship between the target cell type and different types of cells is determined.

[0040] Exemplarily, assuming that the first cell type and the second cell type are initially judged to have an interaction relationship, when the first ratio is greater than the second ratio, the final judgment result is that the first cell type and the second cell type are indeed in an interaction relationship; when the first ratio is less than the second ratio, the final judgment result is that the first cell type and the second cell type are in an avoidance relationship.

[0041] The above-mentioned image-based cell analysis method, device, computer equipment and storage medium determine multiple target images based on the original microscopic image and the preset analysis area; perform cell segmentation and annotation on each of the target images to determine all cells and the cell types of all cells in each of the target images; use the imaging mass spectrometry cell analysis method to determine the initial interaction relationship type between the first cell type and the second cell type in each of the target images based on all cells and the cell types of all cells in each of the target images; the first cell type and the second cell type are any two cell types among all cell types; the interaction relationship type includes an interaction relationship and an avoidance relationship; based on the initial interaction relationship type between the first cell type and the second cell type in each of the target images, determine the first number of interaction relationships between the first cell type and the second cell type; based on the initial interaction relationship type between the first cell type and the second cell type in each of the target images, determine the second number of avoidance relationships between the first cell type and the second cell type; based on the first number, the second number and the number of target images, determine the final interaction relationship type between the first cell type and the second cell type. The above-mentioned image-based cell analysis method can achieve a comprehensive analysis of the cell microenvironment, accurately judge the interaction relationship between cells, accurately determine the results of cell changes, and make a comprehensive prediction and evaluation of drugs.

[0042] In one embodiment, a heat map of cell interactions is also visualized and significant markers are added. Figure 3 As shown, Figure 3 is a schematic diagram of the cell interaction heat map in the target image; see Figure 4 As shown, Figure 4 is a heat map of the average number of cell interactions in the target image; see Figure 5 As shown, Figure 5 It is a schematic diagram of the heat map between cell interaction arrays in the target image.

[0043] In one embodiment, the method further includes: determining the proportion of each cell type in each target image based on all cells and cell types of all cells in each target image; performing neighborhood clustering on all cells and cell types of all cells in each target image to obtain neighborhood clustering results.

[0044] Specifically, after obtaining and summarizing the data related to all cells in the target image and the cell types of all cells, the proportion of each cell type in the target image can be determined.

[0045] Specifically, in some embodiments, the proportion of each cell type in the target images of different groups is analyzed respectively, wherein the grouping criteria may be based on the staining time, staining degree, etc., which will not be described one by one here, and finally a grouped bar chart is used to display the result. In order to show the difference between the overall proportions of different cell types, refer to Figure 6 As shown, Figure 6 It is a box plot of the proportion of all cell types in the target image. The box plot is drawn using ggplot2 and sorted according to the median size of the proportion of each cell type.

[0046] See also Figure 7 As shown, Figure 7 It is a bar graph showing the proportion of all cell types in the target image. Among all lymphocytes, the median proportion of B cells in all target images presents a larger value.

[0047] See also Figure 8 As shown, Figure 8 It is a bar graph of the proportion of all cell types in the target image in different groups. According to the grouping of the target image, the t-test was used to calculate the significant difference p value of each cell type in different groups. Statistics generally believe that when p < 0.05, there is a significant difference between the two groups. The p value of the significant difference is displayed through a box plot. Based on the above analysis results, refer to Fig. 9 As shown, Fig. 9 It is a box plot of the significance of each cell type grouping in the target image in different groups.

[0048] Neighborhood clustering is performed on all cells and their cell types in the target image, with the goal of grouping cells with similar characteristics, such as close locations and / or the same cell types. Neighborhood clustering usually combines spatial information and cell type properties, and the results of neighborhood clustering can be understood as the enrichment of cells.

[0049] In the above embodiment, accurate analysis of the proportion of each cell type in the target image and the enrichment status in the target image is achieved.

[0050] In one embodiment, determining a plurality of target images based on the original microscopic image and the preset analysis area includes: Based on the original microscopic image and the preset analysis purpose, a plurality of preset analysis areas are determined; Based on the plurality of preset analysis areas and the original microscopic image, a plurality of target images are determined.

[0051] Specifically, the preset analysis purpose should be clearly defined. The purpose may include cell morphology analysis, cell counting, cell tracking, localization of specific protein or gene expression, etc. Clarifying the purpose will help determine the focus of subsequent analysis and the information required.

[0052] It can be understood that the original microscopic image can be preprocessed. The specific processing methods may include: grayscale transformation, adjusting the brightness and contrast of the image to make the cell structure clearer; filtering, removing noise in the image, such as Gaussian filtering, median filtering, etc.; color space transformation: converting the RGB image to a color space that is more suitable for subsequent processing, such as HSI, LAB or HSV; histogram equalization: improving the contrast of the image to make the details in the image more prominent.

[0053] Specifically, the analysis purpose is preset, and multiple preset analysis areas are determined on the preprocessed image. These areas may be determined based on the following factors: morphological features: determined based on the morphology, size, shape and other characteristics of the cell; location information: if the approximate location of the target structure in the image is known, the area can be delineated based on this information; color or texture features: the target area is determined using methods such as color segmentation or texture analysis.

[0054] Specifically, when the plurality of preset analysis areas and the original microscopic image are determined, the image of the preset analysis area selected in the original image is the finally determined target image.

[0055] In the above embodiment, the corresponding target images of interest are determined according to different preset analysis purposes, so that the image analysis process is more targeted and more accurate.

[0056] In one embodiment, performing cell segmentation and annotation on each of the target images to determine all cells and cell types of all cells in each of the target images comprises: Performing cell segmentation on the target image based on a deepcell-based cell segmentation algorithm to determine all cells; Based on the target image and all cells, determining the fluorescence intensity values ​​of all proteins in each cell; Cell annotation was performed based on the fluorescence intensity values ​​of all proteins in each cell and the annotation table to determine the cell type of all cells.

[0057] Specifically, cell segmentation is performed based on the DeepCell cell segmentation algorithm. DeepCell is a tool that uses deep learning technology, especially convolutional neural networks, CNN, to perform cell image segmentation. The algorithm can automatically learn features in images to distinguish cells from backgrounds or other non-cellular structures. The steps are as follows: Input: target images, which have been preprocessed, such as noise removal, contrast enhancement, etc.; Processing: The DeepCell model receives the target image as input and extracts high-level features from the image through multiple layers of convolutional layers, activation functions, pooling layers and other structures. These features are then used to generate a segmentation mask, mask, in which each pixel is marked as belonging to a cell or background; Output: The segmented image, in which each cell is clearly marked to form an independent area.

[0058] Specifically, determine the fluorescence intensity values ​​of all proteins in each cell. In the cell image, if fluorescent labeling technology is used to label specific proteins, the fluorescence intensity of these proteins can be measured by image processing. Multi-channel image analysis: If the target image is multi-channel, for example, each channel corresponds to a different fluorescent label, then each channel needs to be analyzed separately; Fluorescence intensity calculation: For each fluorescent channel in each cell, calculate the fluorescence intensity value of all pixels in the channel. In some embodiments, background correction and normalization processing can also be performed. Background correction is to remove the background background fluorescence intensity. Normalization processing is to compare the measured fluorescence intensity with the standard color to finally determine the fluorescence difference; Result summary: Generate a list or vector containing the fluorescence intensity values ​​of all labeled proteins for each cell.

[0059] For example, see Fig.10 As shown, Fig.10 It is a schematic diagram of the target image under different channels. Use Qupath, v0.4.4, software to open the qptiff image of each sample tissue. According to the staining condition of the sample tissue, select the appropriate area on the image as the target image, and export each channel image of each area into a tiff format image.

[0060] Specifically, cell annotation is performed based on the fluorescence intensity values ​​of all proteins in each cell and the annotation table. Cell annotation is the process of classifying cells into different cell types based on their characteristics, such as protein expression patterns; constructing an annotation table: the annotation table is a database or table containing known cell types and their corresponding fluorescence intensity characteristics. These features may be based on previous experimental data or literature reports; feature matching: for each cell, its fluorescence intensity characteristics are compared with the characteristics in the annotation table to find the best matching cell type; classification decision: based on the results of feature matching, each cell is classified into the corresponding cell type. This may require the use of classification algorithms such as k-nearest neighbors, decision trees, random forests, etc., or simple threshold judgments; output annotation results: generate an annotation result containing all cells and their cell types for each target image. This result can be in the form of an image, such as labeling its type next to each cell, or in the form of a data table, such as listing the coordinates, size, fluorescence intensity value and cell type of each cell, see Fig.11 As shown, Fig.11 is a cell annotation image, see Fig.12 As shown, Fig.12 It is a schematic diagram of the heat map after the cell cluster is scaled.

[0061] In the above embodiment, each target image is systematically segmented and annotated for cells, so that the position, size, fluorescence intensity value and cell type of all cells in the image can be accurately determined.

[0062] In one embodiment, determining the proportion of each cell type in each target image based on all cells and cell types of all cells in each target image includes: Based on all cells and cell types of all cells in each of the target images, determining the proportion of each cell type in each of the target images; Group all target images; A t-test calculation is performed according to the proportion of each cell type in each target image to determine the significant difference p value of each cell type in different groups.

[0063] Specifically, determine the proportion of each cell type in each target image. The first step is to count the number of cells: First, for each target image, count the number of cells of each cell type. This is usually done after the cell segmentation and annotation steps, in which each cell is assigned a specific cell type. The second step is to calculate the proportion: Then, for each cell type, calculate its proportion of all cells in the target image. This can be achieved by dividing the number of cells of each cell type by the total number of cells in the image. In some cases, a third step of recording the results can also be included: the proportion of each cell type is recorded, which may be presented in the form of a table or chart for subsequent analysis.

[0064] Specifically, all target images are grouped and grouping criteria are determined: According to the purpose of the study, appropriate grouping criteria are determined. These criteria may be based on experimental conditions, time points, tissue types, disease states, etc., where examples of experimental conditions include treatment groups and control groups. Grouping: According to the selected criteria, all target images are assigned to different groups and the number of samples between each group is as balanced as possible.

[0065] As can be understood, the t-test calculation is performed according to the proportion of each cell type in each target image, and for each cell type, its proportion data in different groups is sorted out. Make sure that the data appears in pairs, that is, each group has a corresponding proportion value. Use the t-test to test whether there is a significant difference in the proportion of each cell type between different groups. Among them, the t-test, also known as the student t test, can be said to be a very common test method in statistical inference, which is used when the statistic follows a normal distribution but the variance is unknown. The basic assumption here is that if there is no difference in the proportion of a certain cell type between two groups, their means should be equal; if there is a difference, the means are not equal. Calculate the p-value, and the t-test will output a p-value, indicating the probability that the observed difference is accidental. Usually, if the p-value is less than the set significance level, such as 0.05, it is considered that there is a significant difference between the two groups in the proportion of the cell type. Finally, record the results, record the t-test p-value between different groups for each cell type, and whether these differences are considered significant.

[0066] It can be understood that based on the results of the t-test, it is possible to explain whether the difference in the proportion of each cell type between different groups is significant and explore the possible reasons for these differences. The t-test is a type of parametric test, which requires the data to meet certain assumptions, such as normality and homogeneity of variance. If the data does not meet these conditions, you may need to consider using non-parametric tests or other statistical methods. In addition, for comparisons between multiple groups, more complex statistical methods such as analysis of variance, ANOVA, etc. may be needed.

[0067] In the above embodiment, after collecting the proportion of each cell type in each of the target images, the difference between each group is judged, thereby achieving accurate judgment of the cell difference between each group and improving the accuracy of cell analysis and judgment.

[0068] In one embodiment, performing neighborhood clustering on all cells and cell types of all cells in each of the target images to obtain neighborhood clustering results includes: Determine the spatial information of each cell in the target image according to all cells and cell types of all cells in each of the target images; Based on the kmeans algorithm and the spatial information of each cell, all cells are clustered to obtain the neighborhood clustering results.

[0069] Specifically, the neighborhood clustering result can be understood as the enrichment of cells, that is, the clustering area of ​​different cells in the image. All cells and their cell types in each target image are subjected to neighborhood clustering to obtain clustering results based on spatial information. The k-means algorithm itself does not directly consider the concept of "neighborhood", but clusters based on distance, such as Euclidean distance. The detailed steps include: Determine the spatial information of each cell: Coordinate extraction, extract the coordinates of each cell from each target image, usually x and y coordinates, which represent the spatial position of the cell in the image.

[0070] In other embodiments, there may be other spatial features. In addition to coordinates, other features related to spatial positions may also be considered, such as the distance between cells, the relative position of cells, such as whether they are near blood vessels, whether they are at the tissue boundary, etc., but these usually require additional image analysis or annotation.

[0071] Exemplarily, in one embodiment, the buildSpatialGraph method in imcRtools is used to find the 20 cells closest to each cell, and the aggregateNeighbors method is used to count the cell type proportions of the 20 cells around each cell, and the proportions are used as the spatial information of each cell.

[0072] Specifically, in some embodiments, a preprocessing process is also included, spatial weight: In order to simulate the "neighborhood" effect, the spatial information of each cell can be weighted before clustering. For example, the weight can be adjusted according to the distance between cells so that cells with closer distances have higher similarity when clustered. Feature expansion: In addition to the spatial coordinates of the cell, the cell type information of the cell can also be added to the clustering process as one of the features. This can be achieved by encoding the cell type as a numerical value or a one-hot encoding.

[0073] Specifically, clustering is performed based on the k-means algorithm, and the k value is selected: Before applying the k-means algorithm, the number of clusters k needs to be determined. This usually needs to be determined based on the purpose of the experiment, prior knowledge, or clustering effectiveness indicators, such as the silhouette coefficient. Clustering process: All cells are clustered using the k-means algorithm. In each iteration, the algorithm calculates the distance to the cluster center based on the spatial information of each cell and, optionally, the cell type information, and reassigns the cell to the nearest cluster center. This process is repeated until the cluster center no longer changes or the predetermined number of iterations is reached.

[0074] Specifically, in one embodiment, the kmeans algorithm is used to cluster cells and construct 15 different CNs. By analyzing the proportion of different cell types in each CN, the corresponding cell enrichment can be obtained, and the cell enrichment can be displayed in the form of a heat map using the pheatmap method in pheatmap (v1.0.12).

[0075] See also Fig.13 As shown, Fig.13 is a heat map of cell enrichment of different CNs; see Fig.14 As shown, Fig.14 This is a CN voronoi diagram of the target image; in this case, B cells are mostly enriched in CN1, CN2, CN10, and CN15, and CN4 is a CN enriched with PD-1+CD8T cells. At the same time, the ggvoronoi, v0.8.6, package was used to draw the voronoi images of all CNs in each target image to clearly show the CN situation in each ROI. By analyzing the proportion of the number of cells in each CN to the total number of cells, a t-test was performed on the CN differences between different groups according to the grouping situation, and the significance of the differences between different groups in different CNs was analyzed. Box plots were used to better display them. See Fig.15 As shown, Fig.15 It is a schematic diagram showing the significant difference between the AB groups of CN15.

[0076] In the above embodiments, the common characteristics and spatial distribution patterns of cells in different clusters are accurately explained based on the clustering results, and the clustering results can be applied to subsequent biological analysis.

[0077] In one embodiment, determining the spatial information of each cell in the target image according to all cells and cell types of all cells in each target image includes: Obtaining the Euclidean distance between any two cells in the target image; According to the Euclidean distance between two cells, determine the preset number of cells closest to each cell and the corresponding cell type; The preset number of cells and the corresponding cell types are used as the spatial information of the corresponding cells.

[0078] Specifically, obtain the Euclidean distance between two cells in the target image. Image preprocessing ensures that the position of each cell, usually the x and y coordinates, has been accurately extracted from the image. This may require the use of techniques such as image segmentation, edge detection, or machine learning to identify and locate cells. Distance calculation, for each pair of cells in the image, calculate the Euclidean distance between them. Euclidean distance is a measure of the straight-line distance between two points and is applicable to points in two-dimensional space, such as the coordinates of cells on an image.

[0079] Specifically, based on the Euclidean distance between two cells, determine the preset number of cells closest to each cell and the corresponding cell type. Select a preset number, such as k, based on the size of the "neighborhood", which determines how many of the nearest cells each cell will consider to build its spatial information. For each cell in the image, select the k nearest cells as neighbor cells based on its Euclidean distance to other cells. For each selected neighbor cell, record its cell type. This usually means encoding the cell type as a numeric value or string for use in subsequent analysis.

[0080] It can be understood that the preset number of cells and the corresponding cell types are used as the spatial information of the corresponding cells, and the spatial information is constructed, and the k nearest neighbors of each cell and their corresponding cell types are combined to form the spatial information of the cell. This can be a list, an array or any other data structure, depending on the needs of subsequent analysis.

[0081] In some cases, a distance threshold can also be set to restrict the selection of neighbor cells. This helps to exclude cells that are close in distance order but are actually not biologically related, for example, cells located in different tissue regions. For images containing a large number of cells, calculating the distances between all pairs of cells can be very time-consuming. In this case, consider using more efficient data structures, such as KD trees or ball trees, to speed up the nearest neighbor search. When interpreting the spatial information of cells, it is important to consider that cell types and the distances between them may together reflect some biological pattern or relationship. This helps to gain a deeper understanding of the distribution and function of cells in tissues.

[0082] In the above embodiments, when determining the spatial information of each cell in the target image, the physical distance between the cells is combined, for example, by calculating the Euclidean distance and the cell type information, so as to specifically and effectively construct the spatial environment of each cell and accurately acquire the spatial information.

[0083] In one embodiment, a cell staining method is also included, and the specific experimental steps are as follows: In a specific cell microenvironment, some antibodies can be used to study the infiltration of T cells, B cells, regulatory T cells, Treg cells, and other lymphocytes, which helps to evaluate the potential of immunotherapy. Or in viral infections or other bacterial infections, these antibodies can help study the response of immune cells, including the activation of CD8+T cells, the expression of PD-1, etc., which helps to understand the host immune response.

[0084] For example, when intestinal tissue is used for staining, the specific experimental steps are as follows: 1. Experimental steps: Dewaxing of baked slices: bake the slices at 62℃ for 1.5 hours; dewaxing: xylene I for 10 minutes, xylene II for 10 minutes, 95% ethanol for 5 minutes, 90% ethanol for 5 minutes, 85% ethanol for 5 minutes, 80% ethanol for 5 minutes. Wash with water 3 times, 5 minutes each time. 2. Fixation: add 4% paraformaldehyde fixative to cover the tissue completely, fix for 15-20 minutes, wash with water 3 times, 2 minutes each time. 3. Repair: put the tissue slide into a staining jar filled with sufficient Tris-EDTA antigen repair solution, put it in the rice cooker before boiling, turn to keep warm for 30 minutes, and cool at room temperature. Wash with PBS 3 times, 5 minutes each time. 4. Blocking: add 3% hydrogen peroxide and incubate in the dark for 25 minutes, and wash with PBS 3 times, 5 minutes each time. 5. Blocking: Use a tissue pen to outline the tissue specimen, add blocking solution to the tissue, cover it completely, and block it at room temperature for 10 minutes. 6. Primary antibody incubation: Dilute the antibody in proportion, drain the blocking solution, add CD4 antibody to completely cover the tissue section, and incubate it at room temperature in the dark at 4° overnight. Then wash 3 times with 1X TBST, 2 minutes each time. 7. Secondary antibody incubation: Add secondary antibody, completely cover the tissue section, and incubate it at room temperature in the dark for 15 minutes. Then wash 3 times with 1X TBST, 2 minutes each time. 8. Fluorescence incubation: Dilute the fluorescence in proportion, add fluorescence 520, and incubate it at room temperature for 15 minutes. Then wash 3 times with 1X TBST, 2 minutes each time. 9. Repair: Put the tissue slide into a staining jar filled with sufficient Tris-EDTA antigen repair solution, put it in a rice cooker before boiling, and start the water for 10 minutes, then turn to keep warm for 30 minutes, and cool it at room temperature. Rinse once with 1X TBST, 2 minutes. 10. Restart steps 5-9 and stop repeating until all five targets are labeled. The labeling order is: PD-1-570, Foxp3-620, CD19-690, CD8-480.

[0085] Continue to step 8 to complete the Opal Polaris780 markings.

[0086] The kit used was Opal Polaris™ 7-Color Manual Immunohistochemistry Kit (NEL861001KT) See Table 1 for the components of the manual immunohistochemistry kit.

[0087] Table 1

[0088] 11. Blocking: Rinse once with 1X TBST for 2 minutes. Add blocking solution to the tissue, cover it completely, and block it at room temperature for 10 minutes. 12. Primary antibody incubation: dilute the antibody in proportion, drain the blocking solution, add CD3 to completely cover the tissue sections, incubate at room temperature in the dark for 1 hour or at 4° overnight. Then wash 3 times with 1X TBST, 2 minutes each time. 13. Secondary antibody incubation: Add secondary antibody, completely cover the tissue sections, incubate at room temperature in the dark for 15 minutes. Then wash 3 times with 1X TBST, 2 minutes each time. 14. TSA-DIG: Add Opal TSA-DIG working solution, diluted 1:100, to each tissue, and incubate the slides at room temperature for 10 minutes. Then wash 3 times with 1X TBST, 2 minutes each time. 15. Repair: Place the tissue slide in a staining jar with sufficient Tris-EDTA antigen repair solution, place in a rice cooker and boil for 10 minutes before boiling, then switch to keep warm for 30 minutes, and cool at room temperature. Rinse once with 1X TBST for 2 minutes. 16. Add OpalPolaris780 to the tissue and incubate at room temperature for 1 hour. Then wash 3 times with 1X TBST, 2 minutes each time. 17. Nuclear staining: Add the prepared DAPI to the tissue slide and incubate at room temperature for 5 minutes. Rinse with 1X TBST for 2 minutes, then wash with water for 2 minutes. 18. After the final staining, seal the slide, refer to Fig.16 As shown, Fig.16 It is the image after cell staining, that is, the original microscopic image.

[0089] In one embodiment, lymphocytes mainly include: T lymphocytes, B lymphocytes, NK cells; 6 protein markers for identifying lymphocytes: CD8, CD4, PD-1, Foxp3, CD19, CD3.

[0090] CD8: A cell surface molecule that is primarily expressed on CD8-positive cells. It is commonly used to locate and count CD8-positive cells in tissues, primarily cytotoxic T cells. Cytotoxic T cells play an important role in immune responses, as they are able to recognize and kill infected cells, tumor cells, or other abnormal cells. Therefore, the detection of CD8-positive cells can provide important information about immune responses and disease states. By analyzing the presence and distribution of CD8-positive cells in tissues, pathologists can assess the extent of the immune response, understand the role of immune cells in the disease process, and guide the diagnosis and treatment of the disease.

[0091] CD4: Commonly used to detect CD4-positive T lymphocytes, also known as helper T cells, T-helper cells. Helper T cells play a key role in the immune system. They help regulate and activate other immune cells, promote and regulate immune responses. By detecting CD4-positive T lymphocytes in pathological tissues, doctors can evaluate the distribution of immune cells, understand the extent of immune response, and help diagnose and evaluate various immune-related diseases, such as autoimmune diseases, infections, and tumors.

[0092] PD-1: is an immune checkpoint molecule that is usually used to mark the PD-1 receptor on the surface of T cells. PD-1 detection is mainly used to evaluate immune cells, especially activated T cells, and their role in the tumor microenvironment and immune-related diseases.

[0093] Foxp3: is a transcription factor that is usually used to mark regulatory T cells, Treg cells. By detecting Foxp3-positive cells in pathological tissues, especially Treg cells, doctors can evaluate the level of regulatory immune cells in tissues and understand the state of immune balance and immune regulation. This is of great significance for understanding the role of immune response in disease development, especially in autoimmune diseases, tumors and inflammatory diseases.

[0094] CD19: used to mark B cells, it is part of the B cell-specific antigen. In pathological histology, the detection of CD19 is often used to locate and count B cells in tissues. B cells are an important type of immune cells, mainly responsible for producing antibodies and participating in humoral immune responses. By detecting CD19-positive cells in pathological tissues, doctors can evaluate the level and distribution of B cells in tissues, understand the extent of immune response and the role of B cells in the disease process.

[0095] CD3: CD3 is often used to mark T cells, a key type of cell in the immune system that is primarily responsible for cell-mediated immune responses. By detecting CD3-positive cells in pathological tissues, doctors can evaluate the presence and distribution of T cells in tissues. T cells play an important role in immune responses, including functions such as anti-infection, anti-tumor, and regulation of immune responses. Therefore, the detection of CD3 can provide information about the active state of immune cells in tissues, which helps to understand the extent and nature of immune responses. In pathology, the detection of CD3 can provide important diagnostic and prognostic information in various disease states, such as autoimmune diseases, infectious diseases, organ transplant rejection, and tumors. The localization and analysis of CD3-positive cells helps doctors understand the role of immune cells in the disease process and guide the formulation of treatment plans.

[0096] In this embodiment, referring to Table 2, the expression of different proteins is determined by the fluorescence emitted by different channels.

[0097] Table 2

[0098] This embodiment constructs a complete set of analysis processes from selecting regions to interaction analysis, covering multiple aspects of selecting regions, ROI, cell segmentation, cell clustering, cell annotation, cell proportion analysis, neighborhood analysis, and interaction analysis.

[0099] The selection area is provided by Qupath, v0.4.4. The software selects the appropriate number of regions at the appropriate positions in each target image, hereinafter referred to as ROI, and determines the appropriate grouping information for each ROI, so as to make the analysis results more accurate. Specifically, ROI is the target image.

[0100] The deepcell algorithm is used for cell segmentation. The fluorescence expression of the DAPI channel is used to locate the position of each cell nucleus, and the cell nucleus is used as the cell center to find the cell boundary. Finally, a complete cell information is obtained. The average expression level of each protein marker in each cell is used as the expression level of the cell on each protein marker. This value will be used for subsequent cell clustering and cell type judgment.

[0101] Specifically, protein marker is also a protein marker.

[0102] Construct data information tables: Construct data information tables info.csv (see Table 3) and marker.csv (see Table 4) for bioinformatics analysis based on sample scanning conditions and protein marker conditions, as follows: Table 3

[0103] Table 4

[0104] Stores the cell information, protein expression information, grouping information, batch information, and target image area information of each target image.

[0105] After the cell segmentation is completed, due to the high background signal intensity in the multiple immunohistochemical fluorescence scheme, mIHC, and the background signal intensity varies to varying degrees between different protein markers and even different samples, it is difficult to determine an overall threshold to remove the background signal intensity. This embodiment uses the annotation table to accurately annotate the cell type through the CELESTA, v0.0.0.9000, algorithm. After verification, refer to Table 5. After testing, it is verified that the results of the classification annotation of the cell type in Table 5 are consistent with the actual image.

[0106] Table 5

[0107] Among them, in Table 5, the first column is the cell type, the second column is the custom detection code of the cell, and the third column is the cell staining effect for CD8 stain. Similarly, the following columns are the cell staining effects of CD3, CD4, CD8, Foxp3, and PD-1 stains. Among them, 1 represents that the color development can be used as a basis for judging the cell type, NA represents that the color development cannot be used as a basis for judging the cell type, which can also be understood as whether the color is developed or not, and it does not affect the judgment of the cell type; 0 represents no color development.

[0108] After the cell annotation is completed, the present invention analyzes the cell type proportion, cell neighborhood, and interaction of each ROI, wherein the ROI is also the target image.

[0109] Cell type proportion: The proportion of each cell type in each ROI was counted, and ggplot2, v3.5.0, was used to draw a bar graph to clearly show the proportion of different cell types in each ROI. At the same time, each ROI was distinguished according to the grouping situation, so as to conveniently view the differences between the cell proportions in different groups. In addition, the present invention uses a t-test to test the differences in the proportions of the same cell types in different groups, and uses the p-value to judge whether there is a significant difference between the cell proportions of different groups in a certain cell type. In statistics, we usually think that p<=0.05 represents a significant difference between groups. The analysis is intuitively displayed in the form of a box plot.

[0110] Cell neighborhood analysis: The single-cell distribution obtained by clustering cells based on the environment can be interpreted as cellular neighborhoods (CN), which can represent the site of unique local processes. The present invention uses the buildSpatialGraph method in imcRtools, v1.6.5, to find the 20 cells closest to each cell in a Knn manner, and uses the aggregateNeighbors method to count the cell type proportions of the 20 cells around each cell, and uses the proportion as the spatial information of each cell. The kmeans, k=15, algorithm is used to cluster the cells to obtain 15 different neighborhood structures, CN. By analyzing the proportions of different cell types in each neighborhood structure, CN, the corresponding cell enrichment can be obtained. The pheatmap method in pheatmap, v1.0.12, is used to display the cell enrichment in the form of a heat map. At the same time, the ggvoronoi, v0.8.6 package is used to draw all domain structures, CN, and the voronoi image in each ROI to clearly display the CN situation in each ROI. By analyzing the proportion of the number of cells in each neighborhood structure to the total number of cells, a t-test was performed on the CN differences between different groups to analyze the significance of the differences between different groups in different CNs, and a box plot was drawn for better display.

[0111] Interaction relationship analysis: referred to as interaction analysis. In this example, the testInteractions() method in imcRtools, v1.6.5, was used to determine whether the interaction / avoidance between each cell type in each CN occurred more frequently than random observation, thereby determining the type of interaction relationship between each cell, whether there was an interaction relationship, Interaction, or avoidance, Aviodance, and at the same time calculating the average number of each interaction pair around each cell and the multiple of interaction and avoidance to better display the interaction relationship between cell types, and drawing a heat map to display the above information.

[0112] In some embodiments, the method includes staining the sample cells based on functional protein markers and obtaining the original microscopic image; the functional protein markers include at least one of CD45, CD45RA, CD45RO, CD3, CD4, CD8, CD25, FoxP3, CCR7, CD127, PD-1, LAG3, Tim3, ICOS, CD69, CD103, Granzyme B, CD79A, CD79B, CD19, CD20, CD21, CD138, CXCR5, CD27, CD38, IGD, IGM, CD56, CD161, CD57, and CD94.

[0113] The protein marker panel is applied to all available fluorescent channels, including but not limited to DAPI, 480, 520, 540, 570, 620, 650, 690, 780.

[0114] Specifically, the process of staining sample cells with functional protein markers is a complex and delicate experimental step that aims to identify, locate and quantitatively analyze functional proteins in cells through specific markers. Preliminary preparation Select markers: Select appropriate functional protein markers according to the purpose of the experiment. These markers are usually antibodies or other molecules that can specifically bind to specific functional proteins, and often carry fluorophores or other detectable labels.

[0115] Sample processing: The sample cells are processed as necessary, such as washing, fixation and permeabilization. The fixation step usually uses chemical reagents such as formaldehyde or paraformaldehyde to maintain cell morphology and protein structure, and the permeabilization step helps the markers enter the cells.

[0116] Staining step: Mix the marker with the sample; Mix the marker (such as fluorescent antibody) with the sample cells to ensure that the marker can fully contact and penetrate into the cells. Incubation: Incubate the mixture of sample and marker under appropriate conditions (such as temperature and time) so that the marker can specifically bind to the target functional protein. The length of incubation time depends on the characteristics of the marker and the experimental requirements. Washing: After the incubation, wash the sample cells with a buffer (such as PBS) to remove unbound markers and background noise of non-specific binding.

[0117] Nuclear staining: If you need to observe the morphology of the nucleus at the same time, you can use dyes such as DAPI or Hoechst to stain the nucleus. Sealing: Place the stained sample cells on a slide, add an appropriate amount of sealing fluid (such as fluorescent sealing fluid), and then cover with a coverslip for sealing. This step helps protect the sample cells from the external environment and facilitates subsequent microscopic observation.

[0118] Microscopic observation and image acquisition: Microscopic observation; observe the sealed samples using a fluorescence microscope or other appropriate microscope. By adjusting the microscope's light source, filters, magnification and other parameters, the distribution and localization of the markers in the cells can be clearly observed. Image acquisition; use the microscope's built-in image capture system or external digital cameras and other devices to obtain original microscopic images. These images will be used for subsequent data analysis and result interpretation. Throughout the staining process, experimental conditions such as temperature, time and pH should be strictly controlled to ensure the accuracy and reproducibility of the experimental results. The use of high-quality markers and reagents is essential for obtaining clear staining results. Appropriate washing steps are very important for removing background noise from non-specific binding, which helps to improve the signal-to-noise ratio and resolution of the image.

[0119] Through the above steps, functional protein markers can successfully stain sample cells, thereby achieving the identification, positioning and quantitative analysis of specific functional proteins in cells.

[0120] Specifically, CD45 is an important phosphatase that is involved in regulating signal transduction of T cells and B cells. It plays a key role in the activation, proliferation and differentiation of immune cells. Different isoforms of CD45, such as CD45RA and CD45RO, can be used to distinguish T cells at different stages of maturity.

[0121] CD45RA: Mainly expressed on naive T cells. CD45RA is an isoform of CD45 and is often used to mark naive T cells. These cells have not yet encountered a specific antigen and therefore have a greater potential for antigen-specific responses.

[0122] CD45RO: Mainly expressed on memory T cells. CD45RO is another isoform of CD45 and is often used to mark memory T cells. These cells have encountered an antigen before and are able to respond quickly to encountering the same antigen again.

[0123] CD25: on activated T cells and regulatory T cells, such as Tregs. CD25 is the alpha chain of the IL-2 receptor, expressed on activated T cells and essential for T cell proliferation and survival. On Tregs, CD25 is an essential component of their function.

[0124] CCR7: Mainly expressed on naive T cells and some B cells. CCR7 is a chemokine receptor that plays an important role in the migration of naive T cells and B cells to the T cell zone of the lymph nodes of secondary lymphoid organs. It helps these cells locate to where they encounter antigens and become activated.

[0125] CD127: Mainly expressed on T cells. CD127 is the α chain of the IL-7 receptor and is essential for the proliferation and survival of T cells. Its expression level can be used to distinguish different T cell subsets, such as regulatory T cells usually express lower levels of CD127.

[0126] LAG3: Mainly expressed on activated T cells. LAG3, lymphocyte activation gene 3, is another immune checkpoint protein that is upregulated when T cells are activated and inhibits T cell function by binding to MHC class II molecules.

[0127] Tim3: Mainly expressed on activated T cells and some natural killer cells. Tim3, T cell immunoglobulin and mucin domain protein 3, is an immunoregulatory molecule whose expression is associated with T cell exhaustion, i.e., the loss of T cell function in chronic infection or tumor environments.

[0128] ICOS: Mainly expressed on activated T cells. ICOS, an inducible co-stimulatory molecule, is a member of the CD28 family and plays an important role in the activation and differentiation of T cells, especially in the functions of regulatory T cells and helper T cells.

[0129] CD69: Mainly expressed on activated T cells, B cells and natural killer cells. CD69 is a marker of cell activation, which is rapidly expressed after immune cell activation. CD69 participates in the proliferation and differentiation of immune cells.

[0130] CD103: Mainly expressed on intraepithelial lymphocytes, some peripheral blood lymphocytes and activated lymphocytes. CD103 is an αEβ7 integrin that plays an important role in the homing of intestinal immune cells and maintaining intestinal barrier function.

[0131] Granzyme B: Mainly expressed on cytotoxic lymphocytes, such as cytotoxic T cells and natural killer cells. Granzyme B is a serine protease that can induce apoptosis of target cells and is a key effector molecule for cytotoxic lymphocytes to kill target cells.

[0132] CD79A: Mainly expressed on B cells. CD79A is the signaling subunit of the B cell receptor, BCR, and is essential for the development and antigen recognition of B cells.

[0133] CD79B: Same as CD79A, mainly expressed on B cells. Together with CD79A, CD79B forms the signal transduction complex of the B cell receptor and participates in the activation and differentiation of B cells.

[0134] CD20: Mainly expressed on mature B cells. CD20 is a surface marker of B cells and is related to the growth and differentiation of B cells. It is also a target for the treatment of B-cell non-Hodgkin's lymphoma.

[0135] CD21: Mainly expressed on mature B cells. CD21, complement receptor 2, CR2, participates in the binding of B cells to complement fragments and plays an important role in B cell activation and antigen processing and presentation.

[0136] CD138: Mainly expressed on plasma cells. CD138, Syndecan-1, is a marker of plasma cells, involved in cell adhesion and signal transduction, and has an impact on the survival and function of plasma cells.

[0137] CXCR5: Mainly expressed on follicular helper T cells and B cells. CXCR5 is a chemokine receptor that plays an important role in the migration of follicular helper T cells and B cells into lymphoid follicles and participates in germinal center reactions and antibody production.

[0138] CD27: Mainly expressed on activated T cells and B cells. CD27 is a co-stimulatory molecule that plays an important role in the activation, proliferation and differentiation of T cells and B cells, especially in the formation of memory cells.

[0139] CD38: Mainly expressed on activated immune cells, such as T cells, B cells and plasma cells. CD38 is a cell surface enzyme that participates in cell signal transduction and calcium ion regulation, and has an impact on the activation and function of immune cells.

[0140] IGD: Mainly expressed on mature B cells. IGD is immunoglobulin D, a component of the B cell receptor, and although its function is not as clear as other immunoglobulin subclasses, it may play a role in the early development of B cells and certain immune responses.

[0141] IGM: Mainly expressed on mature B cells. IGM is the earliest immunoglobulin subclass produced, usually first appears in the primary immune response, and plays an important role in early anti-infection immunity.

[0142] CD56: Mainly expressed on natural killer cells. CD56 is a marker of natural killer cells, involved in cell-to-cell adhesion and signal transduction, and has an impact on the recognition and killing function of natural killer cells.

[0143] CD161: Mainly expressed on natural killer cells and some T cells. CD161 is a member of the C-type lectin family, involved in the activation of natural killer cells and T cells, and may be related to the immune response to certain viral and parasitic infections.

[0144] CD57: Mainly expressed on natural killer cells and some T cells. CD57 is a marker for natural killer cells and T cells, and is usually associated with the maturation and activation status of these cells.

[0145] CD94: Mainly expressed on natural killer cells. CD94 is a protein that forms a complex with NKG2 family members and is involved in the activation of natural killer cells, especially in the process of recognizing and killing abnormal cells such as tumor cells.

[0146] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0147] Based on the same inventive concept, the embodiment of the present application also provides an image-based cell analysis device for implementing the above-mentioned. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations of one or more image-based cell analysis device embodiments provided below can refer to the limitations of the image-based cell analysis method above, and will not be repeated here.

[0148] In one embodiment, Fig.17 As shown, an image-based cell analysis device is provided, including: a frame selection module 1710, a segmentation and annotation module 1720, an acquisition module 1730, a first calculation module 1740, a second calculation module 1750 and a determination module 1760, wherein: The frame selection module 1710 is used to determine multiple target images based on the original microscopic image and the preset analysis area.

[0149] The frame selection module 1710 is further used to determine a plurality of preset analysis areas based on the original microscopic image and the preset analysis purpose; and to determine a plurality of target images based on the plurality of preset analysis areas and the original microscopic image.

[0150] The segmentation and annotation module 1720 is used to perform cell segmentation and annotation on each of the target images, and determine all cells and cell types of all cells in each of the target images.

[0151] The segmentation and annotation module 1720 is further used to perform cell segmentation on the target image based on a deepcell cell segmentation algorithm to determine all cells; Based on the target image and all cells, determining the fluorescence intensity values ​​of all proteins in each cell; Cell annotation was performed based on the fluorescence intensity values ​​of all proteins in each cell and the annotation table to determine the cell type of all cells.

[0152] Acquisition module 1730 is used to determine the initial interaction relationship type between the first cell type and the second cell type in each target image based on all cells and cell types of all cells in each target image using an imaging mass spectrometry cell analysis method; the first cell type and the second cell type are any two cell types among all cell types; the interaction relationship type includes an interaction relationship and an avoidance relationship.

[0153] The first calculation module 1740 is used to determine a first number of interaction relationships between the first cell type and the second cell type according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images.

[0154] The second calculation module 1750 is used to determine a second number of avoidance relationships between the first cell type and the second cell type according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images.

[0155] The determination module 1760 is used to determine the final interaction relationship type between the first cell type and the second cell type according to the first number, the second number and the number of target images.

[0156] The above device also includes a proportion calculation module.

[0157] The proportion calculation module is used to determine the proportion of each cell type in each target image based on all cells and cell types of all cells in each target image; perform neighborhood clustering on all cells and cell types of all cells in each target image to obtain neighborhood clustering results.

[0158] The proportion calculation module is also used to determine the proportion of each cell type in each target image based on all cells and cell types of all cells in each target image; group all target images; perform t-test calculations based on the proportion of each cell type in each target image to determine the significant difference p value of each cell type in different groups.

[0159] The proportion calculation module is also used to determine the spatial information of each cell in the target image according to all cells and cell types of all cells in each of the target images; Based on the kmeans algorithm and the spatial information of each cell, all cells are clustered to obtain the neighborhood clustering results.

[0160] The proportion calculation module is also used to obtain the Euclidean distance between any two cells in the target image; According to the Euclidean distance between two cells, determine the preset number of cells closest to each cell and the corresponding cell type; The preset number of cells and the corresponding cell types are used as the spatial information of the corresponding cells.

[0161] Each module in the above-mentioned image-based cell analysis device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0162] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.18 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC, near-field communication, or other technologies. When the computer program is executed by the processor, an image-based cell analysis device is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0163] Those skilled in the art will understand that Fig.18 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0164] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, any image-based cell analysis method in the above embodiments is implemented, which exemplarily includes the following steps: Determine multiple target images based on the original microscopic image and the preset analysis area; Performing cell segmentation and annotation on each of the target images to determine all cells and cell types of all cells in each of the target images; Using an imaging mass spectrometry cell analysis method, based on all cells and cell types of all cells in each of the target images, determining the initial interaction relationship type between a first cell type and a second cell type in each of the target images; the first cell type and the second cell type are any two cell types among all cell types; the interaction relationship type includes an interaction relationship and an avoidance relationship; Determining a first number of interaction relationships between the first cell type and the second cell type according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images; Determining a second number of avoidance relationships between the first cell type and the second cell type according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images; The final interaction relationship type between the first cell type and the second cell type is determined according to the first number, the second number, and the number of target images.

[0165] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. The computer program is executed by a processor to perform any of the image-based cell analysis methods in the above embodiments, which exemplarily includes the following steps: Determine multiple target images based on the original microscopic image and the preset analysis area; Performing cell segmentation and annotation on each of the target images to determine all cells and cell types of all cells in each of the target images; Using an imaging mass spectrometry cell analysis method, based on all cells and cell types of all cells in each of the target images, determining the initial interaction relationship type between a first cell type and a second cell type in each of the target images; the first cell type and the second cell type are any two cell types among all cell types; the interaction relationship type includes an interaction relationship and an avoidance relationship; Determining a first number of interaction relationships between the first cell type and the second cell type according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images; Determining a second number of avoidance relationships between the first cell type and the second cell type according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images; The final interaction relationship type between the first cell type and the second cell type is determined according to the first number, the second number, and the number of target images.

[0166] It should be noted that the user information involved in this application, including but not limited to user device information, user personal information, etc., and data, including but not limited to data used for analysis, stored data, displayed data, etc., are all information and data authorized by the user or fully authorized by all parties.

[0167] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0168] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0169] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image-based cell analysis method, characterized in that: include: Determine multiple target images based on the original microscopic image and the preset analysis area; Performing cell segmentation and annotation on each of the target images to determine all cells and cell types of all cells in each of the target images; Determine the initial interaction relationship type between the first cell type and the second cell type in each of the target images based on all cells and cell types of all cells in each of the target images using an imaging mass spectrometry cell analysis method; The first cell type and the second cell type are any two cell types among all cell types; the interaction relationship types include interaction relationships and avoidance relationships; Determining a first number of interaction relationships between the first cell type and the second cell type according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images; Determining a second number of avoidance relationships between the first cell type and the second cell type according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images; The final interaction relationship type between the first cell type and the second cell type is determined according to the first number, the second number, and the number of target images.

2. The image-based cell analysis method according to claim 1, characterized in that: The method further comprises: Based on all cells and cell types of all cells in each of the target images, determining the proportion of each cell type in each of the target images; Neighborhood clustering is performed on all cells and cell types of all cells in each of the target images to obtain neighborhood clustering results.

3. The image-based cell analysis method according to claim 1, characterized in that: The determining of the plurality of target images based on the original microscopic image and the preset analysis area comprises: Based on the original microscopic image and the preset analysis purpose, a plurality of preset analysis areas are determined; Based on the plurality of preset analysis areas and the original microscopic image, a plurality of target images are determined.

4. The image-based cell analysis method according to claim 1, characterized in that: The performing cell segmentation and annotation on each of the target images to determine all cells and cell types of all cells in each of the target images comprises: Performing cell segmentation on the target image based on a deepcell-based cell segmentation algorithm to determine all cells; Based on the target image and all cells, determining the fluorescence intensity values ​​of all proteins in each cell; Cell annotation was performed based on the fluorescence intensity values ​​of all proteins in each cell and the annotation table to determine the cell type of all cells.

5. The image-based cell analysis method according to claim 2, characterized in that: Determining the proportion of each cell type in each target image based on all cells and cell types of all cells in each target image includes: Based on all cells and cell types of all cells in each of the target images, determining the proportion of each cell type in each of the target images; Group all target images; A t-test calculation is performed according to the proportion of each cell type in each target image to determine the significant difference p value of each cell type in different groups.

6. The image-based cell analysis method according to claim 2, characterized in that: The performing neighborhood clustering on all cells and cell types of all cells in each of the target images to obtain neighborhood clustering results includes: Determine the spatial information of each cell in the target image according to all cells and cell types of all cells in each of the target images; Based on the kmeans algorithm and the spatial information of each cell, all cells are clustered to obtain the neighborhood clustering results.

7. The image-based cell analysis method according to claim 6, characterized in that: Determining the spatial information of each cell in the target image according to all cells and cell types of all cells in each target image includes: Obtaining the Euclidean distance between any two cells in the target image; According to the Euclidean distance between two cells, determine the preset number of cells closest to each cell and the corresponding cell type; The preset number of cells and the corresponding cell types are used as the spatial information of the corresponding cells.

8. The image-based cell analysis method according to claim 1, characterized in that: The method comprises: The sample cells are stained based on functional protein markers, and the original microscopic image is obtained; the functional protein markers include at least one of CD45, CD45RA, CD45RO, CD3, CD4, CD8, CD25, FoxP3, CCR7, CD127, PD-1, LAG3, Tim3, ICOS, CD69, CD103, Granzyme B, CD79A, CD79B, CD19, CD20, CD21, CD138, CXCR5, CD27, CD38, IGD, IGM, CD56, CD161, CD57, and CD94; The protein marker panel is applied to all available fluorescent channels, including but not limited to DAPI, 480, 520, 540, 570, 620, 650, 690, 780.

9. An image-based cell analysis device, characterized in that: The device comprises: A frame selection module is used to determine multiple target images based on the original microscopic image and the preset analysis area; A segmentation and annotation module, used to perform cell segmentation and annotation on each of the target images, and determine all cells and cell types of all cells in each of the target images; An acquisition module, for determining the initial interaction relationship type between a first cell type and a second cell type in each of the target images based on all cells and cell types of all cells in each of the target images by using an imaging mass spectrometry cell analysis method; the first cell type and the second cell type are any two cell types among all cell types; the interaction relationship type includes an interaction relationship and an avoidance relationship; A first calculation module, configured to determine a first number of interaction relationships between the first cell type and the second cell type according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images; A second calculation module, for determining a second number of avoidance relationships between the first cell type and the second cell type according to the initial interaction relationship type between the first cell type and the second cell type in each of the target images; The determination module is used to determine the final interaction relationship type between the first cell type and the second cell type according to the first number, the second number and the number of target images.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images