Multi-immunohistocyte image segmentation method

Abstract

The invention discloses a multi-immunohistocyte image segmentation method, and relates to the technical field of image processing, and the method comprises the steps: obtaining a to-be-segmented multi-immunohistocyte image, and carrying out the image focusing processing of the multi-immunohistocyte image, so as to obtain a target multi-immunohistocyte image containing a cell internal region and a background region; performing uniform sampling processing on the target multi-immune tissue cell image to form a prompt point set; carrying out mapping processing on the prompt point set through a pre-training cell image segmentation model, and carrying out feature extraction on the multi-immune tissue cell image to obtain a prompt embedding vector and an image embedding vector; and performing mask processing of global semantic features and local detail features on the prompt embedding vector and the image embedding vector, generating a segmentation mask corresponding to each prompt point, integrating the segmentation masks corresponding to the prompt points, and outputting a cell image segmentation result of the multiple immune tissue cell images. And accurate identification of the multi-scale cells is realized.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for segmenting multiple immune tissue cell images. Background Technology

[0002] Multiplex immunoimaging, through in situ co-labeling of multiple targets, enables simultaneous detection of multiple proteins while preserving the complete morphology and spatial structure of tissues, providing crucial spatial information for analyzing the heterogeneity of the tumor microenvironment. The accuracy of cell segmentation in cell images directly affects tumor cell phenotypic analysis, intercellular interactions, and tumor microenvironment characterization. While existing cell segmentation methods can segment cells in tissue images, their accuracy and computational efficiency are low, thus impacting the accuracy and reliability of downstream analyses.

[0003] Traditional cell segmentation algorithms, such as the Watershed Algorithm (WTS) and semi-automated tools (CellProfiler), have inherent limitations in cell segmentation of multi-tissue images. WTS, relying on gradient thresholds, is susceptible to noise interference (such as imaging artifacts) and struggles to adapt to the morphological heterogeneity of tumor cells. While CellProfiler supports manual parameter tuning, its static parameter settings are incompatible with the dynamically changing cell density, nucleocytoplasmic ratio, and other characteristics of the tumor microenvironment. These limitations have directly driven the iterative development of deep learning models in this field. For example, StarDist optimizes dense cell nucleus segmentation through star-convex geometry modeling, solving WTS's sensitivity to morphological variations. Cellpose, through vector gradient representation and training with diverse data, overcomes the bottlenecks of traditional algorithms in generalization and complex morphology handling. Mesmer utilizes millions of labeled data points to train a network to capture complex cell boundaries, compensating for the shortcomings of manual parameter design. However, while these deep learning methods improve cell segmentation accuracy, they are limited by complex architecture design, highly dependent on labeled data, and lack generalization in scenarios with blurred boundaries or overlapping cells.

[0004] In summary, how to develop accurate cell segmentation with low annotation costs, while overcoming multi-scale cell recognition and boundary adhesion ambiguity in dense tissue images, is a technical problem that needs to be solved in this field. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a multi-scale immune tissue cell image segmentation method that can achieve accurate cell segmentation with low annotation costs, while overcoming the challenges of multi-scale cell recognition and boundary blurring in dense tissue images. The specific solution is as follows:

[0006] In a first aspect, this application discloses a method for segmenting multiple immune tissue cell images, including:

[0007] Acquire multiple immune tissue cell images to be segmented, and perform image focusing processing on the multiple immune tissue cell images to obtain target multiple immune tissue cell images containing cell interior regions and background regions;

[0008] The target multiple immune tissue cell image is uniformly sampled according to a preset grid size to form a cue point set;

[0009] The multiple immune tissue cell images and the cue point set are input into a pre-trained cell image segmentation model so that the pre-trained cell image segmentation model can perform mapping processing on the cue point set and feature extraction on the multiple immune tissue cell images to obtain cue embedding vectors and image embedding vectors; wherein, the image embedding vectors are used to characterize the spatial and scale features of cells in the image;

[0010] The pre-trained cell image segmentation model performs masking processing on the cue embedding vector and the image embedding vector using global semantic features and local detail features to generate a segmentation mask corresponding to each cue point; the segmentation mask is used to represent the boundary contour of the cell where the cue point is located.

[0011] The pre-trained cell image segmentation model integrates the segmentation masks corresponding to all cue points to output the cell image segmentation results of the multiple immune tissue cell images.

[0012] Optionally, the step of uniformly sampling the target multiplex immune tissue cell image according to a preset grid size to form a cue point set includes:

[0013] The intracellular and background regions of the target multiple immune tissue cell image are uniformly sampled according to a 32×32 grid size to obtain cue points carrying corresponding foreground point labels or background point labels, and a cue point set is formed based on all the cue points.

[0014] The background region is an image region whose texture entropy value is less than a preset threshold.

[0015] Optionally, feature extraction is performed on the multiple immune tissue cell images to obtain image embedding vectors, including:

[0016] The image encoder of the pre-trained cell image segmentation model extracts features from the multiple immune tissue cell images to obtain a first image feature vector, a second image feature vector, and a third image feature vector.

[0017] The first image feature vector, the second image feature vector, and the third image feature vector are fused to obtain an image embedding vector.

[0018] Optionally, the image encoder is generated based on the MoE-Conv module, and the image encoder contains at least three convolutional expert layers;

[0019] Accordingly, the image encoder of the pre-trained cell image segmentation model extracts features from the multiple immune tissue cell images to obtain a first image feature vector, a second image feature vector, and a third image feature vector, including:

[0020] The small cell features of the multiple immune tissue cell images are extracted using a first convolutional expert layer with a 3×3 convolutional kernel to obtain a first image feature vector;

[0021] The cell boundary features of the multiple immune tissue cell images are extracted using a second convolutional expert layer with a 5×5 convolutional kernel to obtain a second image feature vector.

[0022] The large cell features of the multiple immune tissue cell images are extracted using a third convolutional expert layer with a 7×7 convolutional kernel to obtain a third image feature vector.

[0023] Optionally, the step of performing global semantic feature and local detail feature masking on the cue embedding vector and the image embedding vector using the pre-trained cell image segmentation model to generate a segmentation mask corresponding to each cue point includes:

[0024] The original SAM decoder and the newly added decoder of the pre-trained cell image segmentation model are used to perform masking processing on the cue embedding vector and the image embedding vector for global semantic features and local detail features, generating a segmentation mask corresponding to each cue point.

[0025] Optionally, the step of performing global semantic feature and local detail feature masking on the cue embedding vector and the image embedding vector using the original SAM decoder and the newly added decoder of the pre-trained cell image segmentation model to generate a segmentation mask corresponding to each cue point includes:

[0026] The original SAM decoder decodes the global semantic features in the image embedding vector and the cue embedding vector to generate an initial segmentation mask corresponding to the cue point by modeling the global correlation of the overall morphology and spatial distribution of the cell. The initial segmentation mask is used to characterize the cell outline and overall spatial position of the cell.

[0027] The newly added decoder decodes the local detail features in the image embedding vector and the cue embedding vector to capture the local topological structure of the cell boundary, and refines the boundary region of the initial segmentation mask to generate a segmentation mask with enhanced boundaries.

[0028] Optionally, before generating a segmentation mask for each cue point by performing global semantic feature and local detail feature masking on the cue embedding vector and the image embedding vector using the original SAM decoder and the newly added decoder of the pre-trained cell image segmentation model, the method further includes:

[0029] Input the image training set, which includes training images and corresponding target cue points, into the initial cell image segmentation model, and freeze the parameters of the current newly added decoder in the initial newly added decoder.

[0030] The first gradient information is calculated based on the difference between the current segmentation mask and the real mask output by the current original SAM decoder, and the parameters of the current original SAM decoder are updated based on the first gradient information.

[0031] Freeze the current original SAM decoder parameters, calculate the second gradient information based on the boundary difference between the current refined mask output by the initial new decoder and the real mask, update the current new decoder parameters based on the second gradient information, until the preset co-optimization condition is met, and output the pre-trained cell image segmentation model.

[0032] Optionally, before inputting the image training set, which includes training images and corresponding target cue points, into the initial cell image segmentation model, the method further includes:

[0033] Obtain raw cell images containing different tissue types and carrying corresponding cell nucleus annotation tags;

[0034] The original cell images are preprocessed to obtain training images in a uniform format;

[0035] A preset focusing sampling algorithm is used to randomly extract target regions from the training image to obtain several target cue points.

[0036] Optionally, the cue point set is mapped to obtain a cue embedding vector, including:

[0037] The cue point set is mapped to a cue embedding vector; wherein the cue embedding vector is used to instruct the pre-trained cell image segmentation model to focus on the cell region at the corresponding location.

[0038] Optionally, the step of integrating the segmentation masks corresponding to all cue points through the pre-trained cell image segmentation model to output the cell image segmentation result of the multiple immune tissue cell images includes:

[0039] Redundancy is eliminated from the segmentation mask to retain the segmentation mask with the highest confidence, so as to output the cell image segmentation result of the multiple immune tissue cell images.

[0040] Secondly, this application discloses a multi-image segmentation device for immune tissue cells, comprising:

[0041] The image acquisition module is used to acquire multiple immune tissue cell images to be segmented, and to perform image focusing processing on the multiple immune tissue cell images to obtain a target multiple immune tissue cell image containing the cell interior region and the background region.

[0042] The point sampling module is used to uniformly sample the target multiple immune tissue cell image according to a preset grid size to form a prompt point set;

[0043] The feature extraction module is used to input the multiple immune tissue cell images and the cue point set into a pre-trained cell image segmentation model, so that the pre-trained cell image segmentation model can perform mapping processing on the cue point set and feature extraction on the multiple immune tissue cell images to obtain cue embedding vectors and image embedding vectors; wherein, the image embedding vectors are used to characterize the spatial and scale features of cells in the image;

[0044] The segmentation processing module is used to perform masking processing on the cue embedding vector and the image embedding vector using the pre-trained cell image segmentation model to generate a segmentation mask corresponding to each cue point; the segmentation mask is used to represent the boundary contour of the cell where the cue point is located.

[0045] The result output module is used to integrate the segmentation masks corresponding to all cue points through the pre-trained cell image segmentation model to output the cell image segmentation results of the multiple immune tissue cell images.

[0046] Thirdly, this application discloses an electronic device, including:

[0047] Memory, used to store computer programs;

[0048] A processor is configured to execute the computer program to implement the steps of the aforementioned disclosed method for segmenting multiple immune tissue cell images.

[0049] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the aforementioned disclosed method for segmenting multiple immune tissue cell images.

[0050] As can be seen, this application discloses a method for segmenting multiple immune tissue cell images, including: acquiring multiple immune tissue cell images to be segmented, and performing image focusing processing on the multiple immune tissue cell images to obtain a target multiple immune tissue cell image containing cell interior regions and background regions; uniformly sampling the target multiple immune tissue cell image according to a preset grid size to form a cue point set; inputting the multiple immune tissue cell image and the cue point set into a pre-trained cell image segmentation model, so that the pre-trained cell image segmentation model performs mapping processing on the cue point set and feature extraction on the multiple immune tissue cell image to obtain a cue embedding vector and an image embedding vector; wherein, the image embedding vector is used to characterize the spatial and scale features of cells in the image; performing global semantic feature and local detail feature masking processing on the cue embedding vector and the image embedding vector through the pre-trained cell image segmentation model to generate a segmentation mask corresponding to each cue point; the segmentation mask is used to represent the boundary contour of the cell where the cue point is located; integrating the segmentation masks corresponding to all cue points through the pre-trained cell image segmentation model to output the cell image segmentation result of the multiple immune tissue cell image. Therefore, the image embedding vectors obtained by feature extraction from images using a pre-trained model can characterize the spatial and scale features of cells, solving the problem of adapting a fixed-scale feature pyramid to cell sizes with varying sizes, and achieving accurate identification of cells at multiple scales. By collaboratively processing the cue point set and the image embedding vectors to generate a segmentation mask, and combining global semantics with local detail features, cell boundaries are effectively refined, solving the problem of mask fragmentation in dense regions and clearly distinguishing the boundaries of adhered or overlapping cells. By uniformly sampling a pre-defined grid size to form the cue point set, combined with the efficient fine-tuning strategy of the pre-trained model, high-precision segmentation can be achieved with only a small amount of labeled data, significantly reducing labeling costs. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0052] Figure 1 This is a flowchart of a method for segmenting multiple immune tissue cells images disclosed in this application;

[0053] Figure 2 This application discloses a flowchart of the training and inference process for a cell image segmentation model.

[0054] Figure 3This is a comparison image of multiple immune tissue cell image segmentation results disclosed in this application;

[0055] Figure 4 This is a schematic diagram of the structure of a multi-immunocellular image segmentation device disclosed in this application;

[0056] Figure 5 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0057] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0058] Multiplex immunoimaging, through in situ co-labeling of multiple targets, enables simultaneous detection of multiple proteins while preserving the complete morphology and spatial structure of tissues, providing crucial spatial information for analyzing the heterogeneity of the tumor microenvironment. The accuracy of cell segmentation in cell images directly affects tumor cell phenotypic analysis, intercellular interactions, and tumor microenvironment characterization. While existing cell segmentation methods can segment cells in tissue images, their accuracy and computational efficiency are low, thus impacting the accuracy and reliability of downstream analyses.

[0059] Traditional cell segmentation algorithms, such as the Watershed Time Transmission (WTS) algorithm and the semi-automated tool CellProfiler, have inherent limitations in cell segmentation of multi-tissue images. WTS, relying on gradient thresholds, is susceptible to noise interference (such as imaging artifacts) and struggles to adapt to the morphological heterogeneity of tumor cells. While CellProfiler supports manual parameter tuning, its static parameter settings are incompatible with the dynamically changing cell density, nucleocytoplasmic ratio, and other characteristics of the tumor microenvironment. These limitations have directly driven the iterative development of deep learning models in this field. For example, StarDist optimizes dense cell nucleus segmentation through star-convex geometry modeling, solving WTS's sensitivity to morphological variations. Cellpose, through vector gradient representation and training with diverse data, overcomes the bottlenecks of traditional algorithms in generalization and complex morphology handling. Mesmer utilizes millions of labeled data points to train a network to capture complex cell boundaries, compensating for the shortcomings of manual parameter design. However, while these deep learning methods improve cell segmentation accuracy, they are limited by complex architecture design, highly dependent on labeled data, and lack generalization in scenarios with blurred boundaries or overlapping cells.

[0060] To address this, the present invention provides a multi-immune tissue cell image segmentation scheme that can achieve accurate cell segmentation with low annotation costs, while overcoming the challenges of multi-scale cell recognition and boundary blurring in dense tissue images.

[0061] Reference Figure 1 As shown, this embodiment of the invention discloses a method for segmenting multiple immune tissue cell images, including:

[0062] Step S11: Obtain the multiple immune tissue cell images to be segmented, and perform image focusing processing on the multiple immune tissue cell images to obtain a target multiple immune tissue cell image containing the cell interior region and the background region.

[0063] In this embodiment, when acquiring multiplexed immunohistocellular images to be segmented, images of tissue samples captured from different imaging platforms, such as CODEX, CyCIF, and Vectra, can be collected, covering one or more tissue types, including pancreas, immune system, gastrointestinal tract, breast, and skin. These multiplexed immunohistocellular images must contain complete tissue morphology and spatial structure information to meet the cell analysis requirements under multiplexed immunohistochemical imaging with multi-target in situ co-labeling technology. Furthermore, while the acquired multiplexed immunohistocellular images may have imaging artifacts and uneven brightness, they should clearly present cell morphology, distribution, and boundary adhesion characteristics.

[0064] In this embodiment, after acquiring the multiple immune tissue cell image, it undergoes preprocessing and focusing. Specifically, pixel normalization is adjusted for the image and corresponding mask, such as converting the image to unit8 format, normalizing it to the range of 0 to 255, limiting brightness and contrast to 0.1, and randomizing the scale, for example, scaling and cropping ratios ranging from 0.3 to 1.0, cropping aspect ratios ranging from 0.75 to 1.33, and rotation ranging from -0.5 to 0.5, etc. After the above image preprocessing operations, an image focusing algorithm is further used to process the preprocessed multiple immune tissue cell image to distinguish the intracellular and background regions in the multiple immune tissue cell image, thereby obtaining the target multiple immune tissue cell image.

[0065] Step S12: Perform uniform sampling processing on the target multiple immune tissue cell image according to the preset grid size to form a prompt point set.

[0066] In this embodiment, the intracellular and background regions of the target multiplex immune tissue cell image are uniformly sampled according to a 32×32 grid size to obtain cue points carrying corresponding foreground or background point labels, forming a cue point set based on all such cue points; wherein, the background region is an image region with a texture entropy value less than a preset threshold. It is understood that, when processing the target multiplex immune tissue cell image, an optimized grid sampling strategy is employed to generate a high-quality cue point set to guide subsequent segmentation. Specifically, this strategy uniformly samples the intracellular and background regions of the image according to a 32×32 grid size to generate cue points with clearly defined labels.

[0067] Specifically, the preprocessed 512×512 pixel image is divided into regions. Intracellular regions refer to areas containing complete or partial cell bodies, typically characterized by complex textures and rich colors; background regions refer to image areas with texture entropy values ​​less than a preset threshold (e.g., 0.5), characterized by simple, uniform textures and generally lacking cellular structures. Texture entropy is quantified by calculating the information entropy of local region pixel grayscale values. After determining the region type, uniform sampling is performed using a 32×32 grid. Each grid point serves as a candidate cue point, assigned a foreground label (intracellular) or a background label based on its region type. This uniformly distributed cue point coverage ensures the model considers all regions where cells may exist.

[0068] It is important to note that this grid size is the optimal parameter verified through experiments. For a standard input image of 512×512 pixels, a 32×32 grid (approximately 16×16 pixels per grid) ensures sufficient spatial resolution to capture cell details while controlling computational complexity. Compared to random sampling, this uniform grid sampling strategy can more systematically cover the image space, avoiding the omission of small cells or edge regions. The final generated cue point set serves as the input to the model, guiding the pre-trained cell image segmentation model to focus on cell-related regions, significantly improving segmentation efficiency and accuracy. In practical applications, this strategy effectively reduces the cost of manual annotation while maintaining the model's generalization ability, and is particularly suitable for scenarios with dense cells and blurred boundaries in multiple immune tissue images.

[0069] Step S13: Input the multiple immune tissue cell images and the cue point set into the pre-trained cell image segmentation model so that the pre-trained cell image segmentation model can perform mapping processing on the cue point set and feature extraction on the multiple immune tissue cell images to obtain cue embedding vectors and image embedding vectors; wherein, the image embedding vectors are used to characterize the spatial and scale features of cells in the image.

[0070] In this embodiment, the cue point set is mapped to a cue embedding vector; wherein, the cue embedding vector is used to indicate the cell region at the corresponding location that the pre-trained cell image segmentation model focuses on. It is understood that the two-dimensional coordinates of each cue point are converted into a high-dimensional vector through position encoding, for example, by encoding the coordinates using sine and cosine functions to obtain a coordinate encoded vector. Then, a corresponding label embedding vector is generated for the label of each cue point; for example, a learnable lookup table is used to map the binary labels to label embedding vectors. Finally, the coordinate encoded vector and the label embedding vector are concatenated or added to form the final cue embedding vector.

[0071] This approach transforms the discrete cue point set into a feature representation that the model can understand, guiding the pre-trained cell image segmentation model to focus on cell regions at specific locations, thereby improving segmentation accuracy. In multiple immune tissue images, cells are densely distributed and morphologically diverse. Cue embedding vectors explicitly indicate which regions the model should focus on (e.g., cell interiors or boundaries), reducing misclassification of irrelevant background. The label information of the cue points (foreground / background points) is transformed into semantic vectors and deeply fused with image features. This avoids indiscriminate processing of the entire image and reduces computational complexity through sparse cue points. Furthermore, this mapping mechanism enables the model to efficiently locate cell regions using a small number of cue points (e.g., a 32×32 grid), showing a significant advantage, especially when dealing with cells with blurred or overlapping boundaries.

[0072] In this embodiment, the image encoder of the pre-trained cell image segmentation model extracts features from the multiple immune tissue cell images to obtain a first image feature vector, a second image feature vector, and a third image feature vector; the first image feature vector, the second image feature vector, and the third image feature vector are then fused to obtain an image embedding vector.

[0073] Specifically, the image encoder is generated based on the MoE-Conv (Mixture of Experts Convolution) module, and the image encoder contains at least three convolutional expert layers. Correspondingly, the image encoder, through the pre-trained cell image segmentation model, extracts features from the multiple immune tissue cell image to obtain a first image feature vector, a second image feature vector, and a third image feature vector. This includes: extracting features of small cells from the multiple immune tissue cell image using a first convolutional expert layer with a 3×3 convolutional kernel to obtain the first image feature vector; extracting features of cell boundary features from the multiple immune tissue cell image using a second convolutional expert layer with a 5×5 convolutional kernel to obtain the second image feature vector; and extracting features of large cells from the multiple immune tissue cell image using a third convolutional expert layer with a 7×7 convolutional kernel to obtain the third image feature vector. It can be understood that the 512×512 pixel multiple immune tissue cell image after resolution adjustment is divided into multiple non-overlapping or partially overlapping image blocks. These image patches serve as the basic input units for the MoE-Conv module. Each patch contains local cellular information, such as partial cell morphology and local texture features. The initial convolutional layers in the module perform preliminary convolution operations on each patch to extract the most basic low-level features, such as edges and corners, providing a foundational feature representation for further processing by expert submodules. Based on the initially extracted features, the router mechanism in the MoE-Conv module begins to function. The router calculates an expert probability distribution for each patch through a series of learnable parameters and computational processes. Specifically, the router inputs the feature vector of each patch into its internal fully connected layers and other structures for transformation calculations, resulting in an output vector with the same dimension as the number of experts. Each element in the vector represents the probability that the image patch is assigned to the corresponding expert submodule. Typically, strategies are employed, such as selecting the top K experts with the highest probabilities (commonly K=1 or K=2) to actually process the image patch, thereby achieving sparse activation and avoiding unnecessary computational overhead. For example, if there are 8 expert submodules, after calculation by the router, a certain image patch may be assigned to the 3rd and 5th expert submodules for subsequent processing.

[0074] The selected expert submodules perform further deep convolution processing and feature refinement on the assigned image patch features. Different expert submodules, due to their independent parameters, can learn and capture different aspects of the features in the image patch. Some experts may excel at extracting the textural details of cells, while others are better at extracting the morphological contours of cells. The features processed by each expert submodule are then fused using methods such as weighted summation. The weights are derived from the probability distribution calculated by the router; the higher the probability of an expert submodule, the greater the weight of its processed features in the fusion result. For example, if an image patch is assigned to experts 3 and 5, and the router calculates the probability of expert 3 as 0.6 and expert 5 as 0.4, then expert 3's feature contribution will be greater when fusing the features processed by both. The image patch features processed and fused by the expert submodules are then globally integrated at a higher level in the MoE-Conv module. Through global pooling operations (such as average pooling, max pooling, etc.) or further transformations in fully connected layers, the features of all image patches are integrated into a single feature vector, which is the final generated image embedding representation. The image embedding vector contains multi-scale features of cells (from low-level edge features extracted initially to high-level complex features refined by expert submodules) and local topological features (reflecting the spatial relationships and organization between image patches). It can comprehensively and efficiently characterize the key information of cells in the original multiple immune tissue cell images, providing rich and valuable input information for other parts of the subsequent cell image segmentation model (such as mask decoders).

[0075] Step S14: The pre-trained cell image segmentation model is used to perform masking processing on the cue embedding vector and the image embedding vector using global semantic features and local detail features to generate a segmentation mask corresponding to each cue point; the segmentation mask is used to represent the boundary contour of the cell where the cue point is located.

[0076] In this embodiment, the pre-trained cell image segmentation model uses its original SAM (Segment Anything Model) decoder and a new decoder to perform masking processing on the cue embedding vector and the image embedding vector, combining global semantic features and local detail features, to generate a segmentation mask corresponding to each cue point. Specifically, the original SAM decoder decodes the global semantic features in the image embedding vector and the cue embedding vector to generate an initial segmentation mask corresponding to the cue point by modeling the global correlation of the overall cell morphology and spatial distribution. This initial segmentation mask represents the cell contour and overall spatial position. The new decoder decodes the local detail features in the image embedding vector and the cue embedding vector to capture the local topological structure of the cell boundary and refine the boundary region of the initial segmentation mask, generating a segmentation mask with enhanced boundaries. It is understood that the pre-trained cell image segmentation model employs a dual-decoder structure. The original SAM decoder retains the decoder architecture from the SAM model, excelling at processing global semantic information and responsible for generating the initial cell contour. The new decoder is specifically designed for cell boundary refinement, improving boundary segmentation accuracy by capturing local topological features. After feature separation, the image embedding vector is input into two decoders to extract global semantic features and local detail features, respectively. Global semantic features are derived from deep features of the image encoder (such as high-level outputs of ViT (Vision Transformer), encompassing overall cell morphology, spatial distribution, and contextual relationships. Local detail features are derived from shallow features of the image encoder (such as early convolutional layer outputs), containing high-resolution details such as edge texture and local contrast. Specifically, the processing flow of the original SAM decoder is as follows: The original SAM decoder receives the global semantic features and the cue embedding vector, and generates an initial segmentation mask through the following steps:

[0077] Cross-attention mechanism: The cue embedding vector acts as a query, interacting with the key and value of the image embedding vector to locate the cell region associated with the cue point.

[0078] Global Relationship Modeling: Through a multi-layer Transformer decoder, the spatial relationships between cells are integrated to generate an initial mask that reflects the overall outline of the cells.

[0079] Probabilistic prediction: Output the probability distribution of each pixel belonging to the target cell to form the initial segmentation mask.

[0080] The boundary refinement process for the new decoder is as follows: the new decoder receives local detail features and cue embedding vectors, and refines the boundary regions of the initial mask:

[0081] Boundary localization: Extract local features of the boundary region (e.g., boundary expansion of 3-5 pixels) based on the initial mask.

[0082] Topology analysis: Analyze the local texture and curvature changes of the boundary region using convolutional neural networks (such as variants of U-Net) to identify adhesion points and separation boundaries between cells.

[0083] Mask refinement: Generate boundary enhancement masks by updating the initial mask through a fusion function (such as weighted averaging).

[0084] In this embodiment, before generating a segmentation mask for each cue point by performing global semantic feature and local detail feature masking on the cue embedding vector and the image embedding vector using the original SAM decoder and the newly added decoder of the pre-trained cell image segmentation model, the method further includes: inputting an image training set containing training images and corresponding target cue points into the initial cell image segmentation model, and freezing the current newly added decoder parameters of the initial newly added decoder; calculating first gradient information based on the difference between the current segmentation mask output by the current original SAM decoder and the real mask, and updating the current original SAM decoder parameters based on the first gradient information; freezing the current original SAM decoder parameters, calculating second gradient information based on the boundary difference between the current refined mask output by the initial newly added decoder and the real mask, and updating the current newly added decoder parameters based on the second gradient information, until a preset collaborative optimization condition is met, so as to output the pre-trained cell image segmentation model. It can be understood that the dual decoders achieve collaborative optimization through alternating gradient updates. Specifically, the optimization conditions and steps of the original decoder are: fixing the newly added decoder, updating the original decoder parameters based on the complete mask loss, and strengthening global semantic understanding. The optimization conditions and steps for the new decoder are as follows: The original decoder is fixed, and the loss is calculated only for the boundary regions to improve local detail processing capabilities. Through alternating optimization, the two decoders gradually become complementary, preserving the accuracy of the overall cell morphology while enhancing the precision of boundary segmentation.

[0085] In this embodiment, before inputting the image training set, which includes training images and corresponding target cue points, into the initial cell image segmentation model, the method further includes: acquiring original cell images containing different tissue types and carrying corresponding cell nucleus annotation labels; performing image preprocessing on each of the original cell images to obtain training images in a unified format; and randomly extracting target regions from the training images using a preset focusing sampling algorithm to obtain several target cue points. It is understood that the preprocessing of the training images before training the initial cell image segmentation model specifically involves collecting cell image data from different imaging platforms (such as CODEX, CyCIF, and Vectra) and tissue types (including pancreatic, immune, gastrointestinal, breast, and skin tissues). Specifically, this embodiment uses cell nucleus annotations from 14 of the 19 TissueNet 1.1 subtypes. Each subtype contains more than 10 original cell images. Subsequently, the original cell images and corresponding masks are pixel-normalized. Specifically, the original cell images are converted to unit8 format, normalized to the range of 0 to 255, and the brightness and contrast are limited to 0.1. The size of the original cell images is also randomly scaled, for example, the scaling and cropping ratio ranges from 0.3 to 1.0, the cropping aspect ratio ranges from 0.75 to 1.33, and the rotation ranges from -0.5 to 0.5, etc., to obtain training images in a unified format. Then, a focused sampling algorithm is used to limit the sampling of information-rich regions to the first 20% of random points during the current training phase, obtaining target cue points to address the problem of capturing features in dense regions. Meanwhile, in multiple immune tissue images, the proportion of effective cell regions is usually <30% (e.g., tumor microenvironment). If random global sampling is used, 70% of the sampling points will fall into invalid background areas, leading to model learning bias and wasted annotation resources in non-critical areas. Selecting the first 20% of focused sampling forces the model to focus on morphologically significant regions (cell nuclei / boundaries), thus avoiding the aforementioned problems of model learning bias and wasted annotation resources. Furthermore, the first 20% of probability points actually cover more than 90% of cell adhesion scenarios.

[0086] Step S15: Integrate the segmentation masks corresponding to all cue points using the pre-trained cell image segmentation model to output the cell image segmentation results of the multiple immune tissue cell images.

[0087] In this embodiment, redundancy is eliminated from the segmentation mask to retain the segmentation mask with the highest confidence, thereby outputting the cell image segmentation result of the multiple immune tissue cell images. It is understood that the segmentation result output by the pre-trained cell image segmentation model includes a segmentation mask and cell probability corresponding to each cue point. The segmentation mask characterizes the boundary contour of the cell containing the corresponding cue point, and the cell probability is the predicted intersection-union ratio (IU / R) probability value, representing the likelihood that the cue point is located inside a cell. Then, a non-maximum suppression algorithm is used to integrate the segmentation masks. The integration process involves eliminating redundant masks with an overlap exceeding a preset threshold and retaining the segmentation mask with the highest confidence, forming the final cell image segmentation result.

[0088] Reference Figure 2 As shown, this invention discloses a method for training and inference of a cell image segmentation model, specifically, as follows: Figure 2The training process first involves acquiring target cue points obtained under focused sampling as sample points. These sample points contain multiple immune tissue image fragments of cells and background regions. These sample points are input into a low-rank matrix of an image encoder (which employs a MoE-Conv-LoRA structure with a built-in MoE-Conv hybrid expert convolution module, embedding ViT) to adjust the image encoder parameters and enhance its ability to model cell size differences and local spatial correlations. Simultaneously, grid point information corresponding to the sample points is extracted. Subsequently, the input image resolution is reduced from 1024×1024 to 512×512 pixels to reduce memory usage, and bilinear interpolation is applied to facilitate model adaptation to the downsampling changes. The image encoder outputs shallow and deep features for subsequent feature fusion. The grid points are processed by the cue encoder to generate cue embedding vectors. Pairs of images and points are input into the image encoder to generate embedding representations. Based on the designed MoE-Conv hybrid expert convolutional module, the embedding representation is embedded into a ViT low-rank matrix to adjust the image encoder parameters. This enhances ViT's ability to model cell size differences and local spatial correlations in tissue images with only a 0.65% parameter adjustment, minimizing the domain gap between pre-training data and multiple tissue image data. Secondly, this scheme proposes a dual-decoder boundary optimization strategy. Based on HQ (High Quality)-SAM, the mask decoder is modified. The gradient of the dual-decoder optimization framework alternately updates the parameters of the original SAM and the newly added decoder, achieving collaborative optimization of global semantics (deep contextual features) and local details (shallow edge texture features) to enhance the restoration and refinement of the boundary prediction mask, resulting in high-quality mask predictions. Finally, the original output of SAM is reinterpreted as the cell probability of the predicted intersection-union ratio, indicating whether the current point is inside a cell, enabling simultaneous segmentation and cell / background classification. During the model inference stage, the automatic prompting generation mode of SAM is utilized to uniformly sample 32×32 grid points from the regions representing cell interiors and clear backgrounds obtained through a focused sampling algorithm in the input image. For each point, the model generates a mask and its associated cell probability. An optimized non-maximum suppression algorithm is used to integrate all predicted masks to generate the final predicted segmentation result for the cell image. Specifically, the predicted mask is compared with the ground truth mask, and the error is calculated using MSE Loss (mean squared error loss) and CE Loss (cross-entropy loss). Backpropagation is then used to optimize model parameters, improving segmentation accuracy and reducing annotation costs. The specific formulas for calculating the mean squared error loss and cross-entropy loss are as follows:

[0089] ;

[0090] ;

[0091] in, Indicates the mean square error loss. This represents the binary cross-entropy loss, where N is the number of samples. For the The real cell label corresponding to each sample (1 for positive, 0 for negative). For the predicted first Each sample corresponds to a cell probability under a cell label. It's important to note that for negative samples, the total loss is only calculated by... For positive samples, the final loss is and sum.

[0092] In the training process described above, the relevant training details are as follows: First, the input image resolution is reduced from 1024×1024 to 512×512 pixels to reduce memory usage; then, the MoE-Conv module is applied to the Q and V projection layers in the Transformer layers of the image encoder, mask decoder, and HQ decoder; finally, all experimental training strategies set the LoRA rank to 4 and the dropout rate to 0.1; the OneCycleLR strategy is adopted, with the maximum learning rate set to 0.003 and the peak percentage set to 0.3; the AdamW optimizer uses beta values ​​(0.9, 0.999) and weight decay to 0.01; the batch size during training is 4, the gradient accumulation factor is 8, and therefore the effective batch size is 32.

[0093] like Figure 2 As shown in the inference phase, after model training is complete, the inference operation is performed. Specifically, firstly, global sampling is performed on the multiple immune tissue cell images to obtain the corresponding cue point set. It is important to note that the sampling in the inference phase ensures that no cell instances are missed in the entire image, unlike the sampling of the first 20% of target cue points in the training phase. Then, the subsequent inference phase process is the same as the aforementioned training phase processing flow, but there is no model parameter update process. That is: the cue encoder generates a cue embedding vector; the image is input to the image encoder, and the image embedding vector (including cell space and scale features) is extracted. The cue embedding vector and the image embedding vector are input to the mask decoder. The decoder performs masking processing on both for global semantics and local detail features, generating a segmentation mask for each cue point to represent the cell boundary contour. All cue point segmentation masks are integrated, and after optimization, non-maximum suppression, and other operations, the cell segmentation result of the multiple immune tissue cell images is output, achieving accurate cell segmentation.

[0094] like Figure 3As shown, a comprehensive quantitative experiment was conducted on the TissueNet cell nucleus dataset (from TSN-1 to TSN-14 subsets) using the multiple immune tissue cell image classification method of this invention. Different training paradigms were employed for each benchmark method: SAM with zero-image fine-tuning, CellSAM with over one million cell annotations, Cellpose-cyto2 with all images on the training set, and CellSeg1 with a single image on the training set. To ensure fairness and consistency in the comparison results, our proposed method, CegSAM, used single-image training and fine-tuning, similar to CellSeg1, and adopted mAP0.5 as the evaluation metric, consistent with benchmark methods such as Cellpose. The superiority of our proposed method, CegSAM, was compared with the four benchmark methods, as shown in Table 1.

[0095] Table 1

[0096]

[0097] To evaluate the model's single-image training performance on different subsets within the same dataset and its cross-domain generalization ability across different subsets, the above two sets of mAP0.5 metrics were compared. Whether training and testing on a single image within a subset or training and testing on other subsets, the experimental results show that the proposed method outperforms the benchmark method trained on the complete dataset across all 14 subsets. This further demonstrates the importance of the proposed method in practical applications. Despite the challenges of dense multi-scale cells and blurred boundaries in multi-tissue images, the proposed method CegSAM can achieve powerful segmentation capabilities while significantly reducing the workload and cost of annotation.

[0098] To better evaluate the effectiveness of CegSAM under single-image training conditions, qualitative experiments were conducted on the TissueNet cell nucleus dataset. Tests were performed on two subsets, TSN-1 and TSN-8, representing dense and complex scenes from multiple tissue images of the breast and colorectal region, respectively. Figure 3 As shown. Among them. Figure 3 The second row shows a magnified view of the area corresponding to the rectangle in the first row; the fourth row shows a magnified view of the area corresponding to the rectangle in the third row. From the magnified view, it is clear that the CegSAM method in this paper performs better than the benchmark method in terms of boundary adhesion fuzziness and multi-scale cells. With the same inference time, the accuracy is improved to a certain extent.

[0099] In this way, based on the SAM basic model, LoRA is used to efficiently fine-tune the parameters of the image encoder and decoder for cell segmentation, which greatly reduces the cost of data annotation. The MoE-Conv hybrid expert convolution module is designed to solve the problem that the SAM architecture lacks dynamic adaptability to the large variations in cell size in tumor tissue due to the fixed-scale feature pyramid. A dual-decoder boundary optimization strategy is proposed to solve the problem that the SAM architecture is difficult to model the local topology between cells due to the global attention mechanism, resulting in the fragmentation of the prediction mask of the overlapping and adhesion boundary between cells in dense areas.

[0100] As can be seen, this application discloses a method for segmenting multiple immune tissue cell images, including: acquiring multiple immune tissue cell images to be segmented, and performing image focusing processing on the multiple immune tissue cell images to obtain a target multiple immune tissue cell image containing cell interior regions and background regions; uniformly sampling the target multiple immune tissue cell image according to a preset grid size to form a cue point set; inputting the multiple immune tissue cell image and the cue point set into a pre-trained cell image segmentation model, so that the pre-trained cell image segmentation model performs mapping processing on the cue point set and feature extraction on the multiple immune tissue cell image to obtain a cue embedding vector and an image embedding vector; wherein, the image embedding vector is used to characterize the spatial and scale features of cells in the image; performing global semantic feature and local detail feature masking processing on the cue embedding vector and the image embedding vector through the pre-trained cell image segmentation model to generate a segmentation mask corresponding to each cue point; the segmentation mask is used to represent the boundary contour of the cell where the cue point is located; integrating the segmentation masks corresponding to all cue points through the pre-trained cell image segmentation model to output the cell image segmentation result of the multiple immune tissue cell image. Therefore, the image embedding vectors obtained by feature extraction from images using a pre-trained model can characterize the spatial and scale features of cells, solving the problem of adapting a fixed-scale feature pyramid to cell sizes with varying sizes, and achieving accurate identification of cells at multiple scales. By collaboratively processing the cue point set and the image embedding vectors to generate a segmentation mask, and combining global semantics with local detail features, cell boundaries are effectively refined, solving the problem of mask fragmentation in dense regions and clearly distinguishing the boundaries of adhered or overlapping cells. By uniformly sampling a pre-defined grid size to form the cue point set, combined with the efficient fine-tuning strategy of the pre-trained model, high-precision segmentation can be achieved with only a small amount of labeled data, significantly reducing labeling costs.

[0101] Reference Figure 4 As shown, this embodiment of the invention also discloses a multiplex immune tissue cell image segmentation device, comprising:

[0102] Image acquisition module 11 is used to acquire multiple immune tissue cell images to be segmented, and to perform image focusing processing on the multiple immune tissue cell images to obtain target multiple immune tissue cell images containing cell interior regions and background regions.

[0103] Point sampling module 12 is used to uniformly sample the target multiple immune tissue cell image according to a preset grid size to form a prompt point set;

[0104] The feature extraction module 13 is used to input the multiple immune tissue cell images and the cue point set into a pre-trained cell image segmentation model, so that the pre-trained cell image segmentation model can perform mapping processing on the cue point set and feature extraction on the multiple immune tissue cell images to obtain cue embedding vectors and image embedding vectors; wherein, the image embedding vectors are used to characterize the spatial and scale features of cells in the image;

[0105] The segmentation processing module 14 is used to perform masking processing on the cue embedding vector and the image embedding vector using the pre-trained cell image segmentation model to generate a segmentation mask corresponding to each cue point; the segmentation mask is used to represent the boundary contour of the cell where the cue point is located.

[0106] The result output module 15 is used to integrate the segmentation masks corresponding to all cue points through the pre-trained cell image segmentation model to output the cell image segmentation result of the multiple immune tissue cell images.

[0107] As can be seen, this application discloses the following steps: acquiring multiple immune tissue cell images to be segmented; performing image focusing processing on the multiple immune tissue cell images to obtain a target multiple immune tissue cell image containing cell interior regions and background regions; uniformly sampling the target multiple immune tissue cell image according to a preset grid size to form a cue point set; inputting the multiple immune tissue cell image and the cue point set into a pre-trained cell image segmentation model, so that the pre-trained cell image segmentation model performs mapping processing on the cue point set and feature extraction on the multiple immune tissue cell image to obtain cue embedding vectors and image embedding vectors; wherein, the image embedding vector is used to characterize the spatial and scale features of cells in the image; performing global semantic feature and local detail feature masking processing on the cue embedding vectors and the image embedding vectors through the pre-trained cell image segmentation model to generate a segmentation mask corresponding to each cue point; the segmentation mask is used to represent the boundary contour of the cell where the cue point is located; integrating the segmentation masks corresponding to all cue points through the pre-trained cell image segmentation model to output the cell image segmentation result of the multiple immune tissue cell image. Therefore, the image embedding vectors obtained by feature extraction from images using a pre-trained model can characterize the spatial and scale features of cells, solving the problem of adapting a fixed-scale feature pyramid to cell sizes with varying sizes, and achieving accurate identification of cells at multiple scales. By collaboratively processing the cue point set and the image embedding vectors to generate a segmentation mask, and combining global semantics with local detail features, cell boundaries are effectively refined, solving the problem of mask fragmentation in dense regions and clearly distinguishing the boundaries of adhered or overlapping cells. By uniformly sampling a pre-defined grid size to form the cue point set, combined with the efficient fine-tuning strategy of the pre-trained model, high-precision segmentation can be achieved with only a small amount of labeled data, significantly reducing labeling costs.

[0108] Furthermore, the embodiment of the present application also discloses an electronic device, Figure 5 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0109] Figure 5 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the multiple immune tissue cell image segmentation method disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0110] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0111] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0112] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0113] The operating system 221 manages and controls the various hardware devices and computer programs 222 on the electronic device 20 to enable the processor 21 to perform calculations and processing on the massive amounts of data 223 in the memory 22. It can be Windows Server, Netware, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the multiple immune tissue cell image segmentation method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the electronic device from external devices, as well as data collected by its own input / output interface 25.

[0114] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned disclosed method for segmenting multiple immune tissue cell images. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0115] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0116] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly in hardware, software modules executed by a processor, or a combination of both. The software module may be located in random access memory (RAM), memory, read-only memory (ROM), electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, removable disks, CD-ROMs (compact disc-read-only memory), or any other form of storage medium known in the art.

[0117] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0118] The solution provided by the present invention has been described in detail above. Specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for segmenting multiple immune tissue cell images, characterized in that, include: Acquire multiple immune tissue cell images to be segmented, and perform image focusing processing on the multiple immune tissue cell images to obtain target multiple immune tissue cell images containing cell interior regions and background regions; The target multiple immune tissue cell image is uniformly sampled according to a preset grid size to form a cue point set; The multiple immune tissue cell images and the cue point set are input into a pre-trained cell image segmentation model so that the pre-trained cell image segmentation model can perform mapping processing on the cue point set and feature extraction on the multiple immune tissue cell images to obtain cue embedding vectors and image embedding vectors; wherein, the image embedding vectors are used to characterize the spatial and scale features of cells in the image; The pre-trained cell image segmentation model performs masking processing on the cue embedding vector and the image embedding vector using global semantic features and local detail features to generate a segmentation mask corresponding to each cue point; the segmentation mask is used to represent the boundary contour of the cell where the cue point is located. The pre-trained cell image segmentation model integrates the segmentation masks corresponding to all cue points to output the cell image segmentation results of the multiple immune tissue cell images.

2. The method for segmenting multiple immune tissue cell images according to claim 1, characterized in that, The step of uniformly sampling the target multiple immune tissue cell image according to a preset grid size to form a cue point set includes: The intracellular and background regions of the target multiple immune tissue cell image are uniformly sampled according to a 32×32 grid size to obtain cue points carrying corresponding foreground point labels or background point labels, and a cue point set is formed based on all the cue points. The background region is an image region whose texture entropy value is less than a preset threshold.

3. The method for segmenting multiple immune tissue cell images according to claim 1, characterized in that, Feature extraction is performed on the multiple immune tissue cell images to obtain image embedding vectors, including: The image encoder of the pre-trained cell image segmentation model extracts features from the multiple immune tissue cell images to obtain a first image feature vector, a second image feature vector, and a third image feature vector. The first image feature vector, the second image feature vector, and the third image feature vector are fused to obtain an image embedding vector.

4. The method for segmenting multiple immune tissue cell images according to claim 3, characterized in that, The image encoder is generated based on the MoE-Conv module and contains at least three convolutional expert layers. Accordingly, the image encoder of the pre-trained cell image segmentation model extracts features from the multiple immune tissue cell images to obtain a first image feature vector, a second image feature vector, and a third image feature vector, including: The small cell features of the multiple immune tissue cell images are extracted using a first convolutional expert layer with a 3×3 convolutional kernel to obtain a first image feature vector; The cell boundary features of the multiple immune tissue cell images are extracted using a second convolutional expert layer with a 5×5 convolutional kernel to obtain a second image feature vector. The large cell features of the multiple immune tissue cell images are extracted using a third convolutional expert layer with a 7×7 convolutional kernel to obtain a third image feature vector.

5. The method for segmenting multiple immune tissue cell images according to claim 1, characterized in that, The step of performing global semantic feature and local detail feature masking on the cue embedding vector and the image embedding vector using the pre-trained cell image segmentation model to generate a segmentation mask corresponding to each cue point includes: The original SAM decoder and the newly added decoder of the pre-trained cell image segmentation model are used to perform masking processing on the cue embedding vector and the image embedding vector for global semantic features and local detail features, generating a segmentation mask corresponding to each cue point.

6. The method for segmenting multiple immune tissue cell images according to claim 5, characterized in that, The step involves performing global semantic feature and local detail feature masking on the cue embedding vector and the image embedding vector using the original SAM decoder and the newly added decoder of the pre-trained cell image segmentation model to generate a segmentation mask corresponding to each cue point, including: The original SAM decoder decodes the global semantic features in the image embedding vector and the cue embedding vector to generate an initial segmentation mask corresponding to the cue point by modeling the global correlation of the overall morphology and spatial distribution of the cell. The initial segmentation mask is used to characterize the cell outline and overall spatial position of the cell. The newly added decoder decodes the local detail features in the image embedding vector and the cue embedding vector to capture the local topological structure of the cell boundary, and refines the boundary region of the initial segmentation mask to generate a segmentation mask with enhanced boundaries.

7. The method for segmenting multiple immune tissue cell images according to claim 5, characterized in that, Before generating a segmentation mask for each cue point by performing global semantic feature and local detail feature masking on the cue embedding vector and the image embedding vector using the original SAM decoder and the newly added decoder of the pre-trained cell image segmentation model, the method further includes: Input the image training set, which includes training images and corresponding target cue points, into the initial cell image segmentation model, and freeze the parameters of the current newly added decoder in the initial newly added decoder. The first gradient information is calculated based on the difference between the current segmentation mask and the real mask output by the current original SAM decoder, and the parameters of the current original SAM decoder are updated based on the first gradient information. Freeze the current original SAM decoder parameters, calculate the second gradient information based on the boundary difference between the current refined mask output by the initial new decoder and the real mask, update the current new decoder parameters based on the second gradient information, until the preset co-optimization condition is met, and output the pre-trained cell image segmentation model.

8. The method for segmenting multiple immune tissue cell images according to claim 7, characterized in that, Before inputting the image training set, which includes training images and corresponding target cue points, into the initial cell image segmentation model, the process further includes: Obtain raw cell images containing different tissue types and carrying corresponding cell nucleus annotation tags; The original cell images are preprocessed to obtain training images in a uniform format; A preset focusing sampling algorithm is used to randomly extract target regions from the training image to obtain several target cue points.

9. The method for segmenting multiple immune tissue cell images according to claim 1, characterized in that, The cue point set is mapped to obtain a cue embedding vector, including: The cue point set is mapped to a cue embedding vector; wherein the cue embedding vector is used to instruct the pre-trained cell image segmentation model to focus on the cell region at the corresponding location.

10. The method for segmenting multiple immune tissue cell images according to any one of claims 1 to 9, characterized in that, The step of integrating the segmentation masks corresponding to all cue points through the pre-trained cell image segmentation model to output the cell image segmentation result of the multiple immune tissue cell images includes: Redundancy is eliminated from the segmentation mask to retain the segmentation mask with the highest confidence, so as to output the cell image segmentation result of the multiple immune tissue cell images.

Images