A method and system for tumor tissue MPIF-HE image registration and cell segmentation

Abstract

The application discloses a kind of tumor tissue MPIF-HE image registration and cell segmentation method and system, the method includes: obtaining multiple immunofluorescence MPIF dyeing tumor tissue section image and pre-processing and multi-channel separation, obtain the single-channel MPIF image after pre-processing;Based on HE dyeing image extraction tumor infiltration front area as registration reference, carry out registration to HE dyeing image and the single-channel MPIF image after pre-processing, obtain the HE image segmentation result after registration;Based on the HE image segmentation result after registration, demarcate region of interest, and carry out segmentation and phenotype classification to the immune cell in this region of interest by pre-trained cell segmentation model, obtain cell segmentation result.The application can improve the cell segmentation precision of cell concentration and complex background.The application can be widely applied to tumor pathological image analysis technical field as a kind of tumor tissue MPIF-HE image registration and cell segmentation method and system.

Description

Technical Field

[0001] This invention relates to the field of tumor pathological image analysis technology, and in particular to a method and system for MPIF-HE image registration and cell segmentation of tumor tissue. Background Technology

[0002] Multiplex immunofluorescence (MPIF) technology can simultaneously label multiple immune cell markers (such as CD4, CD8, and B cells) in the tumor microenvironment, providing multi-dimensional data support for exploring cell-cell interaction mechanisms. However, existing MPIF-related image processing techniques have the following key limitations, such as: 1) Insufficient multi-channel separation accuracy: Crosstalk is prone to occur in the channels of 6-color MPIF staining images. Traditional threshold segmentation methods are difficult to achieve high-purity single-channel extraction, which leads to increased errors in subsequent cell identification. 2) HE-MPIF registration lacks targeting: Existing image registration techniques mostly adopt global registration strategies, which do not focus on key functional areas such as the tumor invasion front, resulting in large registration errors (usually >3 pixels), which cannot meet the needs of microscopic feature and spatial location correlation analysis; 3) Weak anti-interference ability of cell segmentation: The tumor microenvironment is dense with cells and has a complex background. Conventional segmentation algorithms (such as traditional Cellpose and U-Net) are easily affected by tissue debris and fluorescent background interference. The accuracy of immune cell segmentation is less than 85%, which makes it difficult to support accurate cell interaction analysis. Summary of the Invention

[0003] To address the aforementioned technical problems, the present invention aims to provide a method and system for MPIF-HE image registration and cell segmentation of tumor tissue, which can improve the segmentation accuracy of cells with dense cell density and complex backgrounds.

[0004] The first technical solution adopted in this invention is: a method for MPIF-HE image registration and cell segmentation of tumor tissue, comprising the following steps: Multiplex immunofluorescence (MPIF) stained tumor tissue slices were acquired, preprocessed, and separated into multiple channels to obtain preprocessed single-channel MPIF images. Based on the tumor invasion front region extracted from the HE staining image as the registration benchmark, the HE staining image and the preprocessed single-channel MPIF image were registered to obtain the registered HE image segmentation result. Based on the segmentation results of the registered HE image, the region of interest is delineated, and the immune cells in the region of interest are segmented and classified by phenotype using a pre-trained cell segmentation model to obtain the cell segmentation results. The cell segmentation results are converted into structured data and output, including immune cell segmentation masks, cell phenotype classification results, and quantitative data on TLS cell composition and spatial distribution.

[0005] Furthermore, the step of acquiring multiplex immunofluorescence (MPIF) stained tumor tissue slice images and performing preprocessing and multichannel separation to obtain preprocessed single-channel MPIF images specifically includes: Multiplex immunofluorescence (MPIF) stained tumor tissue slices were acquired and image denoising was performed to obtain denoised MPIF images; Acquire HE-stained images and perform cross-channel alignment correction on the denoised MPIF images to obtain aligned MPIF images; Based on the characteristic spectral library of fluorescent dyes, a linear spectral unmixing algorithm is used to separate the channels of the aligned and corrected MPIF image to obtain the initial single-channel MPIF image. The signal purity of the initial single-channel MPIF image is calculated and filtered according to a preset purity threshold to obtain the preprocessed single-channel MPIF image.

[0006] Furthermore, the step of acquiring the HE-stained image and performing cross-channel alignment correction processing on the denoised MPIF image to obtain the aligned and corrected MPIF image specifically includes: Tumor tissue sections were stained with hematoxylin and eosin (HE) to obtain HE-stained images; Affine transformation matrix is ​​generated by registering the cell nuclear staining features in the HE staining image with the DAPI channel in the denoised MPIF image. The matrix is ​​mapped to the other marker channels of the denoised MPIF image, and channel space alignment is performed to obtain the aligned and corrected MPIF image.

[0007] Furthermore, the step of extracting the tumor invasion front region from the HE-stained image as a registration benchmark, registering the HE-stained image with the preprocessed single-channel MPIF image, and obtaining the registered HE image segmentation result specifically includes: Based on HE-stained images, pixel segmentation is performed to obtain the final tumor solid region mask and the effective normal tissue region mask. Dynamic boundary detection was used to identify the infiltration front boundary between the final tumor solid region mask and the effective normal tissue region mask. Centered on the boundary line of the infiltration front, a strip-shaped region of a preset physical width is generated on both sides as the registration reference; Based on the registration benchmark, feature key points are extracted, and the HE-stained image is registered with the preprocessed single-channel MPIF image through a two-step strategy of rigid registration and elastic registration to obtain the registered HE image segmentation result.

[0008] Furthermore, the step of performing pixel segmentation based on HE-stained images to obtain the final tumor solid region mask and the effective normal tissue region mask specifically includes: Bilinear interpolation downsampling was performed on the HE staining image to obtain a low-resolution HE staining image; Based on the RGB color space threshold, all background regions of the low-resolution HE staining image are segmented, and the segmentation results are logically inverted to obtain a binary mask containing all effective tissue regions. Based on the RGB values ​​of tumor-specific staining markers, pixel-level thresholding is performed on the binary mask containing all effective tissue regions to obtain the initial binary mask of the tumor region. Based on the initial binary mask image of the tumor, all holes inside the mask are filled and all connected region contours are detected. The pixel area of ​​each contour is calculated, and discrete noise points with an area smaller than the first preset pixel are filtered out to obtain the final tumor solid region mask after removing holes and noise. The binary mask containing all valid tissue regions is segmented at the pixel level based on the RGB values ​​of the normal tissue marker color to obtain the initial binary mask image of the normal tissue region; Based on the initial binary mask image of the normal tissue region, the contours of all connected regions are detected, and the pixel area of ​​each contour is calculated. Small tissue fragments with an area smaller than a second preset pixel are filtered out to obtain the final effective normal tissue region mask.

[0009] Furthermore, the step of defining regions of interest based on the registered HE image segmentation results, and segmenting and classifying immune cells within these regions of interest using a pre-trained cell segmentation model to obtain cell segmentation results, specifically includes: The preprocessed single-channel MPIF image was manually delineated, and the local sub-image of the TLS region was cropped by combining the segmentation results of the registered HE image. Tumor tissue slice images based on multiplex immunofluorescence MPIF staining were used to segment cell nuclei using a pre-trained Cellpose model to obtain an initial cell nucleus mask. For the initial cell nuclear mask, its corrected intensity score on each marker channel is calculated, and phenotypic classification is performed according to the preset positive threshold and channel priority rules to obtain the cell segmentation result.

[0010] Furthermore, the step of manually delineating the preprocessed single-channel MPIF image and cropping a local sub-image of the TLS region based on the segmentation results of the registered HE image specifically includes: The TLS region is obtained by manually delineating the preprocessed single-channel MPIF image; By using distance measurement tools, the straight-line distance from the center of each TLS region to the nearest tumor infiltration edge is obtained, thus acquiring the spatial location information of the tumor body; Based on the spatial location information of the tumor, and combined with the segmentation results of the registered HE image, auxiliary cropping is performed to obtain a local sub-image of the TLS region.

[0011] The second technical solution adopted in this invention is: a system for MPIF-HE image registration and cell segmentation of tumor tissue, comprising: The first module is used to acquire multiplex immunofluorescence (MPIF) stained tumor tissue slice images and perform preprocessing and multichannel separation to obtain preprocessed single-channel MPIF images. The second module is used to extract the tumor invasion front region based on the HE staining image as the registration benchmark, register the HE staining image with the preprocessed single-channel MPIF image, and obtain the registered HE image segmentation result. The third module is used to delineate regions of interest based on the segmentation results of the registered HE image, and to segment and classify the immune cells in the regions of interest using a pre-trained cell segmentation model to obtain cell segmentation results. The fourth module is used to convert cell segmentation results into structured data and output them. The cell segmentation results include immune cell segmentation masks, cell phenotype classification results, and quantitative data on TLS cell composition and spatial distribution.

[0012] The beneficial effects of the method and system of this invention are as follows: This invention acquires tumor tissue slice images stained with multiplex immunofluorescence (MPIF) and performs preprocessing and multi-channel separation to obtain a preprocessed single-channel MPIF image, eliminating the dual interference of spatial misalignment and signal crosstalk from the source; further, based on the HE-stained image, the tumor invasion front region is extracted as a registration benchmark, and the HE-stained image and the preprocessed single-channel MPIF image are registered to obtain the registered HE image segmentation result. The "rigid + elastic" two-step registration strategy with the tumor invasion front as the core meets the accurate requirements of microscopic feature and spatial location correlation analysis; then, based on the registered HE image segmentation result, the region of interest is delineated, and the immune cells in the region of interest are segmented and phenotypically classified by a pre-trained cell segmentation model to obtain the cell segmentation result. Through region masking and algorithm parameter optimization, the segmentation problem caused by dense cells and complex backgrounds is solved; finally, the cell segmentation result is converted into structured data and output, which can improve the cell segmentation accuracy in dense cells and complex backgrounds. Attached Figure Description

[0013] Figure 1 This is a flowchart of the steps of a method for MPIF-HE image registration and cell segmentation of tumor tissue according to the present invention; Figure 2 This is a structural block diagram of a system for MPIF-HE image registration and cell segmentation of tumor tissue according to the present invention; Figure 3 This is a schematic diagram of the overall process of image registration and cell segmentation provided in a specific embodiment of the present invention; Figure 4 This is a schematic diagram of 6-color MPIF high-purity single-channel separation provided in a specific embodiment of the present invention; Figure 5 This is a schematic diagram of the MPIF-HE target registration principle provided in a specific embodiment of the present invention; Figure 6 This is a schematic diagram of precise segmentation of immune cells provided in a specific embodiment of the present invention. Detailed Implementation

[0014] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are only for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.

[0015] Reference Figure 1 This invention provides a method for MPIF-HE image registration and cell segmentation of tumor tissue, the method comprising the following steps: S100. Obtain images of tumor tissue sections stained with multiplex immunofluorescence (MPIF) and perform preprocessing and multichannel separation to obtain preprocessed single-channel MPIF images. In this embodiment, as Figure 4 As shown, firstly, the MPIF image is preprocessed for noise reduction; then, the nuclear staining features in the HE-stained image are registered with the DAPI channel in the MPIF image to generate an affine transformation matrix, which is then mapped to other marker channels in the MPIF image to achieve spatial alignment among all MPIF channels; based on the characteristic spectral library of fluorescent dyes, a linear spectral unmixing algorithm is used to separate the channels of the spatially aligned MPIF image and deduct inter-channel crosstalk signals; finally, the signal purity of each single-channel image after separation is calculated, and single-channel images with purity not lower than a preset threshold are selected for subsequent analysis.

[0016] S110. Obtain images of tumor tissue sections stained with multiplex immunofluorescence (MPIF) and perform image denoising to obtain denoised MPIF images; In this embodiment, the tumor tissue slice image stained with 6 colors MPIF is subjected to noise reduction processing (using Gaussian filtering algorithm with a filter kernel size of 3×3) to remove background noise.

[0017] S120. Obtain the HE staining image and perform cross-channel alignment correction on the denoised MPIF image to obtain the aligned MPIF image. In this embodiment, the deep staining features of cell nuclei in H&E staining are used to target and align with the nuclear staining of the DAPI channel in MPIF staining. The registration relationship between the two is established using the WSIReg tool, generating an affine transformation matrix. This affine matrix is ​​then mapped to the remaining 5 labeled channels (CD4, CD8A, CD20, CD21, PANCK) of the same MPIF slice, achieving precise alignment of all MPIF channels (ensuring that the positional deviation of cell nuclei between channels is ≤0.5 pixels), laying a spatial consistency foundation for subsequent channel separation and cell localization.

[0018] S130. Based on the characteristic spectral library of fluorescent dyes, the alignment-corrected MPIF image is separated into channels using a linear spectral unmixing algorithm to obtain the initial single-channel MPIF image. In this embodiment, based on the characteristic spectral library of six fluorescent dyes, a linear spectral demixing algorithm is used to perform channel separation on the preprocessed and aligned MPIF image and simultaneously deduct channel crosstalk signals.

[0019] S140. Calculate the signal purity of the initial single-channel MPIF image and filter it according to the preset purity threshold to obtain the preprocessed single-channel MPIF image.

[0020] In this embodiment, the signal purity of each single-channel image is calculated (the signal purity calculation formula is the sum of the signal strength of the target channel and the signal strength of the non-target channels), and single-channel images with a purity ≥ 95% are selected for subsequent analysis.

[0021] S200. Based on the tumor invasion front region extracted from the HE staining image as the registration benchmark, the HE staining image and the preprocessed single-channel MPIF image are registered to obtain the registered HE image segmentation result. In this embodiment, as Figure 5 as well as Figure 6 As shown, the tumor region and normal tissue region are segmented from the HE-stained image. The invasion front boundary line between the tumor region and the normal tissue region is extracted by dynamic boundary detection. A strip-shaped region of a specified physical width is generated on both sides of the invasion front boundary line as the center, which serves as the registration reference. Feature key points are extracted based on the registration reference, and a two-step strategy including rigid registration and elastic registration is adopted to register the HE-stained image and the MPIF image.

[0022] S210. Based on HE staining images, pixel segmentation is performed to obtain the final tumor solid region mask and the effective normal tissue region mask. In this embodiment, multi-scale preprocessing and tissue segmentation are first performed. The 10X full-scan HE image is downsampled to 1.25X (scaling factor 0.125) by bilinear interpolation. The background region is segmented based on the RGB color space threshold [255,255,255]. After logical inversion, a binary mask containing all tissue regions is obtained to provide spatial constraints. Further dual-target region extraction and optimization were performed, including: For the tumor region, an initial mask is generated based on the RGB values ​​[230,0,18] of the specific staining marker. The internal gaps are eliminated by the binary hole filling algorithm of the SciPy library. Discrete noise points are removed by combining OpenCV contour detection and area filtering (threshold 5000 pixels) to retain the main tumor entity. For normal tissue areas, tissue fragments are extracted based on the marker color [255,241,0] and then eliminated using the same contour area filter (threshold 10000 pixels).

[0023] S220. Dynamic boundary detection is used to determine the infiltration front boundary between the final tumor solid region mask and the effective normal tissue region mask. In this embodiment, the optimized tumor region is subjected to adaptive morphological dilation (the size of the structural element is dynamically correlated with the image scale factor), and a logical AND operation is performed with the normal tissue region to obtain the preliminary boundary area. The Douglas-Peucker polygon approximation algorithm is used to simplify the boundary area contour (the approximation accuracy is 0.5% of the contour perimeter), and the continuous boundary and the broken edge are distinguished by the filling area calculation, and the open boundary line representing the infiltration front is extracted.

[0024] S230. A strip-shaped region of a preset physical width is generated on both sides, centered on the boundary line of the infiltration front, as a registration reference. In this embodiment, the band region is first constructed and post-processed. Based on the 10X objective resolution (1 μm / pixel) and the preset frontal physical width (500 μm), the pixel width of the band region is calculated. With the boundary line as the center, bidirectional morphological expansion is performed towards the tumor side and the normal tissue side to generate dual sub-bands. After merging, the intersection with the tissue mask is taken. The original 10X resolution is upsampled back by nearest neighbor interpolation. After morphological closing operation and edge smoothing and contour area filtering (threshold 1000 pixels), a binarized infiltration front mask is output.

[0025] Further visualization and area calculation were performed. The infiltration front mask was overlaid semi-transparently onto the original tissue segmentation image to show the IF position and extent. Pixel-level area calculations were performed on the binarized IF mask (stored as a PNG format, foreground pixel value = 255), and converted to the actual area based on physical spatial resolution: Area( = Number of white pixels × (micrometers / pixel) 2 (2.0 μm / pixel at 5× magnification, 0.5 μm / pixel at 20× magnification), results are summarized in CSV format. Then, cross-scale alignment and TLS filtering are performed. The 10× indexed segmentation map (TLS regions are labeled with index value 8) is downsampled to 1 / 4 of its original size through nearest neighbor interpolation and aligned with the 2.5× immersion front mask resolution. The TLS mask is directly extracted based on index value = 8, and small area noise is removed by hole filling and area filtering (equivalent diameter ≥ 300 μm, corresponding to minimum area ≈ 70,686 μm²) (minimum area threshold 5200 pixels). The TLS mask and the immersion front mask are subjected to a pixel-by-pixel logical AND operation to obtain the TLS regions within the immersion front. The number of TLS is counted by connected component labeling (bwlabel) and region props analysis, and the measurement unit is calibrated to millimeters (mm) based on the 10× physical pixel size (0.25 μm / pixel). Finally, feature key point extraction is performed. Based on the infiltration front mask and internal TLS region extracted above, tissue edge contour points and TLS cell cluster center points are extracted to construct a registration reference set.

[0026] S240. Based on the registration benchmark, feature key points are extracted, and the HE-stained image is registered with the preprocessed single-channel MPIF image through a two-step strategy of rigid registration and elastic registration to obtain the registered HE image segmentation result.

[0027] S300. Based on the segmentation results of the registered HE image, the region of interest is delineated, and the immune cells in the region of interest are segmented and classified phenotyped using a pre-trained cell segmentation model to obtain the cell segmentation results. First, it should be noted that this embodiment aims to accurately identify and segment various immune cells in the tumor microenvironment from multiplex immunofluorescence (MPIF) images, laying the foundation for subsequent quantitative and spatial analysis. In particular, to quantify the cellular composition and spatial distribution characteristics of tertiary lymphoid structures (TLS) in the tumor microenvironment, this embodiment adopts a strategy combining "manual annotation + deep learning-driven multiplex immunofluorescence quantitative analysis".

[0028] In this embodiment, a local sub-image of the TLS region is cropped from the MPIF image based on manually labeled location information; image enhancement and background noise suppression are performed on the local sub-images of each marker channel; based on the DAPI channel image, a pre-trained and parameter-optimized Cellpose model is used for cell nucleus segmentation, and parameter optimization includes adjusting the model nucleus diameter and flow threshold according to the image resolution; for each segmented cell nucleus, its corrected intensity score on each marker channel is calculated; according to the preset positive threshold and channel priority rules, the cells are phenotypically classified based on the intensity score.

[0029] S310. Manually delineate the preprocessed single-channel MPIF image, and combine the segmentation results of the registered HE image to crop out a local sub-image of the TLS region. In this embodiment, region of interest (LOI) masks are generated. To accurately analyze specific microenvironment regions (such as TLS), the LIOIs are first manually labeled and located. Experienced pathologists manually review high-resolution multiplexed slides (including DAPI, CD4, CD8A, CD20, CD21, and PANCK channels) using Olympus whole-slide imaging software, manually delineating all morphologically consistent regions that conform to the TLS definition (such as dense lymphocyte aggregation, clear boundaries, etc.) using rectangular bounding boxes. Simultaneously, using the software's built-in distance measurement tool, the straight-line distance from the center of each TLS region to the nearest tumor invasion edge is precisely recorded, thereby obtaining its spatial location information relative to the tumor body. These labeling results are exported in GeoJSON format as spatial anchors for subsequent automated analysis. In addition, based on the segmentation results of the registered HE images, a tumor invasion front region mask can be generated simultaneously to help limit the effective range of cell segmentation in other analysis scenarios.

[0030] S320. Tumor tissue slice images based on multiplex immunofluorescence MPIF staining are used to segment cell nuclei using a pre-trained Cellpose model to obtain an initial cell nucleus mask. In this embodiment, optimized Cellpose cell segmentation is first performed. Based on manually labeled TLS regions, a fully automated cell segmentation and phenotypic classification process based on the Cellpose deep learning model is constructed. Specifically, this includes: 1) Model parameter adjustment and image preprocessing: First, each MPIF six-channel staining image was separated into six single-channel images: DAPI, CD4, CD8A, CD20, CD21, and PANCK. Then, using the QuPath tool, multi-channel local sub-images of each TLS region were precisely cropped from the original full-image image based on the previously saved GeoJSON file (preserving the original resolution). To eliminate non-specific staining signals, the RGB images of each protein labeling channel (CD4 / CD8A / CD20) were converted to the HSV color space, and a high signal-to-noise ratio binarized mask was generated using the Otsu adaptive thresholding method combined with saturation and value filtering, effectively suppressing background noise. For the MPIF fluorescence image features, the nuclear diameter parameter (nuclear diameter set to 10-15 μm) and flow threshold (0.4-0.6) of the Cellpose algorithm were adjusted to enhance the ability to distinguish dense cells.

[0031] 2) Multi-channel fusion segmentation: Nucleus segmentation is based on the DAPI channel. The pre-processed DAPI sub-image is input into the pre-trained Cellpose "nuclei" model to automatically generate an initial nucleus mask. Subsequently, morphological operations (removing fragments smaller than 15 pixels and relabeling connected components) optimize the segmentation results, and the centroid coordinates and area of ​​each nucleus are extracted. This process integrates the nuclear information from the DAPI channel and, combined with optimized model parameters, achieves accurate segmentation of immune cells such as CD4+ T cells, CD8+ T cells, and B cells.

[0032] S330. For the initial cell nuclear mask, calculate its corrected intensity score on each marker channel, and perform phenotypic classification according to the preset positive threshold and channel priority rules to obtain the cell segmentation result.

[0033] In this embodiment, the segmentation results are verified by manually annotating the gold standard to calculate the Dice coefficient, ensuring that the Dice coefficient of immune cell segmentation is ≥0.90 and the segmentation accuracy is ≥92%.

[0034] Cell phenotype classification is then performed. For each segmented nucleus region, the program calculates its average brightness value (V channel) across the CD4, CD8A, and CD20 protein channels, and subtracts the global background mean of the entire image to obtain a corrected channel-specific intensity score. Cell type determination follows a strict priority rule: if the intensity of a cell exceeds a set threshold (≥0.12) across multiple channels, the dominant channel is selected in the order of CD8A>CD4>CD20; if the intensity difference between the highest-scoring channel and other positive channels is ≤0.01 (i.e., CONFLICT_EPS), it is marked as "conflict"; otherwise, CD8A or CD4 positivity is classified as T cells, CD20 positivity as B cells, and the rest are classified as "unclassified".

[0035] S400. Convert the cell segmentation results into structured data and output them. The cell segmentation results include immune cell segmentation masks, cell phenotype classification results, and quantitative data on TLS cell composition and spatial distribution.

[0036] In this embodiment, the intermediate and final products in the analysis process are directly output for the basic image and segmentation results, including: high-purity MPIF single-channel image, HE-MPIF registered and fused image, immune cell segmentation mask map, and cell-level dataset.

[0037] Output detailed information for each cell in CSV format, including cell type label, spatial location information (centroid coordinates), intensity score, and spatial attributes. Specifically, for each TLS, generate a structured table recording the total number of cells in that TLS, the number of CD20, CD21, CD4, and CD8 cell subpopulations, the coordinates of each cell, and the cell density (cells per million pixels).

[0038] To further improve the automation and standardization of data processing, this study employs an automated process based on marker scoring to perform fine classification and spatial quantization analysis on the entire cohort of cell data.

[0039] Recursively search the specified root directory for CSV files that conform to the naming conventions (e.g., <Patient ID> / tls_* / tls). (cells.csv). For each cell record, a uniform threshold T=0.1 is set for positive determination based on a pre-defined list of immunomarkers (CD4, CD8A, CD20, CD21) and their corresponding score columns. For each cell, only markers with a score ≥ T are retained, and the highest score is selected as its primary phenotype; if no marker exceeds the threshold, it is classified as "Unclassified". Subsequently, the original marker names are mapped to standard cell type names (e.g., CD8A to "CD8+ T cell"), generating a new column cell_type_fine.

[0040] For spatial analysis, the geometric center coordinates of all cells within each TLS are calculated, and the Euclidean distance from each cell to the center is determined accordingly. Based on this, the number of cells, average distance, median distance, and number of cells with a distance ≤ 50 μm for each cell type are summarized by TLS group.

[0041] Finally, all TLS-level statistical results will be integrated into a structured table and saved as a UTF-8 encoded CSV file (e.g., all_tls_spatial_summary_fine.csv). Simultaneously, the same classification logic will be performed on the entire cohort of single-cell datasets, outputting a new file with fine phenotypic labels (e.g., 3rd_all_patients_single_cells_fine_classified_0.1.csv), and the classification success rate and frequency distribution of each class will be calculated. The entire analysis workflow is implemented using Python 3.9, primarily relying on standard libraries such as pandas, numpy, pathlib, and glob.

[0042] In summary, this invention preprocesses 6-color MPIF images and employs Gaussian filtering for noise reduction and linear spectral unmixing algorithms to achieve high-purity single-channel separation (purity ≥ 95%). Through multi-scale preprocessing, dual-target region extraction, dynamic detection of invasion boundaries, and construction of banded regions, a precise binary mask of the tumor invasion front is generated. Combined with cross-scale alignment and TLS screening to obtain the registration benchmark, a two-step strategy of "rigid + elastic" is used to complete targeted registration of MPIF and HE images (registration error ≤ 1.5 pixels). Based on the invasion front region mask, the segmentation range is limited, Cellpose algorithm parameters are optimized, and multi-channel fluorescence signals are fused to achieve precise segmentation of immune cells (Dice coefficient ≥ 0.90). Finally, high-purity single-channel images, registered and fused images, cell segmentation results, and coordinate datasets are output. The embodiments of the present invention can solve the defects of the prior art such as channel crosstalk, insufficient registration targeting, and weak anti-interference ability of segmentation. It realizes integrated automated processing from image preprocessing to result output, improving efficiency by 66.7%. It has strong universality and can be directly connected to downstream applications such as tumor microenvironment mechanism research and clinical prognostic indicator screening, providing standardized technical support for tumor immune-related research.

[0043] Finally, as Figure 3 As shown, the present invention will be further described in conjunction with specific embodiments. The preprocessing of data specifically includes: 1) Sample Source and Processing: Fifty-five pathologically confirmed colorectal cancer tissue samples were collected, including 25 samples with a high number of TLS (≥5 TLS / sample) at the high invasive front and 25 samples with a low number of TLS at the low invasive front (<5 TLS / sample). All tissue samples were fixed in formalin, embedded in paraffin, and prepared into 4 μm thick tissue sections. Each sample underwent 6-color multiplex immunofluorescence (MPIF) staining and routine hematoxylin-eosin (HE) staining. The MPIF staining marker combination included: DAPI (nucleus), CD4 (helper T cells), CD8A (cytotoxic T cells), CD20 (B cells), CD21 (follicular dendritic cells, used for TLS recognition), and PANCK (tumor cytokeratin). Tyramine signal amplification technology was used during staining to enhance the fluorescence signal intensity and ensure the detection rate of low-abundance markers.

[0044] 2) Image Acquisition Equipment: An Olympus FV3000 laser confocal microscope was used for whole-slide scanning imaging. Scanning parameters were set as follows: resolution 1024×1024 pixels, scan mode layer-by-layer scanning, independent acquisition of fluorescence signals for each channel, and exposure time optimized based on signal intensity (50ms for DAPI channel, 100-150ms for CD4 / CD8A / CD20 / CD21 / PANCK channels). After scanning, multi-channel raw image files were obtained and stored in TIFF format.

[0045] 3) Computing platform and software: Computing platform: Intel Core i9-13900K processor (24 cores, 3.0GHz), 64GB DDR5 memory, NVIDIA RTX 4090 graphics card (24GB VRAM), 4TB NVMe SSD storage; Image processing software: Python 3.9 (core analysis framework), ImageJ 1.53 (including linear spectral unmixing plugin), NiftyReg (image registration software), QuPath 0.4.3 (manual annotation tool); Deep learning frameworks: PyTorch 1.13.1 (CUDA 11.7), Cellpose 2.2.2 (cell segmentation model).

[0046] The specific implementation of the data includes: 1) Channel Separation and Image Preprocessing: ImageJ software combined with a linear spectral unmixing algorithm was used. Characteristic spectral reference curves of six fluorescent dyes (DAPI, FITC, Cy3, Cy5, Cy7, AF750) were input to perform spectral unmixing on the original MPIF image, resulting in six single-channel grayscale images (DAPI, CD4, CD8A, CD20, CD21, PANCK). After spectral unmixing, the channel purity was verified by calculating the correlation between each channel and the target spectrum. The results showed an average channel purity ≥96.2% (DAPI channel 97.1%, CD4 channel 96.5%, CD8A channel 96.3%, CD20 channel 96.8%, CD21 channel 95.9%, PANCK channel 96.7%). The separated single-channel images were saved in 16-bit TIFF format, retaining the original resolution (0.5 μm / pixel).

[0047] 2) Manual annotation of Regions of Interest (TLS): Two experienced pathologists independently reviewed high-resolution MPIF-stained slides using Olympus whole-slide imaging software. The annotation criteria were: TLS is defined as a morphologically well-defined, circular or elliptical region of lymphocyte aggregation with a visible CD21+ follicular dendritic cell network structure and a significantly higher cell density than the surrounding tissue. Annotators manually delineated all regions conforming to the TLS definition using rectangles and precisely recorded the straight-line distance from the center of each TLS region to the nearest tumor infiltration edge (based on the boundary of PANCK-positive tumor cell clusters) using the software's built-in distance measurement tool. After annotation, the results from the two annotators were validated for consistency (Kappa coefficient ≥ 0.85), and disputed regions were determined through negotiation. All annotation results were exported in GeoJSON format, including the rectangle coordinates, center coordinates, distance from the tumor edge, and sample ID information for each TLS.

[0048] 3) TLS Sub-image Cropping: Based on the generated GeoJSON annotation file, the script interface of QuPath 0.4.3 software is used to accurately crop multi-channel sub-images of each TLS region from the original full-image. The original resolution (0.5μm / pixel) is preserved during cropping, and the edges are extended by 100 pixels to ensure complete inclusion of the TLS region and its surrounding microenvironment. Each TLS sub-image is named as follows: <Patient ID>_tls<number>.tif, where the numbers are ordered sequentially from left to right and top to bottom according to the TLS spatial location within the sample (1-9). For samples with more than 9 TLS regions, the numbers are sequentially increased.

[0049] 4) HE-MPIF Image Registration: To achieve the spatial correspondence between HE and MPIF images, multimodal image registration was performed using NiftyReg software, specifically including: Key point extraction: 300 key points of invasion front features were extracted from HE images, with a focus on high curvature points, vascular branching points and densely celled areas in the tumor-stromal junction region; Rigid registration: Rigid registration is performed first to correct translation and rotation errors caused by sample preparation and scanning. Mutual information is used as a similarity measure, and the number of optimization iterations is set to 200. Elastic registration: Based on rigid registration, elastic registration is performed. The B-spline free deformation model is used to optimize local deviations. The grid spacing is controlled to be 16×16 pixels and the regularization parameter λ=0.01. Registration result verification: The mutual information value (MI) of the infiltration front region after registration is 0.89±0.03, and the registration error (calculated by the Euclidean distance between manually labeled landmark pairs) is 1.2±0.3 pixels, which verifies that the registration accuracy meets the requirements of subsequent analysis.

[0050] 5) Image Enhancement and Background Suppression: For the cropped TLS sub-image, image enhancement processing is performed on each protein labeling channel (CD4, CD8A, CD20, CD21), specifically including: Color space conversion: Convert RGB format images to HSV color space, separating the three channels of hue (H), saturation (S), and value (V); Background noise suppression: The Otsu adaptive thresholding method is used in conjunction with the saturation (S) and brightness (V) channels for background screening. Specifically, the Otsu thresholds of the S and V channels are calculated to generate a binary mask (pixel values ​​≥ the threshold are foreground, and < the threshold are background). This mask is then multiplied with the original image to effectively remove non-specific staining signals and background noise. Intensity Normalization: Z-score normalization is performed on each channel image to eliminate batch-to-batch staining differences. The normalization formula is: ,in and These are the mean and standard deviation of the image in that channel, respectively.

[0051] 6) Cell nucleus segmentation (optimized Cellpose): Cell nucleus segmentation based on DAPI channel images, specifically including: Model loading: Load the pre-trained Cellpose "nuclei" model (trained based on over 70,000 manually labeled cell nuclei); Parameter adjustments: Based on the MPIF fluorescence image features, adjust the Cellpose core parameters as follows: set the nuclear diameter to 12μm (corresponding to 24 pixels in the image, converted at 0.5μm / pixel), the flow threshold to 0.5, the cell probability threshold to 0.0, and the isotropic parameter (diameter) to True. Cell nucleus segmentation: The preprocessed DAPI subgraph is input into the optimized Cellpose model to automatically generate the initial cell nucleus mask; Morphological optimization: Post-processing of the initial mask: First, remove fragments with an area less than 15 pixels (corresponding to a physical area of ​​approximately 3.75 μm², which is the minimum cell nucleus area threshold); then, merge adjacent nucleus regions with a distance ≤ 3 pixels by relabeling connected components; finally, use the watershed algorithm to separate adhered cell nuclei. Feature extraction: Extract the centroid coordinates of each segmented cell nucleus ( Area (number of pixels), roundness ( Morphological features such as ( ).

[0052] 7) Cell phenotype classification: For each segmented nuclear region, multi-channel intensity analysis and phenotype determination are performed, specifically including: Intensity Calculation: For the CD4, CD8A, and CD20 protein channels, the average brightness value (V channel) within a 5-pixel extension of the nuclear region is calculated, and the global background mean of the entire image (calculated using the average intensity of the non-cellular region of 100×100 pixels in the four corners of the image) is subtracted to obtain the corrected channel-specific intensity score. ; Positive threshold setting: Based on the intensity distribution of negative control samples (isotype control antibody staining), a positive judgment threshold is set. (Normalized intensity value); Priority classification rules: If a cell has an intensity ≥0.12 on multiple channels, the dominant channel is selected according to the priority order of CD8A>CD4>CD20 (i.e., CD8A positivity takes precedence over CD4 positivity, and CD4 positivity takes precedence over CD20 positivity); if the intensity difference between the highest-scoring channel and other positive channels is ≤0.01 (defined as the conflict threshold CONFLICT_EPS), it is marked as "Conflict"; if only CD8A or CD4 is positive, it is classified as a T cell subtype (CD8+ T cell or CD4+ T cell); if only CD20 is positive, it is classified as a B cell; if CD21 is positive (only used for TLS structure recognition, not for cell classification) and the intensity is ≥0.12, combined with morphological characteristics (long axis / short axis ratio ≥1.5), it is classified as a follicular dendritic cell; if no marker exceeds the threshold, it is classified as "Unclassified". Quality control: Positive controls (tonsil tissue sections) and negative controls (isotype controls) are set up for each batch of samples to ensure the reliability of the classification results.

[0053] 8) Segmentation Result Validation: The segmentation and classification results were validated using the gold standard of manual annotation. Ten samples were randomly selected, and pathology researchers manually annotated 500 cell nuclear boundaries and phenotypes (150 CD4+ T cells, 150 CD8+ T cells, 150 B cells, and 50 other cells). The Dice coefficient was calculated to assess segmentation accuracy: the Dice coefficient for CD4+ T cells was 0.93±0.02, for CD8+ T cells it was 0.92±0.03, and for B cells it was 0.91±0.02, with an average segmentation accuracy of 93.5%. The classification accuracy was calculated: compared with the manually annotated phenotypes, the overall classification accuracy was 92.8% (sensitivity 93.2%, specificity 92.5%). Inter-class confusion matrix analysis showed that CD4 and CD8 double-positive cells (accounting for approximately 2.3% of the total cells) were mainly classified into the "Conflict" category, which was in line with expectations.

[0054] Finally, the results are output and spatial quantification analysis is performed. The analysis process for each sample outputs the following data files: 1) High-purity MPIF single-channel images (DAPI, CD4, CD8A, CD20, CD21, PANCK, a total of 6 files); 2) HE-MPIF registered and fused image (1 file); 3) Immune cell segmentation mask (one file per TLS region, PNG format, with different cell types labeled in different colors); 4) Cell-level dataset: Each TLS generates a CSV file (named: tls) The _cells.csv file contains the following fields: cell ID, centroid coordinates (x, y), cell type, intensity score for each channel (score_CD4, score_CD8A, score_CD20, score_CD21), cell area, and cell density (calculated as the number of cells per million pixels within the TLS).

[0055] Then, automated fine-grained classification and spatial quantization are performed. An automated analysis process based on Python 3.9 is used to perform secondary classification and spatial quantization analysis on the entire queue of data, specifically including: Data retrieval: Use the glob module to recursively search CSV files that conform to the naming convention (path pattern: <patient ID> / tls_* / tls). (_cells.csv), the patient ID and TLS number are parsed using pathlib.Path; Fine-grained classification: For each cell record, a uniform threshold T=0.1 is set for positive determination based on a pre-defined list of immunomarkers (CD4, CD8A, CD20, CD21) and their corresponding scoring columns. The marker with the highest score among all markers with a score ≥ T is selected as its primary phenotype. If no marker exceeds the threshold, it is classified as "Unclassified". The original marker names are mapped to standard cell type names (CD8A→"CD8+ T cell", CD4→"CD4+ T cell", CD20→"B cell", CD21→"Follicular DC"), generating a new column cell_type_fine; Spatial distribution calculation: Calculate the geometric center coordinates of all cells within each TLS ( , Based on this, the Euclidean distance from each cell to the center is calculated. ; TLS-level summary: Grouped by TLS, calculate the number, average distance, median distance, standard deviation of distance, and number of cells with a distance ≤50μm for each cell type (defined as TLS core area cells). Full cohort integration: All TLS-level statistical results are integrated into a structured table; at the same time, the same classification logic is performed on the full cohort single-cell dataset, and the classification success rate (average 98.7%) and the frequency distribution of each category are calculated (CD8+ T cells account for 31.2%, CD4+ T cells account for 24.5%, B cells account for 18.3%, follicular dendritic cells account for 2.1%, unclassified accounts for 15.6%, and conflict accounts for 8.3%).

[0056] Further, typical data examples: Taking the patient ID "CRC-1344382" as an example, this sample is a surgical resection specimen of advanced colorectal cancer (pT3N1M0). Five TLS structures (tls1–tls5) were annotated in the tumor invasion front region. After processing using the method of this invention, each TLS region generates a single-cell-level CSV file. For example, some data records in tls1_cells.csv are shown in Table 1: Table 1. Example of partial data from tls1_cells.csv ; TLS1 area statistics include: 1) Total number of cells: 274.

[0057] Number and percentage of each cell type: CD3+ T cells 89 (32.5%), CD8+ T cells 78 (28.5%), B cells 51 (18.6%), tumor cells 9 (3.3%), macrophages 13 (4.7%), unclassified cells 22 (8.0%), conflicting cells 12 (4.4%) Spatial analysis: The geometric center coordinates of TLS1 (x̄=192.3, ȳ=954.2) and the average distance from CD8+ T cells to the center were 38.2 μm, which was significantly smaller than the 52.4 μm of CD3+ T cells (independent samples t test, p=0.028), suggesting that cytotoxic T cells tend to aggregate in the core region of TLS.

[0058] 2) Results of full queue integration: A summary analysis was performed on all 5 TLS (a total of 1,247 cells), and the output was all_tls_spatial_summary_fine.csv.

[0059] The classification success rate reached 97.8%, with CD3+ T cells accounting for 31.2%, CD8+ T cells accounting for 27.5%, B cells accounting for 19.3%, tumor cells accounting for 3.8%, macrophages accounting for 5.2%, unclassified cells accounting for 8.5%, and conflicting cells accounting for 4.5%.

[0060] Spatial quantification showed that the average CD8+ / CD3+ ratio within the TLS was 0.88, which was highly consistent with the results of manual assessment by pathologists (average 0.85) (Pearson correlation coefficient r=0.92).

[0061] In this embodiment, the total processing time for a single sample (containing approximately 12 tumor ROIs and 8 TLSs, totaling approximately 12,000 cells) was 16±3 minutes, a 66.7% reduction compared to the traditional method (channel separation + global registration + conventional Cellpose segmentation) which took 48±6 minutes. Specifically, the ROI sub-image cropping and segmentation steps took 9±1 minutes, the automated classification and spatial quantization steps took 5±1 minutes, and the data integration and output steps took 2±0.5 minutes. The segmentation accuracy was improved by 11.0% compared to the conventional Cellpose algorithm (without optimized parameters and image enhancement) of 82.5%, and the registration error was reduced by 63.6% compared to the global registration method (without elastic registration) which had an error of 3.3±0.6 pixels.

[0062] The core parameters of this invention can be dynamically adjusted according to image resolution and tumor type, as shown in Table 2: Table 2 Core Parameter Adjustment Data Table ; Therefore, the embodiments of the present invention have the following advantages compared with the prior art: 1) Dual Guarantee of Multi-Channel Separation Purity and Spatial Alignment Accuracy: Existing technologies not only suffer from MPIF channel crosstalk but also lack precise alignment correction between channels, resulting in a spatial position deviation of cells ≥1-2 pixels after channel separation, further amplifying errors in subsequent cell identification and interaction analysis. This invention, by adding "WSIReg tool targeted alignment and affine matrix mapping," utilizes the homologous staining features of H&E cell nuclei and MPIF-DAPI channels to achieve precise spatial alignment of all MPIF channels (deviation ≤0.5 pixels). Combined with "Gaussian filtering noise reduction + linear spectral unmixing" technology, it ultimately achieves the dual effect of "precise spatial alignment + high signal purity" (single-channel purity ≥95%). 2) The registration technology achieves a triple breakthrough of "targeting + high precision + high efficiency": Existing registration technologies lack a pre-alignment step between channels, requiring iterative optimization of registration parameters from scratch, resulting in slow convergence speed and a tendency to get trapped in local optima; The embodiment of this invention obtains the initial value of the affine matrix through channel pre-alignment, providing accurate initial constraints for subsequent HE-MPIF registration, thereby improving the convergence speed of rigid registration by 30%. At the same time, combined with the two-step strategy of "immersion front target benchmark + rigid + elastic", the final registration error is ≤1.5 pixels and MI≥0.85.

[0063] 3) Significantly Enhanced Anti-interference Ability of Cell Segmentation: The tumor microenvironment is characterized by dense cells, numerous tissue fragments, and complex fluorescence backgrounds. Existing conventional segmentation algorithms (such as the traditional Cellpose and U-Net) are not optimized for MPIF image features, resulting in segmentation accuracy generally below 85%, making it difficult to distinguish dense immune cells from background noise. This invention achieves accurate segmentation through triple optimization: first, the segmentation range is limited by a registered invasion front mask to eliminate interference from non-target regions; second, the Cellpose algorithm parameters are adjusted to suit the morphological characteristics of immune cells (nuclear diameter 10-15 μm, flow threshold 0.4-0.6); and third, a feature map is constructed by fusing 6-channel fluorescence signals to enhance the distinction between cells and background. The final immune cell segmentation Dice coefficient is ≥0.90, and the segmentation accuracy is ≥92%, representing a 9.3% improvement over the conventional Cellpose algorithm, effectively solving the segmentation problem caused by dense cells and complex backgrounds.

[0064] 4) Integrated Technology Process, Superior Efficiency and Compatibility: In existing technologies, channel separation, image registration, and cell segmentation are mostly independent processes requiring manual connection, which is cumbersome and prone to human error, with a single sample processing time of approximately 45 minutes. This invention constructs an integrated technology chain of "channel separation - targeted registration - precise segmentation - result output," automating the entire process. The total processing time per sample is only 15±2 minutes, improving efficiency by 66.7%, and avoiding systematic errors caused by manual operation. Furthermore, the high-purity single-channel images, registered and fused images, and cell coordinate datasets output by this invention can be directly connected to downstream applications such as tumor microenvironment mechanism research (e.g., TLS cell interaction analysis, immune cell spatial distribution statistics), and clinical prognostic indicator screening. It provides standardized technical tools for fields such as spatial proteomics and tumor immunotherapy target development, offering strong compatibility and wide applicability. 5) Adaptive parameter adjustment with strong universality: Existing technologies are mostly designed for specific tumor types or fixed image resolutions, with fixed parameters, making it difficult to adapt to samples with different magnifications (e.g., 5×, 10×, 20×) and different tumor types. In the embodiments of this invention, the core parameters (e.g., registration structure element size, Cellpose kernel diameter, physical pixel conversion factor) can be dynamically adjusted according to image resolution (e.g., 2.0μm / pixel at 5×, 0.5μm / pixel at 20×) and tumor tissue type. Its effectiveness has been verified in colorectal cancer samples, and it can also be extended to multiple tumor types such as lung cancer and breast cancer.

[0065] Therefore, this embodiment of the invention, by combining a spectral unmixing algorithm, achieves a 6-color MPIF channel separation purity ≥95%, which is 10-15% higher than the traditional threshold segmentation method, effectively avoiding cell recognition errors caused by channel crosstalk; furthermore, the "rigid + elastic" two-step registration strategy centered on the tumor invasion front achieves a registration error ≤1.5 pixels, reducing the error by more than 40% compared to the global registration method, meeting the precise requirements for microscopic feature and spatial location correlation analysis; furthermore, through region masking and algorithm parameter optimization, the tumor microenvironment immune cell segmentation accuracy is ≥92%, solving the segmentation problem caused by dense cells and complex backgrounds; finally, it can also be directly connected to tumor microenvironment mechanism research (such as TLS cell interaction and immune cell spatial distribution analysis), providing core technical support for the precise analysis of spatial proteomics data.

[0066] Reference Figure 2 A system for MPIF-HE image registration and cell segmentation of tumor tissue, comprising: The first module 201 is used to acquire multiplex immunofluorescence (MPIF) stained tumor tissue slice images and perform preprocessing and multichannel separation to obtain preprocessed single-channel MPIF images. The second module 202 is used to extract the tumor invasion front region based on the HE staining image as the registration benchmark, register the HE staining image with the preprocessed single-channel MPIF image, and obtain the registered HE image segmentation result. The third module 203 is used to delineate regions of interest based on the segmentation results of the registered HE image, and to segment and classify the immune cells in the regions of interest using a pre-trained cell segmentation model to obtain cell segmentation results. The fourth module 204 is used to convert the cell segmentation results into structured data and output them. The cell segmentation results include immune cell segmentation masks, cell phenotype classification results, and quantitative data on TLS cell composition and spatial distribution.

[0067] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0068] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this is not intended to limit the scope of the embodiments of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A method for MPIF-HE image registration and cell segmentation of tumor tissue, characterized in that, Includes the following steps: Multiplex immunofluorescence (MPIF) stained tumor tissue slices were acquired, preprocessed, and separated into multiple channels to obtain preprocessed single-channel MPIF images. Based on the tumor invasion front region extracted from the HE staining image as the registration benchmark, the HE staining image and the preprocessed single-channel MPIF image were registered to obtain the registered HE image segmentation result. Based on the segmentation results of the registered HE image, the region of interest is delineated, and the immune cells in the region of interest are segmented and classified by phenotype using a pre-trained cell segmentation model to obtain the cell segmentation results. The cell segmentation results are converted into structured data and output, including immune cell segmentation masks, cell phenotype classification results, and quantitative data on TLS cell composition and spatial distribution.

2. The method for MPIF-HE image registration and cell segmentation of tumor tissue according to claim 1, characterized in that, The step of acquiring multiplex immunofluorescence (MPIF) stained tumor tissue slice images and performing preprocessing and multichannel separation to obtain preprocessed single-channel MPIF images specifically includes: Multiplex immunofluorescence (MPIF) stained tumor tissue slices were acquired and image denoising was performed to obtain denoised MPIF images; Acquire HE-stained images and perform cross-channel alignment correction on the denoised MPIF images to obtain aligned MPIF images; Based on the characteristic spectral library of fluorescent dyes, a linear spectral unmixing algorithm is used to separate the channels of the aligned and corrected MPIF image to obtain the initial single-channel MPIF image. The signal purity of the initial single-channel MPIF image is calculated and filtered according to a preset purity threshold to obtain the preprocessed single-channel MPIF image.

3. The method for MPIF-HE image registration and cell segmentation of tumor tissue according to claim 2, characterized in that, The step of acquiring the HE-stained image and performing cross-channel alignment correction processing on the denoised MPIF image to obtain the aligned and corrected MPIF image specifically includes: Tumor tissue sections were stained with hematoxylin and eosin (HE) to obtain HE-stained images; Affine transformation matrix is ​​generated by registering the cell nuclear staining features in the HE staining image with the DAPI channel in the denoised MPIF image. The matrix is ​​mapped to the other marker channels of the denoised MPIF image, and channel space alignment is performed to obtain the aligned and corrected MPIF image.

4. The method for MPIF-HE image registration and cell segmentation of tumor tissue according to claim 3, characterized in that, The step of extracting the tumor invasion front region from the HE-stained image as a registration benchmark, registering the HE-stained image with the preprocessed single-channel MPIF image, and obtaining the registered HE image segmentation result specifically includes: Based on HE-stained images, pixel segmentation is performed to obtain the final tumor solid region mask and the effective normal tissue region mask. Dynamic boundary detection was used to identify the infiltration front boundary between the final tumor solid region mask and the effective normal tissue region mask. Centered on the boundary line of the infiltration front, a strip-shaped region of a preset physical width is generated on both sides as the registration reference; Based on the registration benchmark, feature key points are extracted, and the HE-stained image is registered with the preprocessed single-channel MPIF image through a two-step strategy of rigid registration and elastic registration to obtain the registered HE image segmentation result.

5. The method for MPIF-HE image registration and cell segmentation of tumor tissue according to claim 4, characterized in that, The step of performing pixel segmentation based on HE-stained images to obtain the final tumor solid region mask and the effective normal tissue region mask specifically includes: Bilinear interpolation downsampling was performed on the HE staining image to obtain a low-resolution HE staining image; Based on the RGB color space threshold, all background regions of the low-resolution HE staining image are segmented, and the segmentation results are logically inverted to obtain a binary mask containing all effective tissue regions. Based on the RGB values ​​of tumor-specific staining markers, pixel-level thresholding is performed on the binary mask containing all effective tissue regions to obtain the initial binary mask of the tumor region. Based on the initial binary mask image of the tumor, all holes inside the mask are filled and all connected region contours are detected. The pixel area of ​​each contour is calculated, and discrete noise points with an area smaller than the first preset pixel are filtered out to obtain the final tumor solid region mask after removing holes and noise. The binary mask containing all valid tissue regions is segmented at the pixel level based on the RGB values ​​of the normal tissue marker color to obtain the initial binary mask image of the normal tissue region; Based on the initial binary mask image of the normal tissue region, the contours of all connected regions are detected, and the pixel area of ​​each contour is calculated. Small tissue fragments with an area smaller than a second preset pixel are filtered out to obtain the final effective normal tissue region mask.

6. The method for MPIF-HE image registration and cell segmentation of tumor tissue according to claim 5, characterized in that, The step of defining regions of interest based on the registered HE image segmentation results, and then segmenting and classifying immune cells within these regions of interest using a pre-trained cell segmentation model to obtain the cell segmentation results, specifically includes: The preprocessed single-channel MPIF image was manually delineated, and the local sub-image of the TLS region was cropped by combining the segmentation results of the registered HE image. Tumor tissue slice images based on multiplex immunofluorescence MPIF staining were used to segment cell nuclei using a pre-trained Cellpose model to obtain an initial cell nucleus mask. For the initial cell nuclear mask, its corrected intensity score on each marker channel is calculated, and phenotypic classification is performed according to the preset positive threshold and channel priority rules to obtain the cell segmentation result.

7. The method for MPIF-HE image registration and cell segmentation of tumor tissue according to claim 6, characterized in that, The step of manually delineating the preprocessed single-channel MPIF image and cropping a local sub-image of the TLS region based on the segmentation results of the registered HE image specifically includes: The TLS region is obtained by manually delineating the preprocessed single-channel MPIF image; By using distance measurement tools, the straight-line distance from the center of each TLS region to the nearest tumor infiltration edge is obtained, thus acquiring the spatial location information of the tumor body; Based on the spatial location information of the tumor, and combined with the segmentation results of the registered HE image, auxiliary cropping is performed to obtain a local sub-image of the TLS region.

8. A system for MPIF-HE image registration and cell segmentation of tumor tissue, characterized in that, Includes the following modules: The first module is used to acquire multiplex immunofluorescence (MPIF) stained tumor tissue slice images and perform preprocessing and multichannel separation to obtain preprocessed single-channel MPIF images. The second module is used to extract the tumor invasion front region based on the HE staining image as the registration benchmark, and register the HE staining image with the preprocessed single-channel MPIF image to obtain the registered HE image segmentation result. The third module is used to delineate regions of interest based on the segmentation results of the registered HE image, and to segment and classify the immune cells in the regions of interest using a pre-trained cell segmentation model to obtain cell segmentation results. The fourth module is used to convert cell segmentation results into structured data and output them. The cell segmentation results include immune cell segmentation masks, cell phenotype classification results, and quantitative data on TLS cell composition and spatial distribution.

Images