Method, system, device and medium for epithelial tissue segmentation on stained converted pathological sections

Abstract

The application discloses a kind of pathological section epithelial tissue segmentation method, system, equipment and medium based on dyeing conversion, method includes: obtaining the HE dyeing image of patient original TMA pathological section and pre-processing, obtain the effective tissue area of HE dyeing image;Effective area tissue of HE dyeing image is cut by sliding window method, obtain multiple HE dyeing image small blocks;HE dyeing image small block is input to pre-trained dyeing style conversion network CS-Net, generate multiple corresponding IHC dyeing image small blocks;Multiple IHC dyeing image small blocks obtained are spliced and assembled, obtain the pseudo IHC dyeing graph consistent with the size of original HE dyeing graph, and the mask graph of epithelial tissue is obtained by color deconvolution, noise reduction processing and pixel threshold segmentation.The pathological section is converted from HE dyeing to IHC dyeing by dyeing style conversion network, which saves the time, manpower and cost of eluting HE dyeing on pathological section and re-performing IHC dyeing.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and specifically to a method, system, device, and medium for segmenting epithelial tissue from pathological sections based on staining conversion. Background Technology

[0002] Pathological section examination is an essential step in biopsy. After obtaining tissue specimens from the patient's lesion site through surgical excision or needle puncture, the specimens are fixed in shape with chemicals (such as formalin), hardened with paraffin, and then sliced ​​into thin sections using a microtome and adhered to glass slides. Subsequently, various colors are applied using specific reagents as needed, allowing doctors to observe pathological changes under a microscope, make corresponding pathological diagnoses, and provide support and assistance for the patient's subsequent clinical treatment.

[0003] Tissue microarrays (TMA) are a type of pathological section, differing from traditional methods in that, instead of fixing and hardening the diseased tissue specimen in paraffin and then slicing it into thin sections, a hollow needle is used to extract small tissue cores (0.6 mm in diameter) from the ROI (Region of Interest) from the paraffin block. These tissue cores are then inserted at precise intervals into new paraffin blocks to form microarray paraffin blocks, which are then sliced ​​into thin sections using a microtome. Subsequent staining is the same as before. Each microarray paraffin block can be divided into 100-500 segments for independent testing. Since its introduction in 1998, this technology has been widely adopted due to its advantages of large scale, high throughput, and standardization. Its greatest advantage lies in the completely consistent experimental conditions for tissue samples on the chip, ensuring excellent quality control. The time and reagent savings are also obvious.

[0004] HE staining (Hematoxylin and eosin stain) is one of the main staining methods used in histopathology and is the most widely used staining agent in medical diagnosis, often considered the gold standard. It is a combination of two histological staining agents: hematoxylin and eosin. Hematoxylin stains the cell nucleus blue-purple, while eosin stains the extracellular matrix and cytoplasm pink. This allows pathologists to easily observe the overall condition of tissue samples and, in practical applications, can identify abnormal pathological changes such as cell necrosis, edema, degeneration, and inflammatory cell infiltration. HE staining is the primary staining method in histology, partly because it is quick, inexpensive, and the staining results are easily identifiable.

[0005] IHC staining (Immunohistochemistry) involves binding fluorescent or chromogenic chemicals to antibodies, utilizing the specific binding reaction between antigens and antibodies in immunology to detect the presence of target antigens (specific protein markers) in cells or tissues. This method can be used not only to measure the amount of antigen expressed but also to observe the location of the antigen. Any substance that can be bound by antibodies, i.e., antigenic substances, including proteins, nucleic acids, polysaccharides, and pathogens, can be detected. In this experiment, the IHC reagent was used to stain the epithelial tissue of the patient's oropharyngeal carcinoma tumor. Because most malignant tumors originate from epithelial tissue, the analysis of epithelial tissue has significant clinical value for pathological diagnosis.

[0006] HE staining provides a relatively rough identification of different cell morphological changes; while IHC can detect specific changes (number or area) in specific cells by targeting specific cell markers, and can also detect intracellular cytokine translocation and changes in the expression levels of certain specific proteins in tissues. A comprehensive analysis is then performed using an image analysis system to analyze color intensity, distribution area, and other factors.

[0007] Typically, after observing HE-stained sections, if pathologists need to further confirm the results through IHC staining, they can only do so by eluting the HE stain from the section or by taking adjacent tissue samples and repeating the IHC staining experiment. This method has the following drawbacks:

[0008] (1) It increases the time cost and preparation cost for the testers, usually requiring five days, which seriously affects immediate diagnosis;

[0009] (2) After re-staining, the structure and morphology of the sections may differ, and the staining quality varies from person to person and is greatly affected by human factors.

[0010] Therefore, obtaining IHC staining images quickly, which require long processing times, high costs, and stable quality, is a method urgently needed by both doctors and patients. Summary of the Invention

[0011] The main objective of this invention is to overcome the shortcomings and deficiencies of the prior art and provide a method for segmenting epithelial tissue in pathological sections based on staining conversion. This method automatically converts HE staining images into IHC staining images, solving the problems of high time, labor, and financial costs associated with re-staining HE staining on washed pathological sections and then performing IHC staining again.

[0012] To achieve the above objectives, the present invention adopts the following technical solution:

[0013] This invention provides a method for segmenting epithelial tissue in pathological sections based on staining conversion, comprising the following steps:

[0014] HE-stained images of the patient's original TMA pathological sections were obtained and preprocessed to obtain the effective tissue area of ​​the HE-stained images;

[0015] The effective tissue region of the obtained HE staining image was segmented using the sliding window method to obtain multiple HE staining image patches.

[0016] Each HE staining image patch is input into a pre-trained staining style transfer network CS-Net to generate multiple corresponding IHC staining image patches.

[0017] Multiple IHC staining image patches were spliced ​​together to obtain a pseudo IHC staining image with the same size as the original HE staining image. A binary mask image of the epithelial tissue was obtained by color deconvolution, noise reduction and pixel thresholding.

[0018] As a preferred technical solution, the preprocessing involves color normalizing the HE staining image and then cropping it into a square HE staining image with equal length and width.

[0019] As a preferred technical solution, the sliding window method segmentation refers to using a sliding window of size 256*256 for a HE staining image of size 2048*2048, starting from the upper left corner of the HE staining image, sliding from left to right and from top to bottom, with each step of the window sliding forward being half the length of the window.

[0020] As a preferred technical solution, the style transfer network CS-Net is implemented by replacing the encoder of the U-Net network with a VGG16 network to achieve style transfer, and a new attention module, namely CS-Gate, is inserted.

[0021] The pre-trained coloring style transfer network CS-Net is specifically as follows:

[0022] The segmented HE-stained image patches and the corresponding IHC-stained image patches are used as the source and target of the staining style transfer network, respectively, and are simultaneously input into the staining style transfer network for training.

[0023] The VGG16 network extracts features from HE-stained images from shallow to deep layers. Then, in the Decoder stage, the extracted deep features are upsampled step by step. Meanwhile, assuming there are n layers in the Encoder stage, the features extracted in the i-th layer are refined by the CS-Gate and then combined with the ni-th layer of the Decoder through a skip connection to become a new feature map, which is then passed to the next Decoder layer, i.e., the (n-i+1)-th layer. This process is repeated until the last layer. The output feature map is then mapped by the Sigmoid algorithm to obtain the generated IHC staining map.

[0024] The generated IHC staining image and the real IHC staining image are compared using L1Loss to calculate the loss, and the loss is backpropagated together to update the network parameters. The training continues until the loss stabilizes, at which point the training ends.

[0025] The trained staining style transfer network was tested by dividing the HE staining image into several HE staining image blocks using the sliding window method. The segmented HE staining image blocks were then input into the trained staining style transfer network for style transfer, resulting in IHC staining image blocks corresponding to each HE staining image block.

[0026] As a preferred technical solution, the step of splicing and assembling multiple IHC staining image patches to obtain a pseudo IHC staining image of the same size as the original HE staining image specifically involves:

[0027] The obtained IHC staining image patches are stitched together in order from left to right and from top to bottom. For areas where multiple IHC staining image patches overlap, the lowest pixel value among all overlapping parts is selected. After the stitching operation is completed, a pseudo IHC staining image with the same size as the original HE staining image is obtained.

[0028] As a preferred technical solution, the step of obtaining a binary mask image of epithelial tissue through color deconvolution, noise reduction processing, and pixel threshold segmentation specifically involves:

[0029] The obtained pseudo-IHC staining image is denoted as I ihc Select the B channel from the RGB channels as the grayscale image of this image, denoted as I. gray ;

[0030] Set threshold T max =210, change I gray In the middle of 0 to T max Pixel values ​​between these ranges are set to 255, and pixel values ​​outside this range are set to 1, thus obtaining a binary mask image, denoted as I. maskIn the binary mask image, white represents the foreground, indicating epithelial tissue; black represents the background, indicating the stroma and non-epithelial tissues on the slide.

[0031] For the obtained I mask Noise reduction processing is performed, i.e., filling I mask For small pores with an area less than 5000, isolated regions with an area less than 800 are removed, and then closed. This smooths the internal and external contours of the tissue, resulting in a processed binary mask image, denoted as I. bin ;

[0032] For the obtained I bin The small pore areas with an area less than 5000 are filled again to obtain the final binary mask image of the epithelial tissue.

[0033] In another aspect, the present invention provides a pathological slide epithelial tissue segmentation system based on staining conversion, which is applied to the aforementioned pathological slide epithelial tissue segmentation method based on staining conversion, including a preprocessing module, an image segmentation module, a network model training module, and a mask image acquisition module;

[0034] The preprocessing module is used to acquire HE-stained images of the patient's original TMA pathological sections and perform preprocessing to obtain the effective tissue area of ​​the HE-stained image;

[0035] The image segmentation module is used to segment the effective tissue region of the obtained HE staining image using a sliding window method to obtain multiple HE staining image blocks.

[0036] The network model training module is used to input each HE staining image patch into a pre-trained staining style transfer network CS-Net to generate multiple corresponding pseudo IHC staining image patches.

[0037] The mask image acquisition module is used to stitch together multiple pseudo-IHC staining image blocks to obtain a pseudo-IHC staining image of the same size as the original HE staining image, and to obtain a binary mask image of the epithelial tissue through color deconvolution, noise reduction processing and pixel threshold segmentation.

[0038] In another aspect, the present invention provides an electronic device, characterized in that the electronic device comprises:

[0039] At least one processor; and,

[0040] A memory communicatively connected to the at least one processor; wherein,

[0041] The memory stores computer program instructions that can be executed by the at least one processor to enable the at least one processor to perform the staining conversion-based epithelial tissue segmentation method for pathological sections.

[0042] In another aspect, the present invention provides a computer-readable storage medium storing a program that, when executed by a processor, implements the aforementioned method for segmenting epithelial tissue of pathological sections based on staining conversion.

[0043] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0044] (1) This invention proposes a new Attention module, CS-Gate, which is inserted into the neural network to enhance its feature extraction and discrimination capabilities, improves the quality of converting HE staining images to IHC staining images, and provides a more accurate epithelial tissue mask for subsequent color deconvolution and threshold segmentation operations.

[0045] (2) The present invention can automatically generate IHC staining images by computer when only HE staining images are available, providing a channel for testers to obtain relatively reliable IHC staining images in laboratories without IHC staining preparation conditions. In some cases, the computer-generated images are even better than real pathological staining.

[0046] (3) This invention combines shallow features and deep features in the Encoder, and then passes the combined features through the channel attention module and the spatial attention module respectively, and finally encodes them into a feature vector and assigns it to the shallow features, and then merges it with the deep features through skip-connection; the intention of doing so is to make full use of the low-dimensional and high-dimensional features of the same image, and then encode them into a feature vector through the spatial and channel dimensions, so as to extract and identify which features are needed more effectively;

[0047] (4) This invention greatly saves the time and cost of washing HE staining from pathological sections and then re-staining with IHC. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a flowchart of the epithelial tissue segmentation method for pathological sections based on staining conversion, as described in an embodiment of the present invention.

[0050] Figure 2 This is a structural diagram of the coloring style transfer network CS-NET in an embodiment of the present invention;

[0051] Figure 3 This is the CS-Gate module of the pathological section epithelial tissue segmentation method based on staining conversion in an embodiment of the present invention;

[0052] Figure 4 This is a schematic diagram illustrating a method for segmenting a pathological image into stained image blocks in an embodiment of the present invention;

[0053] Figure 5 This is a schematic diagram of the structure in an embodiment of the present invention where the Encoder part in U-Net is replaced with VGG16;

[0054] Figure 6 This is a schematic diagram of the epithelial tissue segmentation system for pathological sections based on staining conversion in this embodiment;

[0055] Figure 7 This is a structural diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0056] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0057] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0058] like Figure 1 As shown, in one embodiment of this application, a method for segmenting pathological section epithelial tissue based on staining conversion is provided, including the following steps:

[0059] 1. Obtain HE-stained images of the patient's original TMA pathological sections and perform preprocessing. The preprocessing involves color normalization of the HE-stained images and then cropping them into square HE-stained images with equal length and width to obtain the effective tissue area of ​​the HE-stained images.

[0060] 2. Prepare a training set, and use the sliding window method to divide the effective region of the obtained HE staining image into small patches to obtain multiple HE staining image patches;

[0061] Furthermore, the sliding window method for segmenting stained image patches refers to using a sliding window of size 256*256 for a HE-stained image of size 2048*2048. Starting from the top left corner of the HE-stained image, the window slides forward from left to right and from top to bottom, with each slide being half the window length, i.e., an overlap of 50%. In the training set, for each HE-stained image patch, a judgment is made: if the foreground portion accounts for more than or equal to 5%, then this stained image patch is used. Figure 4 The patch_a shown; if the foreground portion occupies less than 5%, it will not be used, such as... Figure 4 As shown in patch_b, a total of 29,023 HE-stained image patches were obtained as the training set.

[0062] 3. For example Figure 2 As shown, each HE staining image patch is input into a pre-trained staining style transfer network CS-Net to generate multiple corresponding IHC staining image patches.

[0063] Furthermore, the initial training parameters were initialized using Xavier: lr = 0.0002, batch_size = 16, optimizer = Adam, loss = L1 loss, and the learning strength was halved every two epochs.

[0064] Furthermore, such as Figure 5 As shown, the style transfer network CS-Net achieves style transfer by replacing the encoder of the U-Net network with VGG16 and inserting an attention module, namely CS-Gate. The purpose of using VGG16 as the encoder is to improve the feature extraction capability of U-Net, while the CS-Gate module is used to improve the network's feature recognition capability, that is, emphasizing effective features while suppressing ineffective features, thereby achieving a higher prediction accuracy. Figure 3 As shown.

[0065] The pre-trained coloring style transfer network CS-Net is specifically as follows:

[0066] The segmented HE-stained image patches and the corresponding IHC-stained image patches are used as the source and target of the staining style transfer network, respectively, and are simultaneously input into the staining style transfer network for training.

[0067] The VGG16 network extracts features from HE images from shallow to deep layers. Then, in the Decoder stage, the extracted deep features are upsampled step by step. Meanwhile, assuming that the Encoder stage has n layers, the features extracted in the i-th layer (in this embodiment, i∈{1,2,3,4,5},n=5) are refined by CS-Gate and then combined with the ni-th layer of the Decoder through a skip-connection to become a new feature map, which is then passed to the next Decoder layer, i.e., the n-i+1-th layer. This process is repeated until the last layer. The output feature map is then mapped by Sigmoid to obtain the generated IHC staining map.

[0068] The generated IHC staining map and the real IHC staining map are compared using L1Loss to calculate the loss, and the loss is backpropagated together to update the network parameters. The training continues for the next iteration until the loss tends to stabilize, at which point the training ends.

[0069] 4. To test the trained staining style transfer network, the HE staining image is first divided into several HE staining image patches using the same sliding window method as the training set. However, all staining images in the test set are used, i.e., both patch_a and patch_b are used. Then, the segmented HE staining image patches are input into the trained staining style transfer network for style transfer, resulting in IHC staining image patches corresponding to each HE staining image patch. The resulting IHC staining image patches of size 256*256 are then stitched together from left to right and from top to bottom. For overlapping areas of multiple pseudo-IHC staining image patches, the lowest pixel value among all overlapping parts is selected. After the stitching operation is completed, a pseudo-IHC staining image of the same size as the original HE staining image is obtained.

[0070] 5. The obtained pseudo-IHC staining image is processed by color deconvolution, noise reduction, and pixel thresholding to obtain a mask image of the epithelial tissue.

[0071] Furthermore, the process of obtaining a binary mask image of the epithelial tissue through color deconvolution, noise reduction, and pixel thresholding is specifically as follows:

[0072] The obtained pseudo-IHC staining image is denoted as I ihc Select the B channel from the RGB channels as the grayscale image of this image, denoted as I. gray ;I gray =I ihc [:,∶,3];

[0073] Set threshold T max =210, change I gray In the middle of 0 to T maxPixel values ​​between these ranges are set to 255, and pixel values ​​outside this range are set to 1, thus obtaining a binary mask image, denoted as I. mask I mask =where(0<I gray <T max In the binary mask image, white represents the foreground, indicating epithelial tissue; black represents the background, indicating the stroma and non-epithelial tissues on the slide.

[0074] For the obtained I mask Noise reduction processing is performed, i.e., filling I mask In the small hole region with an area less than 5000, I bin =morphology.remove_small_holes(I mask Remove isolated regions with an area less than 800 (5000); I bin =morphology.remove_small_object(I bin (800); then perform a closing operation on it to smooth the internal and external contours of the tissue, resulting in a processed binary mask image, denoted as I. bin ;

[0075] For the obtained I bin Refill the small hole areas with an area less than 5000, I bin =morphology.remove_small_holes(I mask (,5000); to obtain the final binary mask image of the epithelial tissue.

[0076] This application combines shallow and deep features in the encoder, and then passes the combined features through the channel attention module and the spatial attention module respectively, finally encoding them into a feature vector and assigning it to the shallow features. Then, it merges the feature vector with the deep features through skip-connection. This application makes full use of the low-dimensional and high-dimensional features of the same image, encoding them into a feature vector through spatial and channel dimensions, which can more effectively extract and identify which features are needed.

[0077] It should be noted that, for the sake of simplicity, the aforementioned method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously.

[0078] Based on the same idea as the staining-conversion-based epithelial tissue segmentation method for pathological sections in the above embodiments, the present invention also provides a staining-conversion-based epithelial tissue segmentation system for pathological sections. This system can be used to perform the above-described staining-conversion-based epithelial tissue segmentation method for pathological sections. For ease of explanation, the structural schematic diagram of the embodiment of the staining-conversion-based epithelial tissue segmentation system only shows the parts related to the embodiments of the present invention. Those skilled in the art will understand that the illustrated structure does not constitute a limitation on the device, and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0079] like Figure 6 As shown, in another embodiment of this application, a pathological slide epithelial tissue segmentation system 100 based on staining conversion is provided. The system includes a preprocessing module 101, an image segmentation module 102, a network model training module 103, and a mask image acquisition module 104.

[0080] The preprocessing module 101 is used to acquire HE-stained images of the patient's original TMA pathological sections and perform preprocessing to obtain the effective tissue area of ​​the HE-stained image.

[0081] The image segmentation module 102 is used to segment the effective region tissue of the obtained HE staining image using a sliding window method to obtain multiple HE staining image blocks.

[0082] The network model training module 103 is used to input each HE staining image patch into the pre-trained staining style transfer network CS-Net to generate multiple corresponding IHC staining image patches.

[0083] The mask image acquisition module 104 is used to stitch together multiple IHC staining image blocks to obtain a pseudo IHC staining image of the same size as the original HE staining image, and to obtain a binary mask image of the epithelial tissue through color deconvolution, noise reduction processing and pixel threshold segmentation.

[0084] It should be noted that the pathological slide epithelial tissue segmentation system based on staining conversion of the present invention corresponds one-to-one with the pathological slide epithelial tissue segmentation method based on staining conversion of the present invention. The technical features and beneficial effects described in the embodiments of the pathological slide epithelial tissue segmentation method based on staining conversion described above are applicable to the embodiments of pathological slide epithelial tissue segmentation based on staining conversion. For details, please refer to the description in the embodiments of the method of the present invention, which will not be repeated here.

[0085] Furthermore, in the embodiments of the pathological slide epithelial tissue segmentation system based on staining conversion described above, the logical division of each program module is merely illustrative. In actual applications, the above functions can be assigned to different program modules as needed, for example, for the sake of corresponding hardware configuration requirements or the convenience of software implementation. That is, the internal structure of the pathological slide epithelial tissue segmentation system based on staining conversion can be divided into different program modules to complete all or part of the functions described above.

[0086] like Figure 7 As shown, in one embodiment, an electronic device is provided for implementing a stain-conversion-based epithelial tissue segmentation method for pathological sections. The electronic device 200 may include a first processor 201, a first memory 202, and a bus, and may also include a computer program stored in the first memory 202 and executable on the first processor 201, such as a stain-conversion-based epithelial tissue segmentation program 203 for pathological sections.

[0087] The first memory 202 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the first memory 202 can be an internal storage unit of the electronic device 200, such as the portable hard drive of the electronic device 200. In other embodiments, the first memory 202 can also be an external storage device of the electronic device 200, such as a plug-in portable hard drive, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device 200. Furthermore, the first memory 202 can include both internal storage units and external storage devices of the electronic device 200. The first memory 202 can be used not only to store application software and various types of data installed on the electronic device 200, such as the code of the staining and conversion pathological slide epithelial tissue segmentation program 203, but also to temporarily store data that has been output or will be output.

[0088] In some embodiments, the first processor 201 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The first processor 201 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the first memory 202 and calls data stored in the first memory 202 to perform various functions of the electronic device 200 and process data.

[0089] Figure 7 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 7 The structure shown does not constitute a limitation on the electronic device 200, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0090] The staining-converted pathological slide epithelial tissue segmentation program 203 stored in the first memory 202 of the electronic device 200 is a combination of multiple instructions. When run in the first processor 201, it can achieve the following:

[0091] HE-stained images of the patient's original TMA pathological sections were obtained and preprocessed to obtain the effective tissue area of ​​the HE-stained images;

[0092] The effective tissue region of the obtained HE staining image was segmented using the sliding window method to obtain multiple HE staining image patches.

[0093] Each HE staining image patch is input into a pre-trained staining style transfer network CS-Net to generate multiple corresponding pseudo IHC staining image patches.

[0094] Multiple pseudo-IHC staining image patches were spliced ​​together to obtain a pseudo-IHC staining image of the same size as the original HE staining image. Then, a binary mask image of the epithelial tissue was obtained by color deconvolution, noise reduction and pixel thresholding.

[0095] Furthermore, if the modules / units integrated in the electronic device 200 are implemented as software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0096] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0097] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0098] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for segmenting epithelial tissue in pathological sections based on staining conversion, characterized in that, Includes the following steps: HE-stained images of the patient's original TMA pathological sections were obtained and preprocessed to obtain the effective tissue area of ​​the HE-stained images; The effective tissue region of the obtained HE staining image was segmented using the sliding window method to obtain multiple HE staining image patches. Each HE staining image patch is input into a pre-trained staining style transfer network CS-Net to generate multiple corresponding IHC staining image patches. Multiple IHC staining image patches were spliced ​​together to obtain a pseudo IHC staining image with the same size as the original HE staining image. Then, a binary mask image of the epithelial tissue was obtained by color deconvolution, noise reduction and pixel thresholding. The style transfer network CS-Net is implemented by replacing the encoder of the U-Net network with the VGG16 network and inserting a new attention module, namely CS-Gate. The pre-trained coloring style transfer network CS-Net is specifically as follows: The segmented HE-stained image patches and the corresponding IHC-stained image patches are used as the source and target of the staining style transfer network, respectively, and are simultaneously input into the staining style transfer network for training. The VGG16 network extracts features from HE-stained images from shallow to deep layers, and then upsamples the extracted deep features step by step in the Decoder stage. Meanwhile, the Encoder stage has n layers. The features extracted in the i-th layer are refined by the CS-Gate and then combined with the ni-th layer of the Decoder through the skip-connection to become a new feature map, which is then passed to the next Decoder layer, i.e. the n-i+1-th layer, and so on until the last layer. The output feature map is then mapped by the Sigmoid to obtain the generated IHC staining map. The generated IHC staining image and the real IHC staining image are compared using L1Loss to calculate the loss, and the loss is backpropagated together to update the network parameters. The training continues until the loss stabilizes, at which point the training ends. The trained staining style transfer network was tested. The HE staining image was also divided into several HE staining image blocks using the sliding window method. Then, the segmented HE staining image blocks were input into the trained staining style transfer network for style transfer, and IHC staining image blocks corresponding to each HE staining image block were obtained. Specifically, CS-Gate combines shallow and deep features in the Encoder, then passes the combined features through the channel attention module and the spatial attention module, and finally encodes them into a feature vector which is then assigned to the shallow features. Finally, it merges the shallow features with the deep features through skip-connection.

2. The method for segmenting pathological section epithelial tissue based on staining conversion according to claim 1, characterized in that, The preprocessing involves color normalizing the HE staining image and then cropping it into a square HE staining image with equal length and width.

3. The method for segmenting pathological sections based on staining conversion according to claim 1, characterized in that, The sliding window method segmentation refers to using a sliding window of size 256*256 for a HE-stained image of size 2048*2048. Starting from the top left corner of the HE-stained image, the window slides forward from left to right and from top to bottom, with each slide being half the length of the window.

4. The method for segmenting pathological sections based on staining conversion according to claim 1, characterized in that, The process of stitching together multiple IHC staining image patches to obtain a pseudo-IHC staining image of the same size as the original HE staining image is as follows: The obtained IHC staining image patches are spliced ​​together in order from left to right and from top to bottom. For areas where multiple IHC staining image patches overlap, the lowest pixel value of all overlapping parts is selected. After the splicing operation is completed, a pseudo IHC staining image with the same size as the original HE staining image is obtained.

5. The method for segmenting pathological sections based on staining conversion according to claim 1, characterized in that, The process of obtaining a binary mask image of epithelial tissue through color deconvolution, noise reduction, and pixel threshold segmentation is as follows: The obtained pseudo-IHC staining image is denoted as I ihc Select the B channel from the RGB channels as the grayscale image, denoted as I. gray ; Set threshold T max =210, change I gray In the middle of 0 to T max Pixel values ​​between this range are set to 255, and pixel values ​​outside this range are set to 1, thus obtaining a binary mask image, denoted as I. mask In the binary mask image, white represents the foreground, indicating epithelial tissue; black represents the background, indicating the stroma and non-epithelial tissues on the slide. For the obtained I mask Noise reduction processing is performed, i.e., filling I mask For small pores with an area less than 5000, isolated regions with an area less than 800 are removed, and then closed. This smooths the internal and external contours of the tissue, resulting in a processed binary mask image, denoted as I. bin ; For the obtained I bin The small pore areas with an area less than 5000 are filled again to obtain the final binary mask image of the epithelial tissue.

6. A pathological section epithelial tissue segmentation system based on staining conversion, characterized in that, The method for segmenting epithelial tissue of pathological sections based on staining conversion, applicable to any one of claims 1-5, includes a preprocessing module, an image segmentation module, a network model training module, and a mask image acquisition module; The preprocessing module is used to acquire HE-stained images of the patient's original TMA pathological sections and perform preprocessing to obtain the effective tissue area of ​​the HE-stained image; The image segmentation module is used to segment the effective tissue region of the obtained HE staining image using a sliding window method to obtain multiple HE staining image blocks. The network model training module is used to input each HE staining image patch into a pre-trained staining style transfer network CS-Net to generate multiple corresponding IHC staining image patches. The mask image acquisition module is used to stitch together multiple IHC staining image blocks to obtain a pseudo IHC staining image of the same size as the original HE staining image, and to obtain a binary mask image of the epithelial tissue through color deconvolution, noise reduction processing and pixel threshold segmentation.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the staining conversion-based epithelial tissue segmentation method for pathological sections as described in any one of claims 1-5.

8. A computer-readable storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the epithelial tissue segmentation method for pathological sections based on staining conversion as described in any one of claims 1-5.

Images