Virtual staining method of unstained multispectral kidney cancer tissue HE

Abstract

The invention discloses an unstained multispectral kidney cancer tissue Hamp; the virtual staining method for the kidney cancer tissue slice comprises the following steps: S1, acquiring original data of the kidney cancer tissue slice white slice; s2, preprocessing the original data to obtain a standardized multispectral image; s3, obtaining a corresponding Hamp through an experiment; e, dyeing the slice image as a reference; s4, constructing a standard data set and inputting the standard data set into the deep learning model for training; s5, inputting the data set into the SwinIR-UNet fusion network, and after inputting an undyed multispectral image, outputting the data set and the Hamp; and E, dyeing the target image with the consistent style. S6, training is finished, and kidney cancer tissue Hamp is obtained; and E, virtualizing the dyeing model. The method fuses a global attention mechanism and an encoder-decoder structure, has excellent image conversion capability and robustness, and can effectively restore the dyeing effect, improve the diagnosis efficiency and reduce time consumption, tissue damage and personal errors in traditional dyeing.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of malignant tumor pathological diagnosis, and in particular to a virtual staining method of unstained multi-spectral kidney cancer tissue H&E. BACKGROUND

[0002] Renal cell carcinoma is a common malignant tumor originating from renal tubular epithelial cells, accounting for more than 90% of adult urinary system tumors, and the incidence is increasing year by year. In clinical practice, the final diagnosis of renal cell carcinoma is highly dependent on histopathological analysis, and hematoxylin and eosin (H&E) staining, one of the commonly used staining methods in paraffin section technology, is the most basic and widely used technique in histology, embryology, and pathology teaching and research. It is the current gold standard for the diagnosis of renal cell carcinoma. H&E staining can clearly show the structure of the cell nucleus and cytoplasm, providing important morphological basis for doctors to distinguish between benign and malignant tumors. However, the traditional H&E staining process has many limitations. The operation steps are complex, requiring deparaffinization, hydration, staining, dehydration, and mounting, which are highly dependent on the technical level of professional personnel. The concentration of the staining solution, the staining time, and environmental factors can significantly affect the staining results, leading to unstable slice quality and affecting the consistency and accuracy of the diagnosis. At the same time, the staining process usually takes 30 minutes to several hours, which is not conducive to intraoperative rapid diagnosis and high-throughput processing. In addition, the chemical reagents used in staining are toxic and corrosive, posing potential risks to the laboratory environment and the health of the operators. Therefore, there is an urgent need for a faster, more stable, and safer alternative to achieve visualization and pathological analysis of renal cancer tissue. Therefore, the construction of a virtual staining method for renal cancer tissue based on multi-channel image input and deep neural networks not only effectively overcomes the shortcomings of traditional H&E staining, but also provides a new solution for pathological image analysis, with important scientific research value and clinical significance SUMMARY In order to overcome the shortcomings and deficiencies of the prior art, the present application provides a virtual staining method for unstained multi-spectral kidney cancer tissue H&E.

[0003] The technical solution adopted by the present application is a virtual staining method for unstained multi-spectral kidney cancer tissue H&E, comprising the following steps: Step S1: Obtain the original data of the kidney cancer tissue section white sheet; Step S2: Preprocess the original data to obtain standardized multi-spectral images; Step S3: Obtain the corresponding H&E stained section image as a reference through experiments; Step S4: Construct a standard data set and input it into a deep learning model for training; Step S5: Input the dataset into the SwinIR-UNet fusion network. After inputting the unstained multispectral image, it outputs the target image consistent with the H&E staining style to complete the virtual staining model training. Step S6: After the training is completed, the H&E virtual staining model of kidney cancer tissue is obtained.

[0004] In step S1, a total of 2696 white film images are provided by the hospital, including kidney tissue slice images.

[0005] In step S2, multispectral image acquisition of each white slice is performed using a hyperspectral imaging device with imaging capabilities in the visible to near-infrared band (400nm–1000nm), namely, a Specim IQ hyperspectral camera. The tissue slices are placed in a customized light source device with multiple adjustable light sources for imaging, and spectrally balanced illumination is used to improve image acquisition consistency. The obtained original multispectral image undergoes normalization and spectral alignment of each channel to eliminate spectral deviations caused by different batches or lighting conditions, and finally a standardized multispectral image is obtained for subsequent training and inference of the virtual staining network.

[0006] In step S3, the H&E stained image obtained as a reference image is obtained by staining a paraffin-embedded renal cancer tissue section of the same tissue source as that in step S2, wherein the thickness of the tissue section is 4 μm or 10 μm. The staining step includes: Step B1: After dewaxing and rehydrating the tissue sections, unstained images were collected using the designed computational microscopy imaging system; Step B2: Stain the sections in hematoxylin solution for 5 to 7 minutes, differentiate in hydrochloric acid-alcohol solution for 1 second, rinse with running tap water, and then rinse with distilled water; Step B3: Return to blue in warm water at 49°C for 8 minutes. Observe under a microscope to confirm that the cell nucleus is dark blue and the cytoplasm is unstained. Step B4: Stain the sections in eosin solution for 8 minutes, rinse with tap water for 3 minutes, and rinse again with distilled water; Step B5: Dehydrate in 85%, 95% and 100% ethanol in sequence, with each step lasting 3 seconds. After natural air drying, use neutral quick-drying mounting medium to seal the slide. The obtained H&E staining image is used for subsequent training and verification of the virtual staining model.

[0007] In step S4, the feature extraction, information fusion and RGB virtual staining image generation of the unstained kidney cancer tissue multispectral image are realized by constructing a multi-branch fusion deep learning model; the multi-branch fusion deep learning model takes the 8-channel unstained multispectral image as input, the first 4 channels are sent to the first branch network G_A for global modeling, and the last 4 channels are sent to the second branch network G_B for extracting local structure information, and two 4-channel feature maps are output respectively; the two feature maps are spliced in the channel dimension and then input into the fusion module to further extract cross-scale information and complete the reconstruction of the RGB three-channel virtual staining image; the model adopts an end-to-end training mode, the input is the unstained multispectral image, and the output is a virtual image close to the real H&E staining image, which is supervised by multiple loss functions to improve the color accuracy and tissue structure restoration ability of the generated image.

[0008] In step S4, the first branch network is trained. The first branch network G_A is used to receive the first 4 spectral channels of the input image, first extracts initial texture features through a shallow feature extraction module, and then performs deep feature modeling through a preset number of Swin Transformer modules; each Swin Transformer module includes a local window self-attention mechanism and a sliding window strategy, which can model the long-range dependency relationship between different regions and improve the expression ability of the global color structure; the multi-head attention mechanism and multi-scale residual connection structure are introduced inside the module, so that the network can focus on the key features of the staining area while maintaining the spatial resolution of the image, and finally output a feature map with an input size and a channel number of 4; this branch can effectively extract the spatial spectral distribution of kidney cancer tissue under different wavebands and enhance the model's ability to distinguish the details between tissues in virtual staining.

[0009] In step S4, the second branch network is trained. The second branch network receives the last 4 spectral channels of the input image, and the overall structure adopts an encoder-decoder framework; the encoder part is composed of a preset number of convolution blocks and a maximum pooling layer, which extracts the edge features and local textures of the image layer by layer; the decoder part gradually restores the image resolution through the deconvolution layer, and performs skip connection with the features of the same scale in the encoder to enhance the information fusion of shallow and deep features; the network finally outputs a feature map with the same size as the input image and a channel number of 4, which highlights the expression of key anatomical information such as cell morphology and tissue structure outline in H&E staining; this branch enhances the model's perception of pathological tissue structure details, which helps to improve the realism and diagnosability of the virtual staining image.

[0010] In step S5, the two 4-channel feature maps output by the SwinIR branch and the U-Net branch are spliced in the channel dimension to form an 8-channel fusion feature map; the fusion feature map is input into a convolution fusion module, which is composed of a preset number of 3x3 convolution layers, an activation function and a normalization layer, and finally outputs a 3-channel RGB image with the same size as the virtual staining image; in the training process, an L1 loss function is used to constrain the pixel error between the generated image and the real staining image; a perceptual loss function is introduced to compare the similarity of images in the high-level semantic space through a pre-trained feature network (such as VGG), so as to improve the color and texture performance; further, a multi-scale structural similarity loss (MSSSIM Loss) is introduced in the training process to measure the structural consistency of the generated image and the reference staining image at a preset number of scales, so as to improve the detail restoration ability and overall perception quality of the virtual staining image, and enhance the structural clarity and visual authenticity of the image. This loss can effectively suppress image blur and avoid the shortcomings of traditional loss functions in handling texture details and organizational boundaries; the training uses the Adam optimizer with a learning rate decay mechanism, and iterates on the multi-scale dataset until the model converges and stably generates virtual staining images highly consistent with the real H&E images.

[0011] Beneficial effects: The present application provides a kind of based on SwinIR-UNet structure's un-dyed multi-spectral kidney cancer tissue H&E virtual staining method, the virtual staining method proposed in the present application inputs the first 4 channels of multi-spectral image into SwinIR module, utilizes its hierarchical attention mechanism based on Swin Transformer, can effectively extract long-distance dependent features and spatial structure information in image, enhance the perception ability of model to tumor tissue global characteristics, improve the structural consistency and expression accuracy of staining image. The last 4 channels are input into UNet module, which can effectively capture local texture and edge detail information of the image by means of its symmetric encoder-decoder structure, enhance the recognition ability of the model to the microstructure in pathological section, help to reconstruct high-quality virtual staining images with clear organizational boundaries and morphological characteristics. By fusing the features extracted by SwinIR and UNet modules, global and local information is comprehensively utilized to effectively improve the authenticity and detail retention of the generated image. The model design fully considers the band characteristics of multi-spectral imaging, so that the spectral information of different channels is fully utilized in the network, further improving the color restoration and tissue differentiation ability of the virtual staining result. The present method does not require traditional H&E staining process, and can directly realize high-fidelity virtual staining based on un-dyed multi-spectral images, simplifying the experimental process, saving staining cost, and avoiding tissue damage or information loss caused by staining steps, suitable for auxiliary diagnosis and image analysis tasks of renal cancer and other histopathology, with good clinical application prospect and popularization value. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 Flow chart of method steps of the present application; Figure 2 Network structure diagram of virtual staining algorithm of the present application; Figure 3 Black and white image of 420nm waveband of multispectral unstained kidney cancer tissue section of the present application; Figure 4 H&E virtual staining image generated based on the multi-branch fusion model of the present application; Figure 5 Real H&E staining image of the present application. DETAILED DESCRIPTION

[0013] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict, and the present application will be further described in detail below in combination with the drawings and specific embodiments.

[0014] As shown in Figure 1 An unstained multispectral kidney cancer tissue H&E virtual staining method, comprising the following steps: Step S1: Obtain kidney cancer tissue section white sheet original data, a total of 2696 white sheet images used are provided by a hospital, including kidney tissue section images.

[0015] Step S2: Preprocess the original data to obtain standardized multispectral images. The multispectral image acquisition of each white sheet section is performed by a hyperspectral imaging device with visible light to near-infrared waveband (400nm-1000nm) imaging capability, the device is a Specim IQ hyperspectral camera; the tissue section is placed in a customized light source device with multiple adjustable light sources for shooting, and the image acquisition consistency is improved through spectral equalization lighting; the obtained original multispectral images are subjected to normalization processing and spectral alignment of each channel to eliminate spectral deviations under different batches or lighting conditions, and finally standardized multispectral images are obtained for subsequent virtual staining network training and inference.

[0016] Step S3: Obtain corresponding H&E stained section images as a reference through experiments. It is obtained by staining paraffin-embedded kidney cancer tissue sections from the same source as step S2, the thickness of the tissue section is 4μm or 10μm, and the staining steps include: Step B1: After dewaxing and rehydrating the tissue section, use the designed computational microscopy imaging system to collect the unstained image; Step B2: Stain the section in hematoxylin solution for 5 to 7 minutes, differentiate with hydrochloric acid alcohol for 1 second, then rinse with flowing tap water and again with distilled water; Step B3: return blue for 8 minutes in warm water at 49°C, and confirm under a microscope that the cell nucleus is dark blue and the cytoplasm is not stained; Step B4: stain the slice in an eosin solution for 8 minutes, then rinse with tap water for 3 minutes, and rinse again with distilled water; Step B5: dehydrate in 85%, 95%, and 100% ethanol, each for 3 seconds, then use neutral quick-drying mounting agent for mounting after natural air drying, and the obtained H&E staining image is used for subsequent training and verification of the virtual staining model.

[0017] Step S4: construct a standard data set and input it into a deep learning model for training.

[0018] Step S5: input the data set into the SwinIR-UNet fusion network, output a target image consistent with the H&E staining style after inputting an unstained multispectral image, and complete the virtual staining model training. The SwinIR-UNet model mainly consists of two branch structures, as shown in the network structure diagram Figure 2 , and the SwinIR-Unet mainly consists of G_A and G_B. The SwinIR-UNet model takes an 8-channel unstained multispectral image as input, sends the first 4 channels to the first branch network G_A for global modeling, and sends the last 4 channels to the second branch network G_B to extract local structural information, respectively outputting two 4-channel feature maps; the two feature maps are concatenated in the channel dimension and then input into the fusion module to further extract cross-scale information and complete the reconstruction of the RGB three-channel virtual staining image; the model uses an end-to-end training method, the input is an unstained multispectral image, and the output is a virtual image close to the real H&E staining image, which is supervised by multiple loss functions to improve the color accuracy and tissue structure restoration ability of the generated image.

[0019] For the training process of the first branch network G_A: the first branch network G_A is used to receive the first 4 spectral channels of the input image, first extracts initial texture features through a shallow feature extraction module, and then performs deep feature modeling through a preset number of SwinTransformer modules; each Swin Transformer module includes a local window self-attention mechanism and a sliding window strategy, which can model the long-range dependency relationship between different regions and improve the expression ability of the global color structure; the multi-head attention mechanism and multi-scale residual connection structure are introduced inside the module, so that the network can focus on the key features of the staining area while maintaining the spatial resolution of the image, and finally output a feature map with an input size of 4 and a channel number of 4; this branch can effectively extract the spatial spectral distribution of kidney cancer tissue under different wavebands, enhancing the model's ability to distinguish the details of the virtual staining between tissues.

[0020] For the second branch network training process: the second branch network receives the last 4 spectral channels of the input image, and the overall structure adopts an encoder-decoder framework; the encoder part is composed of a preset number of convolution blocks and a maximum pooling layer, which extracts the edge features and local textures of the image layer by layer; the decoder part gradually restores the image resolution through the deconvolution layer, and performs skip connection with the features of the same scale in the encoder to enhance the information fusion of shallow and deep features; the network finally outputs a feature map with the same size as the input image and 4 channels, highlighting the key anatomical information such as cell morphology and tissue structure outline in H&E staining; the perception ability of this branch enhanced model for pathological tissue structure details helps to improve the realism and diagnosability of the virtual staining image.

[0021] In the training stage, the SwinIR-UNet fusion neural network constructed by the application takes an 8-channel original multispectral image with consistent size as input, which covers the spectral information of 420nm, 460nm, 540nm, 660nm, 780nm, 860nm, 900nm and 960nm, a total of 8 wavebands. As shown in Figure 3 , it comprehensively reflects the spectral response characteristics of kidney cancer tissue in the visible light to near-infrared range.

[0022] In order to realize the collaborative modeling of different scale features in the image and the generation of high-quality virtual staining images, the network structure adopts a double-branch design strategy. The first four wavebands (420nm to 660nm) are fused as a tensor input to the first branch G_A for global context semantic feature extraction; the last four wavebands (780nm to 960nm) are fused as input to the second branch G_B for local edge and detail modeling. This branch strategy fully considers the tissue reflection and absorption characteristics corresponding to different wavebands, and realizes the complementary fusion of frequency domain and spatial domain information.

[0023] The first branch G_A adopts a stacked Swin Transformer module as the core backbone. The input tensor is first subjected to a 3x3 convolution layer for preliminary feature extraction, then introduced into a non-linear mapping capability through a ReLU non-linear activation function, and subjected to layer normalization to standardize the channel distribution at each position. Subsequently, a preset number of residual Swin Transformer blocks (RSTB) are entered, each block including local window partitioning, multi-head self-attention (Multi-Head Self-Attention), window shifting mechanism (Shifted Window), feedforward network (MLP), and residual connection module, ensuring that the network has strong context capture and deep semantic understanding capabilities.

[0024] The second branch G_B is constructed as a typical UNet structure. First, shallow features are extracted from the input image through two successive 3x3 convolution operations with ReLU activation. Then, the image enters the encoder part, which performs two levels of down-sampling through max-pooling layers. Each level of down-sampling includes two convolution layers combined with ReLU activation function modules, which extract higher-level middle and deep features. The decoder part realizes gradual up-sampling of spatial resolution through transposed convolution (Transposed Convolution), and performs Skip Connection with the feature maps of the symmetric layers in the encoder at each level to ensure that the shallow texture and edge information are retained. After each level of decoding, two convolution + ReLU modules are still used to enhance the expression ability, and finally the image is restored to the same spatial size as the input.

[0025] The two branches output middle feature maps of the same size, which are then concatenated in the channel dimension (Channel Axis) to form a fused feature map. The fused tensor is input into the fusion module, which includes several 3x3 convolution layers connected in series. After each convolution operation, a ReLU activation function is connected in sequence to enhance the non-linear expression ability.

[0026] As shown in Figure 4 , the generated image can accurately restore the tissue structure and staining features, and has good visualization performance. As shown in Figure 4 , the lumen structure in the image is complete, the edge profile is clear and smooth, and there are no obvious artifacts or blurred areas, fully demonstrating the reconstruction ability of the present application in texture restoration, boundary reconstruction and local detail preservation. The structural differences between different tissue regions are clearly distinguishable in the image, which is convenient for subsequent pathological analysis and diagnosis, indicating that the present method has significant advantages in image expression quality and clinical practicability. Further, as shown in Figure 4 , the generated image is highly consistent with the real H&E stained image in terms of cell morphology, tissue structure level and boundary detail presentation, verifying the accuracy and stability of the present application in pathological image restoration in the virtual staining task, and demonstrating good technical feasibility and application prospect. In the model training process, the present application uses an L1 pixel-level loss function as a basic constraint means, as shown in Figure 5 , to ensure that the generated image is highly consistent with the corresponding real H&E stained image at the pixel level, thereby effectively improving the overall restoration accuracy of the image.

[0027] Meanwhile, a perception loss function is introduced to compare the high-level semantic feature distribution through a pre-trained classification network such as VGG, so as to improve the performance of the image in terms of texture, color and perceptual quality; in addition, a multi-scale structural similarity loss (MSSSIM) is further introduced to evaluate the fidelity of the generated result on the preset number of spatial scales from the perspective of structural consistency, effectively improving the clarity of the organizational boundary and the overall visual authenticity. In order to improve the training stability and convergence speed, the Adam optimizer is used in the training process, and a dynamic learning rate decay mechanism is set to ensure continuous optimization of network performance on multi-scale and multi-sample training sets, and finally realize the generation effect of the stained image in color restoration, organizational structure identification and visual expression, which is highly consistent with the real H&E image.

[0028] The virtual staining method provided by the application fully excavates the global structural features and local texture information of the image by inputting different band information of the multispectral image into two structural modules of SwinIR and UNet respectively. The SwinIR module enhances the modeling ability of the model for the spatial layout of the tumor tissue and the long-range dependence relationship by means of the hierarchical self-attention mechanism, and improves the performance of the synthesized image in terms of structural fidelity. The UNet module focuses on the extraction of local details, effectively retains the organizational edge and microscopic morphological features, so that the staining result is clearer and more real. The design of the two branch feature fusion makes the model achieve a good balance between global perception and local recognition, and the generated image performs excellently in terms of color restoration, detail integrity and organizational recognition. In addition, the method directly uses the multispectral image for virtual staining, which eliminates the experimental operation and chemical reagents required by traditional H&E staining, reduces the time and economic cost, and reduces the potential damage to the tissue sample, which is helpful to realize a more efficient and safe digital pathology analysis process, especially suitable for auxiliary diagnosis and research work of major diseases such as kidney cancer, and has significant practical application value and popularization potential.

[0029] Although the embodiments of the present application have been shown and described, it can be understood by those skilled in the art that various equivalent changes, modifications, replacements and variations of the embodiments can be made without departing from the principles and spirits of the present application, and the scope of the present application is defined by the appended claims and their equivalent ranges.

Claims

1. A virtual staining method for unstained multispectral renal cancer tissue H&E, characterized in that: The following steps are involved: Step S1: obtaining raw data of white slices of kidney cancer tissue sections; Step S2: preprocessing the raw data to obtain a standardized multispectral image; Step S3: Obtain the corresponding H&E stained slice image as a reference through experiments; Step S4: Construct a standard dataset and input it into the deep learning model for training; Step S5: Input the dataset into the SwinIR-UNet fusion network. After inputting the unstained multispectral image, it outputs the target image consistent with the H&E staining style to complete the virtual staining model training. Step S6: After the training is completed, the H&E virtual staining model of kidney cancer tissue is obtained.

2. The virtual staining method for unstained multispectral renal cancer tissue H&E according to claim 1, characterized in that: In step S1, a total of 2696 white film images are provided by the hospital, including kidney tissue slice images.

3. The virtual staining method for unstained multispectral renal cancer tissue H&E according to claim 1, characterized in that: In step S2, multispectral image acquisition of each white slice is performed using a hyperspectral imaging device with imaging capabilities in the visible to near-infrared band (400nm–1000nm), such as a Specim IQ hyperspectral camera. The tissue slices are placed in a customized light source device with multiple adjustable light sources for imaging, and spectrally balanced illumination is used to improve image acquisition consistency. The original multispectral images obtained are normalized and spectrally aligned for each channel to eliminate spectral deviations caused by different batches or lighting conditions, and obtain standardized multispectral images for subsequent training and inference of the virtual staining network.

4. The virtual staining method for unstained multispectral renal cancer tissue H&E according to claim 1, characterized in that: The H&E stained image obtained in step S3 as a reference image is obtained by staining a paraffin-embedded renal cancer tissue section of the same tissue source as that in step S2, wherein the thickness of the tissue section is 4 μm or 10 μm. The staining step includes: Step B1: After dewaxing and rehydrating the tissue sections, unstained images were collected using the designed computational microscopy imaging system; Step B2: Stain the sections in hematoxylin solution for 5 to 7 minutes, differentiate in hydrochloric acid-alcohol solution for 1 second, rinse with running tap water, and then rinse with distilled water; Step B3: Return to blue in warm water at 49°C for 8 minutes. Observe under a microscope to confirm that the cell nucleus is dark blue and the cytoplasm is unstained. Step B4: Stain the sections in eosin solution for 8 minutes, rinse with tap water for 3 minutes, and rinse again with distilled water; Step B5: Dehydrate in 85%, 95% and 100% ethanol in sequence, each step lasting 3 seconds. After natural air drying, use neutral quick-drying mounting medium to seal the slide. The obtained H&E staining image is used for subsequent training and verification of the virtual staining model.

5. The virtual staining method for unstained multispectral renal cancer tissue H&E according to claim 1, characterized in that: In step S4, a multi-branch fusion deep learning model is constructed to perform feature extraction, information fusion, and RGB virtual stained image generation on the unstained multispectral image of renal cancer tissue; the multi-branch fusion deep learning model takes 8 bands of unstained multispectral images as input, including 420nm, 460nm, 540nm, 660nm, 780nm, 860nm, 900nm, and 960nm, and fuses the first 4 bands into the first branch network G_A for global modeling, and fuses the last 4 bands into the second branch network G_B to extract local structural information, and outputs two 4-channel feature maps respectively; the two feature maps are spliced ​​in the channel dimension and then enter the fusion module to further extract cross-scale information and complete the reconstruction of the RGB three-channel virtual stained image; the multi-branch fusion deep learning model adopts an end-to-end training method, with the input being the unstained multispectral image and the output being a virtual image close to the real H&E stained image, and is jointly supervised by a loss function.

6. The virtual staining method for unstained multispectral renal cancer tissue H&E according to claim 5, characterized in that: The first branch network is constructed based on the SwinIR structure and includes: The first branch network G_A is used to receive the first four spectral channels of the input image, extract the initial texture features through the shallow feature extraction module, and perform deep feature modeling through a preset number of Swin Transformer modules; each Swin Transformer module includes a local window self-attention mechanism and a sliding window strategy to model the long-range dependencies between different regions; a multi-head attention mechanism and a multi-scale residual connection structure are introduced within the module, and the network focuses on the key features of the stained area while maintaining the spatial resolution of the image, and outputs a feature map that maintains the input size and has a channel number of 4.

7. The virtual staining method for unstained multispectral renal cancer tissue H&E according to claim 5, characterized in that: The second branch network G_B is constructed using a U-Net structure, specifically including: The second branch network receives the last four spectral channels of the input image, and the overall structure adopts an encoder-decoder framework; the encoder part consists of a preset number of convolution blocks and maximum pooling layers, which extracts the edge features and local texture of the image layer by layer; the decoder part gradually restores the image resolution through deconvolution layers, and makes jump connections with the features of the same scale in the encoder to enhance the information fusion of shallow and deep features; the network finally outputs a feature map with the same size as the input image and the number of channels of 4, expressing key anatomical information in H&E staining, such as cell morphology and tissue structure contours.

8. The virtual staining method for unstained multispectral renal cancer tissue H&E according to claim 1, characterized in that: The multi-branch fusion deep learning model training process includes: The two 4-channel feature maps output by the SwinIR branch and the UNet branch are spliced ​​in the channel dimension to form an 8-channel fused feature map; the fused feature map is input into the convolutional fusion module, which consists of a preset number of 3×3 convolutional layers, activation functions and normalization layers, and outputs a 3-channel RGB image of unchanged size as a virtual stained image; during the training process, the L1 loss function is used to constrain the pixel error between the generated image and the real stained image; the perceptual loss function is introduced to compare the similarity of images in the high-level semantic space through the pre-trained feature network; during the training process, multi-scale structural similarity loss is introduced to measure the structural consistency between the generated image and the reference stained image at a preset number of scales. The training adopts the Adam optimizer with a learning rate decay mechanism, and iterates repeatedly on the multi-scale dataset until the model converges and stably generates virtual stained images that are highly consistent with the real H&E images.

Images