Self-attention enhanced multiscale histological section immunofluorescence virtual staining method

Abstract

The application provides a self-attention enhanced multiscale tissue section immunofluorescence virtual staining method, and relates to the technical field of a self-attention enhanced UNet++ network, and comprises the following steps: inputting a pretreated staining image into a constructed self-attention enhanced UNet++ network to perform deep feature extraction, processing a deep feature map by adopting a constructed AG-SA module, generating a gating fusion feature map, performing multiscale feature fusion on an original encoding feature map generated by an encoder, performing full-coupling optimization on a decoding map, performing Gaussian weighting splicing on a multi-channel fluorescence prediction probability map output, and performing morphological optimization on the spliced image. By constructing the self-attention enhanced UNet++ network and introducing the AG-SA module, the global self-attention feature and the local convolution feature are adaptively fused by the local variance gating in the deep feature extraction stage, and the modeling capability of the network for the complex structure and the long-range dependence relationship of the tissue morphology is significantly enhanced.

Description

Technical Field

[0001] This invention relates to the field of self-attention-enhanced UNet++ network technology, specifically to a self-attention-enhanced multi-scale tissue section immunofluorescence virtual staining method. Background Technology

[0002] In histopathological analysis and diagnosis, multiplex immunofluorescence staining is highly regarded because it can simultaneously label multiple biomarkers on a single tissue section, revealing the spatial heterogeneity of the tumor immune microenvironment. However, traditional multiplex immunofluorescence staining procedures are extremely cumbersome, time-consuming, and costly, typically requiring multiple rounds of antigen retrieval, antibody elution, and restaining. This not only easily causes tissue section damage but is also limited by spectral overlap and signal crosstalk issues of fluorescent dyes, making it difficult to achieve high-throughput, stable multi-target detection. Existing deep learning-based virtual staining methods, such as the standard U-Net and its variant UNet++, while enhancing the fusion of multi-scale features through nested skip connections and deep supervision mechanisms when processing tissue slice images, still primarily rely on convolutional operations to capture local features, making it difficult to effectively model long-range dependencies and global contextual information between different regions in tissue slices. Furthermore, existing methods often neglect the signal independence between different fluorescence channels when generating multiple fluorescence images, leading to channel crosstalk between different biomarkers. Simultaneously, for block prediction of large-size tissue slice images, existing methods lack effective stitching optimization strategies, easily producing obvious boundary artifacts and stitching marks. Therefore, a self-attention-enhanced multi-scale tissue slice immunofluorescence virtual staining method is urgently needed.

[0003] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] The purpose of this invention is to provide a self-attention-enhanced multi-scale tissue section immunofluorescence virtual staining method to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: The self-attention-enhanced multi-scale tissue section immunofluorescence virtual staining method includes the following steps: S1: Obtain the original stained image of the tissue section and perform preprocessing. Input the preprocessed stained image into the constructed self-attention enhanced UNet++ network for deep feature extraction. In each coding layer, the constructed AG-SA module is used to process the deep feature map extracted in the current layer to generate a gated fusion feature map. The gated fusion feature map is downsampled to output the original coding feature map of the corresponding layer, which is then used as the input of the next coding layer. S2: Perform multi-scale feature fusion on the original encoded feature maps output by each layer of the encoder, and then combine the fused result map with the original encoded features of the corresponding layer. Figure 1 The data is then input into the decoder for decoding to obtain a decoded graph. Fully coupled optimization is performed on the decoded graph, and the total loss parameter is further calculated. S3: Construct a training dataset containing multiple pairs of preprocessed staining images and real multiplex immunofluorescence staining images. Randomly sample a batch of paired samples from the training dataset and input the original staining images into the attention-enhanced UNet++ network. Calculate the total loss parameters according to steps S1 and S2. Process the total loss parameters to obtain the learnable parameters of the attention-enhanced UNet++ network. S4: Based on learnable parameters, predict the image of the preprocessed tissue slice to be detected to obtain a multi-channel fluorescence prediction probability map. Perform Gaussian weighted stitching on the multi-channel fluorescence prediction probability map, perform morphological optimization on the stitched image, and output the final virtual fluorescence image.

[0006] Further, the original stained images of the tissue sections are obtained and preprocessed, specifically as follows: The original stained images of tissue sections are obtained, and their RGB channels are separated. The mean and standard deviation of pixels in each channel are calculated, and the pixel values ​​of each channel are normalized based on the mean and standard deviation. Wavelet domain adaptive denoising is performed on the normalized image, and multi-scale contrast enhancement is performed on the denoised image to obtain the preprocessed stained image.

[0007] Furthermore, a self-attention enhanced UNet++ network is constructed to extract deep features from the preprocessed stained image, specifically as follows: The preprocessed stained image is first subjected to a first-level convolution extraction, that is, using... The system processes features using convolution, batch normalization, and ReLU activation to output the feature map extracted by the first-level convolutional layer. Then, a second-level deep feature extraction is performed on this feature map. Convolution, batch normalization, and ReLU activation function processing are used to obtain feature maps for secondary deep feature extraction.

[0008] Furthermore, the constructed AG-SA module is used to process the depth feature map extracted from the current layer to generate a gated fusion feature map, specifically including: The feature map extracted by the second depth feature extraction is divided into multiple non-overlapping local feature blocks in the height and width directions, and the local variance of each local feature block is calculated separately. The local variances corresponding to each local feature block are arranged according to their spatial positions in the feature map of the secondary deep feature extraction to form a single-channel statistical feature map. The single-channel statistical feature map is then subjected to convolution and activation processing to generate a gating map, which is used to characterize the feature response intensity of each local region. The feature map extracted by the second deep feature extraction is reconstructed to obtain a two-dimensional feature matrix, and query features, key features and value features are obtained by three learnable linear transformations respectively. Self-attention operations are performed based on the query features, key features, and value features to obtain a self-attention feature map; The self-attention feature map is restored to the spatial structure corresponding to the feature map extracted by the second deep feature extraction. The self-attention feature map and the feature map extracted by the second deep feature extraction are then weighted and fused based on the gating map to obtain a gating fused feature map. The weighted fusion is used to adaptively adjust the fusion ratio of the self-attention feature and the original feature according to the statistical features of the local region.

[0009] Furthermore, the original encoded feature map is output, specifically: First, the gated fusion feature map is downsampled to output the first-layer encoded feature map, i.e., a map of size [size missing]. Max pooling is used to downsample the gated fusion feature map to obtain the first layer of encoded feature map; Then, construct the second layer of encoded feature maps: pass the first layer of encoded feature maps sequentially through convolution kernels with a kernel size of... The process involves convolution, batch normalization, and ReLU activation, with 128 convolution kernels; then, it sequentially passes through convolution kernels of size [missing information]. The convolution, batch normalization, and ReLU activation functions are applied, with the number of convolution kernels also being 128. The result is then input into the AG-SA module for further processing, yielding the output of the second-layer AG-SA module. Finally, a kernel of size [size missing] is used... Max pooling is used to downsample the output to obtain the second layer of encoded feature map.

[0010] Next, the third layer of encoded feature maps is constructed: the second layer of encoded feature maps is sequentially processed through convolution kernels with a kernel size of [size missing]. The process involves convolution, batch normalization, and ReLU activation, with a kernel size of 256; then it sequentially passes through convolution kernels of size [missing information]. The convolution, batch normalization, and ReLU activation functions are applied, with the number of convolution kernels also being 256. The result is then input into the AG-SA module for further processing, yielding the output of the third-layer AG-SA module. Finally, a kernel of size [size missing] is used... Max pooling is used to downsample the output to obtain the third layer of encoded feature map.

[0011] Finally, construct the fourth layer of encoded feature maps: pass the third layer of encoded feature maps sequentially through convolution kernels with a kernel size of [size missing]. The process involves convolution, batch normalization, and ReLU activation, with a kernel size of 512; then it sequentially passes through convolution kernels of size [missing information]. The convolution, batch normalization, and ReLU activation functions are applied, with the number of convolution kernels also being 512. The result is then input into the AG-SA module for processing, resulting in the output of the fourth-layer AG-SA module. This output is the fourth-layer encoded feature map, and no further downsampling is performed.

[0012] Furthermore, multi-scale feature fusion is performed on the original encoded feature maps output from each layer of the encoder, specifically including: The original encoded feature maps of each layer are subjected to average pooling at different scales to obtain the corresponding multi-scale pooling features. Multi-scale pooling features of different scales are uniformly adjusted to a spatial size that matches the corresponding coding layer; The multi-scale pooling features with uniform size are spliced ​​together according to the channel dimension to form a multi-scale fused feature map; The multi-scale fused feature map is subjected to convolutional compression and feature integration to obtain the multi-scale feature fusion result map of the corresponding layer.

[0013] Furthermore, the decoded image is obtained, specifically: The multi-scale feature fusion result map output by the encoder and the original encoded feature map of the corresponding layer are input into the decoder for layer-by-layer decoding. Layer-by-layer decoding includes: starting from deep features, upsampling or resizing the current decoded features to make them match the spatial dimensions of the multi-scale feature fusion result map of the adjacent shallow layers and the original encoded feature map; The size-matched features are fused with the multi-scale feature fusion result map of the corresponding layer and the original encoded feature map, and the decoded feature map of the current layer is generated by feature transformation. The decoding feature maps of each decoding layer are obtained layer by layer until the final decoding map is obtained.

[0014] Furthermore, the total loss parameter is output as follows: The decoded image is transformed by channel transformation so that the number of output channels corresponds to the number of target fluorescence channels to obtain the original prediction result. The original prediction result is then activated and mapped to obtain the prediction probability map of each fluorescence channel. Based on the predicted probability map and the actual multiplex immunofluorescence staining image, the weighted cross-entropy loss is calculated; Based on the correlation between each fluorescence channel, the channel penalty loss is calculated; Based on the structural similarity between the predicted probability map and the actual multiplex immunofluorescence staining image, the structural similarity loss is calculated; The weighted cross-entropy loss, channel penalty loss, and structural similarity loss are weighted and combined to obtain the total loss parameter; A training dataset containing multiple pairs of preprocessed stained images and real multiplex immunofluorescence stained images is constructed. Paired samples are randomly sampled from the training dataset and input into the self-attention-enhanced UNet++ network. The learnable parameters of the network are iteratively updated according to the total loss parameter until the preset training termination condition is reached, and the trained self-attention-enhanced UNet++ network is obtained.

[0015] Furthermore, the output multi-channel fluorescence prediction probability map is Gaussian weighted and stitched together, specifically as follows: Based on the learnable parameters of the self-attention-enhanced UNet++ network obtained through training, predictions are made for the preprocessed tissue slice images to be detected. The preprocessed tissue slice image to be detected is divided into multiple image sub-blocks, and each is input into the self-attention enhanced UNet++ network to obtain the multi-channel fluorescence prediction probability map corresponding to each image sub-block. Based on the spatial location of each image sub-block in the original tissue slice image, the multi-channel fluorescence prediction probability map corresponding to each image sub-block is mapped back to the complete image; The prediction results in adjacent or overlapping regions are weighted and fused using Gaussian weights, where the weight of the central region of the image sub-block is higher than that of the edge region. The prediction results of each image sub-block after weighted fusion are stitched together to obtain the multi-channel fluorescence prediction probability map corresponding to the complete tissue slice. Furthermore, morphological optimization is performed on the stitched image to output the final virtual fluorescence image, specifically: The spliced ​​multi-channel fluorescence prediction probability map is processed by opening operations for each channel to remove noise points; After the opening operation, each fluorescence channel image is segmented according to the segmentation threshold of the corresponding channel to obtain a binary image of each fluorescence channel; Based on the mean and standard deviation of the real fluorescence image and the predicted fluorescence image of each fluorescence channel, the intensity scaling parameter and intensity offset parameter of the corresponding channel are determined, and the intensity of each fluorescence channel image after the opening operation is calibrated to obtain the virtual fluorescence image of each fluorescence channel. The virtual fluorescence images of each fluorescence channel are combined into a multi-channel image, which is then output as the final virtual fluorescence image.

[0016] Compared with the prior art, the beneficial effects of the present invention are: By constructing a self-attention enhanced UNet++ network and introducing an AG-SA module in each encoding layer, the network captures global contextual information of the entire feature map through a self-attention mechanism to generate a self-attention feature map. Finally, a gated graph is used to perform element-wise weighted fusion of the self-attention features and the original convolutional features, achieving an adaptive combination of global semantic information and local detail features. This significantly enhances the network's ability to model complex organizational structures and long-range dependencies. Furthermore, multi-scale feature fusion is performed on the original encoded feature maps output from each encoder layer, and the fused result is compared with the original encoded features of the corresponding layer. Figure 1 The data is then input into the decoder for layer-by-layer decoding, effectively integrating multi-scale information from shallow to deep layers and enhancing the network's adaptability to different sizes of tissue structures. Through a Gaussian weighted image sub-block stitching method, which uses a weighting strategy with the center of each image sub-block as the maximum value and gradually decays towards the edge, seamless prediction and smooth stitching of large-size tissue slice images are achieved. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the overall method flow of the present invention; Figure 2 Mapping probability With the original logical value The relationship diagram of changes. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0019] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0020] Example: Please see Figures 1-2 The present invention provides a technical solution: The self-attention-enhanced multi-scale tissue section immunofluorescence virtual staining method includes the following steps: S1: Obtain the original stained image of the tissue section and perform preprocessing. Input the preprocessed stained image into the constructed self-attention enhanced UNet++ network for deep feature extraction. In each coding layer, the constructed AG-SA module is used to process the deep feature map extracted in the current layer to generate a gated fusion feature map. The gated fusion feature map is downsampled to output the original coding feature map of the corresponding layer, which is then used as the input of the next coding layer. In this embodiment, the original stained image of the tissue section is obtained and preprocessed, specifically as follows: The original stained images of tissue sections are obtained, and their RGB channels are separated. The mean and standard deviation of pixels in each channel are calculated, and the pixel values ​​of each channel are normalized based on the mean and standard deviation. Wavelet domain adaptive denoising is performed on the normalized image, and multi-scale contrast enhancement is performed on the denoised image to obtain the preprocessed stained image.

[0021] In the above process, RGB channel separation of the original stained image refers to decomposing the original three-channel color image into three independent single-channel images, corresponding to the red, green, and blue channels, respectively. Specifically, the original stained image is composed of three color channels superimposed, with each channel recording the brightness information of the image at different spectral bands. The red channel mainly reflects the difference between the blue of hematoxylin staining and the red of eosin staining, while the green and blue channels provide complementary tissue staining features. Through RGB channel separation, the originally fused three-channel data is split into three independent two-dimensional matrices. Each pixel value in each matrix represents the intensity value of that channel at that location, thus allowing for independent statistical analysis of each channel. Normalizing the pixel values ​​of each channel involves converting the raw value of each pixel into a standardized form based on the mean and standard deviation of all pixels in that channel. Specifically, first, the arithmetic mean of all pixel values ​​in the current channel is calculated as the mean, and the square root of the dispersion of all pixel values ​​relative to this mean is calculated as the standard deviation. Then, for each pixel value in the channel, the mean of that channel is subtracted, and then divided by the standard deviation of that channel, so that the pixel value distribution of the entire channel after transformation has a mean of zero and a standard deviation of one. This normalization operation can eliminate the inconsistency in pixel intensity distribution caused by differences in lighting conditions and staining concentrations between different staining batches, different scanning devices, and different tissue samples, making the preprocessed stained images statistically consistent and facilitating stable feature learning by the subsequent network model. Adaptive wavelet domain denoising of a normalized image involves transforming the image to the wavelet domain, adaptively suppressing noise components at different scales and directions, and then inversely transforming the processed coefficients back to the spatial domain. Specifically, first, a discrete wavelet transform is performed on the normalized image, decomposing it into a low-frequency approximate sub-band and multiple high-frequency detail sub-bands. The low-frequency detail sub-bands contain the main structural information of the image, while the high-frequency detail sub-bands mainly contain edge texture information and noise components. Then, for each high-frequency detail sub-band, a threshold is adaptively calculated based on the characteristics of the coefficient distribution within that sub-band. Coefficients with absolute values ​​less than the threshold are considered noise components and set to zero, while coefficients greater than the threshold are retained to preserve edge and texture information; the low-frequency sub-band remains unchanged. Finally, an inverse wavelet transform is performed on the processed wavelet coefficients to reconstruct the denoised image. The threshold is adaptively calculated based on the characteristics of the coefficient distribution within the high-frequency detail sub-band. A general thresholding method is typically used. Specifically, the threshold is set as the standard deviation of the wavelet coefficients multiplied by the square root of twice the logarithmic signal length. That is, firstly, the standard deviation of all coefficients within the current high-frequency detail sub-band is calculated. If noise dominates in this sub-band (i.e., the robust standard deviation is estimated by calculating several times the absolute value of the median of the coefficients within the high-frequency detail sub-band, and the ratio of the robust standard deviation to the standard deviation of the coefficients is close to 1), then this standard deviation can be approximated as the standard deviation of the noise. Then, the corresponding scaling factor is calculated based on the signal length to obtain a global threshold that can effectively distinguish between signal and noise. Calculating the scaling factor based on the signal length refers to adjusting the threshold value using the total number of sampling points in the signal, so that the threshold increases appropriately with the increase of signal length. Specifically, the scaling factor is calculated as the square root of twice the logarithm of the signal length, i.e., the scaling factor equals the square root of the logarithm of the signal length with the natural constant e as the base and the signal length as the argument. The signal length is usually taken as the total number of coefficients in the current high-frequency detail subband, which reflects the number of sampling points. This scaling factor design is based on the extremum theory of Gaussian white noise: when a set of independent and identically distributed Gaussian noise signals has a length of N, the expected value of its maximum absolute value increases approximately according to the square root of twice the logarithm N, i.e., as the signal length increases, the maximum amplitude value appearing in pure noise also increases slowly. Therefore, setting the general threshold to the standard deviation of wavelet coefficients multiplied by the square root of twice the logarithmic signal length mathematically ensures that the threshold increases accordingly with the signal length, effectively suppressing potentially large-amplitude spurious signals in the noise. Simultaneously, the slow growth characteristic of the logarithmic function ensures that even as the signal length increases from hundreds to hundreds of thousands, the scaling factor changes relatively gently, preventing the threshold from becoming excessively large and thus overly suppressing the true signal due to the increased signal length. Multi-scale contrast enhancement of denoised images involves constructing Gaussian and Laplacian pyramids, adjusting local contrast at different spatial resolution levels, and then reconstructing the image to simultaneously improve the visual clarity of both large-area brightness distribution and small-scale structural edges. Specifically, the denoised image is first subjected to continuous Gaussian smoothing and downsampling operations. Each smoothing and downsampling operation generates an image layer with half the resolution. All image layers together form a Gaussian pyramid, where the bottom layer is the original resolution image and the top layer is the minimum resolution image. Each layer represents the general information of the original image at different scales. Based on the Gaussian pyramid, each image layer is pixel-wise differencing the upsampled and Gaussian smoothed result from the previous layer to obtain the layers of the Laplacian pyramid. Each layer contains frequency band details at the corresponding scale, such as edges, textures, and local contrast variations. Then, contrast enhancement processing is performed on each layer of the Laplacian pyramid, using the following method: In this process, the value corresponding to each pixel in each layer of the Laplacian pyramid is labeled as a Laplacian coefficient. A histogram of the coefficient distribution for each layer is calculated to determine the upper and lower thresholds for contrast stretching. Coefficients below the lower threshold are mapped to a minimum intensity value, while coefficients above the upper threshold are mapped to a maximum intensity value. Coefficients between these thresholds are stretched using a linear function, thus amplifying local differences at that scale. For the top layer of the Gaussian pyramid, global contrast adjustment is performed based on its overall brightness distribution to improve the dynamic range of the image. Finally, starting from the top layer of the Laplacian pyramid, upsampling is performed layer by layer upwards, and the results are added to the Laplacian details of the corresponding layer to reconstruct an enhanced image at the original resolution. Through this multi-scale processing strategy, small-scale layers enhance the edge sharpness of fine structures such as cell nuclei and cell membranes, medium-scale layers improve the contour clarity of medium-sized tissue structures such as glands and blood vessels, and large-scale layers optimize overall brightness contrast and background uniformity. When enhancing the contrast of coefficients at each level of the Laplace pyramid, the lower and upper thresholds are typically determined adaptively based on the statistical characteristics of the coefficient distribution at that level. First, a histogram of the distribution of all coefficients at that level is plotted, i.e., the frequency of occurrence of coefficients within each numerical interval is counted to obtain the overall distribution of coefficient values ​​from minimum to maximum. Then, thresholds are determined based on preset cutoff percentages: coefficient values ​​whose cumulative frequency in the coefficient distribution histogram reaches a preset lower threshold percentage (e.g., 1%) are set as the lower threshold, and coefficient values ​​whose cumulative frequency reaches a preset upper threshold percentage (e.g., 99%) are set as the upper threshold. A linear function is a function that maps input values ​​to output values ​​in a directly proportional relationship. In the context of contrast enhancement, the specific form of a linear function is: the output coefficient equals the input coefficient minus the lower threshold, multiplied by a fixed scaling factor, and then added to the minimum value of the target range. The scaling factor is equal to the target coefficient range (maximum minus minimum) divided by the effective range of the input coefficients (upper threshold minus lower threshold). Through this linear mapping, coefficients originally distributed between the lower and upper thresholds are uniformly stretched to the entire target coefficient range, for example, from 0 to 255, or from -1 to 1, thereby amplifying local contrast differences at that scale.

[0022] A self-attention-enhanced UNet++ network is used to extract deep features from the preprocessed stained image, specifically as follows: The preprocessed stained image is first subjected to a first-level convolution extraction, that is, using... The system processes features using convolution, batch normalization, and ReLU activation to output the feature map extracted by the first-level convolutional layer. Then, a second-level deep feature extraction is performed on this feature map, i.e., it is processed again using... Convolution, batch normalization, and ReLU activation function processing are used to obtain feature maps for secondary deep feature extraction.

[0023] The above process is designed to gradually abstract and enhance the semantic features of the input image while preserving its spatial resolution, providing the subsequent AG-SA module with feature maps that contain both rich local texture information and preliminary high-order semantic expression. Specifically, through the first-level 3×3 convolution, batch normalization, and ReLU activation function, the network can extract basic features such as low-level edges, corners, and color tones from the input image. Batch normalization accelerates training convergence and stabilizes the feature distribution, while the ReLU activation function introduces non-linear expressive power, outputting the feature map extracted by the first-level convolution layer. Based on this, a second 3×3 convolution, batch normalization, and ReLU activation function are performed, enabling the network to further combine low-level features to form more complex texture patterns and local structures, such as glandular contours and cell nucleus aggregation regions, while keeping the receptive field unchanged, ultimately obtaining the feature map of secondary deep feature extraction. Compared to using larger convolutional kernels (such as 5×5 or 7×7), two consecutive 3×3 convolutions have fewer parameters while maintaining the same receptive field (both are 5×5). At the same time, due to the additional non-linear transformation (ReLU) layer in the middle, their expressive power is stronger. Compared to a single convolution, two consecutive convolutions increase the network depth with fewer parameters, improving the feature abstraction capability. Compared to stacking deeper convolutions, two convolutions avoid prematurely reducing the spatial resolution, preserving enough spatial details for the AG-SA module for subsequent local variance calculation and self-attention modeling.

[0024] The constructed AG-SA module is used to process the depth feature map extracted from the current layer to generate a gated fusion feature map, specifically including: The feature map extracted by the second depth feature extraction is divided into multiple non-overlapping local feature blocks in the height and width directions, and the local variance of each local feature block is calculated separately. The local variances corresponding to each local feature block are arranged according to their spatial positions in the feature map of the secondary deep feature extraction to form a single-channel statistical feature map. The single-channel statistical feature map is then subjected to convolution and activation processing to generate a gating map, which is used to characterize the feature response intensity of each local region. The feature map extracted by the second deep feature extraction is reconstructed to obtain a two-dimensional feature matrix, and query features, key features and value features are obtained by three learnable linear transformations respectively. Self-attention operations are performed based on the query features, key features, and value features to obtain a self-attention feature map; The self-attention feature map is restored to the spatial structure corresponding to the feature map extracted by the second deep feature extraction. The self-attention feature map and the feature map extracted by the second deep feature extraction are then weighted and fused based on the gating map to obtain a gating fused feature map. The weighted fusion is used to adaptively adjust the fusion ratio of the self-attention feature and the original feature according to the statistical features of the local region.

[0025] The formula upon which the above process is based is: in, Indicates the first Local variance of a local feature block; This represents the total number of pixels within each local feature block; Indicates that the local feature block is located within Pixel value at coordinates; , These represent the x and y coordinates of a pixel within a local feature block, respectively. Indicates the first Local feature blocks The mean value of all pixel values ​​within the range; Indicates the first A local feature block; Indicates the index of the local feature block; The local variances of all local feature blocks are arranged into a single-channel feature map according to their spatial positions on the original feature map. Each pixel value in this feature map corresponds to the variance of a local block. The gated graph is generated using convolution and the sigmoid activation function, based on the following formula: in, Represents a gating diagram; Indicates the total number of local feature blocks; In the above process, the feature map extracted from the secondary deep features is first divided into K×K non-overlapping local feature blocks using a uniform grid, and the local variance of each local feature block is calculated. Local variance is a statistic that measures the dispersion of pixel values ​​within a block, effectively characterizing the structural complexity and texture richness of the region: regions with large variance correspond to dense cell areas, tissue edges, and other complex structures, while regions with small variance correspond to flat backgrounds or uniform stroma regions. After arranging the local variances of all local feature blocks into a single-channel feature map according to their spatial location, a gating map G is generated using 1×1 convolution and a sigmoid activation function. The value at each position is mapped to between 0 and 1, indicating the relative weight that the self-attention feature and the convolution feature should be assigned to that local region. In the aforementioned AG-SA module, the number of blocks is a hyperparameter. The specific value is usually determined based on the spatial resolution of the feature map and the desired local receptive field size, and the value range is generally [range missing]. In this embodiment, the default settings are... That is, the feature map is divided into 8 segments in both the height and width directions, resulting in a total of A local feature block. The reason for this value is as follows: First, when At that time, the spatial size of each local feature block is Under typical downsampled feature map sizes (e.g., the size of the third layer feature map in the encoder is...), Each local feature block contains approximately The size of a pixel is such that it captures enough local texture statistics (such as variance variations at cell edges) without causing unstable variance estimation due to too few pixels within the block; secondly, Able to generate The gated map at this resolution preserves the spatial differences between different regions on the feature map (able to distinguish between tissue centers and edges) while avoiding the negative impact of excessively high gated map resolution on subsequent... Third, convolution is computationally too expensive; The value can be adaptively adjusted according to different levels of the encoder; a larger value can be used for shallower feature maps with higher resolution. (like To preserve more detailed spatial location information, deeper feature maps, which have lower resolution, can use smaller [sizes / sizes]. (like This is to avoid having too few pixels in each local feature block.

[0026] Self-attention calculation is performed on the feature map extracted by secondary deep feature extraction. Specifically, the feature map is first flattened in space through a reshaping operation to obtain a two-dimensional feature matrix. Then, the two-dimensional feature matrix is ​​linearly transformed by three learnable linear transformation matrices, namely the first, second, and third linear transformation matrices, to obtain the query matrix, key matrix, and value matrix, respectively. The self-attention feature map is then calculated based on the query matrix, key matrix, and value matrix, using the following formula: This indicates the length of each row vector in the query matrix; , , These represent the query matrix, the transpose of the key matrix, and the value matrix, respectively. Represents a self-attention feature map; The reason for performing self-attention calculation on the feature map extracted from the secondary deep feature extraction process is that convolution operations are limited by the local receptive field and struggle to capture the contextual influence of other regions far from the current pixel on that pixel. Self-attention, on the other hand, can explicitly model global long-range dependencies by calculating pairwise similarities between all positions. Specifically, the feature map is first flattened into a two-dimensional feature matrix through a reshaping operation, so that each row corresponds to a feature vector of a spatial location. Then, this feature matrix is ​​mapped into a query matrix, a key matrix, and a value matrix through three learnable linear transformation matrices. The query matrix represents the information that needs to be queried at each position, the key matrix represents the information that each position can provide, and the value matrix represents the actual feature content at each position. The product of the query matrix and the transpose of the key matrix is ​​calculated and divided by the square root. The matrix is ​​scaled, then normalized using Softmax to obtain the attention weight matrix, and finally multiplied by the value matrix to obtain the self-attention feature map. The feature vector at each position in the feature map is a weighted sum of the feature vectors at all positions in the entire feature map, with the weights determined by the attention distribution. In this way, the self-attention feature map can encode the global context information of the entire map, enabling the network to perceive relevant organizational structures far from the current position during subsequent processing.

[0027] Gated fusion feature maps are generated based on self-attention feature maps, using the following formula: in, Represents the gating fusion feature map; This represents the feature map extracted using secondary deep feature extraction.

[0028] In the above process, finally, the self-attention feature map is based on the gating graph G. and the original convolutional feature map Perform element-wise weighted fusion to generate gated fusion feature maps. The core advantage of this fusion mechanism lies in its spatial adaptability: in complex structural regions with high local variance, the gated graph... When the value approaches 1, the output is mainly composed of self-attention features, enabling the network to utilize global contextual information to understand the positional relationship of the region within the entire tissue slice, thereby generating more accurate structural boundaries and fluorescence signal distributions; in flat regions with low local variance, the gating map... The value of approaches 0, and the output mainly consists of the original convolutional features, allowing the network to rely primarily on local consistency for prediction and avoiding unnecessary noise interference introduced by global information. Through this adaptive fusion, the AG-SA module can dynamically adjust the feature extraction strategy according to the content of the feature map. It retains the efficient encoding capability of convolutional operations for local details while introducing the self-attention capability to model global dependencies, providing richer and higher-quality gated fusion feature maps for subsequent encoding downsampling and decoding recovery.

[0029] The output is the original encoded feature map, specifically: First, the gated fusion feature map is downsampled to output the first-layer encoded feature map, i.e., a map of size [size missing]. Max pooling is used to downsample the gated fusion feature map to obtain the first layer of encoded feature map; Then, construct the second layer of encoded feature maps: pass the first layer of encoded feature maps sequentially through convolution kernels with a kernel size of... The process involves convolution, batch normalization, and ReLU activation, with 128 convolution kernels; then, it sequentially passes through convolution kernels of size [missing information]. The convolution, batch normalization, and ReLU activation functions are applied, with the number of convolution kernels also being 128. The result is then input into the AG-SA module for further processing, yielding the output of the second-layer AG-SA module. Finally, a kernel of size [size missing] is used... Max pooling is used to downsample the output to obtain the second layer of encoded feature map.

[0030] Next, the third layer of encoded feature maps is constructed: the second layer of encoded feature maps is sequentially processed through convolution kernels with a kernel size of [size missing]. The process involves convolution, batch normalization, and ReLU activation, with 256 kernels; then it sequentially passes through convolution kernels of size [missing information]. The convolution, batch normalization, and ReLU activation functions are applied, with the number of convolution kernels also being 256. The result is then input into the AG-SA module for further processing, yielding the output of the third-layer AG-SA module. Finally, a kernel of size [size missing] is used... Max pooling is used to downsample the output to obtain the third layer of encoded feature map.

[0031] Finally, construct the fourth layer of encoded feature maps: pass the third layer of encoded feature maps sequentially through convolution kernels with a kernel size of [size missing]. The process involves convolution, batch normalization, and ReLU activation, with a kernel size of 512; then it sequentially passes through convolution kernels of size... The convolution, batch normalization, and ReLU activation functions are applied, with the number of convolution kernels also being 512. The result is then input into the AG-SA module for processing, resulting in the output of the fourth-layer AG-SA module. This output is the fourth-layer encoded feature map, and no further downsampling is performed.

[0032] The reason for this process is to construct a pyramid of encoded feature maps that gradually increases the number of channels and gradually decreases the spatial resolution from shallow to deep layers. This allows the network to extract information at different levels, from fine local textures to abstract semantic features, at multiple scales. Specifically, the construction of each layer of encoded feature maps follows the basic paradigm of "two convolutions plus an AG-SA module plus max pooling." The use of two consecutive 3×3 convolutions, batch normalization, and the ReLU activation function serves the same purpose as the aforementioned deep feature extraction stage: firstly, two 3×3 convolutions provide the same receptive field with fewer parameters than a single 5×5 convolution, while enhancing the abstract expressive power of features through two nonlinear transformations; secondly, introducing batch normalization after the convolution operation accelerates training convergence and stabilizes the feature distribution, while the ReLU activation function introduces sparsity and enables the network to learn nonlinear mappings. The number of convolutional kernels in each layer gradually increases from 64 in the first layer to 512 in the fourth layer. This is because as the spatial resolution of the feature map decreases layer by layer, the receptive field corresponding to each spatial location continues to expand, and the semantic information becomes more and more abstract. More channels are needed to encode diverse high-order features such as different tissue components, cell types, and spatial relationships. Before each downsampling step, an AG-SA module is introduced, enabling the network to perform self-attention enhancement on the feature map at each spatial resolution level, capturing long-range dependencies at that scale. It also adaptively fuses global contextual information with local convolutional features through local variance gating, effectively mitigating the potential loss of detail caused by subsequent max-pooling downsampling. At the end of each layer, a 2×2 max-pooling downsampling is performed, halving the spatial size of the feature map and gradually expanding the receptive field of each pixel while preserving the basic shape information of the feature map. The final layer does not undergo downsampling to retain the deepest semantic features at the smallest spatial resolution, avoiding excessive spatial information loss due to over-downsampling, which could affect the decoder's ability to recover the overall organizational structure. Through this progressively layered design, the resulting encoded feature maps from layers one to four provide complementary encoded feature representations for subsequent multi-scale feature fusion and the decoder.

[0033] S2: Perform multi-scale feature fusion on the original encoded feature maps output by each layer of the encoder, and then combine the fused result map with the original encoded features of the corresponding layer. Figure 1 The data is then input into the decoder for decoding to obtain a decoded graph. Fully coupled optimization is performed on the decoded graph, and the total loss parameter is further calculated. In this embodiment, multi-scale feature fusion is performed on the original encoded feature maps output by each layer of the encoder, specifically including: The original encoded feature maps of each layer are subjected to average pooling at different scales to obtain the corresponding multi-scale pooling features. Multi-scale pooling features of different scales are uniformly adjusted to a spatial size that matches the corresponding coding layer; The multi-scale pooling features with uniform size are spliced ​​together according to the channel dimension to form a multi-scale fused feature map; The multi-scale fused feature map is subjected to convolutional compression and feature integration to obtain the multi-scale feature fusion result map of the corresponding layer.

[0034] The formula upon which the above process is based is: in, Indicates the first Layer-encoded feature maps use The multi-scale pooling map obtained by average pooling; Indicates the first Layer-encoded feature map, ; Indicates the size of the average pooling kernel; The reason for this process is to extract and fuse contextual information from multiple receptive fields at each layer of the encoded feature map, enabling the network to simultaneously perceive tissue structural features from small to large scales, thereby enhancing its adaptability to cell morphology, glandular structure, and tissue regions of different sizes. Specifically, for each layer of the encoded feature map... Each uses a size of , , and Multi-scale pooling operations are performed using the average pooling kernel to obtain a multi-scale pooling graph. .in, Average pooling is equivalent to not performing pooling, preserving the finest local information at the scale of the original feature map; Average pooling captures local texture and cell edge features within a small area; The average pooling coverage is moderate, enabling the sensing of cell clusters or small gland structures composed of several cells; Average pooling covers a wider range of contexts, enabling it to capture the macroscopic distribution relationships and overall morphological patterns between organizational regions; The average pooling kernel size is set to The design of these four coprime odd numbers is for the following reasons: First, odd-sized pooling kernels ensure that the pooled feature map remains spatially aligned with the original feature map, facilitating subsequent upsampling and concatenation operations. Second, these four sizes, increasing sequentially from 1 to 7 and not overlapping, cover multiple scales from small to large, avoiding redundancy between scales. Third, the maximum pooling kernel size is set to 7 because after four downsampling operations, the spatial resolution of the deepest encoded feature map is already relatively small (e.g., half the original size). ), larger pooling kernels (such as or In deep feature maps, excessive pooling can lead to over-averaging of effective information and loss of important local contrast; while in shallow feature maps, an excessively large pooling kernel size can cause excessive fusion of features from different regions, reducing discriminative power. Instead of consecutive integers, the goal is to strike a balance between computational efficiency and scale coverage, covering the widest possible receptive field with fewer pooling operations.

[0035] The formula used for upsampling and stitching multi-scale pooling maps is: in, This represents the feature map after upsampling and concatenation; Indicates the height of the preprocessed stained image; Indicates the width of the preprocessed stained image; In the above process, after obtaining the multi-scale pooling map Then, an upsampling operation needs to be performed on the pooling results at each scale. Restore the feature maps of all scales to the same level as the encoded feature maps of the current layer. Same space size Then, the components are spliced ​​along the channel dimension. The upsampled and concatenated feature map is obtained. The necessity of this operation lies in the fact that pooling operations at different scales produce feature maps with different spatial dimensions (e.g., using...). The pooled feature map is much smaller than the original feature map, so it is necessary to upsample them to make their sizes uniform before they can be stitched and fused along the channel dimension.

[0036] The formula used for multi-scale feature fusion of the sampled stitched images is as follows: in, Indicates the first The result image after multi-scale feature fusion.

[0037] In the above process, through convolution The stitched multi-scale features are subjected to channel compression and cross-channel information integration to output the first... Result image after multi-scale feature fusion . Convolution plays three roles here: first, it increases the number of channels after concatenation, which is the same as the original number of channels. 4 times dimensionality reduction The system employs three main methods: first, it controls the computational load; second, it uses learnable weights to weight and fuse features at different scales, automatically learning which scale is more important for the current layer and the current task; and third, it introduces nonlinear transformations to enhance the discriminative power of features.

[0038] The decoded image is as follows: The multi-scale feature fusion result map output by the encoder and the original encoded feature map of the corresponding layer are input into the decoder for layer-by-layer decoding. Layer-by-layer decoding includes: starting from deep features, upsampling or resizing the current decoded features to make them match the spatial dimensions of the multi-scale feature fusion result map of the adjacent shallow layers and the original encoded feature map; The size-matched features are fused with the multi-scale feature fusion result map of the corresponding layer and the original encoded feature map, and the decoded feature map of the current layer is generated by feature transformation. The decoding feature maps of each decoding layer are obtained layer by layer until the final decoding map is obtained.

[0039] The processing steps described above are as follows: The multi-scale feature fusion result image generated by the encoder is... Compared with the original encoded feature map of the corresponding layer All data is input into the decoder for layer-by-layer decoding. The decoding process starts from the deepest 4th decoding layer and proceeds upwards layer by layer to the 1st decoding layer; specifically: 4th Decoding Feature Map Multi-scale fusion feature map from the fourth layer It is obtained directly through a 3×3 convolution operation, with a kernel size of 256; subsequently, Perform a 2x upsampling to make its spatial size match that of the third-layer encoded feature map. and the third layer multi-scale fusion feature map The features are matched, then concatenated, and finally subjected to a 3×3 convolution operation with 128 kernels to obtain the output of the third decoding layer, which is the third layer decoding feature map. ; Then, the third layer of decoded feature map Perform a 2x upsampling to make its spatial size match that of the second-layer encoded feature map. and the second layer multi-scale fusion feature map Matching. The upsampled feature map, and The features from the three layers are concatenated, and then a 3×3 convolution operation is performed, with a kernel size of 64, to obtain the output of the second decoding layer, which is the second layer decoding feature map. ; Finally, the second layer of decoded feature maps Perform a 2x upsampling to make its spatial size the same as the first layer encoded feature map. and the first layer multi-scale fusion feature map Matching; upsampling feature maps, and The features from the three layers are concatenated, and then a 3×3 convolution operation is performed, with a kernel size of 64, to obtain the output of the first decoding layer: the first layer decoded feature map. ,Should This is the final output of the decoder, the decoded diagram.

[0040] In the above process, the decoding process starts from the deepest fourth decoding layer and calculates upwards layer by layer to the first decoding layer, making full use of the complementary information provided by each layer of the encoder: deep feature maps. and It possesses a large receptive field and rich semantic information, enabling it to recognize high-level concepts such as tissue type, glandular structure, and cell distribution patterns; shallow feature maps , , This preserves fine spatial location information and edge texture details; In the deepest fourth decoding layer, the fourth decoding feature map Multi-scale fusion feature map from the fourth layer Directly The convolution operation yields a kernel size of 256. The reason for this design is that layer 4 is the deepest layer of the decoder, and no deeper decoding layers can provide upsampled feature maps; therefore, only the multi-scale fused feature map from the deepest layer of the encoder is used as input. Convolution and not Convolution is used to integrate local spatial context of features while maintaining the same number of channels, thereby enhancing the discriminative power of semantic features. Subsequently, in each decoding layer, the decoded feature map of the previous layer is upsampled by a factor of 2, doubling its spatial size, thus gradually restoring the spatial resolution of the feature map; the upsampled feature map is compared with the original encoded feature map of the current layer. and the multi-scale fusion feature map of the current layer splicing the three together The advantage of this three-way concatenation design is that the upsampled decoded feature map provides semantic prior information from deeper layers. It provides contextual information about the current layer after multi-scale pooling fusion. This provides the most original encoded features of the current layer, preserving spatial details that have not undergone any fusion processing; The stitched feature map is passed through Convolutional operations are used for feature integration and channel dimensionality reduction, with the number of convolutional kernels decreasing progressively from 256 in the 4th decoding layer to 64 in the 1st decoding layer. This decreasing design is because as the decoding layers increase, the spatial resolution of the feature map gradually increases, but the semantic abstraction level gradually decreases. Therefore, it is necessary to gradually reduce the number of channels to control the computational load and avoid excessive redundant information in shallow decoding features. Through this progressive design of layer-by-layer upsampling and cross-layer skip connections, the decoder can fully utilize the spatial detail information lost in each level of the encoder, gradually recovering the spatial resolution of the feature map, while also leveraging… and The dual supplementation effectively alleviates the problem of boundary blurring and detail loss caused by multiple downsampling in deep feature maps, resulting in the final first-layer decoded feature map. It possesses both high-resolution spatial structure information and deep semantic discrimination capabilities.

[0041] The total loss parameter is output as follows: The decoded image is transformed by channel transformation so that the number of output channels corresponds to the number of target fluorescence channels to obtain the original prediction result. The original prediction result is then activated and mapped to obtain the prediction probability map of each fluorescence channel. Based on the predicted probability map and the actual multiplex immunofluorescence staining image, the weighted cross-entropy loss is calculated; Based on the correlation between each fluorescence channel, the channel penalty loss is calculated; Based on the structural similarity between the predicted probability map and the actual multiplex immunofluorescence staining image, the structural similarity loss is calculated; The weighted cross-entropy loss, channel penalty loss, and structural similarity loss are combined in a weighted manner to obtain the total loss parameter; A training dataset containing multiple pairs of preprocessed stained images and real multiplex immunofluorescence stained images is constructed. Paired samples are randomly sampled from the training dataset and input into the self-attention-enhanced UNet++ network. The learnable parameters of the network are iteratively updated according to the total loss parameter until the preset training termination condition is reached, and the trained self-attention-enhanced UNet++ network is obtained.

[0042] The formula upon which the above process is based is: in, express The One channel, , represents the index of the number of fluorescence channels, and is an integer; In the above process, the first step is to adopt convolution Transform the number of channels in the decoded image to the required number of fluorescence channels. The original logic value is output. Then apply the Sigmoid function channel by channel. Perform probability mapping to normalize the output value of each channel to... Intervals are used to obtain interpretable multi-channel fluorescence prediction probability maps. The Sigmoid function is chosen because the prediction of each fluorescence channel is essentially a binary classification problem, i.e., each pixel is labeled or unlabeled in that channel. The Sigmoid function can map any real number to the probability space, and its smooth derivative properties are beneficial for gradient backpropagation.

[0043] In the above embodiments, the original logic value is given. and the corresponding mapping probability The 20 sets of data are used to reflect the mapping probability. With the original logical value The changes are as shown in Table 1: Table 1: Mapping Probabilities With the original logical value Change Relationship Table In Table 1 above, as From negative to positive, It monotonically increases from 0 to 1, when hour, This is the center symmetric point of the Sigmoid function; when hour, Rounded to 0.00, the network's prediction for negative regions is close to 0, meaning there is no fluorescent labeling; when hour, After rounding to 0.99 or 1, the network predicts that a positive region is close to 1 and thus fluorescently labeled. This saturation characteristic allows the Sigmoid function to compress any real value of the network output into the probability space.

[0044] Calculate the weighted cross-entropy loss: in, Indicates the first The pixel at the th point The predicted probability on each fluorescence channel is determined by... At pixel The value is obtained at the location; The value can be 0 or 1, where 1 represents the first value. The pixel in the first The fluorescent marker is present in the fluorescent channel, where 0 indicates the first fluorescent channel. The pixel in the first The fluorescent label was not present on any of the fluorescent channels; Indicates the first The pixel in the first Weighting coefficients on each channel; Indicates the weighted cross loss; Indicates the total number of pixels; Indicates the total number of fluorescence channels; In the above process, weighted cross-entropy loss is used to improve pixel-level classification accuracy. As the primary monitoring signal, the standard form of the weighted cross-entropy loss is to calculate the cross-entropy between the true label and the predicted probability pixel-by-pixel and channel-by-channel. A negative sign indicates that a smaller loss value signifies a more accurate prediction. Weighting coefficients. The introduction of this method is to address the class imbalance problem caused by the fact that positive signal pixels (i.e., those with fluorescent labels) are usually far fewer than negative pixels (i.e., those without fluorescent labels) in tissue sections. By assigning higher weights to positive pixels, the network focuses more on sparse target regions, avoiding the model's tendency to predict all negative results. Weighting coefficient The determination method usually adopts the median frequency balancing method. First, the category frequency of all pixels in each channel is counted, and the ratio of the median frequency to the category frequency is taken as the weight, thereby avoiding the dominant influence of extreme frequency values ​​on the weight calculation.

[0045] The formula used to calculate channel penalties is: in, Indicates channel penalty; This represents the Pearson correlation coefficient; In the above process, regarding channel independence, a channel penalty loss is introduced. To suppress signal crosstalk between different fluorescence channels. In ideal multiplex immunofluorescence staining, the expression of different biomarkers should be spatially independent, meaning that the presence of marker A at a certain location should not significantly affect the probability of the presence of marker B. However, since different tissue structures in the input single stained image may overlap or be adjacent, the network may learn inappropriate channel associations. By minimizing the absolute value of the correlation coefficient between the predicted probability maps of different channels, The network is forced to learn independent feature representations for each fluorescent channel, thereby effectively reducing channel crosstalk and improving the specificity of each fluorescent label; The Pearson correlation coefficient measures the linear correlation between the predicted probability maps of two channels. Its calculation process is as follows: The predicted probability maps of the c-th and c'-th channels are each considered as one-dimensional vectors containing N values, each value corresponding to the predicted probability of a pixel in that channel. First, the mean of the predicted probabilities of all pixels in each channel is calculated, i.e., the sum of all pixel values ​​in each channel divided by the total number of pixels. Then, for each pixel position, the difference between the predicted probability of that pixel in each channel and the mean of that channel is calculated, and these two differences are multiplied together. The covariance of the two channels is obtained by summing these products for all pixel positions. Simultaneously, the square root of the sum of the squares of the differences between pixels in each channel is calculated to obtain the standard deviation for each channel. Finally, the covariance is divided by the product of the standard deviations of the two channels to obtain the Pearson correlation coefficient, which ranges from -1 to +1. When the correlation coefficient is +1, it indicates that the two channels are perfectly positively correlated, meaning that when the predicted value of one channel increases, the predicted value of the other channel also increases proportionally. When the correlation coefficient is -1, it indicates that the two channels are perfectly negatively correlated. When the correlation coefficient is zero, it indicates that the two channels are linearly unrelated. In the channel penalty loss, the absolute values ​​of the correlation coefficients of all different channel pairs are summed. When the prediction results of two channels are linearly independent, the absolute value of the correlation coefficient is zero, and no penalty is imposed on the network. When two channels are positively or negatively correlated, the absolute value of the correlation coefficient is greater than zero. At this time, a positive penalty is imposed on the network, forcing the network to learn to reduce the correlation between channels, thereby enhancing the independence of the prediction of each fluorescence channel and effectively suppressing signal crosstalk.

[0046] The formula used to calculate the SSIM loss is: in, Indicates SSIM loss; This represents a true multiplex immunofluorescence staining image; In the above process, SSIM is the structural similarity index, which comprehensively compares the similarity between two images in terms of brightness, contrast, and structure. Its value ranges from [value range missing]. The closer the value is to 1, the more similar the two images are. Compared with pixel-level mean squared error loss, SSIM loss focuses more on the overall visual quality and local structural consistency of the image, effectively avoiding problems such as over-smoothing or blurred edges in the generated image, making the predicted fluorescence image closer to the real staining effect in terms of cell nucleus morphology, membrane structure continuity, etc.

[0047] The total loss parameter is constructed based on weighted cross-entropy loss, channel penalty, and SSIM loss, using the following formula: in, Indicates the total loss parameter; In the above process, The weight is set to 1 as the dominant loss to ensure pixel-level classification accuracy; The weight is set to 0.1 as a regularization term to appropriately constrain channel independence; The weight of is set to 0.2 as a structure fidelity term to improve the visual quality of the generated image. This weighted design allows the network to optimize three objectives simultaneously during training, and the final learned parameters can generate multi-immunofluorescence prediction images with high pixel accuracy, strong channel independence and good structure fidelity from a single stained image; Weighted cross-entropy loss The weight is set to 1 because it directly supervises the classification accuracy of each pixel in each fluorescence channel, which is the fundamental guarantee for generating a reliable virtual fluorescence image and determines whether the prediction result can express the correct fluorescent label in the correct position. SSIM loss The weight is set to 0.2 because it focuses on the overall structural quality and visual consistency of the image. It's a further optimization based on accurate pixel classification, used to improve the realism of the generated image in terms of cell morphology, boundary sharpness, and texture details. While important, it shouldn't overly dominate the training process. Channel penalty loss The weight is set to 0.1 because it serves as an auxiliary regularization mechanism to suppress signal crosstalk between different fluorescence channels and enhance the independence of predictions for each channel.

[0048] A training dataset is constructed, which contains multiple pairs of preprocessed stained images and real multiplex immunofluorescence stained images. A batch of paired samples is randomly sampled from the dataset, and the preprocessed stained images are input into the self-attention-enhanced UNet++ network. The total loss parameter is calculated according to the steps described in S1 and S2. With the goal of minimizing the total loss parameter, the learnable parameters of the network are iteratively updated using the gradient descent method until the preset maximum number of training rounds is reached, and the learnable parameters of the trained self-attention-enhanced UNet++ network are obtained.

[0049] In the above process, the preset maximum number of training epochs is usually set as a large upper limit, such as 100, 200, or 500 epochs. Alternatively, it can be estimated empirically based on the size of the dataset and the complexity of the task. Small datasets typically refer to datasets with hundreds to thousands of training images, such as a dataset containing 500 paired stained images. Due to the limited sample size, the network can easily learn the main patterns in the data in a short time, usually achieving convergence within 50 to 100 training epochs. Increasing the number of training epochs further can easily lead to overfitting. Large datasets typically refer to datasets with tens of thousands or even hundreds of thousands of training images, such as a dataset containing 50,000 paired stained images. Because the samples cover a richer variety of tissue types, staining differences, and structural variations, the network needs more epochs to fully learn these diverse patterns, typically requiring 200 to 500 or even more training epochs to achieve convergence. Gradient descent is a first-order iterative optimization algorithm used to update the learnable parameters of a network by minimizing the loss function. The specific process is as follows: First, the preprocessed colored image is input into an attention-enhanced UNet++ network, and forward propagation is performed according to steps S1 and S2 to calculate the total loss parameter value under the current parameters. Then, using the chain rule, the partial derivatives of the total loss parameter with respect to each learnable parameter are calculated layer by layer from the output to the input. These partial derivatives constitute the gradient vector of the loss function with respect to the parameters, and the gradient points in the direction of the fastest increase in the loss function. Next, each learnable parameter is updated along the opposite direction of the gradient, i.e., the new parameter is equal to the current parameter minus the learning rate multiplied by the gradient value corresponding to that parameter, where the learning rate is a preset positive number used to control the step size of each update. The above three steps of forward propagation, gradient calculation, and parameter update are repeated. In each iteration, a batch of paired samples is randomly sampled from the training dataset, causing the network parameters to gradually move towards the minimum point of the loss function. The learning rate is usually determined by trial and error. First, set an initial learning rate such as 0.001 or 0.0001, start training and observe the trend of the total loss parameter. If the loss decreases slowly and fluctuates greatly, the learning rate may be too small. If the loss oscillates violently or even diverges, i.e. the loss value becomes infinite, the learning rate may be too large. It is necessary to adjust the order of magnitude of the learning rate according to the observation results. Usually, the adjustment factor is 10 times or 0.1 times.

[0050] S3: Construct a training dataset containing multiple pairs of preprocessed staining images and real multiplex immunofluorescence staining images. Randomly sample a batch of paired samples from the training dataset and input the original staining images into the attention-enhanced UNet++ network. Calculate the total loss parameters according to steps S1 and S2. Process the total loss parameters to obtain the learnable parameters of the attention-enhanced UNet++ network. S4: Based on learnable parameters, predict the image of the preprocessed tissue slice to be detected to obtain a multi-channel fluorescence prediction probability map. Perform Gaussian weighted stitching on the multi-channel fluorescence prediction probability map, perform morphological optimization on the stitched image, and output the final virtual fluorescence image.

[0051] The output multi-channel fluorescence prediction probability map is then Gaussian-weighted and stitched together, specifically as follows: Based on the learnable parameters of the self-attention-enhanced UNet++ network obtained through training, predictions are made for the preprocessed tissue slice images to be detected. The preprocessed tissue slice image to be detected is divided into multiple image sub-blocks, and each is input into the self-attention enhanced UNet++ network to obtain the multi-channel fluorescence prediction probability map corresponding to each image sub-block. Based on the spatial location of each image sub-block in the original tissue slice image, the multi-channel fluorescence prediction probability map corresponding to each image sub-block is mapped back to the complete image; The prediction results in adjacent or overlapping regions are weighted and fused using Gaussian weights, where the weight of the central region of the image sub-block is higher than that of the edge region. The prediction results of each image sub-block after weighted fusion are stitched together to obtain the multi-channel fluorescence prediction probability map corresponding to the complete tissue slice.

[0052] The formula used above is: in, in, Indicates the first After each image sub-block is predicted by the network, a multi-channel fluorescence prediction probability map is output. Indicates the first Image sub-blocks; , They represent the first Image sub-blocks In the original complete image The x and y coordinates of the top left corner; This represents the learnable parameters of a trained self-attention-enhanced UNet++ network; Indicates the learnable parameters are A self-attention-enhanced UNet++ network; Indicates the index of a sub-block of the image; In the above process, the preprocessed tissue slice image to be detected is first segmented into several blocks according to a preset step size. Pixel image sub-block Each sub-block is determined by the coordinates of its top-left corner. Unique identifier. For each image sub-block Call the pre-trained network Perform independent predictions to obtain the multi-channel fluorescence prediction probability map corresponding to the sub-block. This block-based prediction strategy enables the network to process large images of any size. However, directly stitching together all the sub-blocks will produce obvious visual discontinuities at the block boundaries, namely stitching artifacts or checkerboard effects. This is because the network cannot perceive the contextual information of adjacent sub-blocks when predicting each sub-block, resulting in jumps in the predicted values ​​on both sides of the boundary.

[0053] The output multi-channel fluorescence prediction probability map is Gaussian weighted and stitched together using the following formula: in, in, This indicates the fluorescence prediction probability map after Gaussian weighted splicing at the pixel level. The value; Indicates the first Image patches at pixel locations Gaussian weighted weights at each location; This represents the Gaussian kernel standard deviation.

[0054] In the above process, to eliminate stitching artifacts caused by block prediction, the output multi-channel fluorescence prediction probability map is Gaussian-weighted and stitched together. The core idea is: each image sub-block... Prediction results The confidence level is higher in the block center region and lower in the block edge region. Therefore, different fusion weights are assigned according to the distance of the pixel position from the block center; specifically, for the first... There are several image sub-blocks, the coordinates of which are: The weighting function has a maximum value (weight 1) at the center of the sub-block, and decreases with a Gaussian curve as pixels move away from the center, approaching 0 at the edges of the sub-block. For each pixel in the complete image... Collect all image sub-blocks that cover the pixel, i.e., all that satisfy the condition. and sub-blocks The predicted values ​​of these sub-blocks at this pixel A weighted average is calculated based on their respective Gaussian weights; The denominator is the sum of all weights, used to normalize the weighted average. Under this stitching strategy, the final predicted value of each pixel is determined by all the sub-blocks covering it, with sub-blocks closer to their center contributing more and those farther away contributing less. Since the center distance between adjacent sub-blocks is the step size (usually set to 128, half the side length of the sub-block), each pixel is located precisely in the intersection area near the center and edges of multiple sub-blocks. Through the smooth transition of Gaussian weights, the predicted values ​​at the block boundaries can be naturally merged, effectively suppressing the jump artifacts caused by direct stitching, resulting in a better overall fluorescence prediction probability map after stitching. Visually continuous and natural, while maintaining the prediction accuracy within each sub-block, providing high-quality full-image prediction results for subsequent morphological optimization.

[0055] Morphological optimization is performed on the stitched images to output the final virtual fluorescence image, specifically: The spliced ​​multi-channel fluorescence prediction probability map is processed by opening operations for each channel to remove noise points; After the opening operation, each fluorescence channel image is segmented according to the segmentation threshold of the corresponding channel to obtain a binary image of each fluorescence channel; Based on the mean and standard deviation of the real fluorescence image and the predicted fluorescence image of each fluorescence channel, the intensity scaling parameter and intensity offset parameter of the corresponding channel are determined, and the intensity of each fluorescence channel image after the opening operation is calibrated to obtain the virtual fluorescence image of each fluorescence channel. The virtual fluorescence images of each fluorescence channel are combined into a multi-channel image, which is then output as the final virtual fluorescence image.

[0056] In the above process, morphological optimization is performed on the stitched image. First, opening operation noise reduction and threshold segmentation are performed, that is, for the first... The predicted probability map of each channel is first opened, that is, isolated noise points are removed by erosion and then the original size is restored by dilation, thus obtaining the opening result map. Subsequently, threshold segmentation is performed on the opening operation result image: for each pixel, if the value of that pixel in the opening operation result image is greater than the value of the first pixel... If the segmentation threshold of the i-th fluorescence channel is met, the indicator function marks that point as 1; otherwise, it marks it as 0. This process ultimately yields the i-th segmented fluorescence channel. A binary image with 1 channel; For the Predicted probability chart for each channel First, an opening operation is performed: First, an erosion operation removes isolated noise points. The erosion operation scans the entire image using a structuring element as a template. Only when all pixel values ​​in the area completely covered by the structuring element are positive are the center pixels of that area retained; otherwise, they are set to zero. This process effectively eliminates single-pixel scattered points or small false positive areas caused by prediction uncertainty. Then, a dilation operation restores the original size. The dilation operation also scans the image using a structuring element as a template. As long as there are positive pixels within the area covered by the structuring element, the center pixel is set to positive. This process restores the eroded and shrunk positive areas to near their original size. The combination of erosion and dilation constitutes the opening operation, which removes isolated noise points and small artifacts while maintaining the overall shape and size of the positive areas. Subsequently, the opening operation result image is analyzed. Threshold segmentation: For each pixel If the value of that point in the result graph of the opening operation is greater than the first... Segmentation threshold for each fluorescence channel If the value is 1, the indicator function marks the point as 1; otherwise, it marks it as 0, thus obtaining the final thresholded segmentation result. The binary image of each channel. This step converts the continuous predicted probability values ​​into binary positive and negative results, so that the positive cell regions of each fluorescence channel are clearly segmented, which facilitates subsequent quantitative analysis.

[0057] Intensity calibration is performed on the opening operation results of each fluorescence channel after threshold segmentation to output the final virtual fluorescence image. The formula used is as follows: in, in, Indicates the first Virtual fluorescence images of each fluorescence channel; Indicates the first Intensity scaling factor for each fluorescence channel; Indicates the first Intensity shift of each fluorescence channel; Indicates the first The standard deviation of all pixels in a real fluorescence image with one fluorescence channel; Indicates the first The standard deviation of all pixels in the predicted fluorescence image of each fluorescence channel; Indicates the first The mean value of all pixels in a real fluorescence image with one fluorescence channel; Indicates the first The mean value of all pixels in the predicted fluorescence image of each fluorescence channel; Indicates the first The result of opening the predicted probability map of each fluorescence channel; This is an index for the fluorescence channels; In the above process, the intensity of the binary image of each fluorescence channel after thresholding is calibrated to output the final virtual fluorescence image. The images after opening and thresholding are binary, while real fluorescence images possess continuously varying intensity information, such as the typical nuclear staining pattern of bright centers and dark edges. Directly outputting a binary image would lose this highly diagnostic intensity information. Therefore, a linear transformation is used to adjust the opening result image. Intensity calibration is performed to ensure that the intensity statistical distribution of the predicted results is consistent with the real fluorescence image. This linear transformation stretches or compresses the standard deviation of the predicted probability map to match that of the real image (through...). Adjustments were made, while the mean was shifted and aligned to the level of the real image (by...). The adjustment process ensures that the calibrated image statistically matches the real fluorescence image in terms of overall brightness and contrast range, preserving the continuous intensity gradient information in the opening operation result image while making the intensity differences between different tissue regions more consistent with biological expectations.

[0058] The virtual fluorescence images of all fluorescence channels are stitched together to form a multi-channel image, and the final virtual fluorescence image is then output as follows: in, This represents the final virtual fluorescence image; Indicates the first Virtual fluorescence images of each fluorescence channel.

[0059] The above process outputs the final virtual fluorescence image. This image possesses both the ability to locate positive regions using binarization and the preservation of continuous signal layers that conform to the true staining intensity distribution. Furthermore, opening operations remove noise artifacts in the prediction, significantly improving the clinical readability and quantitative analysis reliability of the virtual multiplex immunofluorescence staining results.

[0060] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0061] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0062] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A self-attention-enhanced multi-scale tissue section immunofluorescence virtual staining method, characterized in that, include: S1: Obtain the original stained image of the tissue section and perform preprocessing. Input the preprocessed stained image into the constructed self-attention enhanced UNet++ network for deep feature extraction. In each coding layer, the constructed AG-SA module is used to process the deep feature map extracted in the current layer to generate a gated fusion feature map. The gated fusion feature map is downsampled to output the original coding feature map of the corresponding layer, which is then used as the input of the next coding layer. S2: Perform multi-scale feature fusion on the original encoded feature maps output by each layer of the encoder, and input the fused result map and the original encoded feature map of the corresponding layer into the decoder for decoding to obtain the decoded map. Perform full coupling optimization on the decoded map and further calculate the total loss parameter. S3: Construct a training dataset containing multiple pairs of preprocessed staining images and real multiplex immunofluorescence staining images. Randomly sample a batch of paired samples from the training dataset and input the original staining images into the attention-enhanced UNet++ network. Calculate the total loss parameters according to steps S1 and S2. Process the total loss parameters to obtain the learnable parameters of the attention-enhanced UNet++ network. S4: Based on learnable parameters, predict the image of the preprocessed tissue slice to be detected to obtain a multi-channel fluorescence prediction probability map. Perform Gaussian weighted stitching on the multi-channel fluorescence prediction probability map, perform morphological optimization on the stitched image, and output the final virtual fluorescence image.

2. The self-attention-enhanced multi-scale tissue section immunofluorescence virtual staining method according to claim 1, characterized in that, Obtain the original stained images of the tissue sections and perform preprocessing, specifically: Obtain the original stained image of the tissue section, separate its RGB channels, calculate the mean and standard deviation of the pixels in each channel, and normalize the pixel values ​​of each channel based on the mean and standard deviation. The normalized image is subjected to wavelet domain adaptive denoising, and the denoised image is subjected to multi-scale contrast enhancement to obtain the preprocessed stained image.

3. The self-attention-enhanced multi-scale tissue section immunofluorescence virtual staining method according to claim 1, characterized in that, A self-attention-enhanced UNet++ network is used to extract deep features from the preprocessed stained image, specifically as follows: The preprocessed stained image is first subjected to a first-level convolution extraction, that is, using... The system processes features using convolution, batch normalization, and ReLU activation to output the feature map extracted by the first-level convolutional layer. Then, a second-level deep feature extraction is performed on this feature map. Convolution, batch normalization, and ReLU activation function processing are used to obtain feature maps for secondary deep feature extraction.

4. The self-attention-enhanced multi-scale tissue section immunofluorescence virtual staining method according to claim 3, characterized in that, The constructed AG-SA module is used to process the depth feature map extracted from the current layer to generate a gated fusion feature map, specifically including: The feature map extracted by the second depth feature extraction is divided into multiple non-overlapping local feature blocks in the height and width directions, and the local variance of each local feature block is calculated separately. The local variances corresponding to each local feature block are arranged according to their spatial positions in the feature map of the secondary deep feature extraction to form a single-channel statistical feature map. The single-channel statistical feature map is then subjected to convolution and activation processing to generate a gating map, which is used to characterize the feature response intensity of each local region. The feature map extracted by the second deep feature extraction is reconstructed to obtain a two-dimensional feature matrix, and query features, key features and value features are obtained by three learnable linear transformations respectively. Self-attention operations are performed based on the query features, key features, and value features to obtain a self-attention feature map; The self-attention feature map is restored to the spatial structure corresponding to the feature map extracted by the second deep feature extraction. The self-attention feature map and the feature map extracted by the second deep feature extraction are then weighted and fused based on the gating map to obtain a gating fused feature map. The weighted fusion is used to adaptively adjust the fusion ratio of the self-attention feature and the original feature according to the statistical features of the local region.

5. The self-attention-enhanced multi-scale tissue section immunofluorescence virtual staining method according to claim 4, characterized in that, The output is the original encoded feature map, specifically: First, the gated fusion feature map is downsampled to output the first-layer encoded feature map, i.e., a map of size [size missing]. Max pooling is used to downsample the gated fusion feature map to obtain the first layer of encoded feature map; Then, construct the second layer of encoded feature maps: pass the first layer of encoded feature maps sequentially through convolution kernels with a kernel size of... The process involves convolution, batch normalization, and ReLU activation, with 128 convolution kernels; then, it sequentially passes through convolution kernels of size [missing information]. The convolution, batch normalization, and ReLU activation functions are applied, with the number of convolution kernels also being 128. The result is then input into the AG-SA module for further processing, yielding the output of the second-layer AG-SA module. Finally, a kernel of size [size missing] is used... Max pooling is used to downsample the output to obtain the second layer of encoded feature map; Next, the third layer of encoded feature maps is constructed: the second layer of encoded feature maps is sequentially processed through convolution kernels with a kernel size of [size missing]. The process involves convolution, batch normalization, and ReLU activation, with a kernel size of 256; then it sequentially passes through convolution kernels of size [missing information]. The convolution, batch normalization, and ReLU activation functions are applied, with the number of convolution kernels also being 256. The result is then input into the AG-SA module for further processing, yielding the output of the third-layer AG-SA module. Finally, a kernel of size [size missing] is applied. Max pooling is used to downsample the output to obtain the third layer encoded feature map; Finally, construct the fourth layer of encoded feature maps: pass the third layer of encoded feature maps sequentially through convolution kernels with a kernel size of [size missing]. The process involves convolution, batch normalization, and ReLU activation, with a kernel size of 512; then it sequentially passes through convolution kernels of size [missing information]. The convolution, batch normalization, and ReLU activation functions are applied, with the number of convolution kernels also being 512. The result is then input into the AG-SA module for processing, resulting in the output of the fourth-layer AG-SA module. This output is the fourth-layer encoded feature map, and no further downsampling is performed.

6. The self-attention-enhanced multi-scale tissue section immunofluorescence virtual staining method according to claim 5, characterized in that, Multi-scale feature fusion is performed on the original encoded feature maps output from each layer of the encoder, specifically including: The original encoded feature maps of each layer are subjected to average pooling at different scales to obtain the corresponding multi-scale pooling features. Multi-scale pooling features of different scales are uniformly adjusted to a spatial size that matches the corresponding coding layer; The multi-scale pooling features with uniform size are spliced ​​together according to the channel dimension to form a multi-scale fused feature map; The multi-scale fused feature map is subjected to convolutional compression and feature integration to obtain the multi-scale feature fusion result map of the corresponding layer.

7. The self-attention-enhanced multi-scale tissue section immunofluorescence virtual staining method according to claim 6, characterized in that, The decoded image is as follows: The multi-scale feature fusion result map output by the encoder and the original encoded feature map of the corresponding layer are input into the decoder for layer-by-layer decoding. Layer-by-layer decoding includes: starting from deep features, upsampling or resizing the current decoded features to make them match the spatial dimensions of the multi-scale feature fusion result map of the adjacent shallow layers and the original encoded feature map; The size-matched features are fused with the multi-scale feature fusion result map of the corresponding layer and the original encoded feature map, and the decoded feature map of the current layer is generated by feature transformation. The decoding feature maps of each decoding layer are obtained layer by layer until the final decoding map is obtained.

8. The self-attention-enhanced multi-scale tissue section immunofluorescence virtual staining method according to claim 7, characterized in that, The total loss parameter is output as follows: The decoded image is transformed by channel transformation so that the number of output channels corresponds to the number of target fluorescence channels to obtain the original prediction result. The original prediction result is then activated and mapped to obtain the prediction probability map of each fluorescence channel. Based on the predicted probability map and the actual multiplex immunofluorescence staining image, the weighted cross-entropy loss is calculated; Based on the correlation between each fluorescence channel, the channel penalty loss is calculated; Based on the structural similarity between the predicted probability map and the actual multiplex immunofluorescence staining image, the structural similarity loss is calculated; The weighted cross-entropy loss, channel penalty loss, and structural similarity loss are weighted and combined to obtain the total loss parameter; A training dataset containing multiple pairs of preprocessed stained images and real multiplex immunofluorescence stained images is constructed. Paired samples are randomly sampled from the training dataset and input into the self-attention-enhanced UNet++ network. The learnable parameters of the network are iteratively updated according to the total loss parameter until the preset training termination condition is reached, and the trained self-attention-enhanced UNet++ network is obtained.

9. The self-attention-enhanced multi-scale tissue section immunofluorescence virtual staining method according to claim 8, characterized in that, The output multi-channel fluorescence prediction probability map is then Gaussian-weighted and stitched together, specifically as follows: Based on the learnable parameters of the self-attention-enhanced UNet++ network obtained through training, predictions are made for the preprocessed tissue slice images to be detected. The preprocessed tissue slice image to be detected is divided into multiple image sub-blocks, and each is input into the self-attention enhanced UNet++ network to obtain the multi-channel fluorescence prediction probability map corresponding to each image sub-block. Based on the spatial location of each image sub-block in the original tissue slice image, the multi-channel fluorescence prediction probability map corresponding to each image sub-block is mapped back to the complete image; The prediction results in adjacent or overlapping regions are weighted and fused using Gaussian weights, where the weight of the central region of the image sub-block is higher than that of the edge region. The prediction results of each image sub-block after weighted fusion are stitched together to obtain the multi-channel fluorescence prediction probability map corresponding to the complete tissue slice.

10. The self-attention-enhanced multi-scale tissue section immunofluorescence virtual staining method according to claim 9, characterized in that, Morphological optimization is performed on the stitched images to output the final virtual fluorescence image, specifically: The spliced ​​multi-channel fluorescence prediction probability map is processed by opening operations for each channel to remove noise points; After the opening operation, each fluorescence channel image is segmented according to the segmentation threshold of the corresponding channel to obtain a binary image of each fluorescence channel; Based on the mean and standard deviation of the real fluorescence image and the predicted fluorescence image of each fluorescence channel, the intensity scaling parameter and intensity offset parameter of the corresponding channel are determined, and the intensity of each fluorescence channel image after the opening operation is calibrated to obtain the virtual fluorescence image of each fluorescence channel. The virtual fluorescence images of each fluorescence channel are combined into a multi-channel image, which is then output as the final virtual fluorescence image.

Images