Automatic Analysis Method, Medium and Program Product for Multiplex Immunofluorescence Images

Abstract

The present invention discloses an automatic analysis method, medium and program product for multi - immunofluorescence images, including obtaining a DAPI channel image and several non - DAPI channel images by processing the acquired multi - immunofluorescence images; performing segmentation processing on the DAPI channel image to obtain a nuclear region mask; superimposing the nuclear region mask on each non - DAPI channel image respectively to obtain new non - DAPI channel images; performing wavelength encoding processing and position encoding processing on the DAPI channel image and each new non - DAPI channel image in sequence, and sending the processed images into a pre - trained Transformer network, and the Transformer network performs denoising and cross - talk removal processing on the channels to obtain a pure multi - channel image; performing analysis processing on the pure multi - channel image according to preset analysis requirements. The present invention can solve the noise problem and spectral interference problem of multi - immunofluorescence images.

Description

Technical Field

[0001] The present invention belongs to the technical field of image analysis, and particularly relates to a method, medium and program product for automatic analysis of multiplex immunofluorescence images. Background Art

[0002] Immunofluorescence imaging technology is an important biomedical imaging tool that uses fluorescently labeled antibodies to observe the location and distribution of specific molecules or structures in a sample. This technology enables researchers to observe the expression and localization of biomolecules at the cellular or tissue level. However, due to the limitations of existing tools and methods, the information extracted from tissue sections is often restricted. Usually, tissue evaluation is limited to the visual analysis of the expression intensity of a few proteins or DNA / RNA molecules. This highlights the necessity of developing and applying new technologies to extract more comprehensive information from tissue sections.

[0003] The multiplex immunofluorescence staining technology based on the principle of tyramide signal amplification (TSA) can achieve simultaneous staining of multiple markers on the same tissue section. Combined with multispectral imaging technology, researchers can explore the spatial distribution and interactions of different markers within a single cell. This technology provides strong support for biomedical research, especially in the context of the growing demand for multi-channel imaging, making it possible to simultaneously observe and analyze multiple target molecules or different markers.

[0004] Specifically, in multiplex immunofluorescence experiments, multiple fluorescent labels are usually used, each targeting a different molecule or structure, thereby generating immunofluorescence images of multiple different channels. By integrating the fluorescence image data of these different channels for multi-channel image analysis and finally presenting it on the image, researchers can intuitively observe the spatial distribution and co-localization relationship of multiple molecules in cells or tissues, which is of great significance for understanding the interactions between molecules and their functions in biological processes.

[0005] In the study of multiplex fluorescence images, it is usually necessary to simultaneously image and process multiple markers from different channels to achieve the visualization, localization and quantitative analysis of multiple molecules in the same biological sample. Although multiplex immunofluorescence imaging technology has great application potential, it still faces many challenges in actual operation.

[0006] 1. Noise problem: Due to the limited sensor sensitivity of multiplex fluorescence staining scanners, the multi-channel images obtained often contain noise, which affects the data quality.

[0007] 2. Spectral interference problem: Specifically, it includes spectral overlap between different fluorescent dyes and autofluorescence of the sample itself. These factors will cause the mixing of multiple target signals, reduce the signal-to-noise ratio of the target antigen fluorescence signal, and make the result analysis more complicated.

[0008] Therefore, there is an urgent need for advanced fluorescence image processing techniques and multi-channel imaging algorithms to optimize data quality, reduce interference, and improve analysis accuracy. Summary of the Invention

[0009] In view of the above problems, the present invention proposes an automatic analysis method, medium, and program product for multi-channel immunofluorescence images, which can solve the noise problem and spectral interference problem of multi-channel immunofluorescence images.

[0010] To achieve the above technical objectives and effects, the present invention is realized through the following technical solutions:

[0011] In a first aspect, the present invention provides an automatic analysis method for multi-channel immunofluorescence images, including:

[0012] Processing the obtained multi-channel immunofluorescence image to obtain a DAPI channel image and several non-DAPI channel images;

[0013] Performing segmentation processing on the DAPI channel image to obtain a nuclear region mask;

[0014] Overlaying the nuclear region mask with each non-DAPI channel image respectively to obtain a new non-DAPI channel image;

[0015] Performing wavelength encoding processing and position encoding processing on the DAPI channel image and each new non-DAPI channel image in sequence, and sending the processed images into a pre-trained Transformer network, and performing denoising and cross-channel bleeding removal processing by the Transformer network to obtain a pure multi-channel image;

[0016] Analyzing and processing the pure multi-channel image according to a preset analysis requirement.

[0017] In combination with the first aspect, optionally, the wavelength encoding processing specifically includes the following steps:

[0018] Let the channel correspond to the wavelength , and calculate the normalized wavelength ;

[0019] Based on the normalized wavelength , calculate the channel spectral wavelength encoding corresponding to the channel ;

[0020] Adding the channel image of the channel to the channel spectral wavelength encoding corresponding to the channel to obtain a new channel image of the channel , and completing the wavelength encoding processing.

[0021] Combined with the first aspect, optionally, the normalized wavelength has the following calculation formula:

[0022] ;

[0023] In the formula, and are respectively the maximum wavelength value and the minimum wavelength value in all channels;

[0024] The calculation formula for the channel spectral wavelength encoding is:

[0025] ;

[0026] In the formula, is the channel spectral wavelength encoding.

[0027] Combined with the first aspect, optionally, the position encoding process specifically includes the following steps:

[0028] Divide each single-channel image processed by wavelength encoding into image blocks of a preset size;

[0029] Map each image block to a one-dimensional vector, and combine all the one-dimensional vectors into an input sequence of a preset size;

[0030] Based on the input sequence, calculate the position encoding of each image block;

[0031] Add the position encoding after each image block.

[0032] Combined with the first aspect, optionally, the calculation formula for the position encoding of each image block is:

[0033] ;

[0034] ;

[0035] In the formula, represents the position encoding of the image block, represents the position index of the image block in the input sequence; represents the index of the embedding dimension. Since each image block becomes a -dimensional vector after being flattened, so ranges from 0 to ; represents the embedding dimension processed by the Transformer network.

[0036] Combined with the first aspect, optionally, the loss function adopted by the Transformer network during training is:

[0037] ;

[0038] Among them,

[0039] ;

[0040] ;

[0041] ;

[0042] In the formula, is the total loss, is the L1 loss, is the structural similarity loss, is the spectral consistency loss; , respectively represent the network output value and the predicted true value, represents the calculated structural similarity, , respectively represent the predicted value and the true value of the image on the channel with wavelength , is the total number of pixels in the selected area.

[0043] Combined with the first aspect, optionally, the analysis and processing of the pure multi-channel image according to the preset analysis requirements specifically include:

[0044] Quantitatively calculate the average fluorescence intensity, total fluorescence intensity, and maximum fluorescence intensity in the user-selected area based on the pixel brightness of the image of the target channel;

[0045] The calculation formula for the average fluorescence intensity is:

[0046] ;

[0047] The calculation formula for the total fluorescence intensity is:

[0048] ;

[0049] The calculation formula for the maximum fluorescence intensity is:

[0050] ;

[0051] In the formula, is the average fluorescence intensity, is the total fluorescence intensity, is the maximum fluorescence intensity, is the th fluorescence intensity value of the pixel, is the total number of pixels in the selected area.

[0052] In combination with the first aspect, optionally, the analysis and processing of the pure multi-channel image according to a preset analysis requirement further includes:

[0053] Dividing the protein expression level corresponding to the fluorescence intensity into different grades according to a series of thresholds set by the user, splitting or combining the processed pure multi-channel image, visually displaying the co-expression state, and then comprehensively analyzing the spatial distribution and expression intensity relationship of different biomarkers in the multi-fluorescence image.

[0054] In a second aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the multi-immunofluorescence image automatic analysis method according to any one of the first aspect.

[0055] In a third aspect, the present invention provides a computer program product, including a computer program / instructions, and when the computer program / instructions are executed by a processor, it implements the multi-immunofluorescence image automatic analysis method according to any one of the first aspect.

[0056] Compared with the prior art, the beneficial effects of the present invention are:

[0057] In the preprocessing stage of the present invention, the obtained multi-immunofluorescence image is split into multi-channel images. For the DAPI channel image, a segmentation method is used to obtain a nuclear region mask, and the nuclear region mask is used to process other channel images to obtain new channel images. Subsequently, wavelength encoding processing and position encoding processing are performed in sequence, paying attention to the connection between different channels, and the processed image is sent into a pre-trained Transformer network, and the Transformer network performs denoising and channel crosstalk removal processing to obtain a pure multi-channel image, solving the noise problem and spectral interference problem of multi-fluorescence images.

[0058] Furthermore, the present invention also provides a fluorescence intensity evaluation and protein expression level statistical analysis function, which helps users perform statistical processing on data, lays a foundation for subsequent in-depth analysis of cell marker distribution and intensity, and helps researchers explore and discover new cell phenotypes in combination with co-expression combinations in the image. This has important value for deeply understanding the pathogenesis of diseases and finding new therapeutic targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings, where:

[0060] Figure 1 Schematic diagram of the rectification process of an automatic analysis method for multi - immunofluorescence images according to an embodiment of the present invention;

[0061] Figure 2 Detailed flowchart of processing a multi - channel image with noise into a pure multi - channel image according to an embodiment of the present invention. Specific embodiments

[0062] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0063] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, such descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Thus, features defined with "first", "second" may explicitly or implicitly include at least one such feature. In addition, the technical solutions between various embodiments can be combined with each other, but it must be based on the fact that those of ordinary skill in the art can implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0064] Embodiment 1

[0065] An automatic analysis method for multi - immunofluorescence images is provided in an embodiment of the present invention. As Figure 1 shown, it includes the following steps:

[0066] (1) Process the obtained multi - immunofluorescence image to obtain a DAPI channel image and several non - DAPI channel images;

[0067] (2) Perform segmentation processing on the DAPI channel image to obtain a nuclear region mask; in the specific implementation process, the segmentation processing can be implemented using the Stardist algorithm;

[0068] (3) Superimpose the nuclear region mask on each non - DAPI channel image respectively to obtain a new non - DAPI channel image;

[0069] (4) Perform wavelength encoding processing and position encoding processing on the DAPI channel image and each new non-DAPI channel image in sequence, and send the processed images into a pre-trained Transformer network. The Transformer network performs denoising and cross-channel crosstalk removal processing to obtain a pure multi-channel image;

[0070] (5) Analyze and process the pure multi-channel image according to preset analysis requirements.

[0071] In the automatic analysis method for multi-immunofluorescence images in the embodiments of the present invention, in the preprocessing stage, the obtained multi-immunofluorescence image is split into multi-channel images. For the DAPI channel image, a nuclear region mask is obtained by using a segmentation method, and the nuclear region mask is used to process other channel images to obtain new channel images. Subsequently, wavelength encoding processing and position encoding processing are performed in sequence, paying attention to the connection between different channels, and the processed images are sent into a pre-trained Transformer network. The Transformer network performs denoising and cross-channel crosstalk removal processing to obtain a pure multi-channel image, solving the noise problem and spectral interference problem of multi-fluorescence images.

[0072] In a specific implementation manner of the embodiments of the present invention, the wavelength encoding processing specifically includes the following steps:

[0073] Let the wavelength corresponding to channel be , and calculate the normalized wavelength ;

[0074] Based on the normalized wavelength , calculate the channel spectral wavelength encoding corresponding to channel ;

[0075] Add the channel image of channel to the channel spectral wavelength encoding corresponding to channel to obtain the new channel image of channel , completing the wavelength encoding processing.

[0076] In a specific implementation manner of the embodiments of the present invention, the calculation formula for the wavelength is:

[0077] ;

[0078] In the formula, , are respectively the maximum wavelength value and the minimum wavelength value in all channels.

[0079] In a specific implementation manner of an embodiment of the present invention, the calculation formula for the channel spectral wavelength encoding is:

[0080] ;

[0081] In the formula, is the channel spectral wavelength encoding.

[0082] In a specific implementation manner of an embodiment of the present invention, the position encoding processing specifically includes the following steps:

[0083] Segment each single-channel image that has undergone wavelength encoding processing into image blocks of a preset size;

[0084] Map each image block to a one-dimensional vector, and combine all the one-dimensional vectors into an input sequence of a preset size;

[0085] Based on the input sequence, calculate the position encoding of each image block;

[0086] Add the position encoding after each image block.

[0087] In a specific implementation manner of an embodiment of the present invention, the calculation formula for the position encoding of each image block is:

[0088] ;

[0089] ;

[0090] In the formula, represents the position encoding of the image block, represents the position index of the current image block in the input sequence; represents the index of the embedding dimension. Since each image block is flattened into a -dimensional vector after being flattened, so takes values in the range from 0 to ; represents the embedding dimension processed by the Transformer network.

[0091] In a specific implementation manner of an embodiment of the present invention, the loss function adopted by the Transformer network during training is:

[0092]

[0093] Among them,

[0094] ;

[0095] ;

[0096] ;

[0097] In the formula, is the total loss, is the L1 loss, is the structural similarity loss, is the spectral consistency loss; and respectively represent the network output value and the predicted true value, represents the calculated structural similarity, and respectively represent the predicted value and the true value of the image on the channel with wavelength .

[0098] In a specific implementation manner of the embodiment of the present invention, the analysis and processing of the pure multi-channel image according to the preset analysis requirements specifically include:

[0099] Quantitatively calculate the average fluorescence intensity, total fluorescence intensity, and maximum fluorescence intensity in the user-selected area based on the pixel brightness of the image of the target channel;

[0100] The calculation formula of the average fluorescence intensity is:

[0101] ;

[0102] The calculation formula of the total fluorescence intensity is:

[0103] ;

[0104] The calculation formula of the maximum fluorescence intensity is:

[0105] ;

[0106] In the formula, is the average fluorescence intensity, is the total fluorescence intensity, is the maximum fluorescence intensity, is the th fluorescence intensity value of the pixel, is the total number of pixels in the selected area.

[0107] In a specific implementation manner of the embodiment of the present invention, the analysis and processing of the pure multi-channel image according to the preset analysis requirements further include:

[0108] According to a series of thresholds set by the user, divide the protein expression levels corresponding to the fluorescence intensity into different grades, split or combine the processed pure multi-channel image, visually display the co-expression state, and then comprehensively analyze the spatial distribution and expression intensity relationship of different biomarkers in the multi-fluorescence image.

[0109] The following will combine with a specific embodiment to elaborate in detail on the automatic analysis method for multi - immunofluorescence images in the embodiments of the present invention.

[0110] As Figure 1 shown, the automatic analysis method for multi - immunofluorescence images in the embodiments of the present invention includes the following steps:

[0111] (1) Process the acquired multi - immunofluorescence image to obtain a DAPI channel image and several non - DAPI channel images, specifically including:

[0112] After the user imports the immunofluorescence image and the original image is displayed on the user interface, the background enters the processing of multi - immunofluorescence images. First, pre - process the multi - immunofluorescence image through the existing spectral library to obtain a single DAPI channel image and several non - DAPI channel images.

[0113] (2) Segment the DAPI channel image to obtain a nuclear region mask, specifically including:

[0114] Since the original multi - channel image may be interfered by other signals outside the nucleus, such as non - specific staining or background fluorescence, reducing the signal - to - noise ratio. DAPI is a commonly used fluorescent dye that can specifically bind to DNA, thus clearly labeling the nucleus and accurately identifying and locating the boundary of each nucleus. Therefore, in the embodiments of the present invention, the Stardist algorithm commonly used in biological image analysis is used to segment the DAPI channel image to obtain a nuclear region mask, ensuring that only the signals inside the nucleus are analyzed subsequently.

[0115] (3) Superimpose the nuclear region mask on each non - DAPI channel image respectively to obtain a new non - DAPI channel image, specifically including:

[0116] By superimposing the nuclear region mask on the channel images of other channels, a new non - DAPI channel image is obtained, which is used to analyze the distribution and co - localization of target proteins in the nucleus, thereby initially reducing the noise in the channel and excluding the interference of other signals outside the nucleus.

[0117] (4) Perform wavelength encoding processing and position encoding processing on the DAPI channel image and each new non - DAPI channel image in sequence, and send the processed images into a pre - trained Transformer network. The Transformer network performs denoising and channel crosstalk removal processing to obtain a pure multi - channel image, as Figure 2 shown, specifically including:

[0118] According to the type of channel input by the user, after wavelength normalization, encoding is performed to guide the downstream model to focus on the wavelengths of adjacent channels. Specifically, let the channel correspond to the wavelength , and perform wavelength encoding for each channel, calculating the channel spectral wavelength encoding corresponding to the channel . Among them, the normalized wavelength is :

[0119] (1.1)

[0120] The calculation formula for the channel spectral wavelength encoding is:

[0121] (1.2)

[0122] In the formula, , are respectively the maximum wavelength value and the minimum wavelength value in all channels. Formula (1.1) normalizes the wavelength corresponding to each channel to between [0, 1]. Formula (1.2) is the calculation formula for the channel spectral wavelength encoding. The sine and cosine functions are respectively used, and 100 is the scaling factor to control the frequency change of different dimensions. Its processing is similar to the position encoding of Transformer. Since the sine and cosine functions are periodic, they can give a channel a smoothly changing value, making the encoding of adjacent channels ordered, enabling the model to learn the relative relationship of the spectral channels. The new feature of this channel is obtained by adding the original feature and the channel spectral wavelength encoding, and its feature encoding method is also consistent with the position encoding of Transformer:

[0123] ;

[0124] The new feature introduces the relative relationship between different wavelengths, which helps the downstream optimization model to identify the association between different wavelengths (frequencies), and thus makes it easier to learn the information of adjacent channels.

[0125] Subsequently, for each single-channel image, according to the processing steps of Transformer, it is divided into small image blocks with a size of , and each image block is mapped into a one-dimensional vector. Assuming the image size is , and each image block is divided into a size of , it will be divided into image blocks, and each image block (single-channel) contains 256 pixel values. Finally, the picture is flattened into a size of The input sequence. Add position encoding (PE) after each image patch. The specific implementation is similar to ViT. For each image patch of size the position encoding of the image patch can be calculated as follows:

[0126] (1.4)

[0127] where represents the position index of the current image patch in the input sequence, ranging from 0 to 195 (a total of 196 image patches); represents the index of the embedding dimension. Since each image patch is flattened into a 256-dimensional vector after that, so the value range is from 0 to 255; represents the embedding dimension processed by the Transformer, which is 256 here (corresponding to the vector length of 256 in the previous text), and 10000 is the scaling factor to control the frequency variation of different dimensions. The roles of the sine and cosine functions are similar to the spectral wavelength encoding in the previous text. For the entire sequence of size its position index can be expressed as:

[0128] (1.5)

[0129] For the feature of the image patch located in the input sequence on the spectral channel after introducing the spectral wavelength encoding and position encoding, the new feature can be expressed as:

[0130] (1.6)

[0131] Thus, the features with spectral and spatial position information encoding are obtained.

[0132] Send the processed image into a pre-trained Transformer network. The Transformer network performs denoising and cross-channel bleeding removal processing to obtain a pure multi-channel image; the main tasks of the Transformer network in the embodiments of the present invention are denoising and cross-channel bleeding removal. The specific parameter definitions of the Transformer network are shown in Table 1.

[0133] Table 1 Specific parameters of the Transformer network

[0134] parameter definition value d_model Model dimension, the dimension of the embedding vector used in the Transformer model 256 nhead The number of attention heads in the multi-head attention mechanism 8 dim_feedforward Represents the hidden layer dimension of the feed-forward neural network in the encoder and decoder 512 num_layers The number of stacked layers in the encoder and decoder (affects the complexity and performance of the model) 6

[0135] When training the Transformer network, the label values are set by the following method:

[0136] Use PCA decomposition to model autofluorescence and subtract the modeled autofluorescence signal; subsequently, according to the known fluorescence spectrum, use Linear Spectral Unmixing (LSU) to obtain the signal after crosstalk elimination for each channel; finally, use self-supervised denoising based on Noise2Void to reduce the noise in the image. In this way, a pure multi-channel fluorescence image extracted by physical methods can be obtained.

[0137] The loss function adopted by the Transformer network during training is:

[0138] ;

[0139] Among them,

[0140] ;

[0141] ;

[0142] ;

[0143] In the formula, is the total loss, is the L1 loss, is the structural similarity loss, is the spectral consistency loss; 、 respectively represent the network output value and the predicted true value, represents the calculated structural similarity, 、 respectively represent the image predicted value and the true value on the channel with wavelength .

[0144] (5) Analyze and process the pure multi-channel image according to the preset analysis requirements, specifically including:

[0145] (5.1) Immunofluorescence signal intensity evaluation

[0146] Based on the pixel brightness of the pure single-channel image of the target channel, quantitatively calculate the mean fluorescence intensity (Mean Fluorescence Intensity, MFI) in the user-selected area:

[0147] ;

[0148] Among them: is the The fluorescence intensity value of a pixel, is the total number of pixels within the selected region. The average fluorescence signal intensity can directly reflect the density of the actual content of the staining marker in this region.

[0149] Total fluorescence intensity information (Total Fluorescence Intensity, TFI):

[0150] ;

[0151] The total fluorescence intensity can reflect the total amount of the staining marker in this region, and in applications, it can evaluate the expression level of specific proteins or markers in the cell nuclei within the region.

[0152] Maximum fluorescence intensity (Max Fluorescence Intensity, MaxFI)

[0153] ;

[0154] In the formula, is the average fluorescence intensity, is the total fluorescence intensity, is the maximum fluorescence intensity, is the fluorescence intensity value of the th pixel, is the total number of pixels within the selected region.

[0155] The maximum fluorescence intensity can reflect the peak expression of the staining marker in this region, and in practical applications, it can be used as a quantitative analysis index to measure the expression differences of markers in pathological samples.

[0156] Furthermore, fluorescence intensity distribution statistics can also be performed: by calculating and displaying statistics such as the standard deviation and median of the fluorescence intensity, and using a histogram to visually visualize the distribution, so as to evaluate the distribution characteristics of the fluorescence signal, thus supporting subsequent visualization operations.

[0157] (5.2) Integrated display of fluorescence images

[0158] Supports users to set a series of thresholds to divide the protein expression levels corresponding to the fluorescence intensity into different grades, and these thresholds can be set according to actual needs and data distribution. For example, the markers in the cells with the lowest fluorescence intensity values can be classified as the "low expression" grade, the markers in the cells with the highest fluorescence intensity values can be classified as the "high expression" grade, and the cells in between can be classified as the "medium expression" grade, such as low expression (1+), medium expression (2+), and high expression (3+), etc.

[0159] Meanwhile, users can, according to actual needs, use the built-in ticking tool of the patent to split or combine the processed multi-channel data, intuitively display the co-expression status, and then comprehensively analyze the spatial distribution and expression intensity relationship of different biomarkers in multi-fluorescence images.

[0160] In practical applications, this patent can be used for co-localization analysis: statistically analyzing the expression intensities of multiple different markers at different levels on the same cell, providing support for the fine description of cell characteristics. Different cell phenotypes can be defined based on different marker combinations, such as regulatory T cells (CD3+CD4+FOXP3+) and double-negative T cells (CD3+CD4-CD8-), etc.

[0161] Example 2

[0162] In the embodiments of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, it implements the multi-immunofluorescence image automatic analysis method described in any one of Embodiment 1.

[0163] Example 3

[0164] In the embodiments of the present invention, a computer program product is provided, including a computer program / instructions, and when the computer program / instructions are executed by a processor, it implements the multi-immunofluorescence image automatic analysis method described in any one of Embodiment 1.

[0165] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0166] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0167] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a particular manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction means that implements the functions specified in one process or a plurality of processes and / or one block or a plurality of blocks in the flow Figure 1 one process or a plurality of processes and / or Figure 1 one block or a plurality of blocks.

[0168] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operational steps are performed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one process or a plurality of processes and / or one block or a plurality of blocks in the flow Figure 1 one process or a plurality of processes and / or Figure 1 one block or a plurality of blocks.

[0169] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the spirit and scope of the present invention as protected by the claims. All of these fall within the protection scope of the present invention.

[0170] The above has shown and described the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and the description in the specification are only to illustrate the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and all of these changes and improvements fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for automatic analysis of multiple immunofluorescence images, characterized in that: include: Processing the acquired multiple immunofluorescence images to obtain DAPI channel images and several non-DAPI channel images; Segmenting the DAPI channel image to obtain a cell nucleus region mask; Superimposing the cell nucleus region mask with each non-DAPI channel image to obtain a new non-DAPI channel image; The DAPI channel image and each new non-DAPI channel image are sequentially subjected to wavelength encoding processing and position encoding processing, and the processed images are sent to a pre-trained Transformer network, and the Transformer network performs denoising and channel cross-color removal processing to obtain a pure multi-channel image; Analyzing and processing the pure multi-channel image according to preset analysis requirements; The wavelength encoding process specifically comprises the following steps: Let the wavelength corresponding to channel c be λ c , calculate the normalized wavelength Based on the normalized wavelength Calculate the channel spectrum wavelength code corresponding to channel c; The channel image of channel c is added to the channel spectrum wavelength code corresponding to channel c to obtain a new channel image of channel c, thus completing the wavelength coding process; The position coding process specifically comprises the following steps: Each single-channel image processed by wavelength encoding is divided into image blocks of preset size; Map each image block into a one-dimensional vector, and combine all the one-dimensional vectors into an input sequence of a preset size; Based on the input sequence, calculating the position code of each image block; Add position code after each image block; The analyzing and processing the pure multi-channel image according to the preset analysis requirements specifically includes: Based on the pixel brightness of the image of the target channel, the average fluorescence intensity, total fluorescence intensity and maximum fluorescence intensity in the user-selected area are quantitatively calculated; The calculation formula of the mean fluorescence intensity is: The calculation formula of the total fluorescence intensity is: The calculation formula of the maximum fluorescence intensity is: MaxFI=max(I1,I2,...,I N ); Where MFI is the mean fluorescence intensity, TFI is the total fluorescence intensity, MaxFI is the maximum fluorescence intensity, I k is the fluorescence intensity value of the kth pixel, and N is the total number of pixels in the selected area.

2. The method for automatic analysis of multiple immunofluorescence images according to claim 1, characterized in that: The normalized wavelength The calculation formula is: In the formula, max(λ) and min(λ) are the maximum wavelength and minimum wavelength in all channels respectively; The calculation formula of the channel spectrum wavelength encoding is: In the formula, Encode the channel spectrum wavelength.

3. The method for automatic analysis of multiple immunofluorescence images according to claim 1, characterized in that: The calculation formula for the position encoding of each image block is: Where PE′() represents the position encoding of the image block, pos represents the position index of the image block in the input sequence; j represents the index of the embedding dimension. Since each image block becomes an M-dimensional vector after being flattened, the value range of 2j,2j+1 is 0 to M-1; d represents the embedding dimension processed by the Transformer network.

4. The method for automatic analysis of multiple immunofluorescence images according to claim 1, characterized in that: The loss function used by the Transformer network during training is: in, In the formula, is the total loss, is the L1 loss, is the structural similarity loss, is the spectral consistency loss; I pred ,I GT Represent the network output value and the predicted true value, SSIM(I pred ,I GT ) represents the calculated structural similarity, Respectively represent the wavelength λ i The image prediction value and true value on the channel, N is the total number of pixels in the selected area.

5. The method for automatic analysis of multiple immunofluorescence images according to claim 1, characterized in that: The analyzing and processing the pure multi-channel image according to the preset analysis requirements also includes: The protein expression levels corresponding to the fluorescence intensity are divided into different levels according to a series of thresholds set by the user. The processed pure multi-channel images are split or combined to intuitively display the co-expression status, and then the spatial distribution and expression intensity relationship of different biomarkers in the multi-fluorescence images are comprehensively analyzed.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the automatic multiple immunofluorescence image analysis method according to any one of claims 1 to 5 is implemented.

7. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the method for automatic analysis of multiple immunofluorescence images according to any one of claims 1 to 5 is implemented.

Images