Multi-immunofluorescence image automatic analysis method, medium and program product

Abstract

The invention discloses a multi-immunofluorescence image automatic analysis method, a medium and a program product. The multi-immunofluorescence image automatic analysis method comprises the following steps: processing an obtained multi-immunofluorescence image to obtain a DAPI channel image and a plurality of non-DAPI channel images; carrying out segmentation processing on the DAPI channel image to obtain a cell nucleus region mask; performing superposition processing on the cell nucleus region mask and each non-DAPI channel image to obtain a new non-DAPI channel image; sequentially carrying out wavelength coding processing and position coding processing on the DAPI channel image and each new non-DAPI channel image, sending the processed images into a pre-trained Transform network, and carrying out denoising and channel cross color removal processing by the Transform network to obtain a pure multi-channel image; and analyzing and processing the pure multi-channel image according to a preset analysis requirement. According to the method, the noise problem and the spectral interference problem of multiple immunofluorescence images can be solved.

Description

Technical Field

[0001] The invention belongs to the technical field of image analysis, and in particular relates to a multiple immunofluorescence image automatic analysis method, a medium and a program product. Background Art

[0002] Immunofluorescence imaging is an important biomedical imaging tool that uses fluorescently labeled antibodies to visualize the location and distribution of specific molecules or structures in a sample. This technology allows researchers to observe the expression and localization of biomolecules at the cellular or tissue level. However, the information extracted from tissue sections is often limited due to the limitations of existing tools and methods. Typically, tissue assessment is limited to a visual analysis of the expression intensity of a few proteins or DNA / RNA molecules. This highlights the need to develop and apply new technologies to extract more comprehensive information from tissue sections.

[0003] The multiple immunofluorescence staining technique based on the tyramide signal amplification (TSA) principle can achieve simultaneous staining of multiple markers on the same tissue section. Combined with multispectral imaging technology, researchers can explore the spatial distribution of different markers in a single cell and their interactions. This technology provides strong support for biomedical research, especially in the context of the growing demand for multi-channel imaging, making it possible to observe and analyze multiple target molecules or different markers simultaneously.

[0004] Specifically, in multiple immunofluorescence experiments, multiple fluorescent markers are usually used, each targeting a different molecule or structure, thereby generating immunofluorescence images of multiple different channels. By integrating the fluorescence image data of these different channels for multi-channel image analysis and finally presenting them on the image, researchers can intuitively observe the spatial distribution and co-localization relationship of multiple molecules in cells or tissues, which is of great significance for understanding the interactions between molecules and their functions in biological processes.

[0005] In the study of multiple fluorescence images, it is usually necessary to simultaneously image and process multiple markers from different channels to achieve the visualization and quantitative analysis of multiple molecules in the same biological sample. Although multiplex immunofluorescence imaging technology has great application potential, it still faces many challenges in actual operation.

[0006] 1. Noise problem: Due to the limited sensor sensitivity of the multiple fluorescent staining scanner, the acquired multi-channel images often contain noise, which affects the data quality.

[0007] 2. Spectral interference problem: specifically including the spectral overlap between different fluorescent dyes and the autofluorescence of the sample itself. These factors will cause the mixing of multiple target signals, reduce the signal-to-noise ratio of the target antigen fluorescence signal, and make the result analysis more complicated.

[0008] Therefore, advanced fluorescence image processing techniques and multi-channel imaging algorithms are urgently needed to optimize data quality, reduce interference and improve analysis accuracy. Summary of the invention

[0009] In view of the above problems, the present invention proposes a multiplex immunofluorescence image automatic analysis method, medium and program product, which can solve the noise problem and spectral interference problem of multiplex immunofluorescence images.

[0010] In order to achieve the above technical objectives and the above technical effects, the present invention is implemented through the following technical solutions:

[0011] In a first aspect, the present invention provides a method for automatically analyzing multiple immunofluorescence images, comprising:

[0012] Processing the acquired multiple immunofluorescence images to obtain DAPI channel images and several non-DAPI channel images;

[0013] Segmenting the DAPI channel image to obtain a cell nucleus region mask;

[0014] Superimposing the cell nucleus region mask with each non-DAPI channel image to obtain a new non-DAPI channel image;

[0015] The DAPI channel image and each new non-DAPI channel image are sequentially subjected to wavelength encoding processing and position encoding processing, and the processed images are sent to a pre-trained Transformer network, and the Transformer network performs denoising and channel cross-color removal processing to obtain a pure multi-channel image;

[0016] The pure multi-channel image is analyzed and processed according to preset analysis requirements.

[0017] In combination with the first aspect, optionally, the wavelength encoding process specifically includes the following steps:

[0018] Set up channel The corresponding wavelength is , calculate the normalized wavelength ;

[0019] Based on the normalized wavelength , calculate the channel The corresponding channel spectrum wavelength encoding;

[0020] Channel Channel images and channels The corresponding channel spectrum wavelength codes are added to obtain the channel The new channel image is obtained to complete the wavelength encoding processing.

[0021] In combination with the first aspect, optionally, the normalized wavelength The calculation formula is:

[0022] ;

[0023] In the formula, , are the maximum and minimum wavelength values ​​in all channels respectively;

[0024] The calculation formula of the channel spectrum wavelength encoding is:

[0025] ;

[0026] In the formula, Encode the channel spectrum wavelength.

[0027] In combination with the first aspect, optionally, the position coding process specifically includes the following steps:

[0028] Each single-channel image processed by wavelength encoding is divided into image blocks of preset size;

[0029] Map each image block into a one-dimensional vector, and combine all the one-dimensional vectors into an input sequence of a preset size;

[0030] Based on the input sequence, calculating the position code of each image block;

[0031] Add a position code after each image block.

[0032] In combination with the first aspect, optionally, a calculation formula for the position encoding of each image block is:

[0033] ;

[0034] ;

[0035] In the formula, represents the position code of the image block, Indicates the position index of the image block in the input sequence; Represents the index of the embedding dimension. Since each image block is flattened, it becomes a dimensional vector, so The value range is 0 to ; Represents the embedding dimension processed by the Transformer network.

[0036] In combination with the first aspect, optionally, the loss function used by the Transformer network during training is:

[0037] ;

[0038] in,

[0039] ;

[0040] ;

[0041] ;

[0042] In the formula, is the total loss, is the L1 loss, is the structural similarity loss, is the spectral consistency loss; , Represent the network output value and the predicted true value respectively, represents the structural similarity of the calculation, , Respectively represent the wavelength The image prediction value and true value on the channel, is the total number of pixels in the selected area.

[0043] In combination with the first aspect, optionally, the analyzing and processing the clean multi-channel image according to a preset analysis requirement specifically includes:

[0044] Based on the pixel brightness of the image of the target channel, the average fluorescence intensity, total fluorescence intensity and maximum fluorescence intensity in the user-selected area are quantitatively calculated;

[0045] The calculation formula of the mean fluorescence intensity is:

[0046] ;

[0047] The calculation formula of the total fluorescence intensity is:

[0048] ;

[0049] The calculation formula of the maximum fluorescence intensity is:

[0050] ;

[0051] In the formula, is the mean fluorescence intensity, is the total fluorescence intensity, is the maximum fluorescence intensity, For the The fluorescence intensity value of each pixel, is the total number of pixels in the selected area.

[0052] In combination with the first aspect, optionally, the analyzing and processing the clean multi-channel image according to a preset analysis requirement further includes:

[0053] The protein expression levels corresponding to the fluorescence intensity are divided into different levels according to a series of thresholds set by the user. The processed pure multi-channel images are split or combined to intuitively display the co-expression status, and then the spatial distribution and expression intensity relationship of different biomarkers in the multi-fluorescence images are comprehensively analyzed.

[0054] In a second aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for automatic analysis of multiple immunofluorescence images as described in any one of the first aspects.

[0055] In a third aspect, the present invention provides a computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the method for automatic analysis of multiple immunofluorescence images described in any one of the first aspects.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] In the preprocessing stage, the present invention splits the acquired multiple immunofluorescence images into multi-channel images, adopts a segmentation method to obtain a nucleus region mask for the DAPI channel image, uses the nucleus region mask to process other channel images to obtain a new channel image, then performs wavelength encoding processing and position encoding processing in sequence, pays attention to the connection between different channels, and sends the processed images to a pre-trained Transformer network, which performs denoising and channel cross-color removal processing to obtain a pure multi-channel image, thereby solving the noise problem and spectral interference problem of multi-fluorescence images.

[0058] Furthermore, the present invention also provides fluorescence intensity evaluation and protein expression level statistical analysis functions to help users perform statistical processing on data, laying the foundation for subsequent in-depth analysis of cell marker distribution and intensity, and helping researchers explore and discover new cell phenotypes based on co-expression combinations in images. This is of great value for in-depth understanding of the pathogenesis of diseases and finding new therapeutic targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor, among which:

[0060] Figure 1 A schematic diagram of a rectification process of a method for automatic analysis of multiple immunofluorescence images according to an embodiment of the present invention;

[0061] Figure 2 The present invention is a detailed flowchart of processing a noisy multi-channel image into a pure multi-channel image according to an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0064] Example 1

[0065] The present invention provides a method for automatically analyzing multiple immunofluorescence images. Figure 1 As shown, the following steps are included:

[0066] (1) Processing the acquired multiple immunofluorescence images to obtain DAPI channel images and several non-DAPI channel images;

[0067] (2) performing segmentation processing on the DAPI channel image to obtain a cell nucleus region mask; in a specific implementation process, the segmentation processing can be implemented using the Stardist algorithm;

[0068] (3) superimposing the cell nucleus region mask with each non-DAPI channel image to obtain a new non-DAPI channel image;

[0069] (4) performing wavelength encoding processing and position encoding processing on the DAPI channel image and each new non-DAPI channel image in turn, and sending the processed images to a pre-trained Transformer network, which performs denoising and channel cross-color removal to obtain a pure multi-channel image;

[0070] (5) Analyzing and processing the pure multi-channel image according to preset analysis requirements.

[0071] The automatic analysis method for multiple immunofluorescence images in the embodiment of the present invention splits the acquired multiple immunofluorescence images into multi-channel images in the preprocessing stage, adopts a segmentation method to obtain a nucleus region mask for the DAPI channel image, uses the nucleus region mask to process other channel images to obtain a new channel image, then performs wavelength encoding processing and position encoding processing in sequence, pays attention to the connection between different channels, and sends the processed image to a pre-trained Transformer network, which performs denoising and channel cross-color removal processing to obtain a pure multi-channel image, thereby solving the noise problem and spectral interference problem of the multi-fluorescence image.

[0072] In a specific implementation of the embodiment of the present invention, the wavelength encoding process specifically includes the following steps:

[0073] Set up channel The corresponding wavelength is , calculate the normalized wavelength ;

[0074] Based on the normalized wavelength , calculate the channel The corresponding channel spectrum wavelength encoding;

[0075] Channel Channel images and channels The corresponding channel spectrum wavelength codes are added to obtain the channel The new channel image is obtained to complete the wavelength encoding processing.

[0076] In a specific implementation of the embodiment of the present invention, the wavelength The calculation formula is:

[0077] ;

[0078] In the formula, , are the maximum and minimum wavelength values ​​in all channels respectively.

[0079] In a specific implementation of the embodiment of the present invention, the calculation formula of the channel spectrum wavelength encoding is:

[0080] ;

[0081] In the formula, Encode the channel spectrum wavelength.

[0082] In a specific implementation of the embodiment of the present invention, the position coding process specifically includes the following steps:

[0083] Each single-channel image processed by wavelength encoding is divided into image blocks of preset size;

[0084] Map each image block into a one-dimensional vector, and combine all the one-dimensional vectors into an input sequence of a preset size;

[0085] Based on the input sequence, calculating a position code of each image block;

[0086] Add a position code after each image block.

[0087] In a specific implementation of the embodiment of the present invention, the calculation formula for the position code of each image block is:

[0088] ;

[0089] ;

[0090] In the formula, represents the position code of the image block, Indicates the position index of the current image block in the input sequence; Represents the index of the embedding dimension. Since each image block is flattened, it becomes a dimensional vector, so The value range is 0 to ; Represents the embedding dimension processed by the Transformer network.

[0091] In a specific implementation of the embodiment of the present invention, the loss function used by the Transformer network during training is:

[0092]

[0093] in,

[0094] ;

[0095] ;

[0096] ;

[0097] In the formula, is the total loss, is the L1 loss, is the structural similarity loss, is the spectral consistency loss; , Represent the network output value and the predicted true value respectively, represents the structural similarity of the calculation, , Respectively represent the wavelength The predicted and true values ​​of the images on the channels.

[0098] In a specific implementation of the embodiment of the present invention, the analyzing and processing the clean multi-channel image according to the preset analysis requirements specifically includes:

[0099] Based on the pixel brightness of the image of the target channel, the average fluorescence intensity, total fluorescence intensity and maximum fluorescence intensity in the user-selected area are quantitatively calculated;

[0100] The calculation formula of the mean fluorescence intensity is:

[0101] ;

[0102] The calculation formula of the total fluorescence intensity is:

[0103] ;

[0104] The calculation formula of the maximum fluorescence intensity is:

[0105] ;

[0106] In the formula, is the mean fluorescence intensity, is the total fluorescence intensity, is the maximum fluorescence intensity, For the The fluorescence intensity value of each pixel, is the total number of pixels in the selected area.

[0107] In a specific implementation of the embodiment of the present invention, the analyzing and processing the clean multi-channel image according to the preset analysis requirements further includes:

[0108] The protein expression levels corresponding to the fluorescence intensity are divided into different levels according to a series of thresholds set by the user. The processed pure multi-channel images are split or combined to intuitively display the co-expression status, and then the spatial distribution and expression intensity relationship of different biomarkers in the multi-fluorescence images are comprehensively analyzed.

[0109] The following is a detailed description of the multiple immunofluorescence image automatic analysis method in an embodiment of the present invention in conjunction with a specific implementation.

[0110] like Figure 1 As shown, the multiple immunofluorescence image automatic analysis method in the embodiment of the present invention comprises the following steps:

[0111] (1) Process the acquired multiple immunofluorescence images to obtain DAPI channel images and several non-DAPI channel images, including:

[0112] After the user imports the immunofluorescence image and the user interface displays the original image, the background enters the multiple immunofluorescence image processing. First, the multiple immunofluorescence images are preprocessed through the existing spectral library to obtain a single DAPI channel image and several non-DAPI channel images.

[0113] (2) Segmenting the DAPI channel image to obtain a cell nucleus region mask, specifically including:

[0114] Since the original multi-channel image may be interfered by other signals outside the cell nucleus, such as non-specific staining or background fluorescence, the signal-to-noise ratio is reduced. DAPI is a commonly used fluorescent dye that can specifically bind to DNA, thereby clearly marking the cell nucleus and accurately identifying and locating the boundaries of each cell nucleus. For this reason, the Stardist algorithm commonly used in biological image analysis is used in the embodiment of the present invention to obtain a cell nucleus area mask by segmenting the DAPI channel image to ensure that only the signal in the cell nucleus is analyzed later.

[0115] (3) Superimposing the cell nucleus region mask with each non-DAPI channel image to obtain a new non-DAPI channel image, specifically comprising:

[0116] By superimposing the cell nucleus area mask with the channel images of other channels, a new non-DAPI channel image is obtained to analyze the distribution and co-localization of the target protein in the cell nucleus, thereby preliminarily reducing the noise in the channel and eliminating the interference of other signals outside the nucleus.

[0117] (4) The DAPI channel image and each new non-DAPI channel image are subjected to wavelength encoding processing and position encoding processing in turn, and the processed images are sent to a pre-trained Transformer network, which performs denoising and channel cross-color removal to obtain a pure multi-channel image, such as Figure 2 As shown, specifically including:

[0118] According to the channel type input by the user, the wavelength is normalized and encoded to guide the downstream model to pay attention to the wavelength of the adjacent channel. Specifically, set the channel The corresponding wavelength is , encode the wavelength for each channel and calculate the channel The corresponding channel spectrum wavelength encoding, where the normalized wavelength is :

[0119] (1.1)

[0120] The calculation formula for channel spectrum wavelength encoding is:

[0121] (1.2)

[0122] In the formula, , are the maximum and minimum wavelength values ​​in all channels respectively. Formula (1.1) normalizes the wavelength corresponding to each channel to between [0,1]. Formula (1.2) is the calculation formula for channel spectrum wavelength encoding. Sine and cosine functions are used respectively, with a scaling factor of 100 to control the frequency changes in different dimensions. The processing is similar to the position encoding of Transformer. Since the sine and cosine functions are periodic, they can give the channel a smoothly changing value, so that the encoding of adjacent channels is orderly, allowing the model to learn the relative relationship between spectral channels. The new feature of this channel Through the original feature Added with the channel spectrum wavelength encoding, its feature encoding method is also consistent with the position encoding of Transformer:

[0123] ;

[0124] The new features introduce the relative relationship between different wavelengths, which helps the downstream optimization model to identify the association between different wavelengths (frequencies), making it easier to learn the information of adjacent channels.

[0125] Then, for each single-channel image, it is split into sizes according to the processing steps of Transformer small image blocks, and each image block is mapped to a one-dimensional vector. Assume that the image size is , divide each image block into , will be split into image blocks, each The image block (single channel) contains 256 pixel values. Finally, the image is flattened to a size of After each image block, position encoding (PE) is added. Its specific implementation is similar to ViT. For each image block of size Image blocks, which can calculate the position encoding of image blocks :

[0126] (1.4)

[0127] in, Represents the position index of the current image block in the input sequence, ranging from 0 to 195 (a total of 196 image blocks); Represents the index of the embedding dimension. Since each image block is flattened into a 256-dimensional vector, The value range is 0 to 255; represents the embedding dimension processed by the Transformer, which is 256 here (corresponding to the vector length of 256 in the previous text), and 10000 is the scaling factor, which controls the frequency variation of different dimensions. The role of sine and cosine functions is similar to the spectral wavelength encoding role mentioned above. The position index of the entire sequence can be expressed as:

[0128] (1.5)

[0129] For spectral channels Above the input sequence Features of image patches After the introduction of spectral wavelength coding and position coding, the new feature It can be expressed as:

[0130] (1.6)

[0131] Thus, features encoding both spectral and spatial position information are obtained.

[0132] The processed image is sent to a pre-trained Transformer network, which performs denoising and channel cross-color removal to obtain a pure multi-channel image; the main task of the Transformer network in the embodiment of the present invention is denoising and channel cross-color removal. The specific parameter definition of the Transformer network is shown in Table 1.

[0133] Table 1 Transformer network specific parameters

[0134] parameter definition value d_model Model dimensionality, the dimension of the embedding vectors used in the Transformer model 256 nhead The number of attention heads in the multi-head attention mechanism 8 dim_feedforward Represents the hidden layer dimensions of the feedforward neural network in the encoder and decoder 512 num_layers The number of layers stacked in the encoder and decoder (affects the complexity and performance of the model) 6

[0135] When training the Transformer network, set the label value using the following method:

[0136] PCA decomposition is used to model the autofluorescence, and the modeled autofluorescence signal is subtracted; then, based on the known fluorescence spectrum, linear spectral unmixing (LSU) is used to obtain the signal of each channel after eliminating cross-color; finally, self-supervised denoising based on Noise2Void is used to reduce the noise of the image. In this way, a pure multi-channel fluorescence image extracted by physical methods can be obtained.

[0137] The loss function used by the Transformer network during training is:

[0138] ;

[0139] in,

[0140] ;

[0141] ;

[0142] ;

[0143] In the formula, is the total loss, is the L1 loss, is the structural similarity loss, is the spectral consistency loss; , Represent the network output value and the predicted true value respectively, represents the structural similarity of the calculation, , Respectively represent the wavelength The predicted and true values ​​of the images on the channels.

[0144] (5) Analyzing and processing the pure multi-channel image according to preset analysis requirements, specifically including:

[0145] (5.1) Immunofluorescence signal intensity assessment

[0146] Based on the pixel brightness of the pure single-channel image of the target channel, the mean fluorescence intensity (MFI) in the user-selected area is quantified:

[0147] ;

[0148] in: For the The fluorescence intensity value of each pixel, The average fluorescence signal intensity directly reflects the density of the actual content of the staining marker in the area.

[0149] Total Fluorescence Intensity (TFI):

[0150] ;

[0151] The total fluorescence intensity reflects the total amount of staining markers in the area, and can be used to evaluate the expression level of specific proteins or markers in the cell nuclei within the area.

[0152] Maximum fluorescence intensity (MaxFI)

[0153] ;

[0154] In the formula, is the mean fluorescence intensity, is the total fluorescence intensity, is the maximum fluorescence intensity, For the The fluorescence intensity value of each pixel, is the total number of pixels in the selected area.

[0155] The maximum fluorescence intensity can reflect the expression peak of the staining marker in the area and can be used as a quantitative analysis indicator in practical applications to measure the expression differences of markers in pathological samples.

[0156] Furthermore, fluorescence intensity distribution statistics can be performed: by calculating and displaying statistics such as the standard deviation and median of fluorescence intensity, and using histograms to visually visualize the distribution, the distribution characteristics of the fluorescence signal can be evaluated to support subsequent visualization operations.

[0157] (5.2) Fluorescence image integration display

[0158] Users can set a series of thresholds to classify protein expression levels corresponding to fluorescence intensity into different levels. The thresholds can be set according to actual needs and data distribution. For example, the markers in cells with the lowest fluorescence intensity value can be classified as "low expression" level, the cell markers with the highest fluorescence intensity value can be classified as "high expression" level, and the cells between the two can be classified as "medium expression" level, such as low expression (1+), medium expression (2+) and high expression (3+).

[0159] At the same time, users can use the patented built-in check tool to split or combine the processed multi-channel data according to actual needs, intuitively display the co-expression status, and then comprehensively analyze the spatial distribution and expression intensity relationship of different biomarkers in multi-fluorescence images.

[0160] In practical applications, this patent can be used for co-localization analysis: statistical analysis of the expression intensity of different levels of multiple different markers on the same cell to provide support for the detailed description of cell characteristics. Different cell phenotypes can be defined based on different marker combinations, such as regulatory T cells (CD3+CD4+FOXP3+) and double-negative T cells (CD3+CD4-CD8-).

[0161] Example 2

[0162] In an embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method for automatic analysis of multiple immunofluorescence images described in any one of Embodiment 1 is implemented.

[0163] Example 3

[0164] A computer program product is provided in an embodiment of the present invention, including a computer program / instruction, which, when executed by a processor, implements the multiple immunofluorescence image automatic analysis method described in any one of Embodiment 1.

[0165] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0166] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0167] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0168] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0169] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

[0170] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for automatic analysis of multiple immunofluorescence images, characterized in that: include: Processing the acquired multiple immunofluorescence images to obtain DAPI channel images and several non-DAPI channel images; Segmenting the DAPI channel image to obtain a cell nucleus region mask; Superimposing the cell nucleus region mask with each non-DAPI channel image to obtain a new non-DAPI channel image; The DAPI channel image and each new non-DAPI channel image are sequentially subjected to wavelength encoding processing and position encoding processing, and the processed images are sent to a pre-trained Transformer network, and the Transformer network performs denoising and channel cross-color removal processing to obtain a pure multi-channel image; The pure multi-channel image is analyzed and processed according to preset analysis requirements.

2. The method for automatic analysis of multiple immunofluorescence images according to claim 1, characterized in that: The wavelength encoding process specifically comprises the following steps: Set up channel The corresponding wavelength is , calculate the normalized wavelength ; Based on the normalized wavelength , calculate the channel The corresponding channel spectrum wavelength encoding; Channel Channel images and channels The corresponding channel spectrum wavelength codes are added to obtain the channel The new channel image is obtained to complete the wavelength encoding processing.

3. The method for automatic analysis of multiple immunofluorescence images according to claim 2, characterized in that: The normalized wavelength The calculation formula is: ； In the formula, , are the maximum and minimum wavelength values ​​in all channels respectively; The calculation formula of the channel spectrum wavelength encoding is: ； In the formula, Encode the channel spectrum wavelength.

4. The method for automatic analysis of multiple immunofluorescence images according to claim 1, characterized in that: The position coding process specifically comprises the following steps: Each single-channel image processed by wavelength encoding is divided into image blocks of preset size; Map each image block into a one-dimensional vector, and combine all the one-dimensional vectors into an input sequence of a preset size; Based on the input sequence, calculating the position code of each image block; Add a position code after each image block.

5. The method for automatic analysis of multiple immunofluorescence images according to claim 4, characterized in that: The calculation formula for the position encoding of each image block is: ； ； In the formula, represents the position code of the image block, Indicates the position index of the image block in the input sequence; Represents the index of the embedding dimension. Since each image block is flattened, it becomes a dimensional vector, so The value range is 0 to ; Represents the embedding dimension processed by the Transformer network.

6. The method for automatic analysis of multiple immunofluorescence images according to claim 1, characterized in that: The loss function used by the Transformer network during training is: ； in, ； ； ； In the formula, is the total loss, is the L1 loss, is the structural similarity loss, is the spectral consistency loss; , Represent the network output value and the predicted true value respectively, represents the structural similarity of the calculation, , Respectively represent the wavelength The image prediction value and true value on the channel, is the total number of pixels in the selected area.

7. The method for automatic analysis of multiple immunofluorescence images according to claim 1, characterized in that: The analyzing and processing the pure multi-channel image according to the preset analysis requirements specifically includes: Based on the pixel brightness of the image of the target channel, the average fluorescence intensity, total fluorescence intensity and maximum fluorescence intensity in the user-selected area are quantitatively calculated; The calculation formula of the mean fluorescence intensity is: ； The calculation formula of the total fluorescence intensity is: ； The calculation formula of the maximum fluorescence intensity is: ； In the formula, is the mean fluorescence intensity, is the total fluorescence intensity, is the maximum fluorescence intensity, For the The fluorescence intensity value of each pixel, is the total number of pixels in the selected area.

8. The method for automatic analysis of multiple immunofluorescence images according to claim 7, characterized in that: The analyzing and processing the pure multi-channel image according to the preset analysis requirements also includes: The protein expression levels corresponding to the fluorescence intensity are divided into different levels according to a series of thresholds set by the user. The processed pure multi-channel images are split or combined to intuitively display the co-expression status, and then the spatial distribution and expression intensity relationship of different biomarkers in multi-fluorescence images are comprehensively analyzed.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the automatic analysis method of multiple immunofluorescence images according to any one of claims 1 to 8 is implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the method for automatic analysis of multiple immunofluorescence images according to any one of claims 1 to 8 is implemented.

Images