Pathological section dyeing image processing method based on color deconvolution

The pseudo-inverse matrix of the absorbance matrix is ​​solved by decomposition of singular value, and the problem of irreversible absorbance matrix during color deconvolution is solved, and the technical effect of separating individual dyeing components from composite dyeing images is achieved.

CN118941562AInactive Publication Date: 2024-11-12HANGZHOU XINO INTELLIGENT MEDICINE CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411424173.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-12
Publication Date
2024-11-12
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In the prior art, due to the irreversible absorbance matrix of the dye agent during color deconvolution, it is impossible to separate individual dye components from the composite dye image, especially in the case of three or more dyes, there is a lack of effective solutions. .

Method used

The pseudo-inverse matrix of the absorbance matrix of the dye is solved by singular value decomposition (SVD) and used as a deconvolution matrix, thereby achieving deconvolution of the colors of multiple dyes and separating individual dyeing components.

Benefits of technology

The deconvolution matrix of the absorbance matrix is ​​realized in the case of any kind of dyeing agent, and the individual dyeing components are successfully separated from the composite staining image, solving the problem of irreversible absorbance matrix.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941562A_ABST
    Figure CN118941562A_ABST
Patent Text Reader

Abstract

The invention relates to a pathological section dyeing image processing method based on color deconvolution. The method comprises the following steps: determining an absorbance matrix of a plurality of coloring agents for dyeing a pathological section dyeing image, the absorbance matrix being used for representing absorption characteristics of the plurality of coloring agents in an optical density space; performing singular value decomposition on the absorbance matrix to obtain a pseudo-inverse matrix of the absorbance matrix, and taking the pseudo-inverse matrix as a deconvolution matrix of the absorbance matrix; single-channel results of the various coloring agents are determined through the deconvolution matrix, and the single-channel results are single staining layers of the various coloring agents separated from the pathological section staining image. According to the method and the device, the technical problem that an independent dyeing component cannot be separated from a composite dyeing image due to the fact that a deconvolution matrix cannot be solved because an absorbance matrix of a dyeing agent is irreversible is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method for processing pathological section stained images based on color deconvolution. Background Art

[0002] In histopathology, microscopic images of stained cells and tissues play a key role. The color deconvolution algorithm is designed for the color information obtained by the RGB camera, and the effect of each stain on the image is calculated based on the specific absorption of the RGB component light of the stain used. This method is particularly useful in pathological research and can help researchers better understand the morphology and function of cells.

[0003] Color deconvolution algorithms can be applied to H-DAB images and commonly used stains in histopathology (HE, H-AEC, FAST Red, DAB), and are widely used in color separation of pathological section staining images. Usually, there are only two stains for deconvolution, and separation of three or more stains is difficult.

[0004] During color deconvolution, if there are more than or equal to 4 dyes, the absorbance matrix of the dye is irreversible and deconvolution cannot be performed; and even if there are less than or equal to 3 dyes, there is a situation where the absorbance matrix of the dye is irreversible and deconvolution cannot be performed.

[0005] Regarding the problem that during color deconvolution in the related art, the deconvolution matrix cannot be solved because the absorbance matrix of the dye is irreversible, resulting in the inability to separate individual dye components from the composite dye image, and no effective solution has been proposed yet. Summary of the invention

[0006] The embodiment of the present application provides a method for processing pathological section stained images based on color deconvolution, which at least solves the problem that during color deconvolution, the deconvolution matrix cannot be solved due to the irreversibility of the absorbance matrix of the dye, resulting in the inability to separate individual staining components from the composite staining image.

[0007] According to one aspect of an embodiment of the present application, a method for processing a pathological section stained image based on color deconvolution is provided, comprising: determining an absorbance matrix of multiple dyes used to dye the pathological section stained image, wherein the absorbance matrix is ​​used to represent the absorption characteristics of the multiple dyes in the optical density space; performing singular value decomposition on the absorbance matrix to obtain a pseudo-inverse matrix of the absorbance matrix, and using the pseudo-inverse matrix as a deconvolution matrix of the absorbance matrix; and using the deconvolution matrix to respectively determine single-channel results of various dyes, wherein the single-channel results are single dyeing layers of various dyes separated from the pathological section stained image.

[0008] Optionally, singular value decomposition is performed on the absorbance matrix to obtain a pseudo-inverse matrix of the absorbance matrix, including: decomposing the absorbance matrix into a left singular matrix, a singular value diagonal matrix and a right singular matrix, wherein the left singular matrix is ​​the eigenvector of a matrix obtained by multiplying the absorbance matrix with the transposed matrix of the absorbance matrix, the right singular matrix is ​​the eigenvector of a matrix obtained by multiplying the transposed matrix of the absorbance matrix with the absorbance matrix, and the absorbance matrix is ​​the product of the left singular matrix, the singular value diagonal matrix and the transposed matrix of the right singular matrix; and determining the pseudo-inverse matrix of the absorbance matrix according to the left singular matrix, the singular value diagonal matrix and the right singular matrix.

[0009] Optionally, the pseudo-inverse matrix of the absorbance matrix is ​​determined based on the left singular matrix, the singular value diagonal matrix and the right singular matrix, including: calculating the transposed matrix of the singular value diagonal matrix; multiplying the diagonal elements in the transposed matrix of the singular value diagonal matrix to obtain a target matrix; multiplying the right singular matrix, the target matrix and the transposed matrix of the left singular matrix to obtain the pseudo-inverse matrix of the absorbance matrix.

[0010] Optionally, determining the absorbance matrix of multiple dyes used to dye the pathological section stained image includes: determining initial values ​​of the absorbance matrix of multiple dyes used to dye the pathological section stained image; and determining an optimal absorbance matrix according to the initial values ​​of the absorbance matrix.

[0011] Optionally, the initial values ​​of the absorbance matrices of multiple dyes used to dye the pathological section stained image are determined, including: respectively calculating the intensity value of each pixel in the RGB image of each dye, and calculating the absorbance value of each pixel based on the intensity value of each pixel, wherein the RGB image is obtained by separately dyeing different pathological sections with various dyes; standardizing the absorbance value of each pixel to obtain a standardized absorbance vector of each pixel; using the standardized absorbance vector to calculate the initial values ​​of the parameters of the objective function, wherein the objective function is used to optimize the parameters in the deconvolution process; and constructing the initial values ​​of the absorbance matrix based on the initial values ​​of the parameters, wherein each column of the initial values ​​of the absorbance matrix represents the relative absorbance of the pixel corresponding to a dye to the RGB channel.

[0012] Optionally, determining the optimal absorbance matrix according to the initial values ​​of the absorbance matrix includes: calculating optimal parameters of the objective function based on the initial values ​​of the absorbance matrix and a gradient descent algorithm; and calculating the optimal absorbance matrix based on the optimal parameters.

[0013] Optionally, the objective function is given by components, where n is the number of dyes, and each component corresponds to an evaluation index of a dye combination consisting of two different dyes.

[0014] Optionally, the single-channel results of various dyes are determined respectively using a deconvolution matrix, including: calculating the dyeing intensity of each pixel in a single dyeing layer of each dye respectively; converting the dyeing intensity of each pixel in a single dyeing layer of each dye from an optical density value to a light intensity expression.

[0015] According to another aspect of an embodiment of the present application, there is also provided an electronic device, including: a processor, and a memory storing a program, wherein the program includes instructions, and when the instructions are executed by the processor, the processor executes the above method.

[0016] According to another aspect of the embodiments of the present application, a non-transitory machine-readable medium storing computer instructions is also provided, where the computer instructions are used to enable a computer to execute the above method.

[0017] According to another aspect of the embodiments of the present application, a computer program product is provided, including a computer program, which is used to enable the computer to execute the above method when the computer program is executed by a processor of the computer.

[0018] Beneficial effects of the embodiments of the present application: In an embodiment of the present application, a general multi-dye color deconvolution algorithm is proposed. The pseudo-inverse matrix of the absorbance matrix of the dye is solved by singular value decomposition (SVD), and the pseudo-inverse matrix is ​​used as the deconvolution matrix of the absorbance matrix. This achieves the technical effect of calculating the deconvolution matrix of the absorbance matrix for any type and number of dyes when performing color deconvolution, thereby separating individual dye components from a composite dye image.

[0019] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can be obtained based on these drawings without creative work.

[0021] Figure 1 It is a flowchart of a method for processing a pathological section stained image based on color deconvolution according to an embodiment of the present application; Figure 2 It is an RGB image after Pap staining.

[0022] Figure 3ais from Figure 2 Single channel result of hematoxylin separated in the RGB image shown; Figure 3b is from Figure 2 The single channel result of eosin separated from the RGB image shown; Figure 3c is from Figure 2 The single channel result of bright green separated from the RGB image shown; Figure 3d is from Figure 2 The single channel result of orange separated from the RGB image shown; Figure 4 Schematic diagram of the structure of the electronic device of this embodiment. DETAILED DESCRIPTION

[0023] Embodiments of the present embodiment will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present embodiment are shown in the accompanying drawings, it should be understood that the present embodiment can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein, which are instead provided for a more thorough and complete understanding of the present embodiment. It should be understood that the drawings and embodiments of the present embodiment are only for exemplary purposes and are not intended to limit the scope of protection of the present embodiment.

[0024] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are explained as follows: Color deconvolution is a commonly used image analysis method in immunohistochemistry (IHC) and other tissue staining techniques. It is used to separate individual staining components from composite staining images for more accurate quantitative analysis.

[0025] The absorbance matrix, also known as the stain color appearance (SCA), describes the properties of dyes that absorb light. For each dye, each column of the absorbance matrix represents the absorbance value at different wavelengths. This matrix is ​​based on the absorbance spectrum of the dye, that is, the absorption and scattering properties of the dye at different wavelengths, which is the key input of the color deconvolution algorithm. The application of the absorbance matrix includes but is not limited to quantitative analysis, kinetic research, and concentration determination. For example, in the treatment of printing and dyeing wastewater, by measuring the absorbance matrix of the dye, the degradation characteristics of the dye can be understood, and then their concentration can be determined.

[0026] Histopathology is a pathological diagnosis at the tissue level. Biopsy tissue or surgical specimens are made into tissue sections and the morphological changes of tissue cells are observed under a microscope to diagnose the disease.

[0027] Papanicolaou staining is a method used to detect cervical precancerous lesions and cervical cancer. It diagnoses abnormalities by observing the different colors of the nucleus and cytoplasm of cervical cells. The specific operation is to apply cervical cells on a slide, fix and stain them with a special dye. Light colors may be normal, and dark colors may indicate cervical precancerous lesions or cervical cancer.

[0028] H&E staining, also known as hematoxylin-eosin staining, is one of the commonly used staining methods in paraffin section technology. Hematoxylin staining is alkaline, which mainly stains the chromatin in the cell nucleus and the nucleic acid in the cytoplasm purple-blue; eosin is an acidic dye, which mainly stains the components in the cytoplasm and extracellular matrix red.

[0029] WSI, short for Whole Slide Image, is a digital slice (also called virtual slice) in Chinese. It uses a fully automatic microscope scanning system combined with a virtual slice software system to scan and seamlessly splice traditional glass slices to generate a complete digital slice with a full field of view.

[0030] The Beer-Lambert law is the fundamental law of light absorption that applies to all electromagnetic radiation and all light-absorbing substances, including gases, solids, liquids, molecules, atoms, and ions.

[0031] An existing color deconvolution algorithm is an adaptive color deconvolution algorithm for histological WSI normalization (ACD). Before describing the technical solution proposed in this application, a brief introduction to the ACD algorithm is given first.

[0032] Color deconvolution (CD) theory is the basis of ACD. Color deconvolution is proposed based on Lambert-Beer's law. These stains are separated in absorbance space.

[0033] As shown in the above formula, Indicates WSI big picture The pixel RGB channel intensity value, Indicates WSI big picture The absorbance value of the RGB channels of each pixel, Is the color deconvolution matrix, each column represents the deconvolution vector of a dye on the RGB channel, is the output staining intensity, is the light intensity value of the output staining intensity, express The first dye, This is the goal of deconvolution. Represents the intensity of the background, that is, the pixel value when there is no stained tissue. The deconvolution matrix D is determined by the SCA matrix M, and its inverse operation is .

[0034] For H&E staining, the SCA matrix can be decomposed into ,in, (j=h, e, d), defined as the SCA vector. In general, is a unit vector that describes the contribution of the R, G, and B channels to the jth color. , the algorithm uses two angle variables to represent for: Then, the SCA matrix M can be represented by 6 independent angle variables. This angle variable is represented by a set: , The key to the deconvolution algorithm is to solve the SCA matrix M, the deconvolution matrix .

[0035] When solving the SCA matrix M, a variable By analyzing the function, the optimized variables are obtained, and then the adaptive matrix of WSI is determined. and . and Also expressed as and .

[0036] The objective function of the ACD algorithm is designed based on the following prior knowledge: 1) There are two types of stains in H&E stained WSIs. Therefore, ideally, the third channel of the separation result should be zero. 2) H&E staining has high specificity. Hematoxylin mainly stains the nucleus, and eosin mainly stains the cytoplasm and matrix. Therefore, the objective function is defined as: The objective function is designed to optimize the separation of the stain components so that the separated results reflect the stain components of hematoxylin and eosin as accurately as possible. The first term of the above objective function is to minimize the separation outside of H or E (i.e., to minimize the pixel values ​​that do not belong to the H or E stain components), and the second term forces the values ​​of the separated pixels to be assigned to the same stain (i.e., to ensure that the separated results only contain H or E stain components).

[0037] At this point, the optimization goal can be described as: The objective function is continuous and differentiable, so the gradient descent algorithm is used to solve it. The gradient descent algorithm is a commonly used optimization algorithm for solving unconstrained optimization problems, especially for minimizing loss functions in machine learning and deep learning. The basic idea of ​​gradient descent is to iteratively update the parameters along the negative direction of the function gradient until the local minimum is reached or the stopping condition is met.

[0038] During the optimization process, only pixels located on the tissue region are used. The background (unstained) region in WSIs is approximately white, the absorbance of background pixels is close to zero, and the background region does not contribute to the analysis of the staining component. Therefore, the pixels on the tissue region can be filtered by a threshold. Specifically, the average absorbance of the RGB channels is calculated, and the average absorbance of the RGB channels of the pixels in the tissue region must meet the threshold (0.3, log(256 / 30)). A tissue mask for WSI can be obtained by threshold screening, and the pixels used in the optimization are randomly sampled from the tissue region.

[0039] Gradient descent algorithm parameters It is calculated based on the relative absorbance vector of H&E staining. H&E staining is a common pathological staining method. H&E staining has reference values, and the parameters are calculated using the reference values. .

[0040] The above ACD algorithm has the following disadvantages: 1) Deconvolution matrix , since RGB is a three-channel matrix, M is required to be a 3*3 square matrix and reversible. For more than 3 dyes, M is a 3*n (n>=4) matrix, where n is the number of dyes used. A 3*n (n>=4) matrix is ​​definitely not reversible, and even a 3*3 square matrix is ​​not necessarily reversible. In other words, deconvolution is not possible when there are more than or equal to 4 dyes, and even when there are less than or equal to 3 dyes, there is a situation where it is irreversible and cannot be deconvolved.

[0041] 2) The algorithm uses the H&E staining RGB relative absorbance vector reference value to calculate parameters , is not universal because for stains other than H&E staining, there may be no reference values ​​and the RGB relative absorbance vector is unknown.

[0042] In order to solve the above problems, a general multi-dye color deconvolution algorithm is proposed in the embodiment of the present application, the pseudo-inverse matrix of the SCA matrix is ​​solved by SVD, and the corresponding objective function is designed to optimize the deconvolution parameters, and the color deconvolution of Pap staining is practiced. The details are described below.

[0043] Figure 1 is a flow chart of a method for processing a pathological section stained image based on color deconvolution according to an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps: Step S102, determining an absorbance matrix of a plurality of dyes used to dye the pathological section dyeing image, wherein the absorbance matrix is ​​used to represent the absorption characteristics of the plurality of dyes in the optical density space.

[0044] Pathological section staining images usually contain the localization information of specific proteins, which is very important for studying disease mechanisms, diagnosing diseases, and evaluating treatment effects.

[0045] The stained image in this step is stained by using the Papanicolaou staining method. The Papanicolaou staining includes four dyes, namely hematoxylin, eosin, brilliant green and orange. The absorbance matrix is ​​the SCA matrix mentioned above.

[0046] Step S104, performing singular value decomposition on the absorbance matrix to obtain a pseudo-inverse matrix of the absorbance matrix, and using the pseudo-inverse matrix as a deconvolution matrix of the absorbance matrix.

[0047] Singular value decomposition (SVD) is an important matrix decomposition in linear algebra and is widely used in data science and machine learning. It can decompose a matrix into the product of three matrices, one of which is a diagonal matrix containing the important eigenvalues ​​of the original matrix - singular values.

[0048] Since the absorbance matrix is ​​not invertible when there are 4 types of dyes, the inverse matrix of the absorbance matrix cannot be directly calculated. In this step, singular value decomposition is performed on the absorbance matrix to calculate the pseudo-inverse matrix of the absorbance matrix, thereby obtaining the deconvolution matrix of the absorbance matrix.

[0049] Step S106, using the deconvolution matrix to respectively determine the single-channel results of various stains, wherein the single-channel results are single staining layers of various stains separated from the pathological section staining image.

[0050] In color deconvolution, a "single-channel result" usually refers to separating a single stain component of a specific stain from a multi-channel color image. For H&E stained images, this means extracting a single stain layer of hematoxylin or eosin from the original RGB image.

[0051] The above-mentioned scheme provided in the embodiment of the present application solves the pseudo-inverse matrix of the absorbance matrix of the dye through singular value decomposition, and uses the pseudo-inverse matrix as the deconvolution matrix of the absorbance matrix, thereby achieving the technical effect of calculating the deconvolution matrix of the absorbance matrix for any type and number of dyes when performing color deconvolution, thereby separating individual dyeing components from the composite dyeing image.

[0052] According to an optional embodiment of the present application, step S104 is executed to perform singular value decomposition on the absorbance matrix to obtain a pseudo-inverse matrix of the absorbance matrix, including the following steps: decomposing the absorbance matrix into a left singular matrix, a singular value diagonal matrix and a right singular matrix, wherein the left singular matrix is ​​the eigenvector of a matrix obtained by multiplying the absorbance matrix with the transposed matrix of the absorbance matrix, the right singular matrix is ​​the eigenvector of a matrix obtained by multiplying the transposed matrix of the absorbance matrix with the absorbance matrix, and the absorbance matrix is ​​the product of the left singular matrix, the singular value diagonal matrix and the transposed matrix of the right singular matrix; determining the pseudo-inverse matrix of the absorbance matrix according to the left singular matrix, the singular value diagonal matrix and the right singular matrix.

[0053] The SCA matrix M of Papanicolaou staining is a matrix with 3 rows and 4 columns, which cannot be inverted. The pseudo-inverse matrix of the SCA matrix is ​​calculated through SVD singular value decomposition.

[0054] The SCA matrix M is a matrix with full row rank. The singular value decomposition of the matrix M (3*4) is shown in the following formula. U is a 3*3 left singular matrix, S is a 3*4 singular value diagonal matrix, and V is a 4*4 right singular matrix.

[0055] in, , Yes A , A is the matrix of singular values ​​to be decomposed (i.e., matrix M), is the transposed matrix of A.

[0056] , yes The eigenvector of A.

[0057] , , and is a singular value.

[0058] If there are n dyes, , Yes A The eigenvector of , yes The eigenvector of A.

[0059] , the number of columns is n, , and is a singular value.

[0060] As an optional embodiment of the present application, the pseudo-inverse matrix of the absorbance matrix is ​​determined according to the left singular matrix, the singular value diagonal matrix and the right singular matrix, including the following steps: calculating the transposed matrix of the singular value diagonal matrix; multiplying the diagonal elements in the transposed matrix of the singular value diagonal matrix to obtain the target matrix; multiplying the right singular matrix, the target matrix and the transposed matrix of the left singular matrix to obtain the pseudo-inverse matrix of the absorbance matrix.

[0061] The pseudo-inverse matrix of M is: yes The matrix obtained by transposing and inverting (taking the reciprocal) the elements on the diagonal.

[0062] for: If there are n dyes, for: , the number of rows of the matrix is ​​n.

[0063] It should be noted that the absorbance matrix of the dye is usually full row rank, and the above formula assumes full row rank by default. If it is deficient rank, the expression is slightly different, and the same method can be used to solve the pseudo-inverse matrix.

[0064] The method of solving the pseudo-inverse matrix using singular value decomposition is a general deconvolution matrix solving method. This method can solve the problem that the absorbance matrix of more than 3 dyes is irreversible or the absorbance matrix of less than or equal to 3 dyes may also be irreversible.

[0065] According to an optional embodiment of the present application, step S102 is executed to determine the absorbance matrix of multiple dyes used to dye the pathological section stained image, which is achieved by the following method: determining the initial value of the absorbance matrix of multiple dyes used to dye the pathological section stained image; and determining the optimal absorbance matrix based on the initial value of the absorbance matrix.

[0066] As an optional embodiment of the present application, determining the initial values ​​of the absorbance matrix of multiple dyes used to dye a pathological section stained image includes the following steps: respectively calculating the intensity value of each pixel in the RGB image of each dye, and calculating the absorbance value of each pixel based on the intensity value of each pixel, wherein the RGB image is obtained by separately dyeing different pathological sections with various dyes; standardizing the absorbance value of each pixel to obtain a standardized absorbance vector of each pixel; using the standardized absorbance vector to calculate the initial values ​​of the parameters of the objective function, wherein the objective function is used to optimize the parameters in the deconvolution process; and constructing the initial values ​​of the absorbance matrix based on the initial values ​​of the parameters, wherein each column of the absorbance matrix represents the absorbance value of the pixel corresponding to a dye.

[0067] Before explaining how to construct the absorbance matrix, two concepts are explained: The intensity of a pixel usually refers to the grayscale value of the pixel, that is, the brightness level of the pixel, which is usually represented by an integer between 0 and 255.

[0068] Absorbance is a term in physics and chemistry. It refers to the logarithm to the base 10 (i.e. lg(I0 / I1)) of the ratio of the intensity of incident light before it passes through a solution or a substance to the intensity of transmitted light after it passes through the solution or a substance, where I0 is the intensity of incident light and I1 is the intensity of transmitted light. Factors that affect it include solvent, concentration, temperature, etc.

[0069] The following is an example of the technical solution using Papanicolaou staining: First, tissue sections were stained with hematoxylin, eosin, brilliant green and orange, respectively, and RGB images of single staining with the four stains were obtained by scanning.

[0070] Then, the large image pixels are screened and sampled in the tissue area. The average absorbance of the RGB channel of the pixels in the tissue area must meet the threshold (0.3, log(256 / 30)), the background and dirty spots are removed, and then the million pixels are sampled.

[0071] Next, the absorbance of the four dye RGB channels is calculated separately, as shown in the following formula, After sampling, The pixel RGB channel intensity value, After sampling, The absorbance value of each pixel's RGB channel.

[0072] Then, yes Perform standardization, standardization formula: Represents the absorbance value of the jth channel of the ith pixel (j=1, 2, 3 represent RGB channels respectively), express The second-order norm of . is the above-mentioned normalized absorbance vector.

[0073] Calculate the parameters of the objective function from the normalized absorbance vector , the design is versatile for multiple stains.

[0074] According to another optional embodiment of the present application, determining the optimal absorbance matrix according to the initial value of the absorbance matrix includes the following steps: calculating the optimal parameters of the objective function based on the initial value of the absorbance matrix and the gradient descent algorithm; and calculating the optimal absorbance matrix based on the optimal parameters.

[0075] As an optional embodiment of the present application, the objective function is components, where n is the number of dyes, and each component corresponds to an evaluation index of a dye combination consisting of two different dyes.

[0076] In the embodiments of the present application, a corresponding objective function is also designed to optimize the deconvolution parameters. There are 6 kinds, that is, the objective function has 6 components. If there are n kinds of dyes, the objective function has A quantity.

[0077] In the above formula, h, e, g, and o represent hematoxylin, eosin, brilliant green, and orange, respectively.

[0078] In digital image processing, especially for WSI processing, defining an objective function is usually used to quantify the metric of a specific target, such as dyeing consistency, color purity, etc. By optimizing the objective function, a set of optimal parameters can be found to improve the image processing results.

[0079] The objective function can contain multiple components, each corresponding to an evaluation metric for a dye combination. For each pair of dye combinations (a pair of dye combinations includes two types of dyes), a component can be defined to evaluate the effect of the combination. For example, each component can be defined using metrics such as dye consistency, color purity, etc. All components are combined into an overall objective function.

[0080] In some optional embodiments, when all components are combined into an overall objective function, a weighted summation or weighted average may also be involved to reflect the importance and priority of different dye combinations.

[0081] In the embodiments of the present application, It is the parameter that the objective function needs to optimize, and the optimal parameter is calculated using the gradient descent algorithm. , and then based on the optimal parameters , calculate the SCA matrix M.

[0082] Calculate the optimal parameters using the gradient descent algorithm The main ideas are as follows: First, select an initial point (i.e., the initial value of the above parameters) as the starting point of the search, and then calculate the gradient of the objective function at the initial point. Update the parameters along the negative direction of the gradient until a certain stop condition is met, such as the gradient is small enough or the maximum number of iterations is reached, and the optimal parameters are obtained. .

[0083] In an embodiment of the present application, a gradient descent algorithm is used to solve the minimum value of the objective function, and the parameters of the objective function corresponding to the minimum value are the optimal parameters.

[0084] In some optional embodiments of the present application, step S106 is executed to use the deconvolution matrix to respectively determine the single-channel results of various dyes, including the following steps: respectively calculating the dyeing intensity of each pixel point in a single dyeing layer of various dyes; converting the dyeing intensity of each pixel point in a single dyeing layer of various dyes from optical density value to light intensity expression.

[0085] In an embodiment of the present application, the staining intensity is calculated using the deconvolution matrix and the optical density (OD) of the pixel point, and converted into light intensity expression. For each separated stain component, its average OD value can be calculated as a measure of the staining intensity. The staining intensity is then converted from the OD value to light intensity expression.

[0086] Since OD is defined as - , where I is the light intensity after passing through the stained sample, is the incident light intensity, so the OD value can be converted back to light intensity using the following formula: .

[0087] Figure 2 It is an RGB image after Pap staining.

[0088] Figure 3a is from Figure 2 Single channel result of hematoxylin separated in the RGB image shown; Figure 3b is from Figure 2 The single channel result of eosin separated from the RGB image shown; Figure 3c is from Figure 2 The single channel result of bright green separated from the RGB image shown; Figure 3d is from Figure 2 The single channel result of the orange separated from the RGB image is shown.

[0089] from Figure 2 as well as Figure 3a~Figure 3d It can be seen that, through the color deconvolution algorithm proposed in the present application, the individual staining layers of hematoxylin, eosin, brilliant green and orange can be successfully separated from the RGB image after Pap stained.

[0090] The embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication. The memory stores a computer program executable by the at least one processor, and the computer program is used to enable the electronic device to perform the method of the embodiment of the present application when executed by the at least one processor.

[0091] An embodiment of the present application also provides a non-transitory machine-readable medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method of the embodiment of the present application.

[0092] The embodiment of the present application also provides a computer program product, including a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method of the embodiment of the present application.

[0093] refer to Figure 4, the structural block diagram of the electronic device that can be used as the server or client of the embodiment of the present application will now be described, which is an example of the hardware device that can be applied to various aspects of the present application. The electronic device is intended to represent various forms of digital electronic computer equipment, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples, and are not intended to limit the implementation of the present application described and / or required herein.

[0094] like Figure 4 As shown, the electronic device includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 to a random access memory (RAM) 403. In RAM 403, various programs and data required for the operation of the electronic device can also be stored. The computing unit 401, ROM 402, and RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0095] Multiple components in the electronic device are connected to the I / O interface 405, including: an input unit 406, an output unit 407, a storage unit 408, and a communication unit 409. The input unit 406 can be any type of device that can input information to the electronic device, and the input unit 406 can receive input digital or character information, and generate key signal input related to user settings and / or function control of the electronic device. The output unit 407 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 408 can include but is not limited to a disk, an optical disk. The communication unit 409 allows the electronic device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0096] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a CPU, a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the various methods and processes described above. For example, in some embodiments, the method embodiments of the present application may be implemented as a computer program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via ROM 402 and / or communication unit 409. In some embodiments, the computing unit 401 may be configured to perform the above method in any other appropriate manner (e.g., by means of firmware).

[0097] The computer program for implementing the method of the embodiment of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0098] In the context of the embodiments of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable signal medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, or infrared system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0099] It should be noted that the term "including" and its variations used in the embodiments of the present application are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The modifications of "one" and "multiple" mentioned in the embodiments of the present application are illustrative and not restrictive. Those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0100] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0101] The various steps described in the method implementation methods provided in the embodiments of the present application can be performed in different orders and / or in parallel. In addition, the method implementation methods may include additional steps and / or omit the steps shown. The scope of protection of the present application is not limited in this respect.

[0102] The term "embodiment" in this specification refers to specific features, structures or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. The various embodiments in this specification are described in a related manner, and the same or similar parts between the various embodiments refer to each other. In particular, for the device, equipment, and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts refer to the partial description of the method embodiment.

[0103] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A method for processing pathological section stained images based on color deconvolution, characterized in that: include: Determining an absorbance matrix of a plurality of dyes used to dye a pathological section dyeing image, wherein the absorbance matrix is ​​used to represent the absorption characteristics of the plurality of dyes in an optical density space; Performing singular value decomposition on the absorbance matrix to obtain a pseudo-inverse matrix of the absorbance matrix, and using the pseudo-inverse matrix as a deconvolution matrix of the absorbance matrix; The deconvolution matrix is ​​used to respectively determine the single-channel results of various stains, wherein the single-channel results are single staining layers of the various stains separated from the pathological section staining image.

2. The method according to claim 1, characterized in that Performing singular value decomposition on the absorbance matrix to obtain a pseudo-inverse matrix of the absorbance matrix, including: Decomposing the absorbance matrix into a left singular matrix, a singular value diagonal matrix and a right singular matrix, wherein the left singular matrix is ​​an eigenvector of a matrix obtained by multiplying the absorbance matrix by a transposed matrix of the absorbance matrix, the right singular matrix is ​​an eigenvector of a matrix obtained by multiplying the transposed matrix of the absorbance matrix by the absorbance matrix, and the absorbance matrix is ​​the product of the left singular matrix, the singular value diagonal matrix and the transposed matrix of the right singular matrix; A pseudo-inverse matrix of the absorbance matrix is ​​determined according to the left singular matrix, the singular value diagonal matrix, and the right singular matrix.

3. The method according to claim 2, characterized in that Determining a pseudo-inverse matrix of the absorbance matrix according to the left singular matrix, the singular value diagonal matrix, and the right singular matrix includes: Calculating the transposed matrix of the singular value diagonal matrix; Taking the inverse of the diagonal elements in the transposed matrix of the singular value diagonal matrix to obtain a target matrix; The right singular matrix, the target matrix, and the transposed matrix of the left singular matrix are multiplied to obtain a pseudo-inverse matrix of the absorbance matrix.

4. The method according to claim 1, characterized in that: Determine the absorbance matrix of multiple stains used to stain pathology slide images, including: Determining initial values ​​of absorbance matrices of a plurality of dyes used to dye pathological section dyed images; An optimal absorbance matrix is ​​determined according to the absorbance matrix initial value.

5. The method according to claim 4, characterized in that Determine the initial values ​​of the absorbance matrix of multiple stains used to stain pathological section staining images, including: Calculating the intensity value of each pixel in the RGB image of each stain, and calculating the absorbance value of each pixel according to the intensity value of each pixel, wherein the RGB image is obtained by staining different pathological sections with each stain; Standardizing the absorbance value of each pixel to obtain a standardized absorbance vector of each pixel; Calculating initial values ​​of parameters of an objective function using the standardized absorbance vector, wherein the objective function is used to optimize parameters in a deconvolution process; The absorbance matrix initial value is constructed based on the parameter initial value, wherein each column of the absorbance matrix initial value represents the relative absorbance of a pixel point corresponding to a dye to an RGB channel.

6. The method according to claim 5, characterized in that Determining an optimal absorbance matrix according to the absorbance matrix initial value includes: Calculating the optimal parameters of the objective function based on the absorbance matrix initial value and the gradient descent algorithm; The optimal absorbance matrix is ​​calculated based on the optimal parameters.

7. The method according to claim 5, characterized in that The objective function is given by components, wherein n is the number of types of the plurality of dyes, and each component corresponds to an evaluation index of a dye combination consisting of two different dyes.

8. The method according to claim 1, characterized in that The deconvolution matrix is ​​used to determine the single channel results of various dyes, including: Calculating the staining intensity of each pixel in a single staining layer of each staining agent respectively; The staining intensity of each pixel in a single staining layer of the various staining agents is converted from an optical density value to a light intensity expression.

9. An electronic device, comprising: A processor and a memory storing a program, wherein the program comprises instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 8.

10. A non-transitory machine-readable medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 8.

Citation Information

Patent Citations

  • Color normalization method for HE staining pathological images

    CN108172278A

  • Method for expanding digital pathology image dataset sample based on dyeing component adjusting

    CN108229569A

  • An approximation method for fast training a self-encoder based on pseudo-inverse learning

    CN109034387A

  • Target object identification method and device, electronic equipment and storage medium

    CN116246254A

  • Color unmixing and region of interest detection in tissue samples

    EP1636752A2