Immunohistochemical quantitative analysis method based on YOLOv8 and multi-staining separation
By combining the YOLOv8 network and the color deconvolution algorithm, the problems of cell localization accuracy and staining signal overlap in immunohistochemical quantitative analysis were solved, and accurate calculation of Ki-67 and PD-L1 expression was achieved, meeting the needs of clinical analysis.
Patent Information
- Application Number
- CN202510798292.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-16
- Publication Date
- 2025-11-07
AI Technical Summary
Existing immunohistochemical quantitative analysis methods suffer from insufficient cell localization accuracy, overlapping staining signals, and insufficient dynamic adaptability, resulting in poor accuracy and consistency of quantitative analysis.
The YOLOv8 network structure was used to train and test the immunohistochemical staining slide dataset. The color deconvolution algorithm was combined to separate multiple stainings. The model was optimized by a multi-task loss function to realize the detection of Ki-67 and PD-L1 and the calculation of the staining channel ratio.
It improves cell recognition accuracy, reduces staining overlap interference, ensures accurate calculation of Ki-67 and PD-L1 expression levels, and supports the clinical analysis needs of multiple immunohistochemical indicators.
Smart Images
Figure CN120913196A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of pathological image recognition, and in particular to an immunohistochemical quantitative analysis method based on YOLOv8 and multi-stain separation. BACKGROUND
[0002] In recent years, deep learning and digital pathology analysis technology have played an important role in immunohistochemical (IHC) quantitative analysis, and the quantitative evaluation of Ki-67 (a cell proliferation marker) and PD-L1 (programmed death ligand-1) is crucial for tumor grading, prognosis judgment and treatment plan formulation; current clinical practice mainly relies on manual reading by pathologists, which has problems such as inter-observer differences and low efficiency. With the development of digital pathology and artificial intelligence technology, automated analysis based on deep learning has gradually become a research hotspot.
[0003] The technical solutions of the prior art and their defects are as follows:
[0004] The existing immunohistochemical quantitative analysis method is mainly based on traditional image processing or a single deep learning model, and its main defects are as follows:
[0005] Insufficient cell localization accuracy: lack of cell-specific recognition, resulting in large calculation deviation of the corresponding indicators;
[0006] Staining signal overlap: the spectral overlap of Hematoxylin (nuclear staining) and DAB (positive signal) in IHC staining results in insufficient accuracy of traditional threshold segmentation method;
[0007] Insufficient dynamic adaptability: fixed threshold or single model is difficult to adapt to different staining batches and sample types. SUMMARY
[0008] The present application provides an immunohistochemical quantitative analysis method based on YOLOv8 and multi-stain separation
[0009] An immunohistochemical quantitative analysis method based on YOLOv8 and multi-stain separation, comprising:
[0010] S1: obtaining an immunohistochemical staining section data set containing Ki-67 and PD-L1 staining sections, and training and testing the YOLOv8 network structure based on the immunohistochemical staining section data set to obtain a target YOLOv8 model;
[0011] S2: detecting the target YOLOv8 model based on the detected staining section to obtain the detection results of Ki-67 and PD-L1;
[0012] S3: performing staining separation on the detected staining section based on a multi-stain separation algorithm based on a color deconvolution algorithm to obtain a staining channel ratio;
[0013] S4: determining the quantitative analysis result of the to-be-detected staining slice based on the detection results of Ki-67 and PD-L1 and the staining channel ratio.
[0014] Preferably, in S1, the YOLOv8 network structure is trained and tested based on the immunohistochemical staining slice dataset to obtain a target YOLOv8 model, including:
[0015] The immunohistochemical staining slice dataset is divided into a training set and a test set in a ratio of 7:3 by stratified sampling, and a specified random seed is set for the data in the training set and the test set;
[0016] The YOLOv8 network structure is trained based on the training set, and a dynamic sample allocation method is used to automatically adjust the positive and negative sample ratio during the training process;
[0017] In the training parameter setting, a phased learning rate strategy is adopted, and the detection head parameters are mainly optimized in the initial stage, and end-to-end fine-tuning is performed in the later stage;
[0018] The training result is tested and optimized based on the test set to obtain the target YOLOv8 model.
[0019] Preferably, in S2, the target YOLOv8 model is used to detect the to-be-detected staining slice to obtain the detection results of Ki-67 and PD-L1, including:
[0020] The to-be-detected staining slice is input into the target YOLOv8 model for detection to obtain an output result;
[0021] The output result is counted to obtain the expression of Ki-67 and PD-L1 as the final detection result.
[0022] Preferably, the design method of the loss function in the target YOLOv8 model is as follows:
[0023] The expression formula of the loss function is as follows:
[0024] L total =γ1L box +γ2L cls +γ3L seg +γ4L dfl
[0025] Wherein, L total represents a multi-task loss function, L box represents an IoU-based bounding box regression loss, γ1 represents the loss weight of L box in the multi-task loss function, L cls represents an improved binary cross-entropy loss function, and γ2 represents the loss weight of Lcls Loss weight in multi-task loss function, L seg Indicates the introduction of area normalization mechanism, γ3 represents L seg Loss weight in multi-task loss function, L dfl Indicates the distribution focus loss, γ4 represents L dfl Loss weight in multi-task loss function.
[0026] Preferably, in S3, the multi-stain separation algorithm based on color deconvolution algorithm is used to separate the stains of the to-be-detected stained section, and a stain channel ratio is obtained, comprising:
[0027] The to-be-detected stained section is normalized to map the pixel value range from [0, 255] to the [0, 1] interval, and the normalized calculation formula is as follows:
[0028]
[0029] Wherein, I norm Indicates the normalized pixel value of the to-be-detected stained section, I RGB Indicates the initial pixel value of the to-be-detected stained section, and ε represents a minimum value, which is 10 -6 ;
[0030] According to the following formula, the normalized pixel value is processed to obtain the representation of optical density space OD RGB ;
[0031] OD RGB =-ln(I norm )
[0032] The standard dye absorption vector corresponding to the image staining scheme is selected and unit vector normalized to construct a color separation matrix, and the expression of the color separation matrix M is as follows:
[0033]
[0034] Wherein, Indicates the unit vector normalized hematoxylin vector, Indicates the unit vector normalized diaminobenzidine vector, Indicates the unit vector normalized residual lesion vector;
[0035] Based on the optical density space OD RGB And the inverse matrix of the color separation matrix M are multiplied to obtain the corresponding dye concentration response vector;
[0036] Based on the dye concentration response vector, the concentration response of all pixels is restored to a 3-channel image, including a cell nucleus channel C H , and a target protein positive channel CD background and other noise signals C R ;
[0037] For the image block area obtained by the reduction separation according to the concentration response, the staining channel ratio Ratio is calculated according to the following formula:
[0038]
[0039] Wherein, Mean(C D ) represents the mean of non-zero pixels of the target protein positive channel, and Mean(C H ) represents the mean of non-zero pixels of the nucleus channel.
[0040] Preferably, in the S4, the quantitative analysis result of the to-be-detected staining slice is determined based on the detection results of Ki-67 and PD-L1 and the staining channel ratio, including:
[0041] determine whether the staining channel ratio is greater than a preset threshold value;
[0042] If yes, the quantitative analysis result of the to-be-detected staining slice is determined based on the staining channel ratio;
[0043] Otherwise, the quantitative analysis result of the to-be-detected staining slice is determined based on the preset threshold value.
[0044] Preferably, the calculation formula of the detection results of Ki-67 and PD-L1 is as follows:
[0045] The calculation formula of the Ki-67 index is as follows:
[0046]
[0047] Wherein, K represents the Ki-67 index, N represents the total number of tumor cells, and A represents the number of Ki-67 positive tumor cells.
[0048] The calculation formula of the detection result TPS of PD-L1 is as follows:
[0049]
[0050] Wherein, TPS represents the PD-L1 index, and B represents the number of PD-L1 positive tumor cells.
[0051] Preferably, including:
[0052] The quantitative analysis result of the to-be-detected staining slice is determined based on the staining channel ratio, specifically:
[0053] determine whether the detection results of Ki-67 and PD-L1 are greater than the staining channel ratio;
[0054] If yes, it is determined that the to-be-detected staining section is a positive cell, otherwise, it is determined that the to-be-detected staining section is a non-positive cell.
[0055] The determination of the quantitative analysis result of the to-be-detected staining section is based on a preset threshold value:
[0056] Whether the detection results of Ki-67 and PD-L1 are greater than the preset threshold value is determined.
[0057] If yes, it is determined that the to-be-detected staining section is a positive cell, otherwise, it is determined that the to-be-detected staining section is a non-positive cell.
[0058] Preferably, before training and testing the YOLOv8 network structure on the immunohistochemical staining section data set, the immunohistochemical staining section data set also includes image enhancement of the staining sections in the immunohistochemical staining section data set, specifically:
[0059] The staining section is decomposed to obtain a hematoxylin channel, a DAB channel and a background channel, and based on the differences between the hematoxylin channel, the DAB channel and the background channel and the corresponding standard channels, the color disturbance parameters of the hematoxylin channel, the DAB channel and the background channel are processed and integrated respectively to obtain a first enhanced staining section.
[0060] The first enhanced staining section is preliminarily identified to determine cell performance characteristics, basic features and specific features in the cell performance characteristics are obtained, an initial cropping strategy for the first enhanced staining section is established based on the basic features, cropping constraint information is determined based on the specific features, the initial cropping strategy is optimized based on the cropping constraint information to obtain a target cropping strategy, and the first enhanced staining section is cropped according to the target cropping strategy to obtain a plurality of sub-sections.
[0061] The boundary semantic features of the plurality of sub-sections are obtained, and based on the position distribution relationship between the boundary semantic features and the plurality of sub-sections, a plurality of attention weights are given to the boundaries of the plurality of sub-sections, and a plurality of attention heads are given to the boundaries of the plurality of sub-sections based on the attention weights, to obtain a multi-attention head mechanism for the boundaries.
[0062] After adding the multi-attention head mechanism to the mosaic enhancement strategy for enhancing the plurality of sub-sections, a target sub-section is obtained, and then the target sub-section is spliced with a preset overlap probability to obtain a target enhanced staining section.
[0063] Preferably, the boundary semantic features of the plurality of sub-sections are obtained, and based on the position distribution relationship between the boundary semantic features and the plurality of sub-sections, a plurality of attention weights are given to the boundaries of the plurality of sub-sections, including:
[0064] Based on the importance of the boundary semantic features of the plurality of sub-sections for immunohistochemical quantitative analysis, the initial attention weights of the boundaries of the sub-sections are determined.
[0065] Based on the position distribution relationship between the plurality of sub-slices, the initial attention weight of the boundary of the sub-slice is adaptively adjusted according to the standard that the difference of attention weight of splicing two sub-slices does not exceed the preset weight threshold, and the final attention weight is obtained.
[0066] Compared with the prior art, the present application has the following beneficial effects:
[0067] By obtaining an immunohistochemical staining slice dataset containing Ki-67 and PD-L1 staining slices, and training and testing the YOLOv8 network structure based on the immunohistochemical staining slice dataset, a target YOLOv8 model is obtained to ensure the accuracy of the model. Based on the target YOLOv8 model, the detection of the to-be-detected staining slice is carried out to obtain the detection results of Ki-67 and PD-L1, realize the identification of the pathological image cell condition, support the parallel calculation of Ki-67 index and PD-L1 TPS, meet the clinical analysis demand of multiple immunohistochemical indexes, have wide practical value and popularization prospect, provide data basis for immunohistochemical quantitative analysis, carry out staining separation on the to-be-detected staining slice based on the multi-staining separation algorithm of color deconvolution algorithm, obtain the staining channel ratio, accurately distinguish Hematoxylin and DAB signals by using the multi-staining channel separation algorithm based on color deconvolution, effectively reduce the staining overlap interference, improve the reliability of positive interpretation, determine the quantitative analysis result of the to-be-detected staining slice based on the detection results of Ki-67 and PD-L1 and the staining channel ratio, and realize accurate calculation of the expression levels of PD-L1 and Ki-67, to provide reliable data support for pathological analysis.
[0068] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application can be realized and obtained by the structure particularly pointed out in the application.
[0069] The technical solutions of the present application will be further described in detail below with the help of the drawings and examples. BRIEF DESCRIPTION OF DRAWINGS
[0070] The accompanying drawings are included to provide a further understanding of the present application, and constitute a part of the specification, together with the embodiments of the present application, to explain the present application, and do not constitute a limitation on the present application. In the drawings:
[0071] Figure 1 The flow chart of the immunohistochemical quantitative analysis method based on YOLOv8 and multi-staining separation in an embodiment of the present application is shown in the figure.
[0072] Figure 2A flowchart for obtaining a target YOLOv8 model in an embodiment of the present application is shown in the figure.
[0073] Figure 3 A flowchart for image enhancement of a stained section in an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION
[0074] The preferred embodiments of the present application are described below in conjunction with the accompanying drawings, and it should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application.
[0075] Embodiment 1:
[0076] An immunohistochemical quantitative analysis method based on YOLOv8 and multi-stain separation is provided in an embodiment of the present application, as shown in the figure, which comprises: Figure 1
[0077] S1: Obtain an immunohistochemical staining section data set containing Ki-67 and PD-L1 staining sections, and train and test the YOLOv8 network structure based on the immunohistochemical staining section data set to obtain a target YOLOv8 model;
[0078] S2: Detect the target YOLOv8 model based on the target YOLOv8 model to obtain the detection results of Ki-67 and PD-L1;
[0079] S3: Perform stain separation on the target staining section based on the multi-stain separation algorithm based on the color deconvolution algorithm to obtain the staining channel ratio;
[0080] S4: Determine the quantitative analysis result of the target staining section based on the detection results of Ki-67 and PD-L1 and the staining channel ratio.
[0081] In this embodiment, the immunohistochemical staining section data set reviewed by a pathologist is used to ensure the accuracy and reliability of the annotation results. The data set contains Ki-67 and PD-L1 staining sections, wherein the Ki-67 section is annotated with tumor cells and normal cells, and the PD-L1 section only contains tumor cell regions.
[0082] In this embodiment, PD-L1 represents Programmed Death Ligand-1.
[0083] In this embodiment, Ki-67 represents the proportion of tumor cells expressing Ki-67 protein to the total number of tumor cells, and is an indicator for measuring cell proliferation activity.
[0084] In this embodiment, YOLOv8 is an advanced object detection neural network framework with high speed and high precision image object recognition capability, which is used for cell target recognition and segmentation in pathological sections in the present application.
[0085] In this embodiment, the theoretical basis of the color deconvolution algorithm is the Lambert-Beer law, that is, the degree of light absorption is proportional to the dye concentration and the optical path.
[0086] In this embodiment, in pathological image analysis, tissue sections are usually subjected to multiple staining processes, such as Hematoxylin and DAB staining, to distinguish cell nucleus structures and target protein expression regions. Due to mutual interference between multiple staining image channels, direct feature extraction or quantitative analysis on RGB images often leads to insufficient accuracy. Therefore, mathematical separation of the spectral responses of different dyes is a key step to realize accurate image analysis, so the present application uses a multi-staining separation algorithm based on color deconvolution.
[0087] The beneficial effects of the above design scheme are: by obtaining an immunohistochemical staining section data set containing Ki-67 and PD-L1 staining sections, training and testing the YOLOv8 network structure based on the immunohistochemical staining section data set, obtaining a target YOLOv8 model, ensuring the accuracy of the model, detecting the to-be-detected staining section based on the target YOLOv8 model, obtaining the detection results of Ki-67 and PD-L1, realizing the recognition of the pathological image cell situation, supporting the parallel calculation of the Ki-67 index and the PD-L1 TPS, meeting the clinical analysis demand of multiple immunohistochemical indexes, having wide practical value and popularization prospect, providing a data basis for immunohistochemical quantitative analysis, using a multi-staining separation algorithm based on color deconvolution to separate the to-be-detected staining section, obtaining a staining channel ratio, using a multi-staining channel separation algorithm based on color deconvolution to accurately distinguish Hematoxylin and DAB signals, effectively reducing staining overlap interference, and improving the reliability of positive interpretation, determining the quantitative analysis result of the to-be-detected staining section based on the detection results of Ki-67 and PD-L1 and the staining channel ratio, and realizing accurate calculation of the expression levels of PD-L1 and Ki-67, providing reliable data support for pathological analysis.
[0088] Embodiment 2:
[0089] Based on the basis of embodiment 1, the present application provides an immunohistochemical quantitative analysis method based on YOLOv8 and multi-staining separation, as shown in Figure 2 In S1, the YOLOv8 network structure is trained and tested based on the immunohistochemical staining section data set, and a target YOLOv8 model is obtained, including:
[0090] The immunohistochemical staining section data set is divided into a training set and a test set in a ratio of 7:3 by stratified sampling, and a specified random seed is set for the data in the training set and the test set;
[0091] The YOLOv8 network structure is trained based on the training set, and a dynamic sample allocation method is used in the training process to automatically adjust the positive and negative sample ratio.
[0092] In the training parameter setting, a staged learning rate strategy is adopted, and in the initial stage, the detection head parameters are mainly optimized, and in the later stage, end-to-end fine tuning is performed.
[0093] The training result is tested and optimized based on the test set, and the target YOLOv8 model is obtained.
[0094] The beneficial effects of the above design scheme are: the immunohistochemical staining section data set is divided into a training set and a test set in a ratio of 7:3 by hierarchical sampling, which considers the data distribution of different cases and different staining batches, ensures the consistency of the training set and the test set in data features, and at the same time, all data preprocessing processes are set with fixed random seeds to ensure the repeatability of the experiment process. Based on the training process, the dynamic sample allocation method is used to automatically adjust the positive and negative sample ratio to ensure the recognition accuracy of the model for tumor cells. A staged learning rate strategy is adopted, and in the initial stage, the detection head parameters are mainly optimized, and in the later stage, end-to-end fine tuning is performed to optimize the model performance. Based on the test set, the training result is tested and optimized, and the target YOLOv8 model is obtained to ensure the accuracy of the target YOLOv8 model.
[0095] Embodiment 3:
[0096] Based on the basis of embodiment 1, the present application provides an immunohistochemical quantitative analysis method based on YOLOv8 and multi-staining separation. In S2, the target YOLOv8 model is used to detect the detected staining section to obtain the detection results of Ki-67 and PD-L1, which includes:
[0097] The detected staining section is input into the target YOLOv8 model for detection to obtain the output result.
[0098] The output result is counted to obtain the expression of Ki-67 and PD-L1 as the final detection result.
[0099] The beneficial effects of the above design scheme are: by inputting the detected staining section into the target YOLOv8 model for detection, the output result is obtained, and the expression of Ki-67 and PD-L1 is counted as the final detection result, which realizes the recognition of pathological image cell conditions, supports the parallel calculation of Ki-67 index and PD-L1 TPS, and meets the clinical analysis demand of multiple immunohistochemical indicators, has wide practical value and popularization prospect.
[0100] Embodiment 4:
[0101] Based on the basis of Embodiment 3, the present embodiment provides a YOLOv8-based and multi-staining separation immunohistochemical quantitative analysis method, and the design of the loss function in the target YOLOv8 model is as follows:
[0102] The expression formula of the loss function is as follows:
[0103] L total =β1L box +γ2L cls +γ3L seg +γ4L dfl
[0104] Wherein, L total represents a multi-task loss function, L box represents an IoU-based bounding box regression loss, γ1 represents the loss weight of L box in the multi-task loss function, L cls represents an improved binary cross-entropy loss function, γ2 represents the loss weight of L cls in the multi-task loss function, L seg represents the introduction of an area normalization mechanism, γ3 represents the loss weight of L seg in the multi-task loss function, L dfl represents a distribution focal loss, and γ4 represents the loss weight of L dfl in the multi-task loss function.
[0105] In this embodiment, the IoU-based bounding box regression loss is adopted, and the dynamic sampling strategy is combined to optimize the positioning accuracy.
[0106] In this embodiment, the improved binary cross-entropy loss function is used to enhance the classification and discrimination ability of tumor cells and normal cells.
[0107] In this embodiment, the area normalization mechanism is introduced, and the pixel-level weighting strategy is used to balance the segmentation quality of cells of different sizes.
[0108] In this embodiment, the distribution focal loss is used to optimize the probability distribution modeling of the bounding box.
[0109] The beneficial effects of the above design scheme are: through the dynamic weight balance mechanism, the model can maintain the detection accuracy while significantly improving the cell edge segmentation quality. The specially designed area normalization segmentation loss term effectively solves the training imbalance problem caused by the large size difference of cells in the immunohistochemical image, and significantly improves the segmentation performance of the model for weakly stained areas and small cells.
[0110] Embodiment 5:
[0111] Based on the basis of embodiment 1, the embodiment of the application provides an immunohistochemical quantitative analysis method based on YOLOv8 and multi-stain separation, in the S3, the multi-stain separation algorithm based on color deconvolution algorithm is used to separate the staining of the to-be-detected staining section, and the staining channel ratio is obtained, including:
[0112] The to-be-detected staining section is normalized, and the pixel value range is mapped from [0, 255] to [0, 1] interval, and the normalized calculation formula is as follows:
[0113]
[0114] Wherein, I norm represents the normalized pixel value of the to-be-detected staining section, I RGB represents the initial pixel value of the to-be-detected staining section, and ε represents the minimum value, which is 10 -6 .
[0115] According to the following formula, the normalized pixel value is processed to obtain the representation of optical density space OD RGB .
[0116] OD RGB = -ln(I norm )
[0117] The standard dye absorption vector corresponding to the image staining scheme is selected and unit vector normalized to construct a color separation matrix, and the expression of the color separation matrix M is as follows:
[0118]
[0119] Wherein, represents the hematoxylin vector after unit vector normalization, represents the vector after unit vector normalization of diaminobenzidine, and represents the residual lesion vector after unit vector normalization.
[0120] Based on the multiplication of the optical density space OD RGB and the inverse matrix of the color separation matrix M, the corresponding dye concentration response vector is obtained.
[0121] Based on the dye concentration response vector, the concentration response of all pixels is restored to 3-channel images, including the cell nucleus channel C H , the target protein positive channel C D , and the background and other noise signals C R .
[0122] According to the following formula, the dye channel ratio Ratio is calculated for the image block area obtained by separating and restoring the concentration response.
[0123]
[0124] wherein Mean(C D ) represents the mean of non-zero pixels of the target protein positive channel, and Mean(C H ) represents the mean of non-zero pixels of the nucleus channel.
[0125] In this embodiment, the standard dye absorption vector includes hematoxylin, diaminobenzidine and residual lesions.
[0126] In this embodiment, the value of each pixel in space is multiplied by the deconvolution matrix to obtain the corresponding dye concentration response vector:
[0127] C=M -1 *OD RGB
[0128] This process is color deconvolution, and the output includes three values representing the relative concentrations of hematoxylin, DAB and residual components in the pixel.
[0129] In this embodiment, the nucleus channel is blue-violet and the target protein positive channel is brown.
[0130] In this embodiment, the purpose of adding the minimum value is to prevent numerical instability when taking the logarithm later.
[0131] In this embodiment, OD RGB vector can be understood as the total dye absorption of the pixel in the RGB channel.
[0132] The beneficial effects of the above design scheme are: through image preprocessing, dye vector matrix construction, color deconvolution, channel separation post-processing, a multi-staining channel separation algorithm based on color deconvolution is adopted to accurately distinguish Hematoxylin and DAB signals, effectively reduce staining overlap interference, and improve the reliability of positive interpretation.
[0133] Embodiment 6:
[0134] Based on the basis of embodiment 1, the present embodiment provides an immunohistochemical quantitative analysis method based on YOLOv8 and multi-staining separation, wherein in S4, the quantitative analysis result of the to-be-detected staining section is determined based on the detection results of Ki-67 and PD-L1 and the staining channel ratio, including:
[0135] determining whether the staining channel ratio is greater than a preset threshold value;
[0136] If yes, the quantitative analysis result of the to-be-detected staining section is determined based on the staining channel ratio;
[0137] Otherwise, determine the quantitative analysis result of the to-be-detected dyeing slice based on a preset threshold.
[0138] The beneficial effect of the above design scheme is that by judging whether the dyeing channel ratio is greater than the preset threshold, if yes, determining the quantitative analysis result of the to-be-detected dyeing slice based on the dyeing channel ratio, otherwise, determining the quantitative analysis result of the to-be-detected dyeing slice based on the preset threshold, which provides an accurate threshold basis for positive detection of the to-be-detected dyeing slice.
[0139] Embodiment 7:
[0140] Based on the basis of embodiment 3, the embodiment of the application provides a kind of immunohistochemical quantitative analysis method based on YOLOv8 and multiple dyeing separation, the calculation formula of the detection result of Ki-67 and PD-L1 is as follows:
[0141] The calculation formula of Ki-67 index is as follows:
[0142]
[0143] Wherein, K represents Ki-67 index, N represents total tumor cell number, A represents Ki-67 positive tumor cell number;
[0144] The calculation formula of the detection result TPS of PD-L1 is as follows:
[0145]
[0146] Wherein, TPS represents PD-L1 index, B represents PD-L1 positive tumor cell number.
[0147] The beneficial effect of the above design scheme is that by parallel computing Ki-67 index and PD-L1 TPS, the clinical analysis demand of multiple immunohistochemical indexes is met, which has extensive practical value and popularization prospect.
[0148] Embodiment 8:
[0149] Based on the basis of embodiment 3, the embodiment of the application provides a kind of immunohistochemical quantitative analysis method based on YOLOv8 and multiple dyeing separation, including:
[0150] Determine the quantitative analysis result of the to-be-detected dyeing slice based on the dyeing channel ratio specifically:
[0151] Judge whether the detection result of Ki-67 and PD-L1 is greater than the dyeing channel ratio;
[0152] If yes, determine that the to-be-detected dyeing slice is positive cell, otherwise, determine that the to-be-detected dyeing slice is non-positive cell;
[0153] The quantitative analysis result of the to-be-detected dyeing slice is determined based on a preset threshold value, specifically:
[0154] whether the detection results of Ki-67 and PD-L1 are greater than the preset threshold value;
[0155] If yes, the to-be-detected dyeing slice is determined as a positive cell, otherwise, the to-be-detected dyeing slice is determined as a non-positive cell.
[0156] The beneficial effects of the above design scheme are: different judgment methods are adopted based on different situations to accurately calculate the expression levels of PD-L1 and Ki-67, and reliable data support is provided for pathological analysis.
[0157] Embodiment 9:
[0158] Based on the basis of embodiment 1, the present embodiment provides an immunohistochemical quantitative analysis method based on YOLOv8 and multi-dye separation, as shown in Figure 3 Before training and testing the YOLOv8 network structure, the immunohistochemical dyeing slice dataset also includes image enhancement of the dyeing slice in the immunohistochemical dyeing slice dataset, specifically:
[0159] The dyeing slice is decomposed to obtain a hematoxylin channel, a DAB channel and a background channel, and based on the difference features of the hematoxylin channel, the DAB channel and the background channel and the corresponding standard channel, the color disturbance parameters of the hematoxylin channel, the DAB channel and the background channel are respectively processed and integrated to obtain a first enhanced dyeing slice;
[0160] The first enhanced dyeing slice is preliminarily identified to determine the cell performance characteristics, the basic features and specific features in the cell performance characteristics are obtained, the initial cropping strategy of the first enhanced dyeing slice is established based on the basic features, the cropping constraint information is determined based on the specific features, the initial cropping strategy is optimized based on the cropping constraint information, the target cropping strategy is obtained, and the first enhanced dyeing slice is cropped according to the target cropping strategy to obtain a plurality of sub-slices;
[0161] The boundary semantic features of the plurality of sub-slices are obtained, and based on the boundary semantic features and the position distribution relationship between the plurality of sub-slices, a plurality of attention weights are given to the boundaries of the plurality of sub-slices, and a plurality of attention heads are given to the boundaries of the plurality of sub-slices based on the attention weights, to obtain a multi-attention head mechanism for the boundaries.
[0162] After adding the multi-attention head mechanism to the mosaic enhancement strategy to enhance the plurality of sub-slices, a target sub-slice is obtained, and then the target sub-slice is spliced with a preset overlap probability to obtain a target enhanced dyeing slice.
[0163] In this embodiment, the color disturbance parameters of the hematoxylin channel, the DAB channel and the background channel are respectively established, for example, only the contrast stretching is performed on the hematoxylin channel, and the color is unchanged, and for example, only the color adjustment is performed on the background channel.
[0164] In this embodiment, the traditional color disturbance parameter design is based on the whole and cannot be well targeted to the pathological staining characteristics and even destroys the original staining characteristics, and the present application is based on the difference characteristics of the hematoxylin channel, the DAB channel and the background channel from the corresponding standard channel, respectively establishes the color disturbance parameters of the hematoxylin channel, the DAB channel and the background channel for processing and integration, and realizes the specific optimization of the stained slice.
[0165] In this embodiment, the traditional mosaic enhancement strategy for processing the stained slice may cause the cell boundary to be blurred, and the specific reservation of the boundary information is realized through the multi-attention mechanism and the cropping strategy, and the analysis ability of the subsequent model for the stained slice is improved.
[0166] In this embodiment, the boundary semantic features are pathological labels, cell morphology, etc.
[0167] In this embodiment, the multi-attention head mechanism for establishing the boundary is to realize the specific feature determination of the boundary, provide a basis for the specific determination of the mosaic enhancement, and ensure that the mosaic enhancement strategy can reserve important feature information of the clear boundary.
[0168] The beneficial effects of the above design scheme are: the color disturbance parameters are set for the specific channel characteristics of the stained slice, the specific optimization based on the staining characteristics is realized, the slice characteristics are ensured, then the multi-attention mechanism and the cropping strategy are added to the mosaic enhancement strategy, the specific reservation of the boundary information is realized, the analysis ability of the subsequent model for the stained slice is improved, finally, the data enhancement based on the pathological image characteristics is realized, and accurate data basis is provided for the model training.
[0169] Embodiment 10:
[0170] Based on the basis of embodiment 9, the present application provides an immunohistochemical quantitative analysis method based on YOLOv8 and multi-staining separation, acquires the boundary semantic features of the multiple sub-slices, and based on the position distribution relationship between the boundary semantic features and the multiple sub-slices, gives multiple attention weights to the boundaries of the multiple sub-slices, including:
[0171] Based on the importance of the boundary semantic features of the multiple sub-slices for the immunohistochemical quantitative analysis, the initial attention weight of the boundary of the sub-slice is determined;
[0172] Based on the position distribution relationship among the multiple sub-slices, the initial attention weight of the boundary of the fixed sub-slice is adaptively adjusted according to the standard that the attention weight difference of the two sub-slices to be spliced does not exceed the preset weight threshold, and the final attention weight is obtained.
[0173] The beneficial effects of the above design scheme are: based on the importance of the boundary semantic features of multiple sub-slices for immunohistochemical quantitative analysis, the initial attention weight of the boundary of the sub-slice is determined, based on the position distribution relationship among the multiple sub-slices, the initial attention weight of the boundary of the fixed sub-slice is adaptively adjusted according to the standard that the attention weight difference of the two sub-slices to be spliced does not exceed the preset weight threshold, and the final attention weight is obtained, so that the design attention weight considers the semantic features and the position features, ensures the accuracy of the obtained attention weight, and provides a basis for setting the mosaic enhancement strategy.
[0174] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application belong to the scope of the present application and its equivalent technologies, the present application also intends to include these modifications and variations.
Claims
1. A method for immunohistochemical quantification based on YOLOv8 and multi-stain separation, characterized in that, The method comprises the following steps: S1: obtaining an immunohistochemical staining slice data set containing Ki-67 and PD-L1 staining slices, and training and testing a YOLOv8 network structure based on the immunohistochemical staining slice data set to obtain a target YOLOv8 model; S2: detecting the to-be-detected staining slice based on the target YOLOv8 model to obtain detection results of Ki-67 and PD-L1; S3: performing staining separation on the to-be-detected staining slice based on a multi-staining separation algorithm of a color deconvolution algorithm to obtain a staining channel ratio; S4: determining a quantitative analysis result of the to-be-detected staining slice based on the detection results of Ki-67 and PD-L1 and the staining channel ratio.
2. The method according to claim 1, wherein, In S1, the target YOLOv8 model is obtained by training and testing the YOLOv8 network structure based on the immunohistochemical staining slice data set, and the method comprises the following steps: The immunohistochemical staining slice data set is divided into a training set and a test set in a ratio of 7:3 by stratified sampling, and a specified random seed is set for the data in the training set and the test set; The YOLOv8 network structure is trained based on the training set, and a dynamic sample allocation method is used to automatically adjust the positive and negative sample ratio during the training process; In the training parameter setting, a staged learning rate strategy is adopted, the detection head parameters are mainly optimized in the initial stage, and end-to-end fine tuning is performed in the later stage; The training result is tested and optimized based on the test set to obtain the target YOLOv8 model.
3. The method of claim 1, wherein the method is based on YOLOv8 and multi-stain separation for immunohistochemical quantification analysis. In S2, the detection results of Ki-67 and PD-L1 are obtained by detecting the to-be-detected staining slice based on the target YOLOv8 model, and the method comprises the following steps: The to-be-detected staining slice is input into the target YOLOv8 model for detection to obtain an output result; The expression of Ki-67 and PD-L1 is obtained by counting the output result, which is the final detection result.
4. The method according to claim 3, wherein, The design of the loss function in the target YOLOv8 model is as follows: The expression formula of the loss function is as follows: L total = γ1L box + γ2L cls + γ3L seg + γ4L dfl wherein, L total represents a multi-task loss function, L box represents an IoU-based bounding box regression loss, γ1 represents L box a loss weight in the multi-task loss function, L cls represents a modified binary cross-entropy loss function, γ2 represents L cls a loss weight in the multi-task loss function, L seg represents the introduction of an area normalization mechanism, γ3 represents L seg a loss weight in the multi-task loss function, L dfl represents a distribution focus loss, γ4 represents L dfl a loss weight in the multi-task loss function.
5. The method of claim 1, wherein the method is based on YOLOv8 and multi-stain separation for immunohistochemical quantification analysis. In S3, the staining channel ratio is obtained by performing staining separation on the to-be-detected staining slice based on the multi-staining separation algorithm of the color deconvolution algorithm, and the method comprises the following steps: The to-be-detected staining slice is normalized to map the pixel value range from [0, 255] to the [0, 1] interval, and the normalization calculation formula is as follows: Wherein, I norm represents the normalized pixel value of the dyeing section to be detected, I RGB represents the initial pixel value of the dyeing section to be detected, and ε represents a minimum value, which is 10 -6 ; The normalized pixel values are processed according to the following equation to obtain an optical density space OD RGB representation; OD RGB = -ln(I norm ) A standard dye absorption vector corresponding to the image staining scheme is selected and unit vector normalization is performed to construct a color separation matrix, and the expression of the color separation matrix M is as follows: wherein, represents the hematoxylin vector normalized by a unit vector, represents the diamino-benzidine vector normalized by a unit vector, represents the residual lesion vector normalized by a unit vector; Based on the optical density space OD RGB The inverse of the color separation matrix M is multiplied to obtain the corresponding dye concentration response vector; Based on the dye concentration response vector, the concentration response of all pixels is reduced to a 3-channel image, including a nucleus channel C H , a target protein positive channel C D , and a background and other noise signal C R ; The staining channel ratio Ratio is calculated according to the following formula for the image block area separated according to the concentration response reduction; where Mean(C D ) represents the mean of non-zero pixels of the target protein positive channel, and Mean(C H ) represents the mean of non-zero pixels of the nucleus channel.
6. The method of claim 1, wherein the method is based on YOLOv8 and multi-stain separation for immunohistochemical quantification analysis. In S4, the quantitative analysis result of the to-be-detected staining slice is determined based on the detection results of Ki-67 and PD-L1 and the staining channel ratio, and the method comprises the following steps: It is judged whether the staining channel ratio is greater than a preset threshold value; If yes, the quantitative analysis result of the to-be-detected staining slice is determined based on the staining channel ratio; Otherwise, the quantitative analysis result of the to-be-detected staining slice is determined based on the preset threshold value.
7. The method according to claim 3, wherein the method is based on YOLOv8 and multi-stain separation for immunohistochemical quantification analysis. The calculation formula of the detection results of Ki-67 and PD-L1 is as follows: The calculation formula of the Ki-67 index is as follows: Wherein, K represents the Ki-67 index, N represents the total number of tumor cells, and A represents the number of Ki-67 positive tumor cells; The calculation formula of the detection result TPS of PD-L1 is as follows: Wherein, TPS represents the PD-L1 index, and B represents the number of PD-L1 positive tumor cells.
8. The method of claim 3, wherein the method is based on YOLOv8 and multi-stain separation for immunohistochemical quantification analysis. Including: Based on the staining channel ratio, the quantitative analysis result of the to-be-detected staining section is determined, specifically as follows: Determine whether the detection results of Ki-67 and PD-L1 are greater than the staining channel ratio; If yes, determine that the to-be-detected staining section is a positive cell, otherwise, determine that the to-be-detected staining section is a non-positive cell. Based on the preset threshold, the quantitative analysis result of the to-be-detected staining section is determined, specifically as follows: Determine whether the detection results of Ki-67 and PD-L1 are greater than the preset threshold; If yes, determine that the to-be-detected staining section is a positive cell, otherwise, determine that the to-be-detected staining section is a non-positive cell.
9. The method according to claim 1, wherein the method is based on YOLOv8 and multi-stain separation for immunohistochemical quantification analysis. The YOLOv8 network structure of the immunohistochemical staining section data set before training and testing also includes image enhancement of the staining sections in the immunohistochemical staining section data set, specifically as follows: Decompose the staining section to obtain hematoxylin, DAB and background channels, and based on the differences between the hematoxylin, DAB and background channels and the corresponding standard channels, respectively, establish color disturbance parameters for processing and integration of the hematoxylin, DAB and background channels to obtain a first enhanced staining section; Preliminary identification of the first enhanced staining section determines the cell performance characteristics, obtains the basic features and specific features in the cell performance characteristics, establishes an initial cropping strategy for the first enhanced staining section based on the basic features, determines the cropping constraint information based on the specific features, optimizes the initial cropping strategy based on the cropping constraint information, obtains the target cropping strategy, and crops the first enhanced staining section according to the target cropping strategy to obtain a plurality of sub-sections; Obtain the boundary semantic features of the plurality of sub-sections, and based on the position distribution relationship between the boundary semantic features and the plurality of sub-sections, assign a plurality of attention weights to the boundaries of the plurality of sub-sections, and based on the attention weights, assign a plurality of attention heads to the boundaries of the plurality of sub-sections to obtain a multi-attention head mechanism for the boundaries. After adding the multi-attention head mechanism to the mosaic enhancement strategy for enhancing the plurality of sub-sections, a target sub-section is obtained, and then the target sub-section is spliced with a preset overlap probability to obtain a target enhanced staining section.
10. The method of claim 9, wherein the method is based on YOLOv8 and multi-stain separation for immunohistochemical quantification analysis. Obtain the boundary semantic features of the plurality of sub-sections, and based on the position distribution relationship between the boundary semantic features and the plurality of sub-sections, assign a plurality of attention weights to the boundaries of the plurality of sub-sections, including: Based on the importance of the boundary semantic features of the plurality of sub-sections to immunohistochemical quantitative analysis, determine the initial attention weight of the boundary of the sub-section; Based on the position distribution relationship between the plurality of sub-sections, the initial attention weight of the boundary of the sub-section is adaptively adjusted according to the standard that the difference between the attention weights of the two spliced sub-sections does not exceed the preset weight threshold to obtain the final attention weight.