HER2 immunohistochemical pathological image full-automatic quantitative diagnosis method and device and readable storage medium thereof

Abstract

The invention provides an HER2 immunohistochemical pathological image full-automatic quantitative diagnosis method and device and a readable storage medium, the method comprises a foreground tissue segmentation algorithm and an intensity segmentation algorithm, foreground tissue segmentation is based on a Yov8 model, intensity segmentation is based on a PIDNet model, and multi-scale two-stage iteration training is adopted. Through the steps of image screening processing, data labeling, model training and the like, a foreground tissue area can be accurately segmented, the intensity can be subdivided, and meanwhile diagnosis result verification of a pathologist is introduced. According to the method, the defects of a traditional manual film reading and image segmentation method can be overcome, high-precision tissue segmentation and efficient full-automatic quantitative diagnosis are achieved, and the method has wide application prospects in the fields of tumor diagnosis, medicine research and development and the like and has the advantages of being high in automation degree, high in robustness, good in expandability and the like.

Description

Technical Field

[0001] The present application relates to the field of pathological image technology, and in particular to a fully automatic quantitative diagnosis method, device and readable storage medium for HER2 immunohistochemical pathological images, aiming to solve the problems of strong subjectivity, high labor intensity, easy misjudgment and unsatisfactory effect of traditional manual film reading, so as to meet the needs of clinical accurate diagnosis. Background Art

[0002] Immunohistochemistry (IHC) images play an important role in pathological diagnosis. By marking specific antigens in tissues or cells, it can accurately reflect the morphological structure of tissue cells and the expression of related proteins, thus providing an indispensable key basis for disease diagnosis, formulation of treatment plans and prognosis assessment. In immunohistochemistry images, specific sites are stained brown by dimethylbenzidine (DAB), while sites without antigens appear light blue. Antibodies react with existing biomarkers, and different biomarkers characterize the positive conditions of different parts. For example, for tumor areas with positive antibodies, HER2 is distributed in a linear DAB pattern in the cell nucleus, and Ki67 is distributed in a dotted DAB pattern in the cell nucleus. Among them, HER2 (human epidermal growth factor receptor 2), also known as CERBB2, is a proto-oncogene. Its positivity can promote excessive cell division and proliferation. It is a commonly used biomarker. Detecting the expression level of HER2 is of great significance in determining whether cancer patients are suitable for targeted therapy.

[0003] However, traditional HER2 immunohistochemistry image analysis mainly relies on manual reading by pathologists. This method has many disadvantages. It is highly subjective and different doctors may have different judgments. It is labor-intensive and requires doctors to spend a lot of energy. It is also prone to misjudgment, which affects the accuracy of diagnosis. In addition, immunohistochemistry images themselves are highly complex and diverse. While they contain rich information, they also contain many interference factors. For example, the distribution of tissue cells in the image is often uneven, there are differences in staining intensity, and there is large background noise, which makes the accurate segmentation and intensity analysis of different tissue areas in the image face huge challenges.

[0004] With the continuous development of digital pathology technology, computer-assisted analysis of immunohistochemical images has become a research hotspot. However, traditional image segmentation methods, such as threshold segmentation and edge detection, are not satisfactory when processing immunohistochemical images. They are difficult to accurately locate and extract target tissues, nor can they accurately evaluate their intensity characteristics. In addition, they have low consistency with the diagnosis of pathologists and are difficult to meet the needs of clinical precision diagnosis. In this context, with the rapid development of computer technology and artificial intelligence, the development of an efficient, accurate and large-scale reusable fully automatic quantitative diagnosis algorithm has become an urgent need in the field of pathological diagnosis. Summary of the invention

[0005] The embodiments of the present application provide a fully automatic quantitative diagnosis method and device for HER2 immunohistochemical pathology images and a readable storage medium thereof. The existing HER2 immunohistochemical image analysis technology currently exists in the art, which mainly relies on manual film reading, is highly subjective, labor-intensive, and prone to misjudgment. In addition, traditional image segmentation methods are difficult to accurately segment target tissues, evaluate intensity characteristics, and ensure consistency with the pathologist's diagnosis when processing immunohistochemical images, and cannot meet the clinical demand for accurate diagnosis.

[0006] The core technology of the present invention is to use the Yolov8 model for foreground tissue segmentation, the PIDNet model for intensity segmentation, and adopt multi-scale two-stage training, and introduce a fully automatic quantitative diagnosis algorithm for HER2 immunohistochemistry pathology images verified by the pathologist's diagnosis report.

[0007] In a first aspect, the present application provides a method for fully automatic quantitative diagnosis of HER2 immunohistochemical pathological images, the method comprising the following steps: S00, based on the foreground tissue segmentation model, the foreground tissue segmentation model is trained using a HER2 immunohistochemical pathology image labeled with at least two different tissue types, so that the foreground tissue segmentation model can identify and segment the foreground tissue area in the image; S10, using the foreground tissue segmentation model to predict a foreground tissue region, and processing and labeling the foreground tissue region to obtain HER2 immunohistochemical pathology image data labeled with at least five different intensity levels; S20, based on the intensity segmentation model, using HER2 immunohistochemical pathology image data annotated with at least five categories of different intensity levels to train the intensity segmentation model, so that the intensity segmentation model can subdivide the foreground tissue into sub-regions of different intensities according to the image pixel intensity values; S30, introducing the diagnosis result of the pathologist to verify the prediction result of the intensity segmentation model, and adjusting the parameters and structure of the intensity segmentation model according to the verification result; S40, performing foreground tissue region segmentation on the input HER2 immunohistochemistry pathology image using the trained foreground tissue segmentation model, and then predicting the segmented foreground tissue region using the verified intensity segmentation model and outputting the prediction result.

[0008] Furthermore, in step S00, the foreground tissue segmentation model is based on the Yolov8n-seg model.

[0009] Furthermore, in step S00, the Yolov8n-seg model is used as the basic model, and the 5-fold cross-validation method is used to divide the training set and the test set in proportion. The parameters of the Yolov8n-seg model are adjusted during the training process, and the Yolov8n-seg model is optimized based on key evaluation indicators and visualization effects to obtain the foreground tissue segmentation model.

[0010] Furthermore, in step S20, the intensity segmentation model is based on the PIDNet_M_ImageNet model.

[0011] Furthermore, in step S20, the PIDNet_M_ImageNet model unifies the dataset and mask set into different sizes for iterative training, sets the learning rate and the number of categories, and obtains the intensity segmentation model after multiple iterations.

[0012] Further, in step S00, the two different tissue types include foreground tissue and control tissue.

[0013] Furthermore, in step S10, the foreground tissue region is predicted with masks of various sizes and the results are fused, holes in the mask are processed, and the selected region is cut into small blocks of a specific size as data for the intensity segmentation task; The data of the intensity segmentation task are annotated to form the corresponding mask dataset, and the annotations are divided into five different intensity levels according to the preset standards.

[0014] In a second aspect, the present application provides a fully automatic quantitative diagnosis device for HER2 immunohistochemical pathological images, comprising: A foreground tissue segmentation module is used to train the foreground tissue segmentation model based on a HER2 immunohistochemical pathology image labeled with at least two different tissue types, so that the foreground tissue segmentation model can identify and segment the foreground tissue area in the image; and is used to perform foreground tissue area segmentation on the input HER2 immunohistochemical pathology image data; A processing module, using a foreground tissue segmentation model to predict a foreground tissue region, and after processing and labeling the foreground tissue region, obtains HER2 immunohistochemical pathological image data labeled with at least five different intensity levels; The intensity segmentation module is based on the intensity segmentation model, and uses HER2 immunohistochemical pathology image data annotated with at least five different intensity levels to train the intensity segmentation model, so that the intensity segmentation model can subdivide the foreground tissue into sub-regions of different intensities according to the image pixel intensity value; and is used to predict the foreground tissue region segmented by the foreground tissue segmentation module; The debugging module introduces the diagnosis results of pathologists to verify the prediction results of the intensity segmentation model, and adjusts the parameters and structure of the intensity segmentation model according to the verification results; An input module, used to input the HER2 immunohistochemical pathological image input into the foreground tissue segmentation module, and then input the result output by the foreground tissue segmentation module into the intensity segmentation module; Output module, used to output prediction results.

[0015] In a third aspect, the present application provides an electronic device including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the above-mentioned fully automatic quantitative diagnosis method for HER2 immunohistochemical pathology images.

[0016] In a fourth aspect, the present application provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute a process. The process includes the fully automatic quantitative diagnosis method of HER2 immunohistochemistry pathology images according to the above-mentioned method.

[0017] The main contributions and innovations of the present invention are as follows: 1. The algorithm of the present invention can not only achieve accurate segmentation of tissue regions for HER2 immunohistochemical pathological images, but also accurately classify and quantitatively analyze the different intensity parts in the segmented tissue regions. Compared with traditional algorithms, it can accurately locate the foreground tissue region and significantly improve the segmentation accuracy of tissues of various intensities, with an overall accuracy rate of more than 84%. It can provide clearer and more accurate tissue morphology information for pathological diagnosis and provide powerful quantitative indicators for tumor grading and malignancy assessment.

[0018] 2. The pathologist's diagnosis report is innovatively introduced as the final verification result. The model parameters and structure are adjusted based on the pathologist's diagnosis, so that the model prediction results are highly consistent with the pathological diagnosis. This effectively solves the problem of computer algorithms being out of touch with clinical practice in medical image analysis and the low consistency with pathologists' diagnosis, greatly improving the credibility of fully automatic quantitative diagnosis.

[0019] 3. It realizes the fully automatic quantitative diagnosis of HER2 immunohistochemical pathological images. Compared with manual reading, the diagnosis speed can be increased by more than 2 times, which greatly shortens the diagnosis time. It provides strong technical support for large-scale clinical screening and rapid diagnosis, helps doctors to formulate more personalized treatment plans in time, and promotes the development of precision medicine.

[0020] 4. It is highly robust to various noises, uneven staining, and tissue morphology variations in immunohistochemical images. Whether it is low-quality clinical images or special images in experimental research, the algorithm can run stably and give relatively accurate analysis results, with a wider range of applications.

[0021] 5. The algorithm framework has good scalability, which facilitates the subsequent integration of other image processing technologies or bioinformatics analysis methods. It can deeply explore the disease mechanism from multiple levels of gene-protein-tissue morphology to meet the ever-evolving needs of pathological diagnosis and research.

[0022] 6. It has broad application prospects in the diagnosis, treatment selection and prognosis evaluation of HER2-related diseases such as breast cancer. It can assist pathologists to more accurately determine the HER2 expression status and provide a reliable basis for patients to formulate personalized treatment plans, which is expected to improve the treatment effect and quality of life of patients, and lay a solid technical foundation for the widespread application of immunohistochemistry technology in the field of pathological diagnosis.

[0023] 7. In the drug development stage, it can be used as an effective tool to evaluate the effect of drugs on HER2 expression, helping to screen and optimize anticancer drugs. By comparing the changes in tissue immunohistochemical images before and after medication, the effect of drugs on the expression of specific proteins can be analyzed to determine the effectiveness and safety of drugs, providing strong support for drug development.

[0024] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 is a flow chart of a method for fully automatic quantitative diagnosis of HER2 immunohistochemical pathological images according to an embodiment of the present application; Figure 2 is a schematic diagram of a thumbnail image (above) and a labeled mask image (below) according to an embodiment of the present application; Figure 3 is a foreground tissue segmentation result diagram according to an embodiment of the present application; Figure 4 is a foreground tissue mask block diagram according to an embodiment of the present application; Figure 5 is an example of a tissue strength annotation set according to an embodiment of the present application; Figure 6 is the intensity segmentation result according to the embodiment of the present application; Figure 7 It is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0027] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0028] Embodiment 1 This application aims to propose a fully automatic quantitative diagnosis method for HER2 immunohistochemical pathological images. Specifically, refer to Figure 1 , the method comprising: S1. Training steps of foreground tissue segmentation model: screen and process HER2 immunohistochemical pathological images, use open source software to annotate foreground tissue and control tissue area information, scale the annotated image dataset to a specific size, select the Yolov8n-seg model as the benchmark model, use the 5-fold cross-validation method, divide the training set and test set proportionally, adjust the model parameters during the training process, optimize the model according to key evaluation indicators and visualization effects, and obtain the foreground tissue segmentation model; In this embodiment, firstly, immunohistochemical pathology images (WSI images, whole-slice images) with format errors and blurred images are screened and deleted. Secondly, the digital pathology images in sdpc format are converted into thumbnail images in jpg format and annotated using the open source software Qupath. For the training data of the foreground tissue segmentation algorithm, the foreground tissue and control tissue area information are annotated (red represents control tissue, blue represents foreground tissue), as shown in FIG. Figure 2 shown.

[0029] Preferably, the dataset and the labeled mask dataset are uniformly scaled to a size of 2048x2048. The Yolov8n-seg model is selected as the training benchmark, and the 5-fold cross-validation method is used for training. Each time, the training set and the test set are divided into a ratio of 80%:20%, and the training cycle is 100 epochs (referring to the number of times the entire training dataset is traversed during the training process). During the training process, the confidence, intersection-over-union and other parameters of the model are continuously adjusted, and key evaluation indicators such as Precision, Recall and mAP (mean average precision) are observed. If Recall is low, the confidence, intersection-over-union and other thresholds are reduced to reduce false negatives; if Precision is low, the confidence, intersection-over-union and other thresholds are increased to reduce false positives. Combined with the visualization effect, its performance on the training set and the validation set is optimized.

[0030] Therefore, the foreground tissue segmentation model is based on the Yolov8 model, and the Yolov8n-seg model is a high-performance image segmentation model based on the Yolov8 architecture. The pre-trained model is trained by annotating HER2 immunohistochemical pathology images with two different tissue types, so that it can accurately identify and segment the foreground tissue area in the image and separate the control tissue with strong similarity. During the training process, the image is preprocessed, including image size unification, data enhancement and other operations to improve the generalization ability and recognition accuracy of the model. The trained model can quickly and accurately outline the foreground tissue according to the feature information of the image, providing a basis for subsequent analysis and processing.

[0031] S2, intensity segmentation model training steps: Use the foreground tissue segmentation model to predict the foreground tissue area, process the area and divide it into small blocks of specific sizes, use open source software to annotate the intensity categories to form a mask dataset, select the PIDNet_M_ImageNet model as the benchmark model, unify the dataset and mask set into different sizes for iterative training, set the learning rate and number of categories, and obtain the intensity segmentation model after specific epochs; In this embodiment, the foreground tissue segmentation model in the previous step is used to predict the foreground tissue area, such as Figure 3 As shown. Two sizes of 1024 and 2048 are used to predict the mask and the two predicted mask results are merged. Then, the independent masks are merged by judging whether the number of bounding boxes detected by the model is greater than 1. If it is greater than 1, the minimum enclosing rectangle of all bounding boxes is calculated, and each bounding box is looped through to merge the corresponding masks through the logical OR operation. Subsequently, the morphological closing operation is used to fill the holes in the mask. Finally, the selected area is divided into small blocks of 2048*2048 for the next intensity segmentation task, as shown in Figure 4 shown.

[0032] Preferably, in the intensity segmentation stage, Qupath software is further used to annotate the segmented data set (obtained by segmenting the selected area into small blocks of 2048*2048) to form a corresponding mask data set, such as Figure 5 As shown. The HER2 detection guide for breast cancer (2019 edition) can be used to divide the labeling categories into 5 categories: strong positive (Strong), medium positive (Medium), weak medium positive (Weak_medium), weak positive (Weak), and negative (Negative). Different colors can be used for labeling and can be set according to actual needs.

[0033] Preferably, in the intensity segmentation training stage, the PIDNet_M_ImageNet model benchmark is used, and the dataset and the corresponding mask set are unified into 512x512 and 2048x2048 sizes for iterative training. The training uses the default hyperparameter configuration, and the learning rate is set to 0.001, and the category is set to 6 (annotation class plus background class). After 300 epochs, 5 segmentation models with different intensity organizations suitable for the dataset are obtained. The test set image is input into the trained intensity segmentation model to obtain the inference result of the intensity segmentation model, such as Figure 6 shown.

[0034] Therefore, the intensity segmentation model is based on the PIDNet model, and the PIDNet_M_ImageNet model refers to the PIDNet-M model pre-trained on the ImageNet dataset, which has unique advantages in processing image intensity information. The PIDNet model was trained in a multi-scale two-stage iterative manner using HER2 immunohistochemical pathology image data labeled with five different intensity levels, so that the model can learn the relationship between the intensity characteristics of different regions in the image and the pathological significance. After optimized training, the intensity segmentation algorithm can further subdivide the foreground tissue into sub-regions of different intensities according to the intensity values ​​of the image pixels, thereby achieving quantitative analysis of the HER2 expression intensity.

[0035] S3. Verification step: Use the intensity segmentation model to process the HER2 immunohistochemistry pathology image to obtain the prediction result, introduce the diagnosis result of the pathologist to verify the prediction result of the intensity segmentation model, and adjust the parameters and structure of the intensity segmentation model according to the verification result.

[0036] In this embodiment, the HER2 score is inferred based on the HER2 immunohistochemistry evaluation formula for breast cancer. 0 / 1+ represents negative, 2+ requires a FISH experiment to determine whether it is positive or negative, and 3+ is positive. 3+ patients directly use targeted therapy, 2+ patients with FISH amplification use targeted therapy, and those without amplification do not use targeted therapy. At the same time, experienced pathologists are invited to independently diagnose the test set images, and the pathologist's diagnosis results are used as the final verification standard. The prediction results of the integrated model are compared with the pathologist's diagnosis results, and the accuracy evaluation index of the algorithm is calculated. The intensity segmentation model is further optimized and adjusted to ensure the accuracy and reliability of the intensity segmentation model, and to realize a fully automatic quantitative algorithm for embedded foreground tissue segmentation and intensity segmentation.

[0037] Among them, the evaluation formula of HER2 immunohistochemistry for breast cancer is as follows:

[0038] In this embodiment, QuPath is a powerful open source biological image analysis software designed for digital pathology and biological image quantitative analysis, and its principles and functions are not described in detail here.

[0039] S4. Prediction step: After the intensity segmentation model is verified, the data to be tested can be input into the intensity segmentation model, and the prediction results can be output. Based on the prediction results, further analysis can be performed to obtain a more specific report.

[0040] Preferably, the foreground tissue segmentation model and the intensity segmentation model can be integrated into an overall model, and then steps S3 and S4 are performed. Subsequently, the HER2 immunohistochemical pathological image is directly input into the overall model to output the result.

[0041] Embodiment 2 Based on the same concept, the present application also proposes a fully automatic quantitative diagnosis device for HER2 immunohistochemical pathological images, comprising: A foreground tissue segmentation module is used to train the foreground tissue segmentation model based on a HER2 immunohistochemical pathology image labeled with at least two different tissue types, so that the foreground tissue segmentation model can identify and segment the foreground tissue area in the image; and is used to perform foreground tissue area segmentation on the input HER2 immunohistochemical pathology image data; A processing module, using a foreground tissue segmentation model to predict a foreground tissue region, and after processing and labeling the foreground tissue region, obtains HER2 immunohistochemical pathological image data labeled with at least five different intensity levels; The intensity segmentation module is based on the intensity segmentation model, and uses HER2 immunohistochemical pathology image data annotated with at least five different intensity levels to train the intensity segmentation model, so that the intensity segmentation model can subdivide the foreground tissue into sub-regions of different intensities according to the image pixel intensity value; and is used to predict the foreground tissue region segmented by the foreground tissue segmentation module; The debugging module introduces the diagnosis results of pathologists to verify the prediction results of the intensity segmentation model, and adjusts the parameters and structure of the intensity segmentation model according to the verification results; An input module, used to input the HER2 immunohistochemical pathological image input into the foreground tissue segmentation module, and then input the result output by the foreground tissue segmentation module into the intensity segmentation module; Output module, used to output prediction results.

[0042] Embodiment 3 This embodiment also provides an electronic device, referring to Figure 7 , comprises a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above method embodiments.

[0043] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0044] Among them, the memory 404 may include a large capacity memory 404 for data or instructions. For example, but not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In appropriate cases, the memory 404 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 404 may be inside or outside the data processing device. In a specific embodiment, the memory 404 is a non-volatile memory. In a specific embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable Programmable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0045] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0046] The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any of the HER2 immunohistochemical pathological image fully automatic quantitative diagnosis methods in the above embodiments.

[0047] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0048] The transmission device 406 can be used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0049] Input / output devices 408 are used to input or output information.

[0050] Embodiment 4 This embodiment also provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute the process. The process includes the fully automatic quantitative diagnosis method of HER2 immunohistochemical pathology images according to Example 1.

[0051] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0052] In general, various embodiments may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0053] Embodiments of the present invention may be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros may be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof. In addition, at this point, it should be noted that, for example, Figure 1 Any block of the logic flow in the program may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or storage blocks implemented within the processor, magnetic media such as hard disk or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. Physical media are non-transitory media.

[0054] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0055] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A fully automatic quantitative diagnosis method for HER2 immunohistochemical pathological images, characterized in that: The following steps are involved: S00, based on the foreground tissue segmentation model, the foreground tissue segmentation model is trained using a HER2 immunohistochemical pathology image labeled with at least two different tissue types, so that the foreground tissue segmentation model can identify and segment the foreground tissue area in the image; S10, using the foreground tissue segmentation model to predict a foreground tissue region, and processing and labeling the foreground tissue region to obtain HER2 immunohistochemical pathology image data labeled with at least five different intensity levels; S20, based on the intensity segmentation model, using HER2 immunohistochemical pathology image data annotated with at least five categories of different intensity levels to train the intensity segmentation model, so that the intensity segmentation model can subdivide the foreground tissue into sub-regions of different intensities according to the image pixel intensity values; S30, introducing the diagnosis result of the pathologist to verify the prediction result of the intensity segmentation model, and adjusting the parameters and structure of the intensity segmentation model according to the verification result; S40, performing foreground tissue region segmentation on the input HER2 immunohistochemistry pathology image using the trained foreground tissue segmentation model, and then predicting the segmented foreground tissue region using the verified intensity segmentation model and outputting the prediction result.

2. The method for fully automatic quantitative diagnosis of HER2 immunohistochemical pathological images according to claim 1, characterized in that: In step S00, the foreground tissue segmentation model is based on the Yolov8n-seg model.

3. The fully automatic quantitative diagnosis method of HER2 immunohistochemical pathological images according to claim 2, characterized in that: In step S00, the Yolov8n-seg model is used as the basic model. The 5-fold cross-validation method is used to divide the training set and the test set in proportion. The parameters of the Yolov8n-seg model are adjusted during the training process. The Yolov8n-seg model is optimized based on key evaluation indicators and visualization effects to obtain the foreground tissue segmentation model.

4. The method for fully automatic quantitative diagnosis of HER2 immunohistochemical pathological images according to claim 1, characterized in that: In step S20, the intensity segmentation model is based on the PIDNet_M_ImageNet model.

5. The method for fully automatic quantitative diagnosis of HER2 immunohistochemical pathological images according to claim 4, characterized in that: In step S20, the PIDNet_M_ImageNet model unifies the dataset and mask set into different sizes for iterative training, sets the learning rate and number of categories, and obtains the intensity segmentation model after multiple iterations.

6. The method for fully automatic quantitative diagnosis of HER2 immunohistochemical pathological images according to claim 1, characterized in that: In step S00, the two different tissue types include foreground tissue and control tissue.

7. The method for fully automatic quantitative diagnosis of HER2 immunohistochemical pathological images according to any one of claims 1 to 6, characterized in that: In step S10, the foreground tissue area uses multiple sizes to predict masks and fuse the results, process the holes in the mask, and cut the selected area into small blocks of a specific size as data for the intensity segmentation task; The data of the intensity segmentation task are annotated to form the corresponding mask dataset, and the annotations are divided into five different intensity levels according to the preset standards.

8. A fully automatic quantitative diagnostic device for HER2 immunohistochemical pathological images, characterized in that: include: A foreground tissue segmentation module, based on a foreground tissue segmentation model, uses a HER2 immunohistochemical pathology image labeled with at least two different tissue types to train the foreground tissue segmentation model, so that the foreground tissue segmentation model can identify and segment the foreground tissue area in the image; Used to perform foreground tissue area segmentation on the input HER2 immunohistochemistry pathology image data; A processing module, using a foreground tissue segmentation model to predict a foreground tissue region, and after processing and labeling the foreground tissue region, obtains HER2 immunohistochemical pathological image data labeled with at least five different intensity levels; The intensity segmentation module is based on the intensity segmentation model, and uses HER2 immunohistochemical pathology image data annotated with at least five different intensity levels to train the intensity segmentation model, so that the intensity segmentation model can subdivide the foreground tissue into sub-regions of different intensities according to the image pixel intensity value; and is used to predict the foreground tissue region segmented by the foreground tissue segmentation module; The debugging module introduces the diagnosis results of pathologists to verify the prediction results of the intensity segmentation model, and adjusts the parameters and structure of the intensity segmentation model according to the verification results; An input module, used to input the HER2 immunohistochemical pathological image input into the foreground tissue segmentation module, and then input the result output by the foreground tissue segmentation module into the intensity segmentation module; Output module, used to output prediction results.

9. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute the fully automatic quantitative diagnosis method for HER2 immunohistochemical pathological images according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes a program code for controlling a process to execute the process, and the process includes the fully automatic quantitative diagnosis method for HER2 immunohistochemical pathology images according to any one of claims 1 to 7.

Images