Microscope image analysis method and device based on immunohistochemistry and medium

Abstract

The invention discloses a microscope image analysis method and device based on immunohistochemistry and a medium, belongs to the technical field of medical image analysis, and aims to solve the problems that manual interpretation burden of microscope images under existing immunohistochemistry detection is heavy, the problems of man-made subjectivity and consistency are prominent, the stability is easily influenced by illumination and dyeing differences, and the detection efficiency is low. And the intelligent picture analysis cost is high. The method comprises the following steps: carrying out standardization processing on an original collection image collected by a microscope in real-time visual field to obtain a standardized collection image; pixel segmentation and recording under a cell level are carried out on the standardized collected image; carrying out intensity grade discrimination processing of related nuclear staining markers on the cell-level pixel information to generate a cell-level structured quantification result; performing membrane-related phenotype prediction calculation under a related membrane staining marker on the cell-level pixel information to obtain a cell-level prediction result; and carrying out integration processing on the cell-level structured quantification result and the cell-level prediction result to obtain a cell-level analysis result.

Description

Technical Field

[0001] This application relates to the field of medical image analysis technology, and in particular to a microscopic image analysis method, device and medium based on immunohistochemistry. Background Technology

[0002] Immunohistochemistry (IHC) testing has become a crucial tool for assessing molecular subtyping and determining drug indications in precision oncology. In the current process, pathologists typically observe slides using a traditional optical microscope, estimating the proportion of positive cells and staining intensity from multiple fields of view based on experience. This approach presents the following problems: Manual interpretation is a heavy burden: indicators such as Ki-67 require cell-level statistics, and HER2 requires a comprehensive assessment of membrane integrity and strength, making manual interpretation per field of view time-consuming.

[0003] Subjectivity and consistency issues are prominent: there are significant differences between different doctors and between different doctors at different times, which affects diagnostic standardization and quality control.

[0004] Existing AI solutions have high integration costs: some products require replacement or specification of microscope equipment, which is not conducive to implementation in existing departments; or they require manual import of image files, which disrupts the original microscope slide reading workflow and affects doctors' acceptance.

[0005] Differences in illumination and staining affect stability: Different hospitals and different batches of slides have differences in staining depth, light source brightness, and camera parameters, resulting in insufficient stability and generalizability of AI results.

[0006] Therefore, there is an urgent need for a systematic solution that can automatically acquire, analyze, and visualize immunohistochemical images at the field of view level without changing doctors' traditional microscopic image reading habits, using a standard optical microscope, an external acquisition and transmission head, and computer software, while taking into account compatibility, real-time performance, and clinical usability. Summary of the Invention

[0007] This application provides a microscope image analysis method, device, and medium based on immunohistochemistry to solve the following technical problems: existing microscope images under immunohistochemistry detection have a heavy manual interpretation burden, prominent issues of human subjectivity and consistency, and the stability can be easily affected by differences in lighting and staining, while intelligent image analysis is costly.

[0008] The embodiments of this application adopt the following technical solutions: On one hand, embodiments of this application provide a microscope image analysis method based on immunohistochemistry, comprising: standardizing the original acquired image obtained from real-time microscopic field of view to obtain a standardized acquired image; performing cell-level pixel segmentation and recording on the standardized acquired image to output cell-level pixel information; performing intensity level discrimination processing on the cell-level pixel information related to nuclear staining markers to generate cell-level structured quantification results; performing membrane-related phenotypic prediction calculation on the cell-level pixel information related to membrane staining markers to obtain cell-level prediction results; and integrating the cell-level structured quantification results with the cell-level prediction results to obtain the cell-level analysis results of the standardized acquired image.

[0009] This application's embodiments, through cell-level pixel segmentation, can provide more detailed intracellular structural information, facilitating the identification and quantification of intracellular components. Furthermore, by judging the intensity levels of nuclear staining markers, nuclear-related indicators, such as cell cycle status and nuclear size, can be rapidly identified and quantified. Analysis of nuclear staining markers aids in disease diagnosis, prognostic assessment, and efficacy monitoring. Membrane-related phenotypic prediction helps researchers understand cellular biological functions and intercellular interactions. Simultaneously, integrating cell-level structural quantification results with prediction results provides a more comprehensive cellular analysis, offering a scientific basis for clinical treatment.

[0010] In one feasible implementation, the raw acquired images from real-time microscopic field of view acquisition are standardized to obtain standardized acquired images. Specifically, this includes: uniformly acquiring and processing raw acquired images uploaded from different hospitals using a data interface; detecting the field of view brightness distribution in the raw acquired images to obtain field of view brightness distribution information; performing light source compensation processing on uneven brightness areas and color cast areas in the field of view brightness distribution information to obtain a brightness-compensated acquired image with balanced field of view brightness; and performing color separation processing on the brightness-compensated acquired image using a color decomposition method, and then normalizing the separated color classification features through a DAB channel to obtain the standardized acquired image used for cell-level image analysis.

[0011] In one feasible implementation, the standardized acquired image is segmented and recorded at the cell level to output cell-level pixel information. Specifically, this includes: constructing a U-Net convolutional neural network architecture for cell-level segmentation and phenotypic recognition based on a cell-level feature sample library; performing cell-level feature segmentation processing on the standardized acquired image using the U-Net convolutional neural network architecture to obtain individual cell-level features; performing phenotypic recognition processing on the individual cell-level features based on cell contour, center position, morphological representation, and structural features to obtain individual cell phenotypic features; and integrating the individual cell-level features with the individual cell phenotypic features to obtain the cell-level pixel information.

[0012] In one feasible implementation, before performing hierarchical discrimination processing on the cell-level pixel information related to nuclear staining markers and generating cell-level structured quantification results, the method further includes: classifying and identifying the brown DAB staining channel and the blue hematoxylin staining channel in the cell-level pixel information; performing pixel-level semantic segmentation on the hematoxylin staining channel to generate a precise mask for each cell nucleus in the current standardized acquisition image; determining a unique ID for each cell nucleus based on the identified cell nucleus; and performing corresponding image patch cropping processing on the DAB staining channel in the current standardized acquisition image based on the precise mask and the unique ID of each cell nucleus to obtain a nuclear region image patch. Calculate the DAB intensity statistics of pixels within each kernel region image block; wherein the DAB intensity statistics include any one or more of the following: average intensity, median intensity, intensity standard deviation, maximum intensity, and intensity percentile; and determine the kernel intensity features based on the DAB intensity statistics; calculate the kernel texture features of the DAB intensity maps of relevant pixels within each kernel region image block, and calculate the morphological and spatial distribution features of the kernel texture features and kernel spatial moments to obtain kernel morphological and spatial distribution features; wherein the kernel texture features include at least the contrast and homogeneity features of the gray-level co-occurrence matrix (GLCM); calculate the high-dimensional depth feature vector for each kernel region image block using a lightweight feature extraction neural network trained on kernel positive and kernel negative samples, and output the kernel depth features; based on the kernel intensity features, the kernel morphological and spatial distribution features, and the kernel depth features, obtain the multidimensional features within the kernel under the kernel staining marker.

[0013] In one feasible implementation, the cell-level pixel information is processed for grading nuclear staining markers, and cell-level structured quantification results are generated. Specifically, this includes: calculating the positive confidence score of the multidimensional features within the nucleus under expert scoring to obtain a positive confidence score for the cell nucleus in the currently standardized acquired image; using a threshold judgment engine, performing a joint judgment under threshold comparison on the positive confidence score to obtain a nuclear grading result for the nuclear staining markers; wherein the nuclear grading result includes: strong positive, moderate positive, weak positive, and negative; based on the nuclear grading result, and through a sliding window, identifying pixel regions of positive cell density in the currently standardized acquired image to determine and derive positive hotspot regions; based on the nuclear grading result, calculating and deriving the positive cell rate in the currently standardized acquired image, and statistically generating a nuclear grading distribution map for each nuclear grading; and performing structured report generation processing on the nuclear grading result, the positive hotspot regions, the positive cell rate, and the nuclear grading distribution map to obtain the cell-level structured quantification result.

[0014] In one feasible implementation, before performing membrane-related phenotypic prediction calculations on the cell-level pixel information using membrane staining markers to obtain cell-level prediction results, the method further includes: constructing a membrane signal analysis neural network based on cell membrane feature samples and a target detection network; wherein the attributes of the cell membrane feature samples include: presence, continuity, intensity level, and membrane structure localization; through the membrane signal analysis neural network, statistical processing of all cells in the cell-level pixel information under cell membrane features is performed to obtain aggregated cell membrane feature information; wherein the aggregated cell membrane feature information includes: the proportion of cells with complete membrane signals, the average and distribution information of the membrane continuity scores of all positive cells, the average and distribution information of the membrane intensity levels of all positive cells, and the proportion of strongly stained cells; based on the aggregated cell membrane feature information, membrane-related phenotypic information under membrane staining markers is determined.

[0015] In one feasible implementation, the cell-level pixel information is used to perform membrane-related phenotypic prediction calculations under relevant membrane staining markers to obtain cell-level prediction results. Specifically, this includes: using a clinical rule knowledge base to perform prediction and judgment processing on the membrane-related phenotypic information: if the proportion of cells with intact membrane signals is less than 10%, and the positive cell membrane intensity level is extremely low and the positive cell membrane continuity score is low, then the cell-level prediction result is determined as a negative prediction result; if the proportion of cells with intact membrane signals is greater than or equal to 10%, and the positive cell membrane intensity level is low and the proportion of strongly stained cells is high... If the information is absent, the cell-level prediction result is determined as a weak positive prediction result; if the proportion of cells with intact membrane signals is greater than or equal to 10%, and the positive cell membrane intensity level is medium and the proportion of strongly stained cells is within the first preset threshold range, the cell-level prediction result is determined as a moderate positive prediction result; if the proportion of cells with intact membrane signals is greater than or equal to 10%, and the proportion of strongly stained cells is within the second preset threshold range, and the positive cell membrane intensity level is high and the positive cell membrane continuity score is high, the cell-level prediction result is determined as a strong positive prediction result.

[0016] In one feasible implementation, the cell-level structured quantization result and the cell-level prediction result are integrated to obtain the cell-level analysis result of the standardized acquired image. Specifically, this includes: highlighting the precise mask of the cell nucleus corresponding to the cell-level structured quantization result to obtain first highlight information; highlighting the cell membrane prediction mask corresponding to the cell-level prediction result under different colors to obtain second highlight information; generating an intuitive annotation map of the current standardized acquired image and the corresponding cell-level classification result based on the cell-level structured quantization result and the cell-level prediction result, and based on the first highlight information and the second highlight information; and sending the intuitive annotation map and the corresponding cell-level classification result to the front-end interface.

[0017] Secondly, embodiments of this application also provide an immunohistochemistry-based microscope image analysis device, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to execute an immunohistochemistry-based microscope image analysis method as described in any of the above embodiments.

[0018] Thirdly, embodiments of this application also provide a non-volatile computer storage medium, which is a non-volatile computer-readable storage medium storing at least one program, each program including instructions, which, when executed by a terminal, cause the terminal to perform a microscope image analysis method based on immunohistochemistry as described in any of the above embodiments.

[0019] This application provides a microscopic image analysis method, device, and medium based on immunohistochemistry. Compared with the prior art, the embodiments of this application have the following beneficial technical effects: 1. Maintain traditional microscope reading habits: Doctors still observe with the original microscope, and only use "one-click image acquisition + AI prompts" when needed, with minimal changes to the workflow.

[0020] 2. Decoupling of hardware and software, flexible deployment: It can be adapted to microscopes of multiple brands and standard acquisition heads, and can be introduced into departments in the form of "system solutions" to reduce the cost of transformation.

[0021] 3. Real-time analysis capability: Enables single-field analysis within seconds in a standard workstation environment, meeting the needs of daily clinical use.

[0022] 4. Standardization of interpretation results: Through cell-level analysis and rule engine, the quantification and configurable standardization of indicators such as Ki-67, ER / PR, and HER2 are achieved, improving consistency.

[0023] 5. Enhanced robustness and scalability: Color decomposition, illumination correction, and multi-center training strategies significantly reduce the impact of staining differences and equipment differences between different hospitals. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 A flowchart of a microscope image analysis method based on immunohistochemistry is provided for embodiments of this application; Figure 2 A schematic diagram of the overall structure of a microscope image analysis system provided in this application embodiment; Figure 3 This is a schematic diagram of real-time analysis of pathological images provided in an embodiment of this application; Figure 4 This is a schematic diagram of a pathological image analysis result provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a microscope image analysis device based on immunohistochemistry, provided in an embodiment of this application. Detailed Implementation

[0025] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0026] It should be noted that this application mainly addresses the following issues: 1) Achieve AI-assisted interpretation integration without changing the existing microscope working mode: Supports mainstream optical microscopes, connects to digital acquisition heads and computers through standard interfaces, without replacing the entire machine.

[0027] 2) Under “real-time microscope observation conditions”, ensure image quality and robustness of analysis: solve the problems of positive identification and unstable HER2 grading caused by light fluctuations, white balance shifts, and staining differences.

[0028] 3) Enable rapid cell-level segmentation and quantitative analysis: Perform single-field analysis in ≤1 second on a desktop workstation to meet daily auxiliary diagnostic needs.

[0029] 4) Targeted analysis strategies are adopted for different biomarkers and parameters can be configured: including the positive rate statistics and grading of ER / PR / Ki-67, and the comprehensive score of membrane integrity and strength of HER2.

[0030] 5) Provide a unified technical path for subsequent multi-center promotion: ensure traceability of the data collection link, interpretability of the analysis logic, and configurability of standards.

[0031] Therefore, Figure 2 This application provides a schematic diagram of the overall structure of a microscope image analysis system, as shown in the embodiments. Figure 2 As shown, the immunohistochemistry AI-assisted diagnostic system of this application is based on an integrated technical approach of "real-time microscopic field of view acquisition + deep learning cell-level recognition + indicator calculation and visualization". The system operates according to the workflow of "field of view acquisition → standardized preprocessing → cell-level segmentation and phenotypic recognition → indicator statistics and grading → result display and traceability".

[0032] This application provides a microscopic image analysis method based on immunohistochemistry, such as... Figure 1As shown, the immunohistochemistry-based microscopic image analysis method specifically includes steps S101-S105: S101. Standardize the raw images acquired in real-time field of view of the microscope to obtain standardized images.

[0033] Specifically, the first step is to uniformly acquire and process the raw images uploaded by different hospitals using a data interface. Then, the brightness distribution in the field of view is detected in the raw images to obtain the brightness distribution information.

[0034] Furthermore, the uneven brightness areas and color cast areas in the field of view brightness distribution information are then processed by light source compensation to obtain a brightness-compensated image with balanced field of view brightness.

[0035] Furthermore, the brightness-compensated acquired image is subjected to color separation processing by color decomposition method, and the separated color classification features are normalized by DAB channel to obtain a standardized acquired image for cell-level image analysis.

[0036] In one embodiment, after the real-time image acquired by the acquisition and transmission head enters the AI ​​analysis software, in order to ensure the stability of subsequent analysis and cross-device consistency, it is necessary to first perform illumination and background correction, that is, to detect the brightness distribution within the field of view, compensate for obvious unevenness and color cast, and suppress fluctuations caused by light source adjustment. Then, the brightness distribution information of the field of view is subjected to color separation and normalization. Color decomposition methods, including HED, can be used to separate the DAB / brown signal from the hematoxylin background, and the DAB channel is normalized. The separated color classification features are then normalized to obtain a standardized acquisition image for cell-level image analysis.

[0037] S102. Perform cell-level pixel segmentation and recording on the standardized acquired images, and output cell-level pixel information.

[0038] Specifically, it is necessary to first construct a U-Net convolutional neural network architecture for cell-level segmentation and phenotypic recognition based on a cell-level feature sample library.

[0039] Furthermore, the standardized acquired images are segmented using the U-Net convolutional neural network architecture to obtain individual cell-level features.

[0040] Furthermore, individual cell-level features are further processed to identify phenotypic characteristics based on cell outline, center position, morphological features, and structural features, thereby obtaining individual cell phenotypic features.

[0041] Furthermore, the data of individual cell-level features and individual cell phenotypic features are integrated to obtain cell-level pixel information.

[0042] As a feasible implementation method, the standardized acquired images from the preprocessing results also need to be input into the subsequent deep learning model in the form of standardized IHC images. Then, based on the standardized images, this system adopts a unified deep learning framework to achieve cell-level instance segmentation and multi-marker phenotypic recognition. Specifically, an instance segmentation network based on the U-Net / StarDist concept is first used to accurately segment each cell, obtaining individual cell-level features. Then, phenotypic recognition processing related to cell contour, center position, morphological representation, and structural features is performed on these individual cell-level features to obtain individual cell phenotypic features. Based on the data integration of individual cell-level features and individual cell phenotypic features, the contour, center position, and other information of each cell are output as the basic unit for subsequent interpretation, thus obtaining cell-level pixel information.

[0043] S103. Perform intensity level discrimination processing on the cell-level pixel information for relevant nuclear staining markers, and generate cell-level structured quantification results.

[0044] Specifically, the brown DAB staining channel and the blue hematoxylin staining channel in the cell-level pixel information need to be classified and labeled first. Then, the hematoxylin staining channel is subjected to pixel-level semantic segmentation to generate a precise mask for each cell nucleus in the currently standardized acquired image. Based on the identified cell nucleus, a unique ID for each cell nucleus is determined.

[0045] Furthermore, based on the precise mask and unique ID of each cell nucleus, the corresponding image blocks of the DAB staining channels in the currently standardized acquired image are cropped to obtain nuclear region image blocks.

[0046] In one embodiment, the analysis system first automatically identifies and separates the brown DAB staining channel (the colored region) and the blue hematoxylin staining channel (the nucleus region). Then, using a pre-trained deep convolutional neural network (such as a variant based on U-Net++), pixel-level semantic segmentation is performed on the hematoxylin channel to generate a precise mask for each nucleus. Next, instance segmentation is performed to distinguish overlapping or adhered nuclei, assigning a unique ID to each nucleus. Finally, based on the mask for each nucleus, the corresponding nucleus region image patch is cropped from the DAB channel of the original image.

[0047] Furthermore, the DAB intensity statistics of pixels within each kernel region image block are calculated. The DAB intensity statistics include any one or more of the following: average intensity, median intensity, intensity standard deviation, maximum intensity, and intensity percentile; and based on the DAB intensity statistics, the kernel intensity characteristics are determined.

[0048] Furthermore, kernel texture features are calculated for the DAB intensity maps of relevant pixels within each kernel region image block, and the kernel texture features are combined with the kernel spatial moments to calculate morphological and spatial distribution features, thus obtaining kernel morphological and spatial distribution features. The kernel texture features include at least the contrast and homogeneity features of the gray-level co-occurrence matrix (GLCM).

[0049] Furthermore, a lightweight feature extraction neural network trained with nuclear positive and nuclear negative samples is used to calculate high-dimensional deep feature vectors for each nuclear region image block, outputting nuclear depth features.

[0050] Furthermore, based on nuclear intensity characteristics, nuclear morphology and spatial distribution characteristics, and nuclear depth characteristics, multidimensional intranuclear features under nuclear staining markers were finally obtained.

[0051] As a feasible implementation method, the multi-dimensional features within the kernel include: (1) Intensity features: Calculate the DAB intensity statistics of pixels in each kernel region, including but not limited to: average intensity, median intensity, intensity standard deviation, maximum intensity, and intensity percentile (e.g., 75th). (2) Morphological and spatial distribution features: Treat the DAB intensity map as a two-dimensional distribution and calculate its texture features (e.g., contrast and homogeneity of the gray-level co-occurrence matrix GLCM) and the spatial moments of intensity within the kernel. (3) Depth features: Apply a lightweight feature extraction neural network (e.g., the MobileNet backbone) to each kernel region image patch and output a high-dimensional depth feature vector. This network has been pre-trained on a large number of labeled kernel positive / negative samples and can capture subtle patterns that are difficult for the human eye to recognize.

[0052] Furthermore, the positive confidence scores of the multidimensional features within the nucleus must be calculated based on expert ratings to obtain the positive confidence scores of the cell nuclei in the currently standardized acquired images.

[0053] Furthermore, using a threshold judgment engine, the positive confidence scores are jointly judged under threshold comparison, and finally the nuclear grade discrimination results of the relevant nuclear staining markers are obtained. Among them, the nuclear grade discrimination results include: strong positive, moderate positive, weak positive, and negative.

[0054] In one embodiment, the system incorporates a threshold configuration module that allows users to set three thresholds based on laboratory standards or specific protocols: weak positive, moderate positive, and strong positive (e.g., weak = 0.3, moderate = 0.6, strong = 0.9). Cell grades are then dynamically determined based on a comparison of the confidence score with the thresholds: for example, positive confidence score < weak positive: judged as negative; weak positive ≤ positive confidence score < moderate positive: judged as weak positive; moderate positive ≤ positive confidence score < strong positive: judged as moderate positive; positive confidence score ≥ strong positive: judged as strong positive. This decouples the model's generalization ability from the flexibility of clinical rules, allowing users to adjust sensitivity and specificity without retraining the model.

[0055] Furthermore, based on the nuclear grade discrimination results, and through a sliding window, the pixel regions of positive cell density in the current standardized acquired image are identified to determine and derive positive hotspot regions. Moreover, based on the nuclear grade discrimination results, the positive cell rate of the current standardized acquired image is calculated, and the nuclear grade distribution map under each nuclear grade is statistically analyzed.

[0056] Furthermore, by performing structured report generation on the nuclear grade discrimination results, positive hotspot regions, positive cell rate, and nuclear grade distribution map, cell-level structured quantitative results can be obtained.

[0057] In one embodiment, based on the nuclear grade discrimination results, the positive cell rate (number of cells at any positive grade / total number of analyzed cells) is first automatically calculated. Then, the percentages of strong, medium, and weak positive cells are calculated separately. Next, at the WSI level, a sliding window (e.g., 200μm * 200μm) is used to traverse the nuclear grade distribution map, and the positive cell density (number of positive cells / total window area) of each window is calculated. The top N (e.g., 3) windows with the highest density are automatically identified as "hotspot areas," and the average positive rate, highest positive rate, and location coordinates of the positive hotspot areas are output separately. Finally, all statistical indicators, hotspot area location maps, and cell annotation maps with color codes (e.g., red / orange / yellow / blue representing strong / medium / weak / negative) are integrated into a structured report (e.g., JSON / XML format) and visualized to obtain cell-level structured quantitative results.

[0058] S104. Perform membrane-related phenotypic prediction calculations on the cell-level pixel information under relevant membrane staining markers to obtain cell-level prediction results.

[0059] Specifically, a membrane signal parsing neural network needs to be constructed based on cell membrane feature samples and a target detection network. The attributes of the cell membrane feature samples include: presence, continuity, intensity level, and membrane structure localization.

[0060] In one embodiment, a specially designed membrane signal parsing neural network is required. This network takes each cell and its surrounding context region as input, performs multi-task learning, and outputs: Task 1 (Presence): Does the cell have an intact cell membrane? (Binary classification); Task 2 (Integrity / Continuity): Continuity score (0-1) of the membrane staining loop, quantifying the degree of breakage of the staining loop; Task 3 (Intensity): Average intensity level of the membrane staining (e.g., discretized to 1-10 levels); Task 4 (Location): A pixel-level soft mask of the membrane region, highlighting the stained membrane structure.

[0061] Furthermore, a membrane signal analysis neural network is used to perform statistical processing on all cells in the cell-level pixel information based on cell membrane features, resulting in aggregated cell membrane feature information. This aggregated cell membrane feature information includes: the proportion of cells with complete membrane signals, the average and distribution of membrane continuity scores for all positive cells, the average and distribution of membrane intensity levels for all positive cells, and the proportion of strongly stained cells. Based on this aggregated cell membrane feature information, membrane-related phenotypic information under relevant membrane staining markers is then determined.

[0062] Furthermore, it is necessary to use a clinical rule knowledge base to predict and process membrane-related phenotypic information: If the proportion of cells with intact membrane signals is less than 10%, and the positive cell membrane intensity level is very low and the positive cell membrane continuity score is low, then the cell-level prediction result is determined as a negative prediction result.

[0063] If the proportion of cells with intact membrane signals is greater than or equal to 10%, and the positive cell membrane intensity level is low and the information on the proportion of strongly stained cells is not available, then the cell-level prediction result is determined as a weak positive prediction result.

[0064] If the proportion of cells with intact membrane signals is greater than or equal to 10%, and the positive cell membrane intensity level is medium and the proportion of strongly stained cells is within the first preset threshold, then the cell-level prediction result is determined as a medium positive prediction result.

[0065] If the proportion of cells with intact membrane signals is greater than or equal to 10%, and the proportion of strongly stained cells is within the second preset threshold range, and the positive cell membrane intensity level is high and the positive cell membrane continuity score is high, then the cell-level prediction result is determined as a strong positive prediction result.

[0066] In one embodiment, the system incorporates a configurable clinical rule knowledge base whose rules are directly encoded from guidelines such as ASCO / CAP. The rule engine receives the aggregated statistical indicators from the previous step as input and then executes grading logic. Specifically, 0 (negative): The proportion of cells with intact membrane signals is <10%, or the intensity is extremely weak and discontinuous. 1+ (weakly positive): The proportion of cells with intact membrane signals is ≥10%, but the average intensity level is low, and the proportion of strongly stained cells is ≈0%. 2+ (moderately positive): The proportion of cells with intact membrane signals is ≥10%, the intensity level is moderate, and the proportion of strongly stained cells is within the range [X%, Y%] (e.g., 0% < proportion ≤ 30%). 3+ (strongly positive): The proportion of cells with intact membrane signals is ≥10%, and the proportion of strongly stained cells is >Y% (e.g., >30%), and the average membrane continuity score is high.

[0067] S105. Integrate the cell-level structured quantification results with the cell-level prediction results to obtain the cell-level analysis results of the standardized acquired images.

[0068] Specifically, the precise mask of the cell nucleus corresponding to the cell-level structured quantization result is first highlighted to obtain the first highlight information. Then, the cell membrane prediction mask corresponding to the cell-level prediction result is highlighted under different colors to obtain the second highlight information.

[0069] Furthermore, by combining the cell-level structured quantization results and cell-level prediction results, and based on the first and second highlight information, an intuitive annotation map of the current standardized acquired image and the corresponding cell-level grading results are generated.

[0070] Furthermore, the intuitively labeled diagrams and corresponding cell-level grading results are sent to the front-end interface, ultimately achieving a visual presentation of the cell-level analysis results.

[0071] As a feasible implementation method, Figure 3 This is a schematic diagram of real-time analysis of pathological images provided in an embodiment of this application. Figure 4 This is a schematic diagram of a pathological image analysis result provided in an embodiment of this application, such as... Figure 3 as well as Figure 4 As shown, the system generates an intuitive annotation overlay map on the front-end interface, clearly marking positive and negative cells identified by the model using different colors and bounding boxes. For positive cells, their membrane contours are highlighted based on the predicted mask. The system directly displays the final grading results (0 / 1+ / 2+ / 3+) and key supporting data. Furthermore, doctors do not need to perceive the details of the underlying model; what they see is what they get. The system can also be designed to allow clicking on any suspicious cell to view its specific feature values ​​output by the model (such as continuity score and intensity level).

[0072] In addition, embodiments of this application also provide a microscope image analysis device based on immunohistochemistry, such as... Figure 5 As shown, the immunohistochemistry-based microscope image analysis device 500 specifically includes: At least one processor 501; and a memory 502 communicatively connected to the at least one processor 501; wherein the memory 502 stores instructions executable by the at least one processor 501 to enable the at least one processor 501 to execute: The raw images acquired in real-time field of view of the microscope are standardized to obtain standardized images; Perform cell-level pixel segmentation and recording on standardized acquired images, and output cell-level pixel information; Cell-level pixel information is processed to determine the intensity level of relevant nuclear staining markers, and cell-level structured quantification results are generated. Cell-level pixel information is used to perform membrane-related phenotypic prediction calculations based on membrane staining markers to obtain cell-level prediction results. The cell-level structured quantification results are integrated with the cell-level prediction results to obtain the cell-level analysis results of the standardized acquired images.

[0073] This application's embodiments, through cell-level pixel segmentation, provide more detailed intracellular structural information, facilitating the identification and quantification of intracellular components. Furthermore, by judging the intensity levels of nuclear staining markers, nuclear-related indicators, such as cell cycle status and nuclear size, can be rapidly identified and quantified. Analysis of nuclear staining markers aids in disease diagnosis, prognostic assessment, and efficacy monitoring. Membrane-related phenotypic prediction helps researchers understand cellular biological functions and intercellular interactions. Simultaneously, integrating cell-level structural quantification results with prediction results provides a more comprehensive cellular analysis, offering a scientific basis for clinical treatment.

[0074] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and medium embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the description of the method embodiments.

[0075] The devices and media provided in this application are one-to-one with the methods. Therefore, the devices and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0076] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0077] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0078] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0079] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0080] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0081] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0082] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0083] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0084] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of this specification.

Claims

1. A microscopic image analysis method based on immunohistochemistry, characterized in that, The method includes: The raw images acquired in real-time field of view of the microscope are standardized to obtain standardized images; The standardized acquired images are segmented and recorded at the cell level, and cell-level pixel information is output. The cell-level pixel information is processed to determine the intensity level of relevant nuclear staining markers, and cell-level structured quantification results are generated. The cell-level pixel information is used to perform membrane-related phenotypic prediction calculations under relevant membrane staining markers to obtain cell-level prediction results. The cell-level structured quantization results are integrated with the cell-level prediction results to obtain the cell-level analysis results of the standardized acquired images.

2. The microscopic image analysis method based on immunohistochemistry according to claim 1, characterized in that, The raw images acquired during real-time microscopic field of view are standardized to obtain standardized images, specifically including: Unified acquisition and processing of raw images uploaded by different hospitals under a unified data interface; The brightness distribution state of the field of view in the original acquired image is detected to obtain the brightness distribution information of the field of view; The uneven brightness areas and color cast areas in the field of view brightness distribution information are processed by light source compensation to obtain a brightness-compensated image under balanced field of view brightness. The brightness-compensated acquired image is processed by color decomposition, and the separated color classification features are normalized through the DAB channel to obtain the standardized acquired image for cell-level image analysis.

3. The microscopic image analysis method based on immunohistochemistry according to claim 1, characterized in that, The standardized acquired images are segmented and recorded at the cell level to output cell-level pixel information, specifically including: Based on a cell-level feature sample library, a U-Net convolutional neural network architecture for cell-level segmentation and phenotypic recognition was constructed. The U-Net convolutional neural network architecture is used to segment the standardized acquired images for cell-level features to obtain individual cell-level features. The individual cell-level features are subjected to phenotypic recognition processing based on cell outline, center position, morphological characterization, and structural features to obtain individual cell phenotypic features; The cell-level features are integrated with the cell-level phenotypic features to obtain the cell-level pixel information.

4. The microscopic image analysis method based on immunohistochemistry according to claim 1, characterized in that, Before performing hierarchical discrimination processing on the cell-level pixel information for relevant nuclear staining markers and generating cell-level structured quantification results, the method further includes: The brown DAB staining channel and the blue hematoxylin staining channel in the cell-level pixel information are classified and labeled. The hematoxylin staining channel is segmented at the pixel level to generate a precise mask for each cell nucleus in the current standardized acquired image; and based on each identified cell nucleus, a unique ID for each cell nucleus is determined. Based on the precise mask and unique ID of each cell nucleus, the corresponding image block cropping process is performed on the DAB staining channel in the current standardized acquisition image to obtain the nuclear region image block; Calculate the DAB intensity statistics of pixels within each kernel region image block; wherein the DAB intensity statistics include any one or more of the following: average intensity, median intensity, intensity standard deviation, maximum intensity, and intensity percentile; and determine the kernel intensity features based on the DAB intensity statistics. Kernel texture features are calculated for the DAB intensity maps of relevant pixels within each kernel region image block, and the kernel texture features and kernel spatial moments are used to calculate the morphological and spatial distribution features to obtain kernel morphological and spatial distribution features; wherein, the kernel texture features include at least the contrast features and homogeneity features of the gray-level co-occurrence matrix (GLCM). A lightweight feature extraction neural network trained with nuclear positive and nuclear negative samples is used to calculate high-dimensional deep feature vectors for each nuclear region image block and output the nuclear depth features. Based on the nuclear intensity characteristics, nuclear morphology and spatial distribution characteristics, and nuclear depth characteristics, multidimensional intranuclear features under nuclear staining markers are obtained.

5. The microscopic image analysis method based on immunohistochemistry according to claim 4, characterized in that, The cell-level pixel information is subjected to hierarchical discrimination processing based on relevant nuclear staining markers, and cell-level structured quantification results are generated, specifically including: The positive confidence scores of the cell nuclei in the current standardized acquired images are obtained by calculating the positive confidence scores of the multidimensional features within the nucleus under the scoring of relevant experts. The threshold judgment engine performs a joint judgment under threshold comparison on the positive confidence scores to obtain the nuclear grade discrimination results of the relevant nuclear staining markers; wherein, the nuclear grade discrimination results include: strong positive, moderate positive, weak positive and negative; Based on the nuclear level discrimination results, and through a sliding window, the pixel regions of positive cell density in the current standardized acquired image are identified, and positive hotspot regions are determined. Based on the nuclear grade discrimination results, the positive cell rate of the current standardized acquired image is calculated and obtained, and the nuclear grade distribution map under each nuclear grade is statistically analyzed. The nuclear grade discrimination results, the positive hotspot regions, the positive cell rate, and the nuclear grade distribution map are processed into a structured report to obtain the cell-level structured quantitative results.

6. The microscopic image analysis method based on immunohistochemistry according to claim 5, characterized in that, Before performing membrane-related phenotypic prediction calculations on the cell-level pixel information using relevant membrane staining markers to obtain cell-level prediction results, the method further includes: A membrane signal parsing neural network is constructed based on cell membrane feature samples and a target detection network; wherein, the attributes of the cell membrane feature samples include: existence, continuity, intensity level, and membrane structure localization; The membrane signal analysis neural network performs statistical processing on all cells in the cell-level pixel information based on cell membrane features to obtain aggregated cell membrane feature information. The aggregated cell membrane feature information includes: the proportion of cells with complete membrane signals, the average and distribution information of the continuity scores of all positive cell membranes, the average and distribution information of the intensity levels of all positive cell membranes, and the proportion of strongly stained cells. Based on the aggregated information of cell membrane features, membrane-related phenotypic information under relevant membrane staining markers is determined.

7. The microscopic image analysis method based on immunohistochemistry according to claim 1, characterized in that, The cell-level pixel information is used to perform membrane-related phenotypic prediction calculations based on membrane staining markers to obtain cell-level prediction results, specifically including: The membrane-related phenotypic information is predicted and judged using a clinical rule knowledge base. If the proportion of cells with intact membrane signals is less than 10%, and the positive cell membrane intensity level is extremely low and the positive cell membrane continuity score is low, then the cell-level prediction result is determined as a negative prediction result. If the proportion of cells with intact membrane signals is greater than or equal to 10%, and the positive cell membrane intensity level is low and the information on the proportion of strongly stained cells is not available, then the cell-level prediction result is determined as a weak positive prediction result. If the proportion of cells with intact membrane signals is greater than or equal to 10%, and the positive cell membrane intensity level is medium and the proportion of strongly stained cells is within the first preset threshold range, then the cell-level prediction result is determined as a medium positive prediction result. If the proportion of cells with intact membrane signals is greater than or equal to 10%, and the proportion of strongly stained cells is within the second preset threshold range, and the positive cell membrane intensity level is high and the positive cell membrane continuity score is high, then the cell-level prediction result is determined as a strong positive prediction result.

8. The microscopic image analysis method based on immunohistochemistry according to claim 1, characterized in that, The cell-level structural quantification results are integrated with the cell-level prediction results to obtain the cell-level analysis results of the standardized acquired images, specifically including: The precise mask of the cell nucleus corresponding to the cell-level structured quantization result is highlighted to obtain the first highlight information; and the cell membrane prediction mask corresponding to the cell-level prediction result is highlighted under different colors to obtain the second highlight information. Based on the cell-level structured quantization results and the cell-level prediction results, and based on the first highlight information and the second highlight information, an intuitive annotation map of the current standardized acquired image and the corresponding cell-level classification results are generated. The intuitive annotation diagram and the corresponding cell-level grading results are sent to the front-end interface.

9. A microscope image analysis device based on immunohistochemistry, characterized in that, The device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor to enable the at least one processor to perform an immunohistochemistry-based microscopic image analysis method according to any one of claims 1-8.

10. A non-volatile computer storage medium, characterized in that, The storage medium is a non-volatile computer-readable storage medium that stores at least one program, each program including instructions that, when executed by a terminal, cause the terminal to perform a microscope image analysis method based on immunohistochemistry according to any one of claims 1-8.

Images