Intelligent evaluation method and terminal for PD-L1 immunohistochemistry in esophageal cancer tissue

Through digital scanning and automated marking and counting of YOLO models, the subjectivity and efficiency of the PD-L1 immunohistochemistry evaluation method in traditional esophageal cancer tissues was solved, and the evaluation results with high accuracy and efficiency were achieved, which was suitable for screening out PD Potential benefit groups of -1 inhibitors.

CN118443413BActive Publication Date: 2024-11-05THE FIRST AFFILIATED HOSPITAL OF GUANGZHOU MEDICAL UNIV (GUANGZHOU RESPIRATORY CENT)
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410512392.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-26
Publication Date
2024-11-05
Estimated Expiration
2044-04-26

AI Technical Summary

Technical Problem

The traditional PD-L1 immunohistochemistry evaluation method for esophageal cancer tissues is subjective, affecting the accuracy and efficiency of the assessment, and it is difficult to effectively screen out the potential beneficiaries of PD-1 inhibitors.

Method used

Digital scanning and pre-built YOLO model were used to automatically mark and count esophageal cancer tissues, and CPS scores were calculated to evaluate the expression level of PD-L1 immunohistochemistry, reduce the subjectivity and error of artificial operations, and improve the accuracy and efficiency of evaluation.

Benefits of technology

The automation and accuracy of PD-L1 immunohistochemistry evaluation of esophageal cancer tissues is achieved, the work efficiency is improved, and a large amount of pathological section data can be quickly processed, time and labor costs are saved, and the evaluation results are more objective.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118443413B_ABST
    Figure CN118443413B_ABST
Patent Text Reader

Abstract

The present application is in the field of cell recognition technology. The present application provides an intelligent evaluation method and terminal for PD-L1 immunohistochemistry of esophageal cancer tissue, wherein the method comprises: performing PD-L1 immunohistochemical staining on an initial pathological section to obtain a target pathological section, digitally scanning the target pathological section to obtain a full-field pathological image, using a YOLO model to mark and count tumor cells in a tumor area in the digital pathological full-field image to obtain a tumor cell number, and marking and counting positive tumor cells in the tumor area to obtain a positive tumor cell number, using a YOLO model to mark and count positive lymphocytes and positive monocyte macrophages in the tumor area to obtain a positive lymphocyte number and a positive monocyte macrophage number, and calculating a CPS score to achieve intelligent evaluation of PD-L1 immunohistochemistry of esophageal cancer tissue and improve the evaluation accuracy and efficiency of PD-L1 immunohistochemistry of esophageal cancer tissue.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of cell recognition technology. Specifically, the present application relates to an intelligent evaluation method and terminal for PD-L1 immunohistochemistry of esophageal cancer tissue. Background Art

[0002] In the field of tumor immunotherapy, immunotherapy represented by programmed death receptor 1 (PD-1) immune checkpoint inhibitors has made great progress in the treatment of esophageal cancer and gradually rewritten the global treatment model of esophageal cancer. However, only some esophageal cancer patients can significantly benefit from immunotherapy, so how to screen the potential beneficiaries of PD-1 inhibitors is an important challenge currently faced.

[0003] The expression level of programmed death receptor ligand (PD-L1) protein in esophageal cancer is closely related to the efficacy of PD-1 inhibitors. The expression level of PD-L1 (Programmed Death-Ligand 1) is an important biomarker for predicting the patient's response to PD-1 inhibitor treatment. The traditional evaluation method of PD-L1 immunohistochemistry in esophageal cancer tissue relies on the experience of pathologists and is often subjective, which affects the accuracy and efficiency of the evaluation of PD-L1 immunohistochemistry in esophageal cancer tissue. Summary of the invention

[0004] The main purpose of this application is to provide an intelligent evaluation method and terminal for PD-L1 immunohistochemistry of esophageal cancer tissue, so as to realize the intelligent evaluation of PD-L1 immunohistochemistry of esophageal cancer tissue and improve the evaluation accuracy and efficiency of PD-L1 immunohistochemistry of esophageal cancer tissue.

[0005] In order to achieve the above-mentioned invention object, the present application provides an intelligent evaluation method for PD-L1 immunohistochemistry of esophageal cancer tissue, comprising:

[0006] Obtaining an initial pathological section of esophageal cancer tissue for PD-L1 immunohistochemistry, and performing PD-L1 immunohistochemical staining on the initial pathological section to obtain a target pathological section;

[0007] Digitally scanning the target pathological slice to obtain a digital pathological full-field image at a preset magnification field of view, and determining a tumor area in the digital pathological full-field image;

[0008] Using a pre-built YOLO model to mark and count tumor cells in the tumor area to obtain the number of tumor cells, and marking and counting positive tumor cells in the tumor area to obtain the number of positive tumor cells;

[0009] Using the pre-built YOLO model to mark and count the positive lymphocytes and positive monocyte macrophages in the tumor area, to obtain the number of positive lymphocytes and the number of positive monocyte macrophages;

[0010] The CPS score was calculated based on the number of tumor cells, the number of positive tumor cells, the number of positive lymphocytes, and the number of positive monocyte macrophages. The CPS score was used to evaluate the expression level of PD-L1 immunohistochemistry in esophageal cancer tissues:

[0011]

[0012] Wherein, a is the number of positive tumor cells, b is the number of positive lymphocytes, c is the number of positive monocytes and macrophages, and S is the number of tumor cells.

[0013] Preferably, the initial pathological section is subjected to PD-L1 immunohistochemical staining to obtain a target pathological section, comprising:

[0014] Attaching the initial pathological section to a glass slide and placing it in a staining machine;

[0015] Dewaxing the initial pathological section using the staining machine, and rehydrating the section with an alcohol solution to obtain a first pathological section;

[0016] The first pathological section is incubated with a first antibody targeting PD-L1 of esophageal cancer tissue, and is incubated with a second antibody targeting PD-L1 of esophageal cancer tissue to obtain a second pathological section;

[0017] Performing a color reaction treatment on the second pathological section using a DAB color developer, and counter-staining the cell nucleus using hematoxylin to obtain a third pathological section;

[0018] The third pathological section is dehydrated, sealed with resin, and covered with a cover glass to obtain a target pathological section.

[0019] Preferably, the target pathological slice is digitally scanned to obtain a digital pathological full-field image at a preset magnification field of view, including:

[0020] The whole-slice imaging system is set to have multiple scanning stages according to a preset magnification, and the magnification corresponding to each scanning stage is N times or more of the preset magnification. Said N is a positive integer;

[0021] Setting scanning parameters of the whole-slice imaging system based on the magnification and default resolution corresponding to each scanning stage, and controlling the whole-slice imaging system to digitally scan the target pathological slice according to the scanning parameters to obtain a plurality of local microscopic images;

[0022] The multiple local microscopic images are spliced ​​together according to a preset image processing algorithm to obtain a digital pathology full-field image.

[0023] Preferably, the multiple local microscopic images are stitched together according to a preset image processing algorithm to obtain a digital pathology full-field image, including:

[0024] Determine the clarity of each local microscopic image;

[0025] Determining whether the clarity of each of the local microscopic images is higher than a preset clarity;

[0026] When it is determined that the clarity of each of the local microscopic images is higher than the preset clarity, the Canny algorithm is used to perform boundary detection on each of the local microscopic images to obtain the boundary features of each of the local microscopic images. Based on the boundary features of each of the local microscopic images, a scale-invariant feature transformation algorithm is used to determine the matching feature points and transformation matrices between the local microscopic images. Based on the matching feature points and transformation matrices between the local microscopic images, an optimal arrangement and combination is calculated. The multiple local microscopic images are spliced ​​according to the optimal arrangement and combination, and after removing repeated image areas, a digital pathology full-field image is obtained.

[0027] Furthermore, after determining whether the clarity of each of the local microscopic images is higher than a preset clarity, the method further includes:

[0028] When it is determined that the clarity of any of the local microscopic images is lower than the preset clarity, the local microscopic image with a clarity lower than the preset clarity is used as a pending local microscopic image;

[0029] The magnification corresponding to the pending local microscopic image is queried, and the default resolution of the full-slice imaging system is adjusted to the target resolution. Under the target resolution and the magnification corresponding to the pending local microscopic image, the full-slice imaging system is controlled to scan the target pathological slice again to obtain a candidate local microscopic image, and replace the pending local microscopic image.

[0030] Preferably, the Canny algorithm is used to perform boundary detection on each of the local microscopic images to obtain boundary features of each of the local microscopic images, including:

[0031] Performing grayscale processing on each of the local microscopic images using a weighted average method to obtain a plurality of grayscale images;

[0032] Converting the pixel value of each pixel in each of the grayscale images into a weighted average of pixel values ​​in a neighborhood by an adaptive convolution algorithm to generate a plurality of first local microscopic images;

[0033] Dividing each of the first local microscopic images into a plurality of sub-image regions, respectively, and calculating the pixel difference between the central pixel and the surrounding neighborhood pixels of each sub-image region in each of the first local microscopic images, to obtain a pixel difference value set of each of the first local microscopic images;

[0034] Performing gradient calculation on the pixel difference values ​​of the pixel difference value set of each of the first local microscopic images to obtain the gradient amplitude of each of the first local microscopic images in the horizontal direction and the vertical direction;

[0035] The overall gradient amplitude is calculated according to the gradient amplitudes of each of the first local microscopic images in the horizontal direction and the vertical direction;

[0036] Pixel points in each of the first local microscopic images whose pixel values ​​are greater than the overall gradient amplitude are marked as boundary pixel points to form boundary features of each of the local microscopic images.

[0037] Preferably, before using the pre-built YOLO model to mark and count the tumor cells in the tumor area to obtain the number of tumor cells, the method further includes:

[0038] Acquire training samples, wherein the training samples include multiple digital pathology full-field image samples, each of which is labeled with cell type and quantity;

[0039] Constructing an initial YOLO model and a marking rule, configuring the marking rule in the initial YOLO model, inputting the training sample into the initial YOLO model, calling the initial YOLO model to perform a preset number of iterative training on the training sample, and generating a training result after the iterative training;

[0040] According to the training results after iterative training and a preset loss function, calculating the loss value of the initial YOLO model after iterative training;

[0041] Determine whether the loss value after iterative training is lower than a preset loss value;

[0042] When it is determined that the loss value after iterative training is lower than a preset loss value, the initial YOLO model is used as a pre-built YOLO model.

[0043] Preferably, the marking rules include:

[0044] When the tumor cells in the tumor area were located at the center of the field of view at a preset magnification of 20 times, the PD-L1-stained lymphocytes and stained monocytes and macrophages that appeared in the same field of view were recorded as positive lymphocytes and positive monocytes and macrophages, respectively;

[0045] When the tumor area contains PD-L1-stained lymphocytes and stained monocytes and macrophages in the field of view at the preset magnification of 20 times, but no tumor cells appear in the same field of view, the stained lymphocytes in the field of view are not marked as positive lymphocytes, and the stained monocytes and macrophages are not marked as positive monocytes and macrophages;

[0046] When the tumor cells in the tumor area have strong staining or complete linear membrane staining, the tumor cells are marked as positive tumor cells.

[0047] Preferably, determining the tumor area in the digital pathology full-field image includes:

[0048] Acquire a standard digital pathology full-field image, wherein the standard digital pathology full-field image is a preprocessed digital pathology full-field image;

[0049] Adjusting the digital pathology full-field image to an image with the same size as the standard digital pathology full-field image to obtain a first digital pathology full-field image;

[0050] Aligning the first digital pathology full-field image with the standard digital pathology full-field image, and calculating pixel differences of aligned pixels between the first digital pathology full-field image and the standard digital pathology full-field image;

[0051] Calculating a weighted average of all pixel differences, and multiplying the weighted average by lgM to obtain a target pixel difference, where M is the number of pixel pairs of aligned pixels between the first digital pathology full-field image and the standard digital pathology full-field image;

[0052] adjusting the pixel value of each pixel in the first digital pathology full-field image using the target pixel difference to obtain a target digital pathology full-field image;

[0053] Tumor region features and non-tumor region features are extracted from the target digital pathology full-field image, and based on the tumor region features and the non-tumor region features, the target digital pathology full-field image is segmented using a threshold segmentation algorithm to obtain a tumor region.

[0054] The present application also provides a terminal, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0055] The intelligent evaluation method and terminal of PD-L1 immunohistochemistry of esophageal cancer tissue provided in the present application can accurately mark and count tumor cells, positive tumor cells, positive lymphocytes and positive mononuclear macrophages in the tumor area by using digital scanning and pre-built YOLO model. This automated marking and counting process can improve accuracy and precision and reduce the subjectivity and errors of manual operation. At the same time, automatic marking and counting using the pre-built YOLO model can also greatly improve work efficiency; digital scanning and automatic marking and counting methods can also quickly process a large amount of pathological section data, saving time and labor costs. In addition, the CPS score is calculated based on the number of tumor cells, the number of positive tumor cells, the number of positive lymphocytes and the number of positive mononuclear macrophages, making the process of PD-L1 immunohistochemistry evaluation more objective and reducing the impact of human factors on the results. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a flow chart of an intelligent evaluation method for PD-L1 immunohistochemistry of esophageal cancer tissue according to an embodiment of the present application;

[0057] Figure 2 A schematic diagram of a digital pathology full-field image according to an embodiment of the present application;

[0058] Figure 3 A schematic diagram of a digital pathology full-field image according to another embodiment of the present application;

[0059] Figure 4 A schematic diagram of a digital pathology full-field image according to another embodiment of the present application;

[0060] Figure 5 This is a schematic block diagram of the structure of an intelligent evaluation device for PD-L1 immunohistochemistry of esophageal cancer tissue according to an embodiment of the present application;

[0061] Figure 6 A schematic block diagram of the structure of a terminal according to an embodiment of the present application.

[0062] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0064] The present application proposes an intelligent evaluation method for PD-L1 immunohistochemistry of esophageal cancer tissue, in which the execution subject is a terminal, and the terminal can be any type of static or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.) or other types of mobile devices, or a static computing device such as a desktop computer or a PC. The computing device can also be a mobile or static server. The components of the computing device include, but are not limited to, a memory and a processor. The processor is connected to the memory via a bus, and the database is used to store data. The computing device also includes an access device, which enables the computing device to communicate via one or more networks. Examples of these networks include a combination of a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a communication network such as the Internet. The access device may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and the like.

[0065] refer to Figure 1 In one embodiment, the present application provides an intelligent evaluation method for PD-L1 immunohistochemistry of esophageal cancer tissue, the method comprising:

[0066] S11, obtaining an initial pathological section of esophageal cancer tissue for PD-L1 immunohistochemistry, and performing PD-L1 immunohistochemical staining on the initial pathological section to obtain a target pathological section;

[0067] S12, digitally scanning the target pathological section to obtain a digital pathological full-field image at a preset magnification field of view, and determining a tumor area in the digital pathological full-field image;

[0068] S13, using a pre-built YOLO model to mark and count the tumor cells in the tumor area to obtain the number of tumor cells, and marking and counting the positive tumor cells in the tumor area to obtain the number of positive tumor cells;

[0069] S14, using the pre-built YOLO model to mark and count the positive lymphocytes and positive monocyte macrophages in the tumor area to obtain the number of positive lymphocytes and the number of positive monocyte macrophages;

[0070] S15. Calculate the CPS score based on the number of tumor cells, the number of positive tumor cells, the number of positive lymphocytes and the number of positive monocyte macrophages. The CPS score is used to evaluate the expression level of PD-L1 immunohistochemistry in esophageal cancer tissue.

[0071] As described in the above step S11, the initial pathological section refers to a biological tissue sample taken from a patient's body, which is processed and made into a thin slice for examination and analysis under a microscope.

[0072] Staining is the process of making the tissue structure and cell characteristics of the initial pathological section more clearly visible. In this embodiment, when the initial pathological section is subjected to PD-L1 immunohistochemical staining, the initial pathological section can be stained by hematoxylin-eosin (H&E) staining, and the initial pathological section after staining is used as the target pathological section. Among them, hematoxylin is an alkaline dye used to stain the cell nucleus and certain intracellular structures in the initial pathological section. In H&E staining, hematoxylin makes the cell nucleus appear blue or purple. Eosin is an acidic dye used to stain the cytoplasm and other extracellular structures in the initial pathological section. In the H&E staining method, eosin makes the cytoplasm appear red, while the connective tissue and muscle appear pink.

[0073] As described in step S12 above, digital scanning is a process of converting physical documents, images or other visual materials into digital formats. In the medical field, digital scanning particularly refers to converting pathological sections, X-rays, CT scans, MRI images, etc. into digital images that can be viewed, processed and stored on a computer through a dedicated scanning device.

[0074] When the target pathology slice is digitally scanned in this embodiment, the stained target pathology slice can be correctly placed on a glass slide, and a dedicated whole slide scanner (Whole Slide Scanner, WSS) can be used to scan the target pathology slice to obtain a digital pathology full-field image under a preset magnification field of view, so as to capture high-resolution images, realize rapid scanning and image processing, and improve work efficiency. Among them, the whole slide scanner usually provides multiple magnification options, and the preset magnification can be customized, such as set to 200 times.

[0075] In addition, the captured digital pathology full-field images can be subjected to denoising, enhancement and other processing to improve the image quality of the digital pathology full-field images, and the processed digital pathology full-field images can be stored in a server or local database in an appropriate file format to achieve long-term storage of digital images without being easily damaged.

[0076] When determining the tumor area in the digital pathology full-field image, the digital pathology full-field image can be preprocessed first, including denoising, contrast enhancement, brightness and color adjustment, etc., so as to more clearly display the tissue structure; then extract features that help distinguish tumor and non-tumor areas from the preprocessed image, such as using a convolutional neural network (CNN) to perform pixel-level classification on the preprocessed image to distinguish tumor areas from non-tumor areas. The features include shape, texture, color, etc. Finally, use image segmentation algorithms, such as region growing and boundary detection, to separate different areas in the digital pathology full-field image to obtain the tumor area in the digital pathology full-field image, so as to accurately determine the tumor area in the digital pathology image by combining advanced image processing technology and artificial intelligence algorithms, and improve efficiency through automation.

[0077] As described in the above steps S13-S14, the YOLO (You Only Look Once) model is an algorithm for target detection, which is used to convert the target detection task into a regression problem. Different from traditional target detection methods, the YOLO model can predict the target position and category in the image through a single forward propagation, which enables YOLO to achieve fast and efficient target detection.

[0078] The YOLO model divides the tumor area in the input digital pathology full-field image into grids, each of which is responsible for predicting the target in the area. At the same time, each grid also predicts multiple bounding boxes and the confidence of these bounding boxes. The confidence indicates the probability of the bounding box containing the target and the degree of match between the predicted box and the actual box. In addition, each bounding box also predicts the conditional probability of belonging to each category.

[0079] A key feature of the YOLO model is that it uses a convolutional neural network (CNN) to extract image features and use these features to predict bounding boxes and category probabilities, which enables it to handle input images of different sizes and to make a trade-off between detection accuracy and speed. In addition, the YOLO model also includes some techniques such as anchor boxes and non-maximum suppression (NMS) to improve detection accuracy and efficiency.

[0080] In this embodiment, the digital pathology full-field image can be input into the YOLO model, and the YOLO model can be used to mark and count the tumor cells in the tumor area to obtain the number of tumor cells, and the YOLO model can be used to mark and count the positive tumor cells in the tumor area to obtain the number of positive tumor cells; at the same time, the pre-built YOLO model can be used to mark and count the positive lymphocytes and positive monocyte macrophages in the tumor area to obtain the number of positive lymphocytes and positive monocyte macrophages, thereby improving the accuracy and efficiency of target cell detection.

[0081] As described in step S15 above, when calculating the CPS score based on the number of tumor cells, the number of positive tumor cells, the number of positive lymphocytes, and the number of positive monocytes and macrophages, the following formula may be included:

[0082]

[0083] Wherein, a is the number of positive tumor cells, b is the number of positive lymphocytes, c is the number of positive monocytes and macrophages, and S is the number of tumor cells.

[0084] In the field of tumor immunotherapy, CPS stands for "Combined Positive Score". The CPS score is a score used to evaluate the expression level of PD-L1 immunohistochemistry in tumor tissues, which takes into account the expression of PD-L1 on tumor cells and tumor-associated immune cells. Specifically, CPS is calculated by calculating the total number of tumor cells and tumor-associated immune cells that are positive for PD-L1 staining per 100 tumor cells.

[0085] The CPS score can more comprehensively evaluate the expression of PD-L1 in the tumor microenvironment, because in addition to tumor cells themselves, tumor-associated immune cells also express PD-L1 and may be associated with the effectiveness of immunotherapy. The use of the CPS score can help doctors determine whether patients may benefit from immunotherapy with PD-1 / PD-L1 inhibitors. For example, in some clinical trials and treatment guidelines, the CPS score may be used to determine whether a patient is eligible for a specific PD-1 / PD-L1 inhibitor treatment. Different treatments may have different CPS threshold requirements to determine whether a patient is suitable for treatment. The CPS score has therefore become an important biomarker to guide the decision-making process for immunotherapy.

[0086] The intelligent evaluation method of PD-L1 immunohistochemistry of esophageal cancer tissue provided in the present application uses digital scanning and a pre-built YOLO model to accurately mark and count tumor cells, positive tumor cells, positive lymphocytes, and positive mononuclear macrophages in the tumor area. This automated marking and counting process can improve accuracy and precision and reduce the subjectivity and errors of manual operation. At the same time, automatic marking and counting using a pre-built YOLO model can also greatly improve work efficiency; digital scanning and automatic marking and counting methods can also quickly process a large amount of pathological section data, saving time and labor costs. In addition, the CPS score is calculated based on the number of tumor cells, the number of positive tumor cells, the number of positive lymphocytes, and the number of positive mononuclear macrophages, making the process of PD-L1 immunohistochemistry evaluation more objective and reducing the impact of human factors on the results.

[0087] In one embodiment, the initial pathological section is subjected to PD-L1 immunohistochemical staining to obtain a target pathological section, comprising:

[0088] Attaching the initial pathological section to a glass slide and placing it in a staining machine;

[0089] Dewaxing the initial pathological section using the staining machine, and rehydrating the section with an alcohol solution to obtain a first pathological section;

[0090] The first pathological section is incubated with a first antibody targeting PD-L1 of esophageal cancer tissue, and is incubated with a second antibody targeting PD-L1 of esophageal cancer tissue to obtain a second pathological section;

[0091] Performing a color reaction treatment on the second pathological section using a DAB color developer, and counter-staining the cell nucleus using hematoxylin to obtain a third pathological section;

[0092] The third pathological section is dehydrated, sealed with resin, and covered with a cover glass to obtain a target pathological section.

[0093] This embodiment provides a technical solution for immunohistochemical staining of pathological sections, which is used to assist in detecting the expression of PD-L1 protein in esophageal cancer tissue sections.

[0094] In this embodiment, the esophageal cancer tissue sample taken from the patient is fixed and embedded, then cut into thin slices and attached to a glass slide, so that the esophageal cancer tissue sample can be evenly spread on the glass slide for subsequent staining and observation.

[0095] The slide with esophageal cancer tissue attached is placed in a staining machine, and xylene is used to remove the paraffin used for embedding, because paraffin will hinder the binding of antibodies to antigens in the tissue. The dewaxed esophageal cancer tissue sections are rehydrated through a series of alcohol solutions with decreasing concentrations, with the goal of gradually changing the tissue from a hydrophobic state to a hydrophilic state, so that water-soluble antibodies and stains can enter the tissue and react with antigens, thus forming the first pathological section.

[0096] The first pathological section is incubated with a specific first antibody (targeting the PD-L1 protein in esophageal cancer tissue) that specifically binds to the PD-L1 protein in esophageal cancer tissue, which can be a polyclonal antibody or a monoclonal antibody. After the primary antibody incubation, the second antibody is incubated with a second antibody targeting the PD-L1 of esophageal cancer tissue, that is, the second antibody (usually labeled with an enzyme, fluorophore or biotin) is used to bind to the first antibody to form a "bridge" structure to amplify the signal.

[0097] In this embodiment, diaminobenzidine (DAB) color developer can be used to perform a color reaction on the PD-L1 protein site bound to the secondary antibody to produce a brown precipitate, so that the location of the PD-L1 protein can be visualized. At the same time, in order to observe the cell nucleus more clearly, hematoxylin is used to stain the cell nucleus, and the cell nucleus usually appears blue. The third pathological section after color development is dehydrated through a series of alcohol solutions with increasing concentrations, and finally the residual alcohol is removed using solvents such as xylene, and the third pathological section is sealed using a sealing agent such as a resin to protect the stained tissue from damage and increase the stability of the section. Finally, a cover glass is covered on the sealed third pathological section to protect the stained tissue and prevent contamination, and it is also convenient for observation under a microscope.

[0098] This embodiment can complete PD-L1 immunohistochemical staining efficiently and in a standardized manner through automated equipment and optimized staining procedures, thereby improving the accuracy of subsequent intelligent evaluation of PD-L1 immunohistochemical staining of esophageal cancer tissues.

[0099] In one embodiment, digitally scanning the target pathological slice to obtain a digital pathological full-field image at a preset magnification field of view may specifically include:

[0100] The whole-slice imaging system is set to have multiple scanning stages according to a preset magnification, and the magnification corresponding to each scanning stage is N times or more of the preset magnification. Said N is a positive integer;

[0101] Setting scanning parameters of the whole-slice imaging system based on the magnification and default resolution corresponding to each scanning stage, and controlling the whole-slice imaging system to digitally scan the target pathological slice according to the scanning parameters to obtain a plurality of local microscopic images;

[0102] The multiple local microscopic images are spliced ​​together according to a preset image processing algorithm to obtain a digital pathology full-field image.

[0103] In this embodiment, a magnification factor, such as 40X, can be preset according to the size of the cell or tissue structure to be observed. The scanning process of the whole slice imaging system is divided into multiple scanning stages according to the preset magnification factor, such as 5X, 10X, 20X, 100X, etc. Each scanning stage uses N times or 1 / N of the preset magnification factor for scanning, where N is a positive integer.

[0104] For each scanning stage, the scanning parameters are set according to the corresponding magnification and the default resolution of the system, which can be 1920x1080. The scanning parameters also include scanning speed, dots per inch (DPI), scanning mode (such as color or monochrome), etc. After the scanning parameters are set, the whole slice imaging system is used to scan the target pathological slice area by area according to the set scanning parameters. Specifically, the scanning device of the whole slice imaging system moves on the slide and captures images block by block. Each image corresponds to a local microscopic image, and multiple local microscopic images are obtained. After each scanning stage is completed, the system stores the captured local microscopic image as a separate image file.

[0105] When stitching multiple local microscopic images in this embodiment, a preset image processing algorithm may be used to process the captured multiple local microscopic images, which may include image denoising, color correction, contrast enhancement, etc. The processed multiple local microscopic images are then stitched into a complete digital pathology full-field image.

[0106] Specifically, feature point detection and extraction are performed on each local microscopic image. These feature points are significant points in the local microscopic image that can be used for subsequent matching, such as corner points, boundaries, etc. For two or more local microscopic images with overlapping areas, brute force matching, distance-based matching, and FLANN (Fast Library for Approximate Nearest Neighbors) matching algorithms can be used to match their feature points, and fusion and splicing can be performed based on the feature points.

[0107] For example, firstly, the geometric correspondence between the local microscopic images is established, and this process usually involves the calculation of the homography matrix. The homography matrix is ​​a 3x3 transformation matrix used to map points in one local microscopic image to another local microscopic image. At the same time, the RANSAC (Random Sample Consensus) algorithm is used to estimate the homography matrix, and the local microscopic images are subjected to perspective transformation or affine transformation according to the homography matrix so that they can be aligned in the same coordinate system, and the matching points in the local microscopic images are screened to remove outliers and inaccurate matches. The overlapping areas of the local microscopic images are fused to eliminate seams and discontinuities, and the transition areas of the local microscopic images are smoothed using a specific algorithm to improve the naturalness and visual effect of the stitching, and the stitching area is optimized to make the transition between the two local microscopic images more natural and reduce visible seams. Finally, all the processed local microscopic images are synthesized into a high-resolution digital pathology full-field image, which fully displays the details of the entire target pathology slice. Among them, the fusion methods include feathering, pyramid, gradient, etc.

[0108] This embodiment realizes multi-scale scanning of the target pathological slice at different magnifications, thereby obtaining detailed information at different levels. At the same time, through image processing and splicing technology, a complete, high-resolution digital image can be efficiently generated.

[0109] In one embodiment, the multiple local microscopic images are stitched together according to a preset image processing algorithm to obtain a digital pathology full-field image, including:

[0110] Determine the clarity of each local microscopic image;

[0111] Determining whether the clarity of each of the local microscopic images is higher than a preset clarity;

[0112] When it is determined that the clarity of each of the local microscopic images is higher than the preset clarity, the Canny algorithm is used to perform boundary detection on each of the local microscopic images to obtain the boundary features of each of the local microscopic images. Based on the boundary features of each of the local microscopic images, a scale-invariant feature transformation algorithm is used to determine the matching feature points and transformation matrices between the local microscopic images. Based on the matching feature points and transformation matrices between the local microscopic images, an optimal arrangement and combination is calculated. The multiple local microscopic images are spliced ​​according to the optimal arrangement and combination, and after removing repeated image areas, a digital pathology full-field image is obtained.

[0113] In this embodiment, the clarity of each local microscopic image can be quantitatively analyzed by an image clarity evaluation algorithm (such as boundary-based clarity evaluation, frequency domain clarity evaluation, etc.). At the same time, the clarity of each local microscopic image is compared with a preset threshold to ensure that only clear local microscopic images are used for subsequent processing.

[0114] When the clarity of all local microscopic images is higher than the preset clarity, the Canny algorithm is used to perform boundary detection on each local microscopic image to extract the boundary features of the image. The Canny algorithm is a multi-level threshold boundary detection method suitable for image boundary positioning.

[0115] Then, the SIFT algorithm is used to extract key points and local feature descriptors from the boundary features to form SIFT features, and SIFT feature matching is used to determine the matching feature points and transformation matrices between local microscopic images. Among them, SIFT features are scale invariant and can remain stable under image transformations (such as scaling, rotation, etc.).

[0116] Based on the matching feature points and transformation matrix between local microscopic images, the best image arrangement and combination is calculated. This process involves an optimization algorithm to ensure the consistency of image stitching and the minimum overlap error. Finally, according to the calculated optimal arrangement and combination, multiple local microscopic images are stitched into a complete image. This process may include image transformation, resampling, and interpolation. Specifically, through SIFT feature matching, the correspondence between key points of different local microscopic images is determined, so as to infer the matching feature points between local microscopic images. At the same time, using the matching SIFT features, the transformation matrix from one image to another is calculated. The transformation matrix can be a homography matrix, which contains geometric transformation information that maps one plane to another plane. Finally, the optimal image arrangement and combination is calculated with reference to the matching feature points and transformation matrix between local microscopic images.

[0117] For example, suppose there are three local microscopic images A1, A2 and A3, which together cover a large pathological slice area. First, the SIFT algorithm is used to extract key points and feature descriptors from images A1, A2 and A3 to form SIFT features. The feature descriptors between A1 and A2, A1 and A3, and A2 and A3 are matched to find the corresponding matching feature points. At the same time, the homography matrices from A1 to A2, A1 to A3, and A2 to A3 are calculated based on the SIFT features. If based on the corresponding matching feature points between A1, A2 and A3, it can be learned that images A2 and A3 have a large overlapping area, and they jointly supplement the slice area not covered by image A1, it is decided to first splice A1 and A2, and then splice the obtained image with A3. Therefore, the homography matrix can be used to transform image A2 to the same coordinate system as A1, and then the overlapping area is fused to remove the repeated area to obtain the spliced ​​image of A1 and A2. Similarly, image A3 is transformed to the same coordinate system as the newly obtained A1+A2 image and fused, and finally a digital pathology full-field image containing all the information of A1, A2 and A3 is obtained to provide a seamless and continuous field of view for further analysis.

[0118] Through this technical solution, multiple local microscopic images can be effectively and accurately spliced ​​into a complete image, providing more effective information for intelligent evaluation.

[0119] It should be noted that during the stitching process, overlapping areas of the images may be calculated multiple times. Therefore, the non-maximum suppression (NMS) technology can be used to remove or merge these repeated areas to obtain a seamless panoramic image.

[0120] This embodiment can optimize the visual quality of the image and improve the reliability of subsequent evaluation through clarity evaluation and feature matching; at the same time, through Canny boundary detection and removal of repeated areas, the high precision and accuracy of image stitching are ensured, and a full-field digital pathology image without obvious seams is obtained, which is convenient for subsequent analysis.

[0121] In one embodiment, after determining whether the clarity of each of the local microscopic images is higher than a preset clarity, the method further includes:

[0122] When it is determined that the clarity of any of the local microscopic images is lower than the preset clarity, the local microscopic image with a clarity lower than the preset clarity is used as a pending local microscopic image;

[0123] The magnification corresponding to the pending local microscopic image is queried, and the default resolution of the full-slice imaging system is adjusted to the target resolution. Under the target resolution and the magnification corresponding to the pending local microscopic image, the full-slice imaging system is controlled to scan the target pathological slice again to obtain a candidate local microscopic image, and replace the pending local microscopic image.

[0124] This embodiment implements a quality control and image optimization process to improve the quality of pathological slice images. Specifically, the clarity of each local microscopic image is evaluated to determine whether they meet the preset clarity standard. If the clarity of any local microscopic image is found to be lower than the preset clarity, the image is marked as a pending local microscopic image. The preset clarity can be customized.

[0125] For the pending local microscopic image, the corresponding magnification is queried. The magnification is recorded by the system when the pending local microscopic image is scanned, and the default resolution of the full-slice imaging system is adjusted to the target resolution, which is generally higher than the default resolution. The target resolution is a higher resolution setting that the system can provide to improve image quality. At the target resolution and the magnification corresponding to the pending local microscopic image, the full-slice imaging system is controlled to scan the target pathological slice again. A new image is obtained by rescanning, which is called a candidate local microscopic image. The candidate local microscopic image is compared with the original pending local microscopic image. If the quality is improved, the original pending local microscopic image is replaced with the candidate local microscopic image.

[0126] This embodiment can improve the clarity of local microscopic images through rescanning and resolution adjustment, ensure the availability and high value of image data, and ensure that the YOLO model can obtain high-quality images for more accurate analysis and intelligent evaluation. At the same time, the automated image evaluation and optimization process reduces the need for manual review and improves work efficiency.

[0127] In one embodiment, the boundary features of each local microscopic image are obtained by performing boundary detection on each local microscopic image using the Canny algorithm, including:

[0128] Performing grayscale processing on each of the local microscopic images using a weighted average method to obtain a plurality of grayscale images;

[0129] Converting the pixel value of each pixel in each of the grayscale images into a weighted average of pixel values ​​in a neighborhood by an adaptive convolution algorithm to generate a plurality of first local microscopic images;

[0130] Dividing each of the first local microscopic images into a plurality of sub-image regions, respectively, and calculating the pixel difference between the central pixel and the surrounding neighborhood pixels of each sub-image region in each of the first local microscopic images, to obtain a pixel difference value set of each of the first local microscopic images;

[0131] Performing gradient calculation on the pixel difference values ​​of the pixel difference value set of each of the first local microscopic images to obtain the gradient amplitude of each of the first local microscopic images in the horizontal direction and the vertical direction;

[0132] The overall gradient amplitude is calculated according to the gradient amplitudes of each of the first local microscopic images in the horizontal direction and the vertical direction;

[0133] Pixel points in each of the first local microscopic images whose pixel values ​​are greater than the overall gradient amplitude are marked as boundary pixel points to form boundary features of each of the local microscopic images.

[0134] In this embodiment, a weighted average method can be used to convert a local microscopic image into a grayscale image. The weighted average method combines the information of the three color channels of red, green, and blue in the local microscopic image and applies different weights to simulate the human eye's perception of color, thereby obtaining a grayscale image that is closer to visual perception. At the same time, an adaptive convolution algorithm is applied to each grayscale image, and the pixel value of each pixel in the grayscale image is replaced by the weighted average value of the pixel values ​​in the neighborhood, and the first local microscopic image corresponding to each grayscale image is generated respectively, and multiple local microscopic images are obtained to enhance the local features of the image, and to achieve image noise reduction and detail enhancement.

[0135] The pixel value in the neighborhood is the pixel value of all pixels in the preset range of each pixel. The preset range can be customized. The adaptive convolution algorithm is a convolutional neural network (CNN) technology that allows the network to dynamically adjust its convolution kernel parameters according to the input data to better adapt to different input image features.

[0136] Then, each first local microscopic image processed by adaptive convolution is evenly divided into multiple sub-image areas, which is helpful for local analysis of different parts of the image. The pixel value difference between the central pixel and the surrounding neighborhood pixels of each sub-image area is calculated to obtain a pixel difference value set, which includes multiple pixel differences of each first local microscopic image, which is helpful for identifying rapidly changing areas in the image, that is, potential boundaries. Similarly, the surrounding neighborhood pixels are all field pixels centered on the central pixel and within a preset range from the central pixel, and the preset range can be customized.

[0137] The pixel difference of the pixel difference number set of each first local microscopic image is gradient calculated to obtain the gradient amplitude of the image in the horizontal and vertical directions. The magnitude of the gradient amplitude reflects the speed of the image brightness change and is an important indicator for boundary detection. Specifically, the gradient is a measure of the change in image brightness, which can be obtained by calculating the difference between a pixel and its adjacent pixels. For the horizontal and vertical directions, the gradient corresponds to the rate of change of the pixel on the x-axis and y-axis, respectively. For example, the Sobel operator is used, which contains two 3x3 convolution kernels, corresponding to the gradients in the horizontal and vertical directions, respectively. By applying these convolution kernels to the image, the approximate gradient in each direction can be calculated, that is, the above convolution kernel is applied to each pixel of the image, and the gradient values ​​in the horizontal and vertical directions of the position are calculated.

[0138] Finally, the overall gradient amplitude is calculated based on the gradient amplitudes of each image in the horizontal and vertical directions. For example, for each first local microscopic image, the overall gradient amplitude can be obtained by calculating the square root of the sum of the squares of the horizontal gradient and the vertical gradient. This overall gradient amplitude can be used as a threshold for determining whether a pixel is a boundary pixel. Finally, the pixel points in each first local microscopic image whose pixel values ​​are greater than the overall gradient amplitude are marked as boundary pixel points, and the boundary features of the first local microscopic image are formed through these boundary pixel points.

[0139] This embodiment can enhance the local features of the image through grayscale and adaptive convolution processing, making the image boundary and texture information more prominent; at the same time, through the division and analysis of sub-image areas, a more detailed examination of specific parts of the image can be carried out, which provides the possibility for the identification of local pathological features. In addition, by calculating the gradient amplitude and setting the threshold, the image boundary is accurately detected, and the boundary features are effectively extracted from the local microscopic image.

[0140] In one embodiment, the tumor cells in the tumor area are marked and counted using a pre-built YOLO model, and before obtaining the number of tumor cells, the method further includes:

[0141] Acquire training samples, wherein the training samples include multiple digital pathology full-field image samples, each of which is labeled with cell type and quantity;

[0142] Constructing an initial YOLO model and a marking rule, configuring the marking rule in the initial YOLO model, inputting the training sample into the initial YOLO model, calling the initial YOLO model to perform a preset number of iterative training on the training sample, and generating a training result after the iterative training;

[0143] According to the training results after iterative training and a preset loss function, calculating the loss value of the initial YOLO model after iterative training;

[0144] Determine whether the loss value after iterative training is lower than a preset loss value;

[0145] When it is determined that the loss value after iterative training is lower than a preset loss value, the initial YOLO model is used as a pre-built YOLO model.

[0146] In this embodiment, multiple groups of digital pathology full-field image samples can be collected. The number of digital pathology full-field image samples must be greater than a threshold, such as 10 million. These digital pathology full-field image samples have been annotated by professionals with cell types and quantities. The cell types include tumor cells, lymphocytes, positive tumor cells, and positive lymphocytes, etc. These annotations provide the model with reference results required for training. Among them, the training results include: accuracy: the proportion of samples correctly predicted by the model to the total number of samples; precision: the proportion of samples predicted to be positive that are actually positive; recall rate: the proportion of samples that are actually positive that are correctly predicted by the model to be positive; weights and biases: parameters in the model, which are continuously updated during training to minimize losses, etc.

[0147] Initialize a YOLO model, which is a target detection algorithm that predicts the location and category of targets in an image through a single forward propagation. At the same time, define labeling rules that specify how to identify and label different cell types from images, such as lymphocytes, mononuclear macrophages, and cell membrane and cytoplasmic staining (any intensity) in esophageal squamous cell carcinoma nests and adjacent stroma: only lymphocytes and mononuclear macrophages directly associated with the tumor are labeled. Then, configure the labeling rules to the initial YOLO model, input the training samples into the initial YOLO model, and perform a preset number of iterative training. At each iteration, the model adjusts its parameters to reduce the difference between the prediction and the actual annotation.

[0148] In addition, after the initial YOLO model completes a preset number of iterative training, the present embodiment can use a preset loss function (such as mean square error, cross entropy, etc.) to calculate the loss value of the initial YOLO model after iterative training. This loss function measures the difference between the model prediction and the actual annotation. Specifically, the mean square error loss function can be selected, and the trained YOLO model can be used to predict the training samples. According to the definition of the mean square error loss function, the difference between the true value and the predicted value of each sample is calculated, and the loss value of each sample is averaged to obtain the average loss value of the entire training sample.

[0149] The calculated loss value is compared with the preset loss value, which is one of the conditions for training termination and determines the accuracy level that the model needs to achieve. If the model's loss value is lower than the preset loss value, it means that the model has achieved sufficient accuracy. At this point, the YOLO model is saved and marked as a pre-built YOLO model, which can be used for subsequent cell detection tasks.

[0150] This embodiment can quickly train a YOLO model suitable for a specific task through an automated iterative training process; the trained YOLO model can accurately detect the type and number of cells in the image. At the same time, the use of the YOLO model can quickly process a large amount of image data and improve the efficiency of pathological image analysis. The model can also continuously improve its prediction accuracy through continuous training and optimization. In addition, by adjusting the labeling rules and loss functions, the YOLO model can also adapt to different pathological image analysis tasks.

[0151] In one embodiment, the marking rules include the following:

[0152] D1. Reference Figure 2 As shown, when the tumor cells in the tumor area are positioned at the center or approximate center of the field of view at a preset magnification of 20 times, the PD-L1-stained lymphocytes and stained monocytes and macrophages appearing in the same field of view are recorded as positive lymphocytes and positive monocytes and macrophages, respectively.

[0153] D2. Reference Figure 3 As shown, when the tumor area contains PD-L1-stained lymphocytes and stained monocytes and macrophages in the field of view with a preset magnification of 20 times, but no tumor cells appear in the same field of view, the stained lymphocytes in the field of view are not marked as positive lymphocytes, and the stained monocytes and macrophages are not marked as positive monocytes and macrophages.

[0154] D3. Reference Figure 4 As shown, when the tumor cells in the tumor area have staining intensity or complete linear membrane staining, the tumor cells are marked as positive tumor cells. That is, when the tumor cells have visible and reliable partial or complete linear membrane staining of any intensity, the tumor cells should be marked as positive tumor cells. The staining intensity can be divided into A, B and C, A: 1+ staining intensity; B: 2+ staining intensity; C: 3+ staining intensity.

[0155] Among them, complete linear cell membrane staining refers to uniform and continuous cell membrane staining in the digital pathology full-field image. In cell biology, cell membrane staining refers to the use of specific dyes or markers to stain the cell membrane structure for observation and analysis under a microscope.

[0156] In one embodiment, after determining whether the loss value after iterative training is lower than a preset loss value, the following steps may also be performed:

[0157] When it is determined that the loss value is not less than the preset loss value, using the chain rule, back propagation is started from the output layer of the initial YOLO model, and the gradient of the loss value relative to each parameter in the initial YOLO model is calculated layer by layer;

[0158] Based on the gradient of each parameter, each parameter of the initial YOLO model is updated using the Adam algorithm, and the initial YOLO model after the updated parameters is retrained using the training sample until the loss value is lower than the preset loss value, thereby obtaining a pre-built YOLO model.

[0159] This embodiment monitors the change of loss value during training to ensure that the YOLO model is not overfitting or underfitting. If the loss value does not reach the preset loss value, it may be necessary to adjust the model structure, train more iterations, or change hyperparameters.

[0160] Specifically, after iterative training, if the loss value of the YOLO model is not lower than the preset loss value, it means that the performance of the model can be improved. At this time, the chain rule is used to backpropagate from the output layer of the model to calculate the gradient of the loss value relative to each parameter in the model. This is a core step in deep learning training and is used to determine how to adjust the model parameters to reduce prediction errors. Gradients are then calculated layer by layer and parameter by parameter, which indicate the contribution of each parameter to the loss function.

[0161] In this embodiment, the chain rule is a derivation rule used to find the derivative of a composite function. In deep learning and machine learning, the chain rule is the core of the back propagation algorithm, which is used to calculate the gradient of the loss function relative to each parameter in the neural network. The chain rule realizes the decomposition of the derivative of a complex function into the product of a series of derivatives of simple functions.

[0162] This embodiment starts with the loss function and transfers the gradient backwards, and uses the chain rule to calculate the gradient of each parameter. These gradients are then used to update the parameters of the initial YOLO model to reduce the loss value of the loss function, thereby training a YOLO model with high recognition accuracy.

[0163] Among them, the Adam (Adaptive Moment Estimation) algorithm is a gradient descent optimization algorithm that combines the advantages of the two optimization algorithms, Momentum and RMSProp (Root Mean Square Propagation). The Adam algorithm can calculate the adaptive learning rate of each parameter, which is based on the historical gradient information of the parameter and is dynamically adjusted during the training process. Therefore, the Adam algorithm and the calculated gradient of each parameter can be used to update each parameter in the YOLO model to reduce the loss value.

[0164] Finally, the YOLO model is trained again using the training samples with the updated parameters. This process is repeated until the loss value is lower than the preset loss value. When the loss value is lower than the preset loss value, the training is stopped. The YOLO model obtained at this time is the pre-built YOLO model, which can be used for cell type detection tasks.

[0165] In this embodiment, the performance of the model can be optimized and the loss value can be reduced through back propagation and gradient descent. Through the Adam algorithm, the adaptive learning rate of the Adam algorithm helps to converge faster and improve the training efficiency of the model. At the same time, the automated parameter update and training process reduces manual intervention, making the training process more efficient. In addition, the trained YOLO model can more accurately detect targets in images, thereby improving the accuracy and reliability of target detection.

[0166] In one embodiment, determining the tumor area in the digital pathology full-field image includes:

[0167] Acquire a standard digital pathology full-field image, wherein the standard digital pathology full-field image is a preprocessed digital pathology full-field image;

[0168] Adjusting the digital pathology full-field image to an image with the same size as the standard digital pathology full-field image to obtain a first digital pathology full-field image;

[0169] Aligning the first digital pathology full-field image with the standard digital pathology full-field image, and calculating pixel differences of aligned pixels between the first digital pathology full-field image and the standard digital pathology full-field image;

[0170] Calculating a weighted average of all pixel differences, and multiplying the weighted average by lgM to obtain a target pixel difference, where M is the number of pixel pairs of aligned pixels between the first digital pathology full-field image and the standard digital pathology full-field image;

[0171] adjusting the pixel value of each pixel in the first digital pathology full-field image using the target pixel difference to obtain a target digital pathology full-field image;

[0172] Tumor region features and non-tumor region features are extracted from the target digital pathology full-field image, and based on the tumor region features and the non-tumor region features, the target digital pathology full-field image is segmented using a threshold segmentation algorithm to obtain a tumor region.

[0173] This embodiment implements a digital pathology image processing flow, which aims to standardize digital pathology images and segment tumor areas. Specifically, a standard digital pathology full-field image is pre-processed, which is a reference image with high image quality. This image is used as a standard for subsequent size matching and alignment processing. First, the size of the digital pathology full-field image to be processed is adjusted to the same size as the standard digital pathology full-field image by enlarging, reducing or cropping, so as to obtain a first digital pathology full-field image.

[0174] Then, the first digital pathology full-field image is aligned with the standard image to ensure that the pixels of the two images are spatially corresponding, and the pixel difference between the corresponding pixels of the aligned first digital pathology full-field image and the standard image is calculated to obtain multiple pixel differences. After weighted averaging of all pixel differences, a representative error metric, i.e., the weighted average value, is obtained, and the weighted average value is multiplied by lgM to obtain the target pixel difference value. Among them, the lgM is obtained by calculating the number of pixel pairs of aligned pixels between the first digital pathology full-field image and the standard digital pathology full-field image, i.e., the pixel logarithm, and obtaining the logarithm of the number of pixel pairs, so as to further fine-tune the pixel value of the first digital pathology full-field image.

[0175] The target pixel difference is used to adjust the pixel value of each pixel in the first digital pathology full-field image to obtain the target digital pathology full-field image, so that the pixel difference of the aligned pixels between the first digital pathology full-field image and the standard digital pathology full-field image is within the error range. Finally, the tumor area features and non-tumor area features are extracted from the target digital pathology full-field image, and based on the extracted features, the target digital pathology full-field image is segmented using a threshold segmentation algorithm to distinguish between tumor areas and non-tumor areas.

[0176] Among them, the threshold segmentation algorithm is an image segmentation technology in the field of image processing and computer vision. Its core idea is to select one or more thresholds and assign the pixels of the image to different categories, usually foreground and background, according to these thresholds to achieve the segmentation of the image area.

[0177] This embodiment can ensure the consistency between different images through size adjustment and alignment processing, which is convenient for subsequent analysis. The alignment processing also improves the matching accuracy of corresponding pixels between images, providing an accurate basis for pixel value adjustment. At the same time, by calculating the weighted average of pixel differences, the error of image processing is quantified, and the pixel value is adjusted to help improve the image quality, making the target image closer to the standard image. In addition, this embodiment uses feature extraction and threshold segmentation algorithms to automatically segment the tumor area from the image, thereby improving segmentation efficiency and accuracy.

[0178] Reference Figure 5 In an embodiment of the present application, a smart evaluation device for PD-L1 immunohistochemistry of esophageal cancer tissue is also provided, the device comprising:

[0179] An acquisition module 11 is used to acquire an initial pathological section of esophageal cancer tissue PD-L1 immunohistochemistry, and perform PD-L1 immunohistochemistry staining on the initial pathological section to obtain a target pathological section;

[0180] A scanning module 12 is used to digitally scan the target pathological slice to obtain a digital pathological full-field image at a preset magnification field of view, and determine the tumor area in the digital pathological full-field image;

[0181] A first marking and counting module 13 is used to mark and count the tumor cells in the tumor area using a pre-built YOLO model to obtain the number of tumor cells, and to mark and count the positive tumor cells in the tumor area to obtain the number of positive tumor cells;

[0182] A second marking and counting module 14 is used to mark and count the positive lymphocytes and positive monocyte macrophages in the tumor area by using the pre-built YOLO model to obtain the number of positive lymphocytes and the number of positive monocyte macrophages;

[0183] The calculation module 15 is used to calculate the CPS score according to the number of tumor cells, the number of positive tumor cells, the number of positive lymphocytes and the number of positive monocyte macrophages, and the CPS score is used to evaluate the expression level of PD-L1 immunohistochemistry in esophageal cancer tissue.

[0184] As described above, it can be understood that the various components of the intelligent evaluation device for esophageal cancer tissue PD-L1 immunohistochemistry proposed in the present application can realize the functions of any of the intelligent evaluation methods for esophageal cancer tissue PD-L1 immunohistochemistry described above, and the specific structure will not be repeated.

[0185] Reference Figure 6 In the embodiment of the present application, a terminal is also provided, whose internal structure can be as follows Figure 6As shown. The terminal includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the computer-designed processor is used to provide computing and control capabilities. The memory of the terminal includes a storage medium and an internal memory. The storage medium stores an operating system, a computer program and a database. The memory provides an environment for the operation of the operating system and the computer program in the storage medium. The database of the terminal is used to store relevant data of the intelligent evaluation method for PD-L1 immunohistochemistry of esophageal cancer tissue. The network interface of the terminal is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, an intelligent evaluation method for PD-L1 immunohistochemistry of esophageal cancer tissue is implemented.

[0186] In one embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, an intelligent evaluation method for PD-L1 immunohistochemistry of esophageal cancer tissue is implemented.

[0187] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in the present application and used in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0188] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, device, article or method including the element.

[0189] The above description is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An intelligent evaluation method for PD-L1 immunohistochemistry of esophageal cancer tissue, characterized in that: include: Obtaining an initial pathological section of esophageal cancer tissue for PD-L1 immunohistochemistry, and performing PD-L1 immunohistochemical staining on the initial pathological section to obtain a target pathological section; Digitally scanning the target pathological slice to obtain a digital pathological full-field image at a preset magnification field of view, and determining a tumor area in the digital pathological full-field image; Using a pre-built YOLO model to mark and count tumor cells in the tumor area to obtain the number of tumor cells, and marking and counting positive tumor cells in the tumor area to obtain the number of positive tumor cells; Using the pre-built YOLO model to mark and count the positive lymphocytes and positive monocyte macrophages in the tumor area, to obtain the number of positive lymphocytes and the number of positive monocyte macrophages; The CPS score was calculated based on the number of tumor cells, the number of positive tumor cells, the number of positive lymphocytes, and the number of positive monocyte macrophages. The CPS score was used to evaluate the expression level of PD-L1 immunohistochemistry in esophageal cancer tissues: ; Wherein, a is the number of positive tumor cells, b is the number of positive lymphocytes, c is the number of positive monocytes and macrophages, and S is the number of tumor cells; The target pathological slice is digitally scanned to obtain a digital pathological full-field image at a preset magnification field of view, including: Setting multiple scanning stages of the whole-slice imaging system according to a preset magnification, wherein the magnification corresponding to each scanning stage is N times or 1 / N of the preset magnification, where N is a positive integer; Setting scanning parameters of the whole-slice imaging system based on the magnification and default resolution corresponding to each scanning stage, and controlling the whole-slice imaging system to digitally scan the target pathological slice according to the scanning parameters to obtain a plurality of local microscopic images; splicing the multiple local microscopic images according to a preset image processing algorithm to obtain a digital pathology full-field image; The multiple local microscopic images are stitched together according to a preset image processing algorithm to obtain a digital pathology full-field image, including: Determine the clarity of each local microscopic image; Determining whether the clarity of each of the local microscopic images is higher than a preset clarity; When it is determined that the clarity of each of the local microscopic images is higher than the preset clarity, the Canny algorithm is used to perform boundary detection on each of the local microscopic images to obtain the boundary features of each of the local microscopic images, based on the boundary features of each of the local microscopic images, a scale-invariant feature transformation algorithm is used to determine the matching feature points and transformation matrices between the local microscopic images, an optimal arrangement and combination is calculated based on the matching feature points and transformation matrices between the local microscopic images, the multiple local microscopic images are spliced ​​according to the optimal arrangement and combination, and after removing repeated image areas, a digital pathology full-field image is obtained; Wherein, the boundary features of each local microscopic image are obtained by performing boundary detection on each local microscopic image using the Canny algorithm, including: Performing grayscale processing on each of the local microscopic images using a weighted average method to obtain a plurality of grayscale images; Converting the pixel value of each pixel in each of the grayscale images into a weighted average of pixel values ​​in a neighborhood by an adaptive convolution algorithm to generate a plurality of first local microscopic images; Dividing each of the first local microscopic images into a plurality of sub-image regions, respectively, calculating the pixel difference between the central pixel and the surrounding neighborhood pixels of each sub-image region in each of the first local microscopic images, to obtain a pixel difference value set of each of the first local microscopic images; Performing gradient calculation on the pixel difference values ​​of the pixel difference value set of each of the first local microscopic images to obtain the gradient amplitude of each of the first local microscopic images in the horizontal direction and the vertical direction; The overall gradient amplitude is calculated according to the gradient amplitudes of each of the first local microscopic images in the horizontal direction and the vertical direction; Marking pixel points in each of the first local microscopic images whose pixel values ​​are greater than the overall gradient amplitude as boundary pixel points, so as to form boundary features of each of the local microscopic images; Wherein, determining the tumor area in the digital pathology full-field image includes: Acquire a standard digital pathology full-field image, wherein the standard digital pathology full-field image is a preprocessed digital pathology full-field image; Adjusting the digital pathology full-field image under a preset magnification field of view to an image with the same size as the standard digital pathology full-field image to obtain a first digital pathology full-field image; Aligning the first digital pathology full-field image with the standard digital pathology full-field image, and calculating pixel differences of aligned pixels between the first digital pathology full-field image and the standard digital pathology full-field image; Calculating a weighted average of all pixel differences, and multiplying the weighted average by lgM to obtain a target pixel difference, where M is the number of pixel pairs of aligned pixels between the first digital pathology full-field image and the standard digital pathology full-field image; adjusting the pixel value of each pixel in the first digital pathology full-field image using the target pixel difference to obtain a target digital pathology full-field image; Tumor region features and non-tumor region features are extracted from the target digital pathology full-field image, and based on the tumor region features and the non-tumor region features, the target digital pathology full-field image is segmented using a threshold segmentation algorithm to obtain a tumor region.

2. The method according to claim 1, characterized in that The initial pathological section is subjected to PD-L1 immunohistochemical staining to obtain a target pathological section, including: Attaching the initial pathological section to a glass slide and placing it in a staining machine; Dewaxing the initial pathological section using the staining machine, and rehydrating the section with an alcohol solution to obtain a first pathological section; The first pathological section is incubated with a first antibody targeting PD-L1 of esophageal cancer tissue, and is incubated with a second antibody targeting PD-L1 of esophageal cancer tissue to obtain a second pathological section; Performing a color reaction treatment on the second pathological section using a DAB color developer, and counter-staining the cell nucleus using hematoxylin to obtain a third pathological section; The third pathological section is dehydrated, sealed with resin, and covered with a cover glass to obtain a target pathological section.

3. The method according to claim 1, characterized in that After determining whether the clarity of each of the local microscopic images is higher than a preset clarity, the method further includes: When it is determined that the clarity of any of the local microscopic images is lower than the preset clarity, the local microscopic image with a clarity lower than the preset clarity is used as a pending local microscopic image; The magnification corresponding to the pending local microscopic image is queried, and the default resolution of the full-slice imaging system is adjusted to the target resolution. Under the target resolution and the magnification corresponding to the pending local microscopic image, the full-slice imaging system is controlled to scan the target pathological slice again to obtain a candidate local microscopic image, and replace the pending local microscopic image.

4. The method according to claim 1, characterized in that The tumor cells in the tumor area are marked and counted using the pre-built YOLO model. Before obtaining the number of tumor cells, the following steps are also included: Acquire training samples, wherein the training samples include multiple digital pathology full-field image samples, each of which is labeled with cell type and quantity; Constructing an initial YOLO model and a marking rule, configuring the marking rule in the initial YOLO model, inputting the training sample into the initial YOLO model, calling the initial YOLO model to perform a preset number of iterative training on the training sample, and generating a training result after the iterative training; According to the training results after iterative training and a preset loss function, calculating the loss value of the initial YOLO model after iterative training; Determine whether the loss value after iterative training is lower than a preset loss value; When it is determined that the loss value after iterative training is lower than a preset loss value, the initial YOLO model is used as a pre-built YOLO model.

5. The method according to claim 4, characterized in that The marking rules include: When the tumor cells in the tumor area were located at the center of the field of view at a preset magnification of 20 times, the PD-L1-stained lymphocytes and stained monocytes and macrophages that appeared in the same field of view were recorded as positive lymphocytes and positive monocytes and macrophages, respectively; When the tumor area contains PD-L1-stained lymphocytes and stained monocytes and macrophages in the field of view at the preset magnification of 20 times, but no tumor cells appear in the same field of view, the stained lymphocytes in the field of view are not marked as positive lymphocytes, and the stained monocytes and macrophages are not marked as positive monocytes and macrophages; When the tumor cells in the tumor area have strong staining or complete linear membrane staining, the tumor cells are marked as positive tumor cells.

6. A terminal, characterized in that: include: processor; Memory; Wherein, the memory stores a computer program, and when the processor executes the computer program, the intelligent evaluation method for PD-L1 immunohistochemistry of esophageal cancer tissue according to any one of claims 1 to 5 is implemented.