Deep learning-based esophageal squamous cell carcinoma early cancer auxiliary diagnosis system

Abstract

The application discloses an esophageal squamous cell carcinoma early cancer auxiliary diagnosis system based on deep learning, which comprises an esophagoscope image acquisition subsystem, a cloud platform server and a client; the esophagoscope image acquisition subsystem is used to acquire esophagoscope images and transmit the images to the cloud platform server; the cloud platform server comprises a data storage module, a data service processing module and a computing service module, and the computing service module comprises an image quality judgment model based on a deep learning network, an esophageal lesion state detection model and an early cancer grading detection model. The application can solve the missed detection problem caused by uneven doctor level in traditional esophageal squamous cell carcinoma examination, the esophageal lesion state detection model with a bilinear pooling and attention fusion mechanism can identify a lesion site, the early cancer grading detection model constructed by a lightweight residual network with a fusion attention mechanism can realize the grading of esophageal squamous cell carcinoma early cancer, and thus a simple and easy-to-use esophageal squamous cell carcinoma early cancer auxiliary diagnosis platform based on deep learning is formed.

Description

Technical Field

[0001] This invention relates to the field of computer-aided diagnostic technology, and in particular to an auxiliary diagnostic system for early esophageal squamous cell carcinoma based on deep learning. Background Technology

[0002] In the field of computer-aided diagnosis, deep learning technology, represented by convolutional neural networks, has been applied to the analysis of endoscopic images such as upper gastrointestinal endoscopy, colonoscopy, and capsule endoscopy. This application is of great significance for improving the accuracy and efficiency of endoscopic image diagnosis.

[0003] Endoscopy is an effective method for detecting early esophageal squamous cell carcinoma, significantly reducing its incidence and mortality. However, a complete standard upper gastrointestinal endoscopy procedure is time-consuming, involving complex operations such as preoperative preparation, endoscope insertion along the lumen, segmental observation, and modal switching. Therefore, the endoscopic process is affected by various factors, such as procedural standardization, clinical experience, visual fatigue, and individual heterogeneity of lesions. Some early esophageal cancer lesions appear as subtle changes in the mucosa under endoscopy, with inconspicuous visual features, making them difficult to detect. Consequently, a significant number of early esophageal cancer lesions are missed, leading to problems such as low sensitivity, high missed diagnosis rate, and poor endoscopic screening results in clinical practice.

[0004] Furthermore, under the hierarchical medical system that enables nationwide sharing of medical resources, primary care at the grassroots level, two-way referrals, separation of acute and chronic care, and coordinated care between different levels of healthcare institutions, the central hospitals have encountered problems such as diverse structured data and inconsistent quality of document data during the data collection process. This has increased the difficulty of esophageal cancer screening and diagnosis, and brought great challenges to providing continuous and high-quality medical and health services. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide an auxiliary diagnostic system for early esophageal squamous cell carcinoma based on deep learning, addressing the shortcomings of the prior art.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a deep learning-based auxiliary diagnostic system for early esophageal squamous cell carcinoma, comprising: an esophageal endoscopic image acquisition subsystem, a cloud platform server, and a client;

[0007] The esophageal endoscopy image acquisition subsystem is used to acquire esophageal endoscopy images and transmit them to the cloud platform server;

[0008] The cloud platform server includes a data storage module, a data service processing module, and a computing service module. The computing service module includes an image quality assessment model, an esophageal lesion status detection model, and an early cancer grading detection model based on deep learning networks.

[0009] The data service processing module includes a quality control unit, an esophageal squamous cell carcinoma early cancer auxiliary diagnosis unit, and a report generation unit. The quality control unit calls the image quality judgment model to judge the image quality of the input esophageal endoscopy images. Esophageal endoscopy images that meet the quality standards are then input into the esophageal squamous cell carcinoma early cancer auxiliary diagnosis unit. The esophageal squamous cell carcinoma early cancer auxiliary diagnosis unit calls the esophageal lesion status detection model to classify the esophageal endoscopy images as normal esophagus, benign lesions, and esophageal squamous cell carcinoma. When the classification result of the esophageal lesion status detection model is esophageal squamous cell carcinoma, the esophageal squamous cell carcinoma early cancer auxiliary diagnosis unit calls the early cancer grading detection model to further classify the esophageal endoscopy images according to the early cancer level.

[0010] The report generation unit automatically generates and outputs an auxiliary diagnosis report for early esophageal squamous cell carcinoma based on the results of the auxiliary diagnosis unit for early esophageal squamous cell carcinoma.

[0011] Preferably, the esophageal endoscopy image acquisition subsystem is connected to an electronic digestive endoscopy device, which acquires and displays esophageal endoscopy images obtained by the electronic digestive endoscopy device in real time, and uploads them to the cloud platform server after removing non-endoscopic image information.

[0012] The esophageal endoscopy image acquisition subsystem acquires esophageal endoscopy images of the same region, including white light non-magnified esophageal endoscopy images and blue light magnified esophageal endoscopy images, for selection by the esophageal lesion status detection model and the early cancer grading detection model.

[0013] Preferably, the image quality assessment model is a Faster RCNN network model, which gives a quality level score of the input esophageal endoscopy image based on several features that can characterize image quality. Then, the doctor judges whether the image is qualified based on the score result. Only esophageal endoscopy images that are judged to be qualified are sent to the data storage module for storage.

[0014] The features include at least the image's sharpness, contrast, saturation, proportion of reflective areas, proportion of bubble-occluded areas, and proportion of instrument-occluded areas.

[0015] Preferably, the esophageal lesion state detection model is based on the ResNet50 framework and includes an input layer, a residual module, a bilinear pooling module, and an output layer. Each residual module includes a global channel attention module. The esophageal lesion state detection model takes a white light non-magnified esophageal endoscopy image as input and classifies the input image into normal esophagus, benign lesions, and esophageal squamous cell carcinoma based on the features of the esophageal lesion image. The classification result is output to the esophageal squamous cell carcinoma early cancer auxiliary diagnosis unit.

[0016] Preferably, the early cancer grading detection model is based on ResNet50 and includes convolutional layers, pooling layers, GSCBottleneck lightweight residual modules, fully connected layers, and average pooling layers.

[0017] The GSCBottleneck lightweight residual module includes a GhostModule and an SCConv convolution module, and introduces a CA attention mechanism module;

[0018] The early cancer grading detection model takes as input a blue light magnified esophageal endoscopy image at the location corresponding to the white light non-magnified esophageal endoscopy image classified as esophageal squamous cell carcinoma, and further classifies the esophageal endoscopy image into early cancer levels.

[0019] Preferably, the early cancer grading detection model classifies early cancer levels by including type B1 IPCL vascular loops and type B2 IPCL vascular loops.

[0020] Preferably, the data service processing module further includes an image processing unit, a data transmission unit, and an information management unit;

[0021] Preferably, the image processing unit is used to adjust the size, shape, and brightness of the esophageal endoscopy image, as well as to annotate and note the regions of interest in the image;

[0022] The data transmission unit is used to realize data transmission between the esophageal endoscopy image acquisition subsystem and the cloud platform server, as well as data transmission between the cloud platform server and the client.

[0023] Preferably, the working steps of the esophageal endoscopy image acquisition subsystem include:

[0024] 1-1) Receive esophageal image signals output by electronic digestive endoscopy equipment, transcode them, and simultaneously display the real-time images acquired by the electronic digestive endoscopy.

[0025] 1-2) Enter and store the current patient's medical record information;

[0026] 1-3) Acquire and save images of the esophagus of interest. Simultaneously, perform structured processing on the images of the esophagus of interest and the patient's medical record information, and upload the data to the cloud platform server after data packaging and compression.

[0027] Preferably, the working steps of the cloud platform server include:

[0028] 2-1) The data transmission unit receives and parses the data sent by the esophageal endoscopy image acquisition subsystem;

[0029] 2-2) The quality control unit calls the image quality judgment model to judge the image quality of the input esophageal endoscopy image and gives a quality score. The doctor judges whether the image is qualified based on the score. The qualified esophageal endoscopy image is sent to the data storage module for storage and then proceeds to the next step. If the image is unqualified, an image rejection message is sent to the esophageal endoscopy image acquisition subsystem.

[0030] 2-3) The image processing unit receives esophageal endoscopy images that have passed the quality judgment, and adjusts their size, shape, and brightness, as well as marking and annotating the regions of interest in the images;

[0031] 2-4) The esophageal squamous cell carcinoma early cancer auxiliary diagnosis module receives the esophageal endoscopy image processed by the image processing unit, filters out the white light non-magnified esophageal endoscopy image, and then calls the esophageal lesion state detection model to perform change state detection on the white light non-magnified esophageal endoscopy image to realize the classification of normal esophagus, benign lesions and esophageal squamous cell carcinoma.

[0032] When the classification result of the esophageal lesion status detection model is esophageal squamous cell carcinoma, the esophageal squamous cell carcinoma early cancer auxiliary diagnosis unit further filters out blue light magnified esophageal endoscopy images with the same region of interest corresponding to the current white light non-magnified esophageal endoscopy image. Then, it calls the early cancer grading detection model to perform early cancer grading detection on the blue light magnified esophageal endoscopy image to achieve early cancer level classification. The classification result includes type B1 IPCL vascular loops and type B2 IPCL vascular loops.

[0033] 2-5) The report generation unit outputs the change state detection results / early cancer grading detection results of all images and notes the confidence level to form an auxiliary diagnostic report. The auxiliary diagnostic report includes at least the patient's medical record information, esophageal endoscopy images, and the detection results of the esophageal lesion state detection model on the esophageal endoscopy images.

[0034] The client provides an access interface to the cloud platform server to view the patient's auxiliary diagnostic reports and other data in the cloud platform server.

[0035] The beneficial effects of this invention are:

[0036] This invention provides a deep learning-based auxiliary diagnostic system for early esophageal squamous cell carcinoma, which can solve the problem of missed detection in traditional esophageal squamous cell carcinoma examinations due to uneven skill levels among doctors. This invention uses an esophageal lesion status detection model with bilinear pooling and attention fusion mechanisms to intelligently identify lesion sites. The early cancer grading detection model constructed by a lightweight residual network with attention fusion mechanisms can achieve grading of early esophageal squamous cell carcinoma, thus forming a simple and easy-to-use deep learning-based auxiliary diagnostic platform for early esophageal squamous cell carcinoma.

[0037] This invention solves the problem of the process of acquiring esophageal endoscopic images from electronic digestive endoscopy equipment and transmitting them to a cloud platform for auxiliary diagnosis. It can reduce the time spent on esophageal segment examinations during digestive endoscopy and improve the accuracy and efficiency of diagnosing early squamous cell carcinoma of the esophagus. Attached Figure Description

[0038] Figure 1 This is a schematic diagram illustrating the principle and structure of the deep learning-based early esophageal squamous cell carcinoma auxiliary diagnostic system of the present invention.

[0039] Figure 2 This is a schematic diagram of the esophageal lesion state detection model of the present invention;

[0040] Figure 3 This is a comparison of the detection results of the gold standard and the esophageal lesion state detection model in one embodiment of the present invention;

[0041] Figure 4 This is a schematic diagram of the early cancer grading detection model of the present invention;

[0042] Figure 5 The detection result is from an early cancer grading detection model in one embodiment of the present invention;

[0043] Figure 6 This is a flowchart of the deep learning-based auxiliary diagnostic system for early esophageal squamous cell carcinoma of the present invention. Detailed Implementation

[0044] The present invention will be further described in detail below with reference to embodiments, so that those skilled in the art can implement it based on the description.

[0045] It should be understood that terms such as “having,” “comprising,” and “including” as used herein do not exclude the presence or addition of one or more other elements or combinations thereof.

[0046] Example 1

[0047] This embodiment provides a deep learning-based auxiliary diagnostic system for early esophageal squamous cell carcinoma, including: an esophageal endoscopic image acquisition subsystem, a cloud platform server, and a client;

[0048] The esophageal endoscopy image acquisition subsystem is used to acquire esophageal endoscopy images and transmit them to the cloud platform server;

[0049] The cloud platform server includes a data storage module, a data service processing module, and a computing service module. The computing service module includes an image quality assessment model, an esophageal lesion status detection model, and an early cancer grading detection model based on deep learning networks.

[0050] The data service processing module includes a quality control unit, an auxiliary diagnostic unit for early esophageal squamous cell carcinoma, and a report generation unit. The quality control unit calls an image quality assessment model to judge the image quality of the input esophageal endoscopy images. Esophageal endoscopy images that meet the quality standards are then input into the auxiliary diagnostic unit for early esophageal squamous cell carcinoma. The auxiliary diagnostic unit for early esophageal squamous cell carcinoma calls an esophageal lesion status detection model to classify the esophageal endoscopy images as normal esophagus, benign lesions, and esophageal squamous cell carcinoma. When the classification result of the esophageal lesion status detection model is esophageal squamous cell carcinoma, the auxiliary diagnostic unit for early esophageal squamous cell carcinoma calls an early cancer grading detection model to further classify the esophageal endoscopy images according to the early cancer level.

[0051] The report generation unit automatically generates and outputs an auxiliary diagnosis report for early esophageal squamous cell carcinoma based on the results of the auxiliary diagnosis unit for early esophageal squamous cell carcinoma.

[0052] The esophageal endoscopy image acquisition subsystem is deployed in the endoscopy clinic and is directly connected to and used in a one-to-one pair with the electronic digestive endoscopy equipment. It acquires and displays esophageal endoscopy images obtained by the electronic digestive endoscopy equipment in real time, and uploads the data to the cloud platform server after desensitizing the data and removing non-endoscopic image information.

[0053] In a preferred embodiment, the esophageal endoscopy image acquisition subsystem includes an endoscopy image acquisition module, an information management module, and a data transmission module. The information management module is used to store and manage the patient's medical record information, and the data transmission module is used to realize data transmission between the esophageal endoscopy image acquisition subsystem and the electronic digestive endoscopy equipment and cloud platform server.

[0054] In a preferred embodiment, when a doctor uses an electronic digestive endoscope to examine the esophagus, he can acquire images of interest individually or continuously, based on his diagnostic experience.

[0055] The esophageal endoscopy image acquisition subsystem acquires esophageal endoscopy images of the same region, including white light non-magnified esophageal endoscopy images and blue light magnified esophageal endoscopy images, for selection by the esophageal lesion status detection model and the early cancer grading detection model. Specifically, the esophageal lesion status detection model uses white light non-magnified esophageal endoscopy images as input, while the early cancer grading detection model uses blue light magnified esophageal endoscopy images as input.

[0056] In a preferred embodiment, the image quality assessment model is a Faster RCNN network model, which gives a quality level score of the input esophageal endoscopy image based on several features that can characterize image quality. Then, the doctor judges whether the image is qualified based on the score result. Only esophageal endoscopy images that are judged to be qualified are sent to the data storage module for storage.

[0057] The features include image sharpness, contrast, saturation, proportion of reflective areas, proportion of bubble-occluded areas and instrument-occluded areas, edges, and information entropy.

[0058] In a preferred embodiment, the esophageal lesion state detection model is based on the ResNet50 framework, including an input layer, a residual module, a bilinear pooling module, and an output layer. Each residual module incorporates a global channel attention module. The model takes a white light, non-magnified esophageal endoscopy image as input and classifies the input image into normal esophagus, benign lesions, and esophageal squamous cell carcinoma based on the characteristics of the esophageal lesion image. The classification results are output to the esophageal squamous cell carcinoma early cancer auxiliary diagnosis unit. (Refer to...) Figure 2 This is a schematic diagram of the structure of the esophageal lesion state detection model. Figure 2 The attention module first uses 1×1 convolutions to achieve cross-channel interaction and obtain the correlation between channels. Then, matrix multiplication is used to obtain the global representation of each channel, thus achieving global context modeling. Finally, a fully connected layer models the weights between channels, and feature transformation is performed using weight multiplication and weighted feature maps. Bilinear pooling is used with Hadamard product factorization. Because features closer to the back of the neural network contain more high-level semantic information, bilinear pooling is applied. Figure 2 The three activation layers of the Project Layer in the network are relu5_1, relu5_2, and relu5_3. (See reference...) Figure 3 In one embodiment, the results of the gold standard and the esophageal lesion status detection model are compared. From left to right, the figure shows the white light endoscopic image marked by the endoscopist (gold standard), the cancer probability heatmap, the cancer probability heatmap generated by the model, the lesion marked by the model, and the lesion marked by the endoscopist. Figure 3 The results show that the model can effectively locate suspicious lesion areas in lesion images, while covering more than 85% of the real lesion areas.

[0059] The backbone network ResNet50 is used to learn feature maps. Effective features are extracted through a designed global channel attention-residual module, and bilinear pooling is used in the backend to enhance feature representation.

[0060] In a preferred embodiment, the early cancer grading detection model uses a lightweight residual network based on ResNet50 with an attention mechanism, referred to as CALite-ResNet, whose specific structure is as follows: Figure 4 As shown, it includes convolutional layers, pooling layers, GSCBottleneck lightweight residual modules, fully connected layers, and average pooling layers;

[0061] The GSCBottleneck lightweight residual module includes the GhostModule and the SCConv convolution module to reduce the number of model parameters and improve inference efficiency. It also introduces the CA attention mechanism module to construct the CA-GSCBottleneck module to enhance the model's feature learning and representation, thereby improving the model's recognition accuracy. Replacing the original residual structure of ResNet50 with these two modules can reduce the number of model parameters without sacrificing model accuracy, thereby reducing hardware resource requirements and facilitating the model's application in clinical practice.

[0062] The early cancer grading detection model takes the blue light magnified esophageal endoscopy image (BLI-ME) corresponding to the position of the white light non-magnified esophageal endoscopy image classified as esophageal squamous cell carcinoma as input, and further classifies the esophageal endoscopy image into early cancer grades.

[0063] Reference Figure 5 In one embodiment, the detection result is from an early cancer grading detection model. Figure 5 It can be seen that IPCL vascular loops in both dense and discrete distributions can be detected, as can IPCL vascular loops in both large and small regions, and different subtypes of IPCL vascular loops in the same image can be detected.

[0064] In a further preferred embodiment, the early cancer grading detection model classifies early cancer levels as JES classifications, including type B1 IPCL vascular loops and type B2 IPCL vascular loops.

[0065] In a preferred embodiment, the data service processing module further includes an image processing unit, a data transmission unit, and an information management unit.

[0066] The image processing unit adjusts the size, shape, and brightness of esophageal endoscopy images, and annotates and notes regions of interest within the images. The data transmission unit enables data transmission between the esophageal endoscopy image acquisition subsystem and the cloud platform server, as well as between the cloud platform server and the client. The information management unit manages the data on both the cloud platform server and the client.

[0067] In a preferred embodiment, the client is a web client page accessible by a browser.

[0068] Reference Figure 6 In one embodiment, the working steps of this deep learning-based esophageal squamous cell carcinoma early cancer auxiliary diagnostic system include:

[0069] 1) Image acquisition by the esophageal endoscopy image acquisition subsystem:

[0070] 1-1) Receive esophageal image signals output by electronic digestive endoscopy equipment, transcode them, and simultaneously display the real-time images acquired by the electronic digestive endoscopy.

[0071] 1-2) Enter the current patient's medical record information and transmit it to the cloud platform server. Store and manage the medical record information data through the information management module.

[0072] 1-3) Acquire and save images of the esophagus of interest. At the same time, perform structured processing on the images of the esophagus of interest and the patient's medical record information, and upload the data to the cloud platform server after data packaging and compression.

[0073] 2) Data processing on the cloud platform server:

[0074] 2-1) The data transmission unit receives and parses the data sent by the esophageal endoscopy image acquisition subsystem;

[0075] 2-2) The quality control unit calls the image quality judgment model to judge the image quality of the input esophageal endoscopy image and gives a quality score. The doctor judges whether the image is qualified based on the score. The qualified esophageal endoscopy image is sent to the data storage module for storage and then proceeds to the next step. If the image is unqualified, an image return information is sent to the esophageal endoscopy image acquisition subsystem.

[0076] 2-3) The image processing unit receives esophageal endoscopy images that have passed quality assessment, and adjusts their size, shape, and brightness, as well as marking and annotating the regions of interest in the images; in this embodiment, the above image processing is performed manually.

[0077] 2-4) The esophageal squamous cell carcinoma early cancer auxiliary diagnosis module receives the esophageal endoscopy images processed by the image processing unit, filters out the white light non-magnified esophageal endoscopy images, and then calls the esophageal lesion state detection model to perform change state detection on the white light non-magnified esophageal endoscopy images to achieve the classification of normal esophagus, benign lesions and esophageal squamous cell carcinoma.

[0078] When the classification result of the esophageal lesion status detection model is esophageal squamous cell carcinoma, the esophageal squamous cell carcinoma early cancer auxiliary diagnosis unit further filters out the blue light magnified esophageal endoscopy image with the same region of interest corresponding to the current white light non-magnified esophageal endoscopy image. Then, it calls the early cancer grading detection model to perform early cancer grading detection on the blue light magnified esophageal endoscopy image to achieve early cancer level classification. The classification result is the JES classification of early cancer level, including type B1 IPCL vascular loop and type B2 IPCL vascular loop.

[0079] 2-5) The report generation unit outputs the change state detection results / early cancer grading detection results of all images and notes the confidence level to form an auxiliary diagnostic report. This auxiliary diagnostic report includes at least the patient's medical record information, esophageal endoscopy images, and the detection results of the esophageal lesion state detection model on the esophageal endoscopy images. This auxiliary diagnostic report can be used by doctors to provide auxiliary reference for the diagnosis of early squamous cell carcinoma of the esophagus.

[0080] The web client page provides an access interface to the cloud platform server, allowing users to view patients' auxiliary diagnostic reports in list format and visualize various data in the cloud platform server in chart format.

[0081] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details.

Claims

1. A deep learning-based auxiliary diagnostic system for early esophageal squamous cell carcinoma, characterized in that, include: Esophageal endoscopy image acquisition subsystem, cloud platform server and client; The esophageal endoscopy image acquisition subsystem is used to acquire esophageal endoscopy images and transmit them to the cloud platform server; The cloud platform server includes a data storage module, a data service processing module, and a computing service module. The computing service module includes an image quality assessment model, an esophageal lesion status detection model, and an early cancer grading detection model based on deep learning networks. The data service processing module includes a quality control unit, an auxiliary diagnostic unit for early esophageal squamous cell carcinoma, and a report generation unit. The quality control unit calls the image quality judgment model to judge the image quality of the input esophageal endoscopy images. The esophageal endoscopy images that meet the quality requirements are then input into the auxiliary diagnostic unit for early esophageal squamous cell carcinoma. The esophageal squamous cell carcinoma early cancer auxiliary diagnosis unit calls the esophageal lesion status detection model to classify esophageal endoscopy images as normal esophagus, benign lesions, and esophageal squamous cell carcinoma. When the classification result of the esophageal lesion status detection model is esophageal squamous cell carcinoma, the esophageal squamous cell carcinoma early cancer auxiliary diagnosis unit calls the early cancer grading detection model to further classify the esophageal endoscopy images into early cancer levels. The report generation unit automatically generates and outputs an auxiliary diagnosis report for early esophageal squamous cell carcinoma based on the results of the auxiliary diagnosis unit for early esophageal squamous cell carcinoma. The image quality assessment model is a Faster RCNN network model, which gives a quality level score of the input esophageal endoscopy image based on several features that can characterize image quality. Then, the doctor judges whether the image is qualified based on the score result. Only the qualified esophageal endoscopy images are sent to the data storage module for storage. The features include at least the image's sharpness, contrast, saturation, proportion of reflective area, proportion of bubble-occluded area, and proportion of instrument-occluded area. The esophageal lesion state detection model is based on the ResNet50 framework and includes an input layer, a residual module, a bilinear pooling module, and an output layer. Each residual module includes a global channel attention module. The esophageal lesion state detection model takes a white light non-magnified esophageal endoscopy image as input and classifies the input image into normal esophagus, benign lesions, and esophageal squamous cell carcinoma based on the features of the esophageal lesion image. The classification results are output to the esophageal squamous cell carcinoma early cancer auxiliary diagnosis unit. The early cancer grading detection model is based on ResNet50 and includes convolutional layers, pooling layers, GSCBottleneck lightweight residual modules, fully connected layers, and average pooling layers. The GSCBottleneck lightweight residual module includes a GhostModule and an SCConv convolution module, and introduces a CA attention mechanism module; The early cancer grading detection model takes as input a blue light magnified esophageal endoscopy image at the location corresponding to the white light non-magnified esophageal endoscopy image classified as esophageal squamous cell carcinoma, and further classifies the esophageal endoscopy image into early cancer levels.

2. The deep learning-based auxiliary diagnostic system for early esophageal squamous cell carcinoma according to claim 1, characterized in that, The esophageal endoscopy image acquisition subsystem is connected to the electronic digestive endoscopy equipment, and acquires and displays the esophageal endoscopy images obtained by the electronic digestive endoscopy equipment in real time. After removing non-endoscopic image information, it is uploaded to the cloud platform server. The esophageal endoscopy image acquisition subsystem acquires esophageal endoscopy images of the same region, including white light non-magnified esophageal endoscopy images and blue light magnified esophageal endoscopy images, for selection by the esophageal lesion status detection model and the early cancer grading detection model.

3. The deep learning-based auxiliary diagnostic system for early esophageal squamous cell carcinoma according to claim 1, characterized in that, The early cancer grading detection model classifies early cancer levels into B1 type IPCL vascular loops and B2 type IPCL vascular loops.

4. The deep learning-based auxiliary diagnostic system for early esophageal squamous cell carcinoma according to claim 3, characterized in that, The data service processing module also includes an image processing unit, a data transmission unit, and an information management unit.

5. The deep learning-based auxiliary diagnostic system for early esophageal squamous cell carcinoma according to claim 4, characterized in that, The image processing unit is used to adjust the size, shape, and brightness of the esophageal endoscopy image, as well as to mark and annotate the region of interest in the image; The data transmission unit is used to realize data transmission between the esophageal endoscopy image acquisition subsystem and the cloud platform server, as well as data transmission between the cloud platform server and the client.

6. The deep learning-based auxiliary diagnostic system for early esophageal squamous cell carcinoma according to claim 5, characterized in that, The working steps of the esophageal endoscopy image acquisition subsystem include: 1-1) Receive esophageal image signals output by electronic digestive endoscopy equipment, transcode them, and simultaneously display the real-time images acquired by the electronic digestive endoscopy. 1-2) Enter and store the current patient's medical record information; 1-3) Acquire and save images of the esophagus of interest. Simultaneously, perform structured processing on the images of the esophagus of interest and the patient's medical record information, and upload the data to the cloud platform server after data packaging and compression.

7. The deep learning-based auxiliary diagnostic system for early esophageal squamous cell carcinoma according to claim 6, characterized in that, The working steps of the cloud platform server include: 2-1) The data transmission unit receives and parses the data sent by the esophageal endoscopy image acquisition subsystem; 2-2) The quality control unit calls the image quality judgment model to judge the image quality of the input esophageal endoscopy image and gives a quality score. The doctor judges whether the image is qualified based on the score. The qualified esophageal endoscopy image is sent to the data storage module for storage and then proceeds to the next step. If the image is unqualified, an image rejection message is sent to the esophageal endoscopy image acquisition subsystem. 2-3) The image processing unit receives esophageal endoscopy images that have passed the quality judgment, and adjusts their size, shape, and brightness, as well as marking and annotating the regions of interest in the images; 2-4) The esophageal squamous cell carcinoma early cancer auxiliary diagnosis module receives the esophageal endoscopy image processed by the image processing unit, filters out the white light non-magnified esophageal endoscopy image, and then calls the esophageal lesion state detection model to perform change state detection on the white light non-magnified esophageal endoscopy image to realize the classification of normal esophagus, benign lesions and esophageal squamous cell carcinoma. When the classification result of the esophageal lesion status detection model is esophageal squamous cell carcinoma, the esophageal squamous cell carcinoma early cancer auxiliary diagnosis unit further filters out blue light magnified esophageal endoscopy images with the same region of interest corresponding to the current white light non-magnified esophageal endoscopy image. Then, it calls the early cancer grading detection model to perform early cancer grading detection on the blue light magnified esophageal endoscopy image to achieve early cancer level classification. The classification result includes type B1 IPCL vascular loops and type B2 IPCL vascular loops. 2-5) The report generation unit outputs the change state detection results / early cancer grading detection results of all images and notes the confidence level to form an auxiliary diagnostic report. The auxiliary diagnostic report includes at least the patient's medical record information, esophageal endoscopy images, and the detection results of the esophageal lesion state detection model on the esophageal endoscopy images. The client provides an access interface to the cloud platform server to view the patient's auxiliary diagnostic reports and other data in the cloud platform server.

Images