An ampulla adenoma benign and malignant auxiliary diagnosis device and method based on deep learning

Abstract

This invention discloses a deep learning-based auxiliary diagnostic device and method for benign and malignant ampullary adenomas, comprising the following modules: an image preprocessing module, an image quality assessment module, and an image lesion nature assessment module. The image preprocessing module acquires and preprocesses endoscopic images of the patient; the image quality assessment module determines whether the valid examination images output after image preprocessing contain a clear nipple, thus achieving quality screening of the diagnostic images; the image lesion nature assessment module determines the benign or malignant nature of the lesions in the images containing clear nipples output by the image quality assessment module. This invention, by adding an image quality assessment module, solves the problem of low-quality images interfering with diagnostic accuracy in traditional diagnosis, thereby improving the efficiency and accuracy of benign and malignant ampullary adenomas.

Description

Technical Field

[0001] This invention relates to a deep learning-based auxiliary diagnostic method for benign and malignant ampullary adenomas, belonging to the technical field of medical auxiliary diagnostic systems. Background Technology

[0002] Ampullary adenoma is a benign tumor that occurs in the ampulla of Vater of the duodenum. Due to its high potential for malignant transformation, these lesions are considered precancerous. Because of its unique anatomical location, most cases are discovered incidentally during upper gastrointestinal endoscopy. Only some patients present with symptoms such as jaundice or abdominal pain; a minority may initially present with cholangitis or pancreatitis. The 5-year survival rate for ampullary tumors is less than 40%, and early diagnosis is a key independent predictor of survival. Therefore, improving the early detection rate of ampullary adenoma is of significant clinical importance.

[0003] Currently, the diagnosis of ampullary adenomas mainly relies on endoscopy, computed tomography (CT), and magnetic resonance cholangiopancreatography (MRCP), with endoscopy being the most important diagnostic method because it allows direct visualization of the lesion and tissue biopsy. However, due to insufficient observation and understanding, the accuracy rate of endoscopic diagnosis of ampullary tumors is approximately 67.3%, while the accuracy rate of antegrade endoscopy is only 45%. More importantly, the images acquired during endoscopic examination contain a large number of low-quality samples without clear papillae (e.g., blurred vision, lack of target anatomical structures). Directly incorporating these images into subsequent diagnostic procedures can severely interfere with physician judgment or increase the cost of ineffective analysis, leading to reduced diagnostic efficiency and an increased risk of misdiagnosis and missed diagnosis. Furthermore, the accuracy rate of endoscopic biopsy is 38.3%–81.9%, with an underestimation rate of 14.1%–31.9% and an overestimation rate of 0.9%–29.8%, and carries risks of cholangitis, pancreatitis, and bleeding, which may complicate subsequent endoscopic resection. Therefore, it is urgent to add a dedicated quality screening process to filter out low-value images at the source, laying the foundation for accurate diagnosis, and then developing new strategies to improve the efficiency of endoscopic diagnosis of ampullary adenomas. Summary of the Invention

[0004] Objective of the Invention: To address the problems and shortcomings of existing technologies, this invention provides a deep learning-based auxiliary diagnostic device and method for benign and malignant ampullary adenomas. By innovatively adding an image quality assessment module, it achieves pre-screening of endoscopic images, accurately identifying valid images containing clear nipples and eliminating low-quality samples with no diagnostic value. This solves the core pain point of traditional diagnosis—"inconsistent image quality leading to low diagnostic accuracy and efficiency"—and thus efficiently and in real-time assists in the identification of ampullary adenomas during the examination process, comprehensively improving the diagnostic efficiency and accuracy of ampullary adenomas.

[0005] Technical Solution: A deep learning-based auxiliary diagnostic device for benign and malignant ampullary adenomas innovatively constructs a two-tier diagnostic architecture of "pre-screening for quality and post-discrimination for nature," comprising the following modules: an image preprocessing module, an image quality judgment module, and an image lesion nature judgment module. The image preprocessing module acquires and preprocesses endoscopic images of the patient. The image quality judgment module determines whether the valid examination images output after image preprocessing contain a clear nipple. The image lesion nature judgment module determines the benign or malignant nature of the lesion in the image containing a clear nipple output by the image quality judgment module.

[0006] Furthermore, the auxiliary diagnostic device also includes an image acquisition module; the image acquisition module is used to acquire images of the patient's duodenum and output them to the image preprocessing module.

[0007] Furthermore, the image preprocessing module performs preprocessing on the patient's endoscopic examination image, including: cropping the effective area of ​​the image and removing task-irrelevant information borders.

[0008] Furthermore, the image quality judgment module is a deep learning module based on the Swing Transformer algorithm.

[0009] Furthermore, the image lesion nature determination module is a deep learning module based on the Swing Transformer algorithm.

[0010] A method for establishing a deep learning-based auxiliary diagnostic device for benign and malignant ampullary adenomas includes the following steps: Step 1: Acquire endoscopic images of patients to build a dataset; Step 2: Preprocess and label the patient's endoscopic images. First, label all images by quality – whether there is a clear nipple or not. Then, label all images labeled as having a clear nipple by lesion nature – benign or malignant. Step 3: Divide the images labeled as having clear nipples or not having clear nipples into training and testing sets, and use the training set to train an image quality judgment model based on the Swing Transformer algorithm; Step 4: Divide the images labeled as benign or malignant into training and test sets, and use the training set to train an image lesion nature judgment model based on the Swing Transformer algorithm.

[0011] Furthermore, the specific operations for preprocessing and annotating the examination images in step two are as follows: cropping the effective area of ​​the image and removing task-irrelevant information borders; three senior endoscopists annotate the quality of all images, and for clear nipple images, one senior endoscopist annotates the nature of the lesion based on the final diagnosis.

[0012] Furthermore, in steps three and four, the parameters of the model based on the Swing Transformer algorithm are initialized to the parameters of the model pre-trained on the ImageNet dataset.

[0013] Furthermore, in steps three and four, the model training process uses the cross-entropy loss between the model output and the true image label as the target loss function.

[0014] Beneficial Effects: This invention innovatively adds an image quality assessment module, constructing a complete process of "source quality control - subsequent accurate diagnosis." First, it filters out low-quality images without clear nipples at the source, avoiding interference with diagnostic results or increasing unnecessary analysis costs, thus providing high-quality input data for the subsequent lesion nature assessment module. Based on this, the invention can screen clear nipple images in real time and accurately determine the benign or malignant nature of ampullary adenomas. Multi-level experimental results show that this method has universal and robust performance in identifying ampullary adenomas, superior to that of endoscopists. Simultaneously, with the assistance of this method, the diagnostic accuracy, sensitivity, specificity, positive predictive value, and negative predictive value of endoscopists are all improved to varying degrees. The method provided by this invention can significantly improve biopsy efficiency and reduce false negative cases in clinical practice, demonstrating high clinical value. Attached Figure Description

[0015] Figure 1 This is an overall flowchart of the method in this invention; Figure 2 This is a construction diagram of the device module in this invention; Figure 3 This is a schematic diagram and a model heat map of the present invention for determining the benign or malignant nature of nipples; Figure 4 This is a schematic diagram of the diagnostic process on endoscopic video according to the present invention. Detailed Implementation

[0016] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading the present invention, any modifications of the present invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0017] like Figure 1As shown, a deep learning-based auxiliary diagnostic device for benign and malignant ampullary adenomas includes an image acquisition module, an image preprocessing module, an image quality judgment module, and an image lesion nature judgment module.

[0018] The image acquisition module acquires images of the patient's duodenum and outputs them to the image preprocessing module. The image preprocessing module acquires and preprocesses endoscopic images of the patient. The image quality assessment module determines whether the output images from the image preprocessing module contain clear papillae. The image lesion nature assessment module determines the benign or malignant nature of lesions in images containing clear papillae output from the image quality assessment module. The preprocessing performed on the patient's endoscopic images by the image preprocessing module includes: cropping the effective region of the image and removing task-irrelevant bounding boxes.

[0019] The image preprocessing module performs preprocessing on patient endoscopic images including: cropping the effective region of the image and removing task-irrelevant information borders.

[0020] The image quality assessment module is a deep learning module based on the Swing Transformer algorithm.

[0021] The image lesion nature judgment module is a deep learning module based on the Swing Transformer algorithm.

[0022] like Figure 2 As shown, a method for establishing a deep learning-based auxiliary diagnostic device for benign and malignant ampullary adenomas includes the following steps: Step 1: Acquire endoscopic images to build a dataset; Step 2: Preprocess and label the acquired patient examination images. First, label all images for quality – clear nipple or no clear nipple. Then, label all images labeled for clear nipple with lesion nature – benign or malignant. Step 3: Divide the images labeled as having clear nipples or not having clear nipples into training and testing sets, and use the training set to train an image quality judgment model based on the Swing Transformer algorithm; Step 4: Divide the images labeled as benign or malignant into training and test sets, and use the training set to train an image lesion nature judgment model based on the Swing Transformer algorithm.

[0023] The specific operations for preprocessing and labeling the acquired patient examination images in step two are as follows: cropping the effective area of ​​the image and removing task-irrelevant information borders; three senior endoscopists perform quality labeling on all images, and for clear nipple images, one senior endoscopist labels the nature of the lesion based on the final diagnosis.

[0024] In steps three and four, the parameters of the model based on the Swing Transformer algorithm are initialized to the parameters of the model pre-trained on the ImageNet dataset.

[0025] In steps three and four, the model training process uses the cross-entropy loss between the model output and the image's true label as the target loss function.

[0026] The specific steps for testing the diagnostic effectiveness of the auxiliary diagnostic device are as follows: First, the image quality assessment model was tested using an internal test set.

[0027] Then, the lesion nature assessment model was tested using both internal and external test sets. Simultaneously, 100 images (50 benign and 50 malignant) were randomly selected from the internal test set for a human-machine controlled experiment. The model achieved 97% accuracy on these 100 images, surpassing the accuracy of four professional endoscopists. The experimental visualization results are shown below. Figure 3 As shown.

[0028] Finally, as Figure 4 As shown, two deep learning models were combined into a single system and tested on endoscopic videos. For each frame in the video, if both the first model predicted a clear nipple and the second model predicted malignancy, then that frame was considered a malignant ampullary adenoma. The system was tested on 18 endoscopic examination videos, achieving a diagnostic accuracy of 96.3% for malignant ampullary adenomas.

Claims

1. A deep learning-based auxiliary diagnostic device for benign and malignant ampullary adenomas, characterized in that, It includes the following modules: an image preprocessing module, an image quality assessment module, and an image lesion nature assessment module; the image preprocessing module is used to acquire and preprocess endoscopic images of patients; the image quality assessment module is used to determine whether the valid examination images output after the image preprocessing module contain clear nipples, thereby achieving quality screening of diagnostic images; the image lesion nature assessment module is used to determine the benign or malignant nature of lesions in images containing clear nipples output by the image quality assessment module.

2. The deep learning-based auxiliary diagnostic device for benign and malignant ampullary adenomas according to claim 1, characterized in that, The auxiliary diagnostic device also includes an image acquisition module; the image acquisition module is used to acquire images of the patient's duodenum and output them to the image preprocessing module.

3. The deep learning-based auxiliary diagnostic device for benign and malignant ampullary adenomas according to claim 1, characterized in that, The image preprocessing module performs the following preprocessing on the patient's endoscopic examination images: cropping the effective area of ​​the image and removing task-irrelevant information borders.

4. The deep learning-based auxiliary diagnostic device for benign and malignant ampullary adenomas according to claim 1, characterized in that, The image quality assessment module is a deep learning module based on the Swing Transformer algorithm.

5. The deep learning-based auxiliary diagnostic device for benign and malignant ampullary adenomas according to claim 1, characterized in that, The image lesion nature judgment module is a deep learning module based on the Swing Transformer algorithm.

6. A method for establishing a deep learning-based auxiliary diagnostic device for benign and malignant ampullary adenomas, characterized in that, Specifically, the following steps are included: Step 1: Acquire endoscopic images of patients to build a dataset; Step 2: Preprocess and label the patient's endoscopic images. First, label all images by quality – whether there is a clear nipple or not. Then, label all images labeled as having a clear nipple by lesion nature – benign or malignant. Step 3: Divide the images labeled as having clear nipples or not having clear nipples into training and testing sets, and use the training set to train an image quality judgment model based on the Swing Transformer algorithm; Step 4: Divide the images labeled as benign or malignant into training and test sets, and use the training set to train an image lesion nature judgment model based on the Swing Transformer algorithm.

7. The method for establishing a deep learning-based auxiliary diagnostic device for benign and malignant ampullary adenomas according to claim 6, characterized in that, The specific operations for preprocessing and annotating the examination images in step two are as follows: cropping the effective area of ​​the image and removing information borders irrelevant to the task; three senior endoscopists annotate the quality of all images, and for clear nipple images, one senior endoscopist annotates the nature of the lesion based on the final diagnosis.

8. The method for establishing a deep learning-based auxiliary diagnostic device for benign and malignant ampullary adenomas according to claim 6, characterized in that, In steps three and four, the parameters of the model based on the Swing Transformer algorithm are initialized to the parameters of the model pre-trained on the ImageNet dataset.

9. The method for establishing a deep learning-based auxiliary diagnostic device for benign and malignant ampullary adenomas according to claim 6, characterized in that, In steps three and four, the model training process uses the cross-entropy loss between the model output and the image's true label as the target loss function.

Images