Gastrointestinal cancer detection method and apparatus, device, storage medium, and program product

Abstract

A gastrointestinal cancer detection method and apparatus, a device, a storage medium, and a program product, relating to the technical field of image recognition. The method comprises: inputting a gastrointestinal image into an image classification model to obtain an image classification result outputted by the image classification model, wherein the image classification result is used for representing a recognition result of performing cancer recognition on the gastrointestinal image; inputting the gastrointestinal image into a semantic segmentation model to obtain a semantic segmentation result outputted by the semantic segmentation model, wherein the semantic segmentation result is used for indicating whether a lesion region exists in the gastrointestinal image; on the basis of the semantic segmentation result and the image classification result, determining a detection mode for a gastrointestinal part corresponding to the gastrointestinal image; and when the detection mode indicates that detection needs to be performed on the gastrointestinal part corresponding to the gastrointestinal image, outputting prompt information corresponding to the detection mode. The method can achieve efficient, highly sensitive and highly accurate gastrointestinal cancer detection.

Description

Methods, devices, equipment, storage media, and procedures for detecting gastrointestinal cancers.

[0001] Cross-referencing

[0002] This application claims priority to Chinese Patent Application No. 202411949081X, filed on December 27, 2024, entitled “Method, Apparatus, Device, Storage Medium and Procedure Product for Detection of Gastrointestinal Cancer”, the entire disclosure of which is incorporated herein by reference. Technical Field

[0003] This invention relates to the field of image recognition technology, and in particular to a method, apparatus, device, storage medium, and program product for detecting gastrointestinal cancer. Background Technology

[0004] The prognosis of gastrointestinal tumors (such as esophageal cancer, gastric cancer, and colorectal cancer) is closely related to early diagnosis and treatment. Gastroscopy and colonoscopy, as well as endoscopic biopsy, are currently the gold standard for the clinical diagnosis of gastrointestinal tumors. Most guidelines for the diagnosis and treatment of gastrointestinal tumors indicate that the standardized diagnostic process involves detecting lesions with ordinary white light endoscopy, confirming or clarifying the nature of the lesions with chromoendoscopy, and performing biopsy for characterization.

[0005] With the rapid development of artificial intelligence, its application in medical devices is becoming increasingly widespread. AI-based medical image recognition can assist in the diagnosis of gastrointestinal cancers, especially early-stage gastrointestinal cancers, improving diagnostic accuracy.

[0006] Currently, target detection is performed on digestive tract images to determine the presence of lesion areas. Suspected cancerous lesions are outlined with rectangular boxes to assist doctors in further diagnosis. However, the resolution of acquired digestive tract images varies; some are captured with close-up lenses, while others are captured with distant lenses. This reduces the accuracy of target detection using a single model, potentially leading to missed detections. In other words, the sensitivity of current digestive tract cancer identification is not high. Furthermore, target detection uses rectangular boxes to select lesion areas, but lesion areas may not be rectangular. Therefore, the accuracy of lesion localization is not high, requiring doctors to manually determine the lesion area within the rectangular box, resulting in low efficiency in current digestive tract cancer identification. Summary of the Invention

[0007] This invention provides a method, apparatus, device, storage medium, and program product for detecting digestive tract cancer, which addresses the shortcomings of low sensitivity and efficiency in the identification of digestive tract cancer in the prior art, and achieves efficient and highly sensitive detection of digestive tract cancer.

[0008] This invention provides a method for detecting gastrointestinal cancer, comprising:

[0009] A digestive tract image is input into an image classification model to obtain an image classification result output by the image classification model; the image classification result is used to characterize the cancer identification result of the digestive tract image.

[0010] The digestive tract image is input into a semantic segmentation model to obtain the semantic segmentation result output by the semantic segmentation model; the semantic segmentation result is used to indicate whether there is a lesion region in the digestive tract image.

[0011] Based on the semantic segmentation results and the image classification results, the detection method for the digestive tract region corresponding to the digestive tract image is determined;

[0012] When the detection method indicates that the digestive tract region corresponding to the digestive tract image needs to be detected, the prompt information corresponding to the detection method is output.

[0013] According to a method for detecting gastrointestinal cancer provided by the present invention, if the gastrointestinal image is a stained image, the image classification result includes a first classification result or a second classification result, and the method for determining the detection mode of the gastrointestinal region corresponding to the gastrointestinal image based on the semantic segmentation result and the image classification result includes:

[0014] If, based on the semantic segmentation result, it is determined that there is a lesion region in the digestive tract image, and the image classification result is the first classification result, then the detection method is determined to be biopsy.

[0015] If, based on the semantic segmentation result, it is determined that there is a lesion region in the digestive tract image, and the image classification result is the second classification result, then the detection method is determined to be no detection required.

[0016] If, based on the semantic segmentation results, it is determined that there are no lesion areas in the digestive tract image, the detection method is determined to be "no detection required".

[0017] The digestive tract region corresponding to the stained image has been stained; the cancer probability value represented by the first classification result is greater than the cancer probability value represented by the second classification result.

[0018] If the digestive tract image is an unstained image, the image classification result includes a third classification result, a fourth classification result, or a fifth classification result. The method for determining the detection method of the digestive tract region corresponding to the digestive tract image based on the semantic segmentation result and the image classification result includes:

[0019] If, based on the semantic segmentation result, it is determined that there is a lesion region in the digestive tract image, and the image classification result is the third classification result, then the detection method is determined to be biopsy.

[0020] If, based on the semantic segmentation result, it is determined that there is a lesion region in the digestive tract image, and the image classification result is the fourth classification result, then the detection method is determined to be staining;

[0021] If, based on the semantic segmentation result, it is determined that there is a lesion region in the digestive tract image, and the image classification result is the fifth classification result, then the detection method is determined to be no detection required.

[0022] If, based on the semantic segmentation results, it is determined that there are no lesion areas in the digestive tract image, the detection method is determined to be "no detection required".

[0023] Wherein, the digestive tract region corresponding to the unstained image is not stained; the cancer probability value represented by the third classification result is greater than the cancer probability value represented by the fourth classification result, and the cancer probability value represented by the fourth classification result is greater than the cancer probability value represented by the fifth classification result.

[0024] According to a method for detecting gastrointestinal cancer provided by the present invention, if the gastrointestinal image is a stained image, the step of inputting the gastrointestinal image into an image classification model to obtain the image classification result output by the image classification model includes:

[0025] The digestive tract image is input into the first image classification model corresponding to the stained image to obtain the image classification result output by the first image classification model;

[0026] If the digestive tract image is an unstained image, the step of inputting the digestive tract image into an image classification model to obtain the image classification result output by the image classification model includes:

[0027] The digestive tract image is input into the second image classification model corresponding to the unstained image to obtain the image classification result output by the second image classification model;

[0028] If the digestive tract image is a stained image, the step of inputting the digestive tract image into a semantic segmentation model to obtain the semantic segmentation result output by the semantic segmentation model includes:

[0029] The digestive tract image is input into the first semantic segmentation model corresponding to the stained image to obtain the semantic segmentation result output by the first semantic segmentation model;

[0030] If the digestive tract image is an unstained image, the step of inputting the digestive tract image into the semantic segmentation model to obtain the semantic segmentation result output by the semantic segmentation model includes:

[0031] The digestive tract image is input into the second semantic segmentation model corresponding to the unstained image to obtain the semantic segmentation result output by the second semantic segmentation model;

[0032] The digestive tract region corresponding to the stained image is stained, while the digestive tract region corresponding to the unstained image is unstained.

[0033] According to the method for detecting gastrointestinal cancer provided by the present invention, the semantic segmentation model is an integrated semantic segmentation model, which includes multiple different segmentation models;

[0034] For any pixel in the digestive tract image, the integrated semantic segmentation model is used to determine multiple category prediction results for the pixel. Each category prediction result includes a lesion pixel or a non-lesion pixel. If any category prediction result among the multiple category prediction results is a lesion pixel, then the final category prediction result of the pixel is determined to be a lesion pixel. If all category prediction results among the multiple category prediction results are non-lesion pixels, then the final category prediction result of the pixel is determined to be a non-lesion pixel.

[0035] According to a method for detecting gastrointestinal cancer provided by the present invention, before inputting a gastrointestinal image into an image classification model and obtaining the image classification result output by the image classification model, the method further includes:

[0036] The digestive tract image is input into the feature extraction model to obtain the image features output by the feature extraction model;

[0037] Accordingly, the step of inputting the digestive tract image into the image classification model and obtaining the image classification result output by the image classification model includes:

[0038] The image features are input into the image classification model to obtain the image classification result output by the image classification model;

[0039] Accordingly, the step of inputting the digestive tract image into the semantic segmentation model to obtain the semantic segmentation result output by the semantic segmentation model includes:

[0040] The image features are input into the semantic segmentation model to obtain the semantic segmentation result output by the semantic segmentation model.

[0041] According to a method for detecting digestive tract cancer provided by the present invention, the image classification result includes a target classification result or several non-target classification results, wherein the cancer probability values ​​represented by the several non-target classification results are all less than the cancer probability values ​​represented by the target classification result.

[0042] Before determining the detection method for the digestive tract region corresponding to the digestive tract image based on the semantic segmentation result and the image classification result, the method further includes:

[0043] If, based on the semantic segmentation result, it is determined that there is no lesion region in the digestive tract image, and the image classification result is the target classification result, the segmentation threshold of the semantic segmentation model is adjusted to obtain and output the lesion region in the digestive tract image;

[0044] Specifically, for any pixel in the digestive tract image, the semantic segmentation model determines the category prediction result of the pixel based on the segmentation threshold, and the category prediction result of the pixel includes lesion pixels or non-lesion pixels.

[0045] According to a method for detecting gastrointestinal cancer provided by the present invention, adjusting the segmentation threshold of the semantic segmentation model includes:

[0046] The lowest segmentation threshold is set to 0, and the highest segmentation threshold is set as the current segmentation threshold of the semantic segmentation model.

[0047] Determine the sum of the lowest segmentation threshold and the highest segmentation threshold, and adjust the segmentation threshold of the semantic segmentation model to half of the sum.

[0048] The semantic segmentation result is updated based on the adjusted segmentation threshold;

[0049] If, based on the updated semantic segmentation results, it is determined that the proportion of lesion regions in the digestive tract image is greater than the first preset proportion, the minimum segmentation threshold is updated to the adjusted segmentation threshold, and the step of determining the sum of the minimum segmentation threshold and the maximum segmentation threshold and adjusting the segmentation threshold of the semantic segmentation model to half of the sum is returned.

[0050] If, based on the updated semantic segmentation results, it is determined that the proportion of lesion regions in the digestive tract image is less than the second preset proportion, the highest segmentation threshold is updated to the adjusted segmentation threshold, and the step of determining the sum of the lowest segmentation threshold and the highest segmentation threshold and adjusting the segmentation threshold of the semantic segmentation model to half of the sum is returned.

[0051] Specifically, for any pixel in the digestive tract image, the semantic segmentation model outputs a predicted value for the pixel. If the predicted value of the pixel is greater than or equal to the segmentation threshold, the category prediction result of the pixel is determined to be a lesion pixel. The proportion of the lesion region is determined based on the ratio of the lesion region in the digestive tract image to the area of ​​the digestive tract image, and the first preset proportion is greater than the second preset proportion.

[0052] According to a method for detecting gastrointestinal cancer provided by the present invention, the method further includes:

[0053] The digestive tract image is input into the digestive tract location recognition model to obtain the digestive tract location recognition result output by the digestive tract location recognition model;

[0054] Output the prompt information corresponding to the identification result of the digestive tract part to indicate the digestive tract part currently being examined;

[0055] The digestive tract site identification model is trained based on the sample digestive tract image and the corresponding digestive tract site identification result label.

[0056] According to a method for detecting gastrointestinal cancer provided by the present invention, after inputting the gastrointestinal image into a gastrointestinal site recognition model and obtaining the gastrointestinal site recognition result output by the gastrointestinal site recognition model, the method further includes:

[0057] Once the digestive tract sites indicated by the digestive tract site identification results have been examined, the unexamined digestive tract sites are identified from the target digestive tract sites.

[0058] Output the prompt information corresponding to the undetected digestive tract sites.

[0059] According to a method for detecting gastrointestinal cancer provided by the present invention, after inputting the gastrointestinal image into a gastrointestinal site recognition model and obtaining the gastrointestinal site recognition result output by the gastrointestinal site recognition model, the method further includes:

[0060] Based on the historical digestive tract site identification results of multiple acquired digestive tract images, and the acquisition time of the multiple acquired digestive tract images, the time for endoscopic withdrawal of multiple digestive tract sites is determined.

[0061] Based on the withdrawal time of the various digestive tract sites, the total withdrawal time of the endoscope used to acquire digestive tract images is determined.

[0062] According to a method for detecting gastrointestinal cancer provided by the present invention, before inputting a gastrointestinal image into an image classification model and obtaining the image classification result output by the image classification model, the method further includes:

[0063] The digestive tract image is input into the digestive tract mucosal cleanliness recognition model to obtain the digestive tract mucosal cleanliness recognition result output by the digestive tract mucosal cleanliness recognition model; the digestive tract mucosal cleanliness recognition result is used to indicate whether the digestive tract image is a cleanliness qualified image;

[0064] If the digestive tract image is determined to be an image with unacceptable cleanliness based on the digestive tract mucosal cleanliness identification result, a flushing prompt message is output to prompt flushing of the digestive tract area corresponding to the digestive tract image;

[0065] The digestive tract mucosal cleanliness identification model is trained based on the sample digestive tract image and the corresponding digestive tract mucosal cleanliness identification result label.

[0066] The present invention also provides a device for detecting gastrointestinal cancer, comprising:

[0067] An image classification module is used to input a digestive tract image into an image classification model and obtain the image classification result output by the image classification model; the image classification result is used to characterize the cancer identification result of the digestive tract image;

[0068] The semantic segmentation module is used to input the digestive tract image into the semantic segmentation model and obtain the semantic segmentation result output by the semantic segmentation model; the semantic segmentation result is used to indicate whether there is a lesion region in the digestive tract image.

[0069] The method determination module is used to determine the detection method of the digestive tract part corresponding to the digestive tract image based on the semantic segmentation result and the image classification result;

[0070] The prompt output module is used to output prompt information corresponding to the detection method when the detection method indicates that the digestive tract part corresponding to the digestive tract image needs to be detected.

[0071] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the detection method for digestive tract cancer as described above.

[0072] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the detection method for gastrointestinal cancer as described above.

[0073] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the detection method for digestive tract cancer as described above.

[0074] The present invention provides a method, apparatus, device, storage medium, and program product for detecting digestive tract cancer. It inputs a digestive tract image into an image classification model to obtain the image classification result output by the model, and inputs the digestive tract image into a semantic segmentation model to obtain the semantic segmentation result output by the model. The image classification result characterizes the cancer identification result of the digestive tract image, while the semantic segmentation result indicates the presence and location of lesion regions in the digestive tract image. Therefore, fusing semantic segmentation and image classification can improve the accuracy and sensitivity of digestive tract cancer identification. Furthermore, either the image classification result or the semantic segmentation result indicates... In cases where gastrointestinal cancer is present, a preliminary diagnosis can be made, thereby improving the sensitivity of gastrointestinal cancer identification, ensuring a certain biopsy positivity rate, and reducing false negatives. Furthermore, based on semantic segmentation and image classification results, the detection method for the corresponding gastrointestinal region in the image is determined, assisting doctors in deciding on the detection method, thus improving the efficiency of gastrointestinal cancer detection and increasing the sensitivity of gastrointestinal cancer identification, reducing false negatives. Moreover, when the detection method indicates that the corresponding gastrointestinal region in the image needs to be detected, the corresponding prompt information is output to guide the doctor to further detection methods, thereby improving the efficiency of gastrointestinal cancer detection, increasing the sensitivity of gastrointestinal cancer identification, and reducing false negatives. In summary, this invention can achieve efficient, highly sensitive, and highly accurate gastrointestinal cancer detection. Attached Figure Description

[0075] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0076] Figure 1 is one of the flowcharts of the method for detecting digestive tract cancer provided by the present invention.

[0077] Figure 2 is a second schematic flowchart of the method for detecting digestive tract cancer provided by the present invention.

[0078] Figure 3 is a schematic flowchart of the method for detecting digestive tract cancer provided by the present invention.

[0079] Figure 4 is a schematic flowchart of the method for detecting digestive tract cancer provided by the present invention.

[0080] Figure 5 is a schematic flowchart of the method for detecting digestive tract cancer provided by the present invention.

[0081] Figure 6 is a schematic flowchart of the method for detecting digestive tract cancer provided by the present invention.

[0082] Figure 7 is the seventh flowchart of the method for detecting digestive tract cancer provided by the present invention.

[0083] Figure 8 is a schematic diagram of the digestive tract cancer detection device provided by the present invention.

[0084] Figure 9 is a schematic diagram of the structure of the electronic device provided by the present invention.

[0085] Figure 10 is a schematic diagram illustrating the working principle of the early gastric cancer identification model provided by the present invention. Detailed Implementation

[0086] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0087] Currently, target detection is performed on digestive tract images to determine the presence of lesion areas. Suspected cancerous lesions are outlined with rectangular boxes to assist doctors in further diagnosis. However, the resolution of acquired digestive tract images varies; some are captured with close-up lenses, while others are captured with distant lenses. This reduces the accuracy of target detection using a single model, potentially leading to missed detections. In other words, the sensitivity of current digestive tract cancer identification is not high. Furthermore, target detection uses rectangular boxes to select lesion areas, but lesion areas may not be rectangular. Therefore, the accuracy of lesion localization is not high, requiring doctors to manually determine the lesion area within the rectangular box, resulting in low efficiency in current digestive tract cancer identification.

[0088] For example, this method acquires real-time images of the stomach during electronic chromoendoscopy; identifies and labels lesion areas based on a pre-built early gastric cancer recognition model; and determines whether overlapping lesion areas belong to the same lesion area. This approach utilizes target detection technology to simultaneously achieve image recognition and location localization. However, the lesion area is selected using a rectangular bounding box, which is not precise enough for accurate location localization, as the lesion area may not be a rectangular region, requiring the doctor to make further judgments within the bounding box. Furthermore, using only one target detection model makes it difficult to apply simultaneously to both close-up and long-range imaging, resulting in insufficient sensitivity for early cancer recognition and many missed detections, causing economic losses or even life-threatening situations for patients.

[0089] Furthermore, image classification of digestive tract images can help doctors determine whether an early-stage cancer is suspected, thus assisting in further diagnosis. However, relying solely on image classification results for digestive tract cancer identification is not very accurate, potentially leading to missed detections; in other words, the sensitivity of current digestive tract cancer identification is low. Moreover, image classification cannot pinpoint the lesion area, requiring doctors to manually locate the lesion within the digestive tract images, meaning that current digestive tract cancer identification is inefficient.

[0090] For example, a sample set of early gastric cancer images is obtained, and feature recognition is performed on each lesion image in the sample set to obtain the feature vector of each lesion image. Based on the sample set of early gastric cancer images and the feature vectors, a preset initial model is trained to obtain an early gastric cancer recognition model. However, this method only provides one way to recognize early gastric cancer and does not provide the function of locating the early cancer lesion.

[0091] To address the above problems, the present invention proposes the following embodiments. The method for detecting gastrointestinal cancer according to the present invention is described below with reference to Figures 1-7.

[0092] Figure 1 is a schematic flowchart of one of the methods for detecting gastrointestinal cancer provided by the present invention. As shown in Figure 1, the method for detecting gastrointestinal cancer includes the following steps 110, 120, 130 and 140.

[0093] Step 110: Input the digestive tract image into the image classification model to obtain the image classification result output by the image classification model.

[0094] Here, gastrointestinal cancer can refer to early-stage gastrointestinal cancer and advanced-stage gastrointestinal cancer, etc. This embodiment of the invention uses early-stage gastrointestinal cancer (early-stage malignant tumors of the gastrointestinal tract) as an example for illustration.

[0095] Here, the digestive tract image is an image obtained by acquiring data from a portion of the digestive tract. This digestive tract image can be obtained using a digestive tract endoscope.

[0096] For example, the digestive tract sites that can be captured by upper gastrointestinal endoscopy (gastroscopy) include: oropharynx, upper esophagus, middle esophagus, lower esophagus, esophagogastric junction, upper gastric body, posterior wall of upper gastric body, anterior wall of upper gastric body, greater curvature of upper gastric body, lesser curvature of upper gastric body, middle gastric body, posterior wall of middle gastric body, anterior wall of middle gastric body, greater curvature of middle gastric body, lesser curvature of middle gastric body, lower gastric body, posterior wall of lower gastric body, anterior wall of lower gastric body, greater curvature of lower gastric body, lesser curvature of lower gastric body, greater curvature of anterobody junction, gastric angle, posterior wall of gastric angle, anterior wall of gastric angle, gastric antrum, posterior wall of gastric antrum, anterior wall of gastric antrum, greater curvature of gastric antrum, lesser curvature of gastric antrum, pylorus, duodenal bulb, descending duodenum, gastric fundus, posterior wall of gastric fundus, anterior wall of gastric fundus, greater curvature of gastric fundus, lesser curvature of gastric fundus, and cardia. Colonoscopy can capture images of the following parts of the digestive tract: terminal ileum, appendiceal orifice, ileocecal valve, ascending colon, hepatic flexure of the colon, transverse colon, splenic flexure, descending colon, sigmoid colon, rectum, and colectomycorectal junction.

[0097] Furthermore, the acquired digestive tract images undergo image preprocessing. In one embodiment, the acquired digestive tract images are uniformly adjusted to a preset input size, for example, the preset input size is 512×512.

[0098] The image classification result is used to characterize the cancer identification result of the digestive tract image. That is, the image classification model is used to identify cancer in the digestive tract image. Specifically, the image classification model is used to classify the digestive tract image into different cancer probability values, thereby determining whether it is a suspected digestive tract cancer.

[0099] In one embodiment, the image classification model is trained based on a sample digestive tract image and the corresponding image classification result label.

[0100] In some embodiments, it is first determined whether the digestive tract image is a stained image. If the digestive tract image is stained, it is input into a first image classification model corresponding to the stained image to obtain the image classification result output by the first image classification model. If the digestive tract image is unstained, it is input into a second image classification model corresponding to the unstained image to obtain the image classification result output by the second image classification model. The stained image corresponds to a stained portion of the digestive tract, while the unstained image corresponds to an unstained portion. That is, considering the significant difference between stained and unstained images, different image classification models are set for image classification, thereby improving image classification accuracy, which in turn improves the accuracy of digestive tract cancer identification, and ultimately enhances the specificity of digestive tract cancer identification.

[0101] In other embodiments, regardless of whether the digestive tract image is a stained image, the digestive tract image is input into the image classification model to obtain the image classification result output by the image classification model.

[0102] Step 120: Input the digestive tract image into the semantic segmentation model to obtain the semantic segmentation result output by the semantic segmentation model.

[0103] The semantic segmentation result is used to indicate the presence and location of lesion regions in the digestive tract image. The semantic segmentation result is pixel-level, meaning the semantic segmentation model determines the category prediction of each pixel in the digestive tract image, i.e., whether each pixel is a lesion pixel. This allows for the identification of irregular lesion regions, thus improving the accuracy of lesion region localization. Specifically, it helps doctors determine the location of lesions, thereby improving both the accuracy and efficiency of digestive tract cancer identification.

[0104] Furthermore, this semantic segmentation model is an ensemble semantic segmentation model, which includes multiple different segmentation models. Compared to a single semantic segmentation model, the ensemble semantic segmentation model can improve the accuracy of semantic segmentation results, thereby improving the accuracy of gastrointestinal cancer identification. Even further, the ensemble semantic segmentation model is composed of multiple single semantic segmentation models based on the bagging method.

[0105] In one embodiment, the semantic segmentation model is trained based on a sample digestive tract image and the corresponding semantic segmentation result label.

[0106] In some embodiments, it is first determined whether the digestive tract image is a stained image. If the digestive tract image is stained, it is input into a first semantic segmentation model corresponding to the stained image to obtain the semantic segmentation result output by the first semantic segmentation model. If the digestive tract image is unstained, it is input into a second semantic segmentation model corresponding to the unstained image to obtain the semantic segmentation result output by the second semantic segmentation model. The stained image corresponds to a stained portion of the digestive tract, while the unstained image corresponds to an unstained portion. This approach considers the significant differences between stained and unstained images, thus setting different semantic segmentation models for semantic segmentation to improve semantic segmentation accuracy, thereby improving the accuracy of digestive tract cancer identification and ultimately enhancing the specificity of digestive tract cancer identification.

[0107] In other embodiments, regardless of whether the digestive tract image is a stained image, the digestive tract image is input into the semantic segmentation model to obtain the semantic segmentation result output by the semantic segmentation model.

[0108] Step 130: Based on the semantic segmentation result and the image classification result, determine the detection method for the digestive tract region corresponding to the digestive tract image.

[0109] Here, the testing methods include biopsy, staining, or no testing required. This testing method is a further testing method, i.e., a further diagnostic result.

[0110] It should be understood that image classification results are used to characterize the cancer identification results of digestive tract images, while semantic segmentation results are used to indicate the presence of lesion areas in digestive tract images and to locate the location and extent of lesion areas. Therefore, combining semantic segmentation and image classification can improve the accuracy of digestive tract cancer identification. Furthermore, if either the image classification result or the semantic segmentation result indicates the presence of digestive tract cancer, a preliminary judgment can be made that digestive tract cancer is present, thereby improving the sensitivity of digestive tract cancer identification, ensuring a certain biopsy positivity rate, and reducing missed detections. Moreover, based on the semantic segmentation result and the image classification result, the detection method for the corresponding digestive tract location in the digestive tract image can be determined, assisting doctors in judging the detection method and thus improving the detection efficiency of digestive tract cancer.

[0111] In some embodiments, it is first determined whether the digestive tract image is a stained image; if the digestive tract image is a stained image, the detection method of the digestive tract part corresponding to the digestive tract image can be determined based on the semantic segmentation result and the image classification result, which can be referred to in the following embodiments, and will not be described in detail here.

[0112] In other embodiments, regardless of whether the digestive tract image is a stained image, the image classification result includes a third classification result, a fourth classification result, or a fifth classification result, and step 130 includes:

[0113] If, based on the semantic segmentation result, it is determined that there is a lesion region in the digestive tract image, and the image classification result is the third classification result, then the detection method is determined to be biopsy.

[0114] If, based on the semantic segmentation results, it is determined that there are no lesion areas in the digestive tract image, the detection method is determined to be "no detection required".

[0115] If, based on the semantic segmentation result, it is determined that there is a lesion region in the digestive tract image, and the image classification result is the fourth classification result, and the digestive tract image is an unstained image, then the detection method is determined to be staining;

[0116] If, based on the semantic segmentation result, it is determined that there is a lesion region in the digestive tract image, and the image classification result is the fourth classification result, and the digestive tract image is a stained image, then the detection method is determined to be no detection required.

[0117] If, based on the semantic segmentation result, it is determined that there is a lesion region in the digestive tract image, and the image classification result is the fifth classification result, then the detection method is determined to be no detection required.

[0118] In this context, the digestive tract region corresponding to the stained image is stained, the digestive tract region corresponding to the unstained image is unstained, the cancer probability value represented by the third classification result is greater than the cancer probability value represented by the fourth classification result, and the cancer probability value represented by the fourth classification result is greater than the cancer probability value represented by the fifth classification result.

[0119] It should be noted that if the semantic segmentation result determines that there is no lesion area in the digestive tract image, the detection method is determined to be no detection required; if the semantic segmentation result determines that there is a lesion area in the digestive tract image, and the image classification result is the fifth classification result, the detection method is determined to be no detection required. The above two judgment logics are equivalent to determining that if the image classification result is the fifth classification result, the detection method is no detection required.

[0120] For example, as shown in Figure 2, the acquired digestive tract images are preprocessed, and then features are extracted from the preprocessed digestive tract images. These features are then input into a semantic segmentation model and an image classification model, respectively. Next, based on the semantic segmentation results, it is first determined whether there are lesion regions in the digestive tract images. If no lesion regions are found based on the semantic segmentation results, the detection method is set to "no detection required," and no prompt is given. If lesion regions are found based on the semantic segmentation results, further judgment is made based on the image classification results of the image classification model. If lesion regions are found based on the semantic segmentation results... If a lesion area is present in the image and the image classification result is a third category, the detection method is determined to be biopsy, and a biopsy prompt is given accordingly. If a lesion area is determined to be present in the digestive tract image based on semantic segmentation results, and the image classification result is a fourth category, it is first determined whether the digestive tract image is stained. If the digestive tract image is unstained, the detection method is determined to be stained, and a stained prompt is given accordingly. If the digestive tract image is stained, the detection method is determined to be no detection required, and no prompt is given accordingly. If a lesion area is determined to be present in the digestive tract image based on semantic segmentation results, and the image classification result is a fifth category, the detection method is determined to be no detection required, and no prompt is given accordingly.

[0121] Image classification models can output different probabilities. Based on this, if the probability output by the image classification model is greater than the biopsy probability threshold, the image classification result is a third-class result; if the probability output by the image classification model is less than or equal to the biopsy probability threshold but greater than the staining probability threshold, the image classification result is a fourth-class result; and if the probability output by the image classification model is less than or equal to the staining probability threshold, the image classification result is a fifth-class result. Here, the biopsy probability threshold is greater than the staining probability threshold. Therefore, the logic for determining the detection method varies depending on the output probability of the image classification model.

[0122] For example, as shown in Figure 3, the acquired digestive tract images are preprocessed, and then features are extracted from the preprocessed digestive tract images. These features are then input into a semantic segmentation model and an image classification model, respectively. Next, based on the semantic segmentation results, it is first determined whether there are lesion regions in the digestive tract images. If the semantic segmentation results indicate that there are no lesion regions, the detection method is set to "no detection required," and no prompt is given. If the semantic segmentation results indicate that there are lesion regions in the digestive tract images, further judgment is made based on the output probability of the image classification model. If the semantic segmentation results indicate that there are lesion regions in the digestive tract images, and the image classification model... If the model's output probability is greater than the biopsy probability threshold, the detection method is determined to be biopsy, and a biopsy prompt is given accordingly. If the semantic segmentation results indicate the presence of a lesion region in the digestive tract image, and the image classification model's output probability is not greater than the biopsy probability threshold, it is first determined whether the output probability is greater than the staining probability threshold. If the output probability is not greater than the staining probability threshold, the detection method is determined to be no detection required, and no prompt is given accordingly. If the output probability is greater than the staining probability threshold, it is first determined whether the digestive tract image is stained. If the digestive tract image is unstained, the detection method is determined to be staining, and a staining prompt is given accordingly. If the digestive tract image is stained, the detection method is determined to be no detection required, and no prompt is given accordingly.

[0123] In other embodiments, regardless of whether the digestive tract image is a stained image, the image classification result includes a third classification result, a fourth classification result, or a fifth classification result, and step 130 includes:

[0124] If, based on the semantic segmentation result, it is determined that there is a lesion region in the digestive tract image, or if the image classification result is the third classification result, the detection method is determined to be biopsy;

[0125] If, based on the semantic segmentation result, it is determined that there is no lesion region in the digestive tract image, and the image classification result is a fourth classification result, then the detection method is determined to be staining;

[0126] If, based on the semantic segmentation results, it is determined that there are no lesion areas in the digestive tract image, and the image classification result is the fifth category result, then the detection method is determined to be no detection required.

[0127] The cancer probability value represented by the third classification result is greater than the cancer probability value represented by the fourth classification result, and the cancer probability value represented by the fourth classification result is greater than the cancer probability value represented by the fifth classification result.

[0128] It should be understood that if the semantic segmentation results indicate the presence of a lesion area in the digestive tract image, or if the image classification result is a third-class classification result, the detection method is determined to be biopsy. That is, if either the image classification result or the semantic segmentation result indicates the presence of digestive tract cancer, a preliminary judgment can be made that digestive tract cancer exists, thus determining biopsy as the detection method. This improves the sensitivity of digestive tract cancer identification, ensures a certain biopsy positivity rate, and reduces false negatives. Conversely, if the semantic segmentation results indicate the absence of a lesion area in the digestive tract image, and the image classification result is a fifth-class classification result, the detection method is determined to be no detection required. That is, only if all results in the image classification result and the semantic segmentation result indicate the absence of digestive tract cancer, is the detection method determined to be no detection required. This improves the sensitivity of digestive tract cancer identification, ensures a certain biopsy positivity rate, and reduces false negatives.

[0129] For example, as shown in Figure 4, the acquired digestive tract image is preprocessed, and then features are extracted from the preprocessed digestive tract image. These features are then input into a semantic segmentation model and an image classification model, respectively. Next, based on the semantic segmentation results, it is first determined whether a lesion region exists in the digestive tract image. If a lesion region is found, the detection method is determined to be biopsy, and a biopsy prompt is given accordingly. If no lesion region is found, the subsequent judgment is made based on the image classification results of the image classification model. If the image classification result is a third category, the detection method is determined to be biopsy, and a biopsy prompt is given accordingly. If no lesion region is found in the digestive tract image based on the semantic segmentation results, and the image classification result is a fourth category, the detection method is determined to be staining, and a staining prompt is given accordingly. If no lesion region is found in the digestive tract image based on the semantic segmentation results, and the image classification result is a fifth category, the detection method is determined to be no detection required, and no prompt is given accordingly.

[0130] Image classification models can output different probabilities. Based on this, if the probability output by the image classification model is greater than the biopsy probability threshold, the image classification result is a third-class result; if the probability output by the image classification model is less than or equal to the biopsy probability threshold but greater than the staining probability threshold, the image classification result is a fourth-class result; and if the probability output by the image classification model is less than or equal to the staining probability threshold, the image classification result is a fifth-class result. Here, the biopsy probability threshold is greater than the staining probability threshold. Therefore, the logic for determining the detection method varies depending on the output probability of the image classification model.

[0131] For example, as shown in Figure 5, the acquired digestive tract image is preprocessed, and then features are extracted from the preprocessed digestive tract image. These features are then input into a semantic segmentation model and an image classification model, respectively. Next, based on the semantic segmentation results, it is first determined whether a lesion region exists in the digestive tract image. If a lesion region is found, the detection method is determined to be biopsy, and a biopsy prompt is given accordingly. If no lesion region is found, further judgment is made based on the output probability of the image classification model. If the output probability is greater than the biopsy probability threshold, the detection method is determined to be biopsy, and a biopsy prompt is given accordingly. If the output probability is not greater than the biopsy probability threshold, it is determined whether the output probability is greater than the staining probability threshold. If the output probability is greater than the staining probability threshold, the detection method is determined to be staining, and a staining prompt is given accordingly. If the output probability is not greater than the staining probability threshold, the detection method is determined to be no detection required, and no prompt is given accordingly.

[0132] Step 140: If the detection method indicates that the digestive tract region corresponding to the digestive tract image needs to be detected, output the prompt information corresponding to the detection method.

[0133] For example, if the detection method is biopsy, the corresponding prompt message will be a biopsy prompt message to encourage further biopsy; if the detection method is staining, the corresponding prompt message will be a staining prompt message to encourage further staining; if the detection method is no detection required, no prompt message will be output.

[0134] It should be understood that directly outputting the prompts corresponding to the detection method can prompt doctors to use further detection methods, assist doctors in making further diagnoses, and thus improve the detection efficiency of gastrointestinal cancers.

[0135] The digestive tract cancer detection method provided in this invention involves inputting a digestive tract image into an image classification model to obtain the image classification result output by the model, and inputting the digestive tract image into a semantic segmentation model to obtain the semantic segmentation result output by the model. The image classification result characterizes the cancer identification result of the digestive tract image, while the semantic segmentation result indicates the presence of lesion areas in the digestive tract image. Therefore, fusing semantic segmentation and image classification can improve the accuracy and sensitivity of digestive tract cancer identification. Furthermore, if either the image classification result or the semantic segmentation result indicates the presence of digestive tract cancer, the method will be effective. This invention can preliminarily determine the presence of gastrointestinal cancer, thereby improving the sensitivity of gastrointestinal cancer identification, ensuring a certain biopsy positive rate, and reducing false negatives. Furthermore, based on semantic segmentation and image classification results, it determines the detection method for the corresponding gastrointestinal region in the image, assisting doctors in choosing the appropriate detection method, thus improving the detection efficiency and sensitivity of gastrointestinal cancer identification, and reducing false negatives. When the detection method indicates that the corresponding gastrointestinal region in the image needs to be detected, it outputs corresponding prompts to guide doctors on further detection methods, thereby improving the detection efficiency and sensitivity of gastrointestinal cancer identification, and reducing false negatives. In summary, this invention can achieve efficient, highly sensitive, and highly accurate gastrointestinal cancer detection.

[0136] Based on any of the above embodiments, if the digestive tract image is a stained image, and the image classification result includes a first classification result or a second classification result, then step 130 includes:

[0137] If, based on the semantic segmentation result, it is determined that there is a lesion region in the digestive tract image, and the image classification result is the first classification result, then the detection method is determined to be biopsy.

[0138] If, based on the semantic segmentation result, it is determined that there is a lesion region in the digestive tract image, and the image classification result is the second classification result, then the detection method is determined to be no detection required.

[0139] If, based on the semantic segmentation results, it is determined that there are no lesion areas in the digestive tract image, the detection method is determined to be "no detection required".

[0140] In this context, the digestive tract region corresponding to the stained image has been stained; the cancer probability value represented by the first classification result is greater than the cancer probability value represented by the second classification result. For example, the first classification result is suspected cancer, and the second classification result is no cancer.

[0141] Considering that the digestive tract images are stained images, that is, the detection method of staining has been performed, it is only necessary to further determine whether a biopsy is needed. Based on this, it is only necessary to perform binary classification on the digestive tract images, that is, the image classification model is a binary classification model.

[0142] In some embodiments, a digestive tract image is input into a staining recognition model to obtain a staining recognition result output by the model. This staining recognition result indicates whether the digestive tract image is a stained image. Further, the staining recognition result can also indicate whether the digestive tract image is an electrostained image or a chemically stained image, where an electrostained image corresponds to electronic staining and a chemically stained image corresponds to chemical staining.

[0143] In one embodiment, the staining recognition model is a binary classification model, and the classification result includes stained images or unstained images. Further, the staining recognition model is a tri-class classification model, and the classification result includes electro-stained images, chemically stained images, or white light images.

[0144] In one embodiment, the staining recognition model is trained based on sample digestive tract images and their corresponding staining recognition result labels.

[0145] Considering that if the semantic segmentation results indicate the presence of lesions in the digestive tract image, and the image classification result is the first classification result, since the digestive tract image is already stained, i.e., the staining detection method has already been performed, the further detection method is determined to be biopsy.

[0146] Considering that if the semantic segmentation results indicate the presence of lesions in the digestive tract image, and the image classification result is a second classification result, since the digestive tract image is already stained, i.e., the staining detection method has already been performed, the further detection method is determined to be no detection required.

[0147] Considering that semantic segmentation models are based on pixel-level semantic segmentation, meaning that semantic segmentation results are more sensitive, if it is determined from the semantic segmentation results that there are no lesion areas in the digestive tract image, the detection method is directly determined to be no detection required.

[0148] If the digestive tract image is an unstained image, and the image classification result includes a third classification result, a fourth classification result, or a fifth classification result, then step 130 above includes:

[0149] If, based on the semantic segmentation result, it is determined that there is a lesion region in the digestive tract image, and the image classification result is the third classification result, then the detection method is determined to be biopsy.

[0150] If, based on the semantic segmentation result, it is determined that there is a lesion region in the digestive tract image, and the image classification result is the fourth classification result, then the detection method is determined to be staining;

[0151] If, based on the semantic segmentation result, it is determined that there is a lesion region in the digestive tract image, and the image classification result is the fifth classification result, then the detection method is determined to be no detection required.

[0152] If, based on the semantic segmentation results, it is determined that there are no lesion areas in the digestive tract image, the detection method is determined to be "no detection required".

[0153] In this context, the digestive tract portion corresponding to the unstained image is not stained; the cancer probability value represented by the third classification result is greater than the cancer probability value represented by the fourth classification result, and the cancer probability value represented by the fourth classification result is greater than the cancer probability value represented by the fifth classification result. For example, the third classification result is suspected cancer, the fourth classification result is low-probability cancer, and the fifth classification result is no cancer.

[0154] Considering that the digestive tract images are unstained images (white light images), that is, without the performance of staining detection, it is necessary to determine whether a biopsy is required and whether staining is required. Based on this, the digestive tract images need to be classified into three categories, that is, the image classification model is a three-category model.

[0155] Considering that if the semantic segmentation results indicate the presence of lesions in the digestive tract image, and the image classification result is a fourth category result, since the digestive tract image is an unstained image, i.e. no staining detection method was performed, the further detection method is determined to be staining.

[0156] Considering that semantic segmentation models are based on pixel-level semantic segmentation, meaning that semantic segmentation results are more sensitive, if it is determined from the semantic segmentation results that there are no lesion areas in the digestive tract image, the detection method is directly determined to be no detection required.

[0157] For example, as shown in Figure 6, the collected digestive tract images are preprocessed, then the preprocessed digestive tract images are feature extracted, and then the image features of the digestive tract images are input into the staining recognition model. If the digestive tract image is determined to be a white light image (unstained image) based on the staining recognition model, the image features are input into the semantic segmentation model and image classification model corresponding to the unstained image, respectively. First, the presence of lesion areas in the digestive tract image is determined based on the semantic segmentation results. If no lesion areas are found in the digestive tract image based on the semantic segmentation results, the detection method is determined to be "no detection required," and no prompt is given. If lesion areas are found in the digestive tract image based on the semantic segmentation results, subsequent judgment is made based on the image classification results of the image classification model. If lesion areas are found in the digestive tract image based on the semantic segmentation results, and the image classification result is a third-class result, the detection method is determined to be "biopsy," and a biopsy prompt is given accordingly. If lesion areas are found in the digestive tract image based on the semantic segmentation results, and the image classification result is a fourth-class result, the detection method is determined to be "staining," and a staining prompt is given accordingly. If lesion areas are found in the digestive tract image based on the semantic segmentation results, and the image classification result is a fifth-class result, the detection method is determined to be "no detection required," and no prompt is given accordingly. If the gastrointestinal image is determined to be stained based on the staining recognition model, the image features are input into the semantic segmentation model and image classification model corresponding to the stained image, respectively. First, the presence of lesion areas in the gastrointestinal image is determined based on the semantic segmentation results. If no lesion areas are found in the gastrointestinal image based on the semantic segmentation results, the detection method is determined to be no detection required, and no prompt is given. If lesion areas are found in the gastrointestinal image based on the semantic segmentation results, subsequent judgment is made based on the image classification results of the image classification model. If lesion areas are found in the gastrointestinal image based on the semantic segmentation results, and the image classification result is the first classification result, the detection method is determined to be biopsy, and a biopsy prompt is given accordingly. If lesion areas are found in the gastrointestinal image based on the semantic segmentation results, and the image classification result is the second classification result, the detection method is determined to be no detection required, and no prompt is given accordingly.

[0158] Image classification models can output different probabilities. Based on this, if the probability output by the image classification model is greater than the biopsy probability threshold, the image classification result is either the first or third classification result. If the probability output by the image classification model is less than or equal to the biopsy probability threshold, the image classification result is the second classification result. If the probability output by the image classification model is less than or equal to the biopsy probability threshold but greater than the staining probability threshold, the image classification result is the fourth classification result. If the probability output by the image classification model is less than or equal to the staining probability threshold, the image classification result is the fifth classification result. Here, the biopsy probability threshold is greater than the staining probability threshold. Therefore, the logic for determining the detection method varies depending on the output probability of the image classification model.

[0159] For example, as shown in Figure 7, the collected digestive tract images are preprocessed, then the preprocessed digestive tract images are feature extracted, and then the image features of the digestive tract images are input into the staining recognition model. If the staining recognition model determines that the digestive tract image is a white light image (unstained image), the image features are input into the semantic segmentation model and image classification model corresponding to the unstained image, respectively. First, based on the semantic segmentation result, it is determined whether there is a lesion region in the digestive tract image. If the semantic segmentation result determines that there is no lesion region in the digestive tract image, the detection method is determined to be no detection required, and no prompt is given accordingly. If the semantic segmentation result determines that there is a lesion region in the digestive tract image, further judgment is made based on the output probability of the image classification model. If the semantic segmentation result determines that there is a lesion region in the digestive tract image, and the output probability is greater than the biopsy probability threshold, the detection method is determined to be biopsy, and a biopsy prompt is given accordingly. If the semantic segmentation result determines that there is a lesion region in the digestive tract image, and the output probability is not greater than the biopsy probability threshold, it is determined whether the output probability is greater than the staining probability threshold. If the output probability is greater than the staining probability threshold, the detection method is determined to be staining, and a staining prompt is given accordingly. If the output probability is not greater than the staining probability threshold, the detection method is determined to be no detection required, and no prompt is given accordingly. If the gastrointestinal image is determined to be stained based on the staining recognition model, the image features are input into the semantic segmentation model and image classification model corresponding to the stained image, respectively. First, the presence of lesion areas in the gastrointestinal image is determined based on the semantic segmentation results. If no lesion areas are found in the gastrointestinal image based on the semantic segmentation results, the detection method is determined to be no detection required, and no prompt is given accordingly. If lesion areas are found in the gastrointestinal image based on the semantic segmentation results, subsequent judgments are made based on the output probability of the image classification model. If lesion areas are found in the gastrointestinal image based on the semantic segmentation results, and the output probability is greater than the biopsy probability threshold, the detection method is determined to be biopsy, and a biopsy prompt is given accordingly. If the output probability is not greater than the biopsy probability threshold, the detection method is determined to be no detection required, and no prompt is given accordingly.

[0160] The digestive tract cancer detection method provided in this invention, through the above-mentioned approach, first determines whether the digestive tract image is a stained image, and then determines the specific detection method accordingly. This improves the efficiency and accuracy of detection method determination, thereby increasing the detection efficiency and sensitivity of digestive tract cancer identification. Simultaneously, the above-mentioned detection method determination logic improves the accuracy of detection method determination. Furthermore, the detection method is determined to be biopsy only when the presence of a lesion region in the digestive tract image is determined based on semantic segmentation results, and the image classification result is either a first or third classification result. This reduces false positives and improves the specificity of cancer identification. Even if the presence of a lesion region in the digestive tract image is determined based on semantic segmentation results, if the image classification result is a second or fifth classification result, the detection method is determined to be "no detection required," thus reducing false positives and improving the specificity of cancer identification. Conversely, if the absence of a lesion region in the digestive tract image is determined based on semantic segmentation results, the detection method is determined to be "no detection required," without needing to determine the image classification result. This further reduces false positives and improves the specificity of cancer identification.

[0161] Based on any of the above embodiments, considering the significant difference between stained and unstained images, if the digestive tract image is a stained image, step 110 includes: inputting the digestive tract image into a first image classification model corresponding to the stained image to obtain the image classification result output by the first image classification model; if the digestive tract image is an unstained image, step 110 includes: inputting the digestive tract image into a second image classification model corresponding to the unstained image to obtain the image classification result output by the second image classification model.

[0162] Considering the significant difference between stained and unstained images, if the digestive tract image is a stained image, step 120 above includes: inputting the digestive tract image into a first semantic segmentation model corresponding to the stained image to obtain the semantic segmentation result output by the first semantic segmentation model; if the digestive tract image is an unstained image, step 120 above includes: inputting the digestive tract image into a second semantic segmentation model corresponding to the unstained image to obtain the semantic segmentation result output by the second semantic segmentation model.

[0163] The digestive tract region corresponding to the stained image is stained, while the digestive tract region corresponding to the unstained image is unstained.

[0164] In some embodiments, a digestive tract image is input into a staining recognition model to obtain a staining recognition result output by the staining recognition model, which is used to indicate whether the digestive tract image is a stained image.

[0165] Furthermore, the staining recognition result can also indicate whether the digestive tract image is an electrostained image or a chemically stained image. Considering the differences between electrostained and chemically stained images, based on this, if the digestive tract image is an electrostained image, step 110 above includes: inputting the digestive tract image into a first image classification model corresponding to the electrostained image to obtain the image classification result output by the first image classification model; if the digestive tract image is a chemically stained image, step 110 above includes: inputting the digestive tract image into a first image classification model corresponding to the chemically stained image to obtain the image classification result output by the first image classification model. If the digestive tract image is an electrostained image, step 120 above includes: inputting the digestive tract image into a first semantic segmentation model corresponding to the electrostained image to obtain the semantic segmentation result output by the first semantic segmentation model; if the digestive tract image is a chemically stained image, step 120 above includes: inputting the digestive tract image into a first semantic segmentation model corresponding to the chemically stained image to obtain the semantic segmentation result output by the first semantic segmentation model. Therefore, different image classification models are set up to classify the images, thereby improving the accuracy of image classification, which in turn improves the accuracy of gastrointestinal cancer identification and ultimately enhances the sensitivity of gastrointestinal cancer identification.

[0166] The digestive tract cancer detection method provided in this invention improves image classification accuracy by setting different image classification models to classify stained or unstained images, thereby improving the accuracy of digestive tract cancer identification and ultimately enhancing the sensitivity of digestive tract cancer identification. Furthermore, it improves semantic segmentation accuracy by setting different semantic segmentation models to perform semantic segmentation on stained or unstained images, thereby improving the accuracy of semantic segmentation, thereby enhancing the accuracy of digestive tract cancer identification and ultimately enhancing the sensitivity of digestive tract cancer identification.

[0167] Based on any of the above embodiments, in this method, the semantic segmentation model is an integrated semantic segmentation model, which includes multiple different segmentation models.

[0168] For any pixel in the digestive tract image, the integrated semantic segmentation model determines multiple category prediction results for that pixel. Each category prediction result includes either a lesion pixel or a non-lesion pixel. If any of the multiple category prediction results indicates a lesion pixel, then the final category prediction result for that pixel is determined to be a lesion pixel. If all of the multiple category prediction results indicate a non-lesion pixel, then the final category prediction result for that pixel is determined to be a non-lesion pixel. In other words, this embodiment of the invention uses the union of results from multiple single semantic segmentation models to determine the final result.

[0169] Existing bagging ensemble learning methods use voting to determine the majority result. This invention, considering the greater harm of missed detections of gastrointestinal cancer, designates a positive result if any one of the predicted categories is positive. That is, if any category prediction among multiple categories identifies a lesion pixel, the final category prediction for that pixel is determined as a lesion pixel, thereby improving the sensitivity of gastrointestinal cancer identification and reducing missed detections.

[0170] The digestive tract cancer detection method provided in this invention uses an ensemble semantic segmentation model, which includes multiple different segmentation models. Compared to a single semantic segmentation model, the ensemble semantic segmentation model can improve the accuracy of semantic segmentation results, thereby improving the accuracy of digestive tract cancer identification. For any pixel in a digestive tract image, the ensemble semantic segmentation model is used to determine multiple category prediction results for the pixel. Each category prediction result includes lesion pixels or non-lesion pixels. If any category prediction result among the multiple category prediction results is a lesion pixel, then the final category prediction result of the pixel is determined to be a lesion pixel. If all category prediction results among the multiple category prediction results are non-lesion pixels, then the final category prediction result of the pixel is determined to be a non-lesion pixel, thereby improving the sensitivity of digestive tract cancer identification and reducing missed detections.

[0171] Based on any of the above embodiments, before step 110, the method further includes: inputting the digestive tract image into a feature extraction model to obtain image features output by the feature extraction model.

[0172] Accordingly, step 110 includes: inputting the image features into an image classification model to obtain the image classification result output by the image classification model.

[0173] Accordingly, step 120 includes: inputting the image features into the semantic segmentation model to obtain the semantic segmentation result output by the semantic segmentation model.

[0174] In other words, the semantic segmentation model and the image classification model share a single feature extraction model. For example, the semantic segmentation model and the image classification model share a backbone network, and two detection heads are connected to the backbone network to achieve semantic segmentation and image classification respectively.

[0175] Furthermore, the feature extraction model is used to extract high-level abstract features from digestive tract images. High-level abstract features have stronger semantic expressive power and stronger robustness than low-level features, and can be applied to various complex digestive tract image acquisition scenarios, thus achieving higher accuracy and better generalization ability. In particular, it performs better for image classification models, ultimately improving the accuracy of digestive tract cancer identification.

[0176] Furthermore, this feature extraction model can also be shared by other models such as staining recognition model, digestive tract site recognition model, and digestive tract mucosal cleanliness recognition model.

[0177] The digestive tract cancer detection method provided in this embodiment of the invention uses a semantic segmentation model and an image classification model to share a single feature extraction model, thereby effectively reducing the computational load of model inference, thus improving the detection efficiency of digestive tract cancer, and reducing the storage space occupied by the model, making it suitable for devices with limited hardware resources.

[0178] Based on any of the above embodiments, the image classification result includes a target classification result or several non-target classification results, wherein the cancer probability values ​​represented by the several non-target classification results are all less than the cancer probability values ​​represented by the target classification result.

[0179] Here, the number of non-target classification results can be one or more. For example, when the digestive tract image is a stained image, the target classification result is the first classification result, and the non-target classification results include the second classification result, where the cancer probability value represented by the first classification result is greater than that represented by the second classification result; when the digestive tract image is an unstained image, the target classification result is the third classification result, and the non-target classification results include the fourth and fifth classification results, where the cancer probability value represented by the third classification result is greater than that represented by the fourth classification result, and the cancer probability value represented by the fourth classification result is greater than that represented by the fifth classification result.

[0180] Accordingly, considering that the image classification result is the target classification result, i.e., a high probability of cancer, the semantic segmentation model fails to identify the lesion region, thus requiring doctors to further locate the lesion region, leading to a decrease in the detection efficiency of gastrointestinal cancer; based on this, before step 130 above, the method further includes:

[0181] If, based on the semantic segmentation result, it is determined that there is no lesion region in the digestive tract image, and the image classification result is the target classification result, the segmentation threshold of the semantic segmentation model is adjusted to obtain and output the lesion region in the digestive tract image.

[0182] Specifically, for any pixel in the digestive tract image, the semantic segmentation model determines the category prediction result of the pixel based on the segmentation threshold, and the category prediction result of the pixel includes lesion pixels or non-lesion pixels.

[0183] For example, in cases where an image classification model identifies a suspected cancerous area but the semantic segmentation model does not output the lesion region, the threshold of the semantic segmentation model is dynamically adjusted. By dynamically adjusting the segmentation threshold of the semantic segmentation model, the lesion region is output, allowing doctors to locate and analyze the lesion region.

[0184] The digestive tract cancer detection method provided in this embodiment of the invention, through the above-described method, when the image classification result is the target classification result, i.e., a high cancer probability, even if the semantic segmentation model fails to identify the lesion area, can adjust the segmentation threshold of the semantic segmentation model to obtain and output the lesion area in the digestive tract image, assisting doctors in further accurately locating the lesion area, thereby improving the detection efficiency of digestive tract cancer, increasing the sensitivity of digestive tract cancer identification, and reducing missed detections.

[0185] Based on any of the above embodiments, in this method, adjusting the segmentation threshold of the semantic segmentation model includes:

[0186] The lowest segmentation threshold is set to 0, and the highest segmentation threshold is set as the current segmentation threshold of the semantic segmentation model.

[0187] Determine the sum of the lowest segmentation threshold and the highest segmentation threshold, and adjust the segmentation threshold of the semantic segmentation model to half of the sum.

[0188] The semantic segmentation result is updated based on the adjusted segmentation threshold;

[0189] If, based on the updated semantic segmentation results, it is determined that the proportion of lesion regions in the digestive tract image is greater than the first preset proportion, the minimum segmentation threshold is updated to the adjusted segmentation threshold, and the step of determining the sum of the minimum segmentation threshold and the maximum segmentation threshold and adjusting the segmentation threshold of the semantic segmentation model to half of the sum is returned.

[0190] If, based on the updated semantic segmentation results, it is determined that the proportion of lesion regions in the digestive tract image is less than the second preset proportion, the highest segmentation threshold is updated to the adjusted segmentation threshold, and the step of determining the sum of the lowest segmentation threshold and the highest segmentation threshold and adjusting the segmentation threshold of the semantic segmentation model to half of the sum is returned.

[0191] Specifically, for any pixel in the digestive tract image, the semantic segmentation model outputs a predicted value for the pixel. If the predicted value of the pixel is greater than or equal to the segmentation threshold, the category prediction result of the pixel is determined to be a lesion pixel. The proportion of the lesion region is determined based on the ratio of the lesion region in the digestive tract image to the area of ​​the digestive tract image, and the first preset proportion is greater than the second preset proportion.

[0192] Here, the highest segmentation threshold is greater than the lowest segmentation threshold. Both the highest and lowest segmentation thresholds can be updated dynamically.

[0193] Considering that the current segmentation threshold of the semantic segmentation model does not output the lesion region, while if the segmentation threshold is 0, the lesion region output by the semantic segmentation model is the entire digestive tract image, based on this, the initial minimum segmentation threshold is set to 0, and the initial maximum segmentation threshold is the current segmentation threshold of the semantic segmentation model.

[0194] Here, the first and second preset percentages can be set according to the actual situation. For example, according to statistical analysis, the lesion area in a positive digestive tract image generally accounts for 5%-80% of the entire positive digestive tract image. Based on this, the first preset percentage is 80%, and the second preset percentage is 5%.

[0195] For example, assuming the current segmentation threshold of the semantic segmentation model is threshold_init, the first preset percentage is 80%, and the second preset percentage is 5%, the segmentation threshold of the semantic segmentation model is adjusted within the open interval (0, threshold_init) until the semantic segmentation model outputs the lesion region. Furthermore, using a binary search approach can improve the accuracy and rationality of lesion region determination. Specifically, the steps are as follows: First, set the lowest segmentation threshold threshold_low = 0 and the highest segmentation threshold threshold_high = threshold_init. Second, set the segmentation threshold of the semantic segmentation model to threshold = (threshold_low + threshold_high) / 2. Third, based on the adjusted segmentation threshold, update the semantic segmentation result, i.e., obtain the lesion region according to the new segmentation threshold and determine the area ratio P of the lesion region to the entire digestive tract image. Fourth, if 0.05 ≤ P ≤ 0.80, determine the current threshold as a reasonable segmentation threshold and output the lesion region corresponding to this segmentation threshold. If P > 0.80, then threshold_low = threshold, and return to step two, until the finally obtained lesion region satisfies 0.05 ≤ P ≤ 0.80. If P < 0.05, then threshold_high = threshold, and return to step two, until the finally obtained lesion region satisfies 0.05 ≤ P ≤ 0.80.

[0196] The digestive tract cancer detection method provided in this invention determines the sum of the minimum and maximum segmentation thresholds through the above-described method, and adjusts the segmentation threshold of the semantic segmentation model to half of the sum. This allows for accurate and dynamic adjustment of the segmentation threshold, thereby improving the sensitivity of lesion region localization and thus increasing the detection rate of digestive tract cancer. Furthermore, it ensures that the proportion of lesion region is between the first and second preset proportions, further improving the accuracy of lesion region localization and thus further improving the detection accuracy of digestive tract cancer.

[0197] Based on any of the above embodiments, the method further includes:

[0198] The digestive tract image is input into the digestive tract location recognition model to obtain the digestive tract location recognition result output by the digestive tract location recognition model;

[0199] Output the prompt information corresponding to the identification result of the digestive tract part to indicate the digestive tract part currently being examined.

[0200] The digestive tract site identification model is trained based on the sample digestive tract image and the corresponding digestive tract site identification result label.

[0201] For example, if the digestive tract location identification result is the upper anterior wall of the stomach body, the corresponding prompt message can be "The digestive tract location currently being examined is the upper anterior wall of the stomach body".

[0202] Furthermore, the digestive tract site recognition model can share a feature extraction model with the image classification model and the semantic segmentation model. That is, the image features of the digestive tract image are input into the digestive tract site recognition model to obtain the digestive tract site recognition result output by the digestive tract site recognition model.

[0203] The digestive tract cancer detection method provided in this invention inputs a digestive tract image into a digestive tract site recognition model to obtain the digestive tract site recognition result output by the digestive tract site recognition model; and outputs prompt information corresponding to the digestive tract site recognition result to indicate the digestive tract site currently being examined, thereby eliminating the need for doctors to further determine the digestive tract site to be examined and improving the detection efficiency of digestive tract cancer.

[0204] Based on any of the above embodiments, after inputting the digestive tract image into the digestive tract location recognition model and obtaining the digestive tract location recognition result output by the digestive tract location recognition model, the method further includes:

[0205] Once the digestive tract sites indicated by the digestive tract site identification results have been examined, the unexamined digestive tract sites are identified from the target digestive tract sites.

[0206] Output the prompt information corresponding to the undetected digestive tract sites.

[0207] In this embodiment of the invention, a doctor can determine whether the digestive tract part indicated by the digestive tract part identification result has been examined, or the examination can be determined when the digestive tract endoscope no longer takes pictures of the digestive tract part. Of course, other methods can also be used to determine whether the digestive tract part has been examined, which are not specifically limited here.

[0208] Here, the target digestive tract site set includes multiple sites of the digestive tract to be examined.

[0209] In one embodiment, the target set of digestive tract sites can be pre-defined, i.e., the digestive tract sites to be examined are pre-defined. In another embodiment, the target set of digestive tract sites includes all digestive tract sites. In yet another embodiment, the target set of digestive tract sites can be updated in real time, such as deleting a digestive tract site after it has been examined.

[0210] Here, the output displays a prompt message for any unchecked parts of the digestive tract, indicating which parts still need to be checked. The number of unchecked parts can be one or more; if multiple parts are unchecked, all unchecked parts can be displayed, thus indicating any missed areas.

[0211] The digestive tract cancer detection method provided in this invention involves inputting a digestive tract image into a digestive tract site recognition model to obtain digestive tract site recognition results output by the model. Once the digestive tract sites indicated by the recognition results have been examined, unexamined digestive tract sites are identified from the target digestive tract site set. Corresponding prompts for the unexamined digestive tract sites are output to indicate the currently unexamined sites, thus eliminating the need for further medical evaluation of these sites, improving the efficiency of digestive tract cancer detection, avoiding missed detections of some digestive tract sites, and enhancing the comprehensiveness of digestive tract cancer detection.

[0212] Based on any of the above embodiments, after inputting the digestive tract image into the digestive tract location recognition model and obtaining the digestive tract location recognition result output by the digestive tract location recognition model, the method further includes:

[0213] Based on the historical digestive tract site identification results of multiple acquired digestive tract images, and the acquisition time of the multiple acquired digestive tract images, the time for endoscopic withdrawal of multiple digestive tract sites is determined.

[0214] Based on the withdrawal time of the various digestive tract sites, the total withdrawal time of the endoscope used to acquire digestive tract images is determined.

[0215] It should be noted that during the digestive tract examination, each part of the digestive tract is examined sequentially, that is, images of each part of the digestive tract are acquired sequentially. Based on this, the historical digestive tract part identification results of multiple acquired digestive tract images, as well as the acquisition time of multiple acquired digestive tract images, can determine the time of entry into a digestive tract part and the time of exit from that digestive tract part. Thus, the time of withdrawal from the endoscope for that digestive tract part can be determined based on these two times.

[0216] Here, the total withdrawal time is greater than or equal to the sum of the withdrawal times of multiple digestive tract sites.

[0217] In one embodiment, if the endoscope is a gastroscope, the total withdrawal time refers to the time it takes for the endoscope to be withdrawn from the descending part of the duodenum to the outside of the body. If the endoscope is a colonoscope, the total withdrawal time refers to the time it takes for the endoscope to be withdrawn from the ileocecal region, which refers to the area of ​​the terminal ileum, the appendix orifice, and the ileocecal valve.

[0218] For example, the withdrawal time for multiple digestive tract sites includes the withdrawal time for the ascending colon, transverse colon, descending colon, and sigmoid colon / rectum. The total withdrawal time is the sum of the withdrawal times for the ascending colon, transverse colon, descending colon, and sigmoid colon / rectum.

[0219] Furthermore, the time required to withdraw the endoscope from any part of the digestive tract should be reduced by the time spent on biopsy, staining, or surgery. Based on this, it is necessary to determine the biopsy time, staining time, or surgery time for that part of the digestive tract.

[0220] After obtaining the total withdrawal time and the withdrawal time for multiple digestive tract sites, an examination quality control report can be generated based on at least one of the two for subsequent reading by doctors or patients.

[0221] The method for detecting gastrointestinal cancer provided in this invention determines the withdrawal time of multiple gastrointestinal sites based on the historical gastrointestinal site identification results of multiple acquired gastrointestinal images and the acquisition time of the multiple acquired gastrointestinal images; based on the withdrawal time of multiple gastrointestinal sites, the total withdrawal time of the endoscope is automatically monitored, thereby improving the detection quality of gastrointestinal cancer.

[0222] Based on any of the above embodiments, prior to step 110, the method further includes:

[0223] The digestive tract image is input into the digestive tract mucosal cleanliness recognition model to obtain the digestive tract mucosal cleanliness recognition result output by the digestive tract mucosal cleanliness recognition model; the digestive tract mucosal cleanliness recognition result is used to indicate whether the digestive tract image is a cleanliness qualified image;

[0224] If the digestive tract image is determined to be an image with unacceptable cleanliness based on the digestive tract mucosal cleanliness identification result, a flushing prompt message is output to prompt flushing of the digestive tract area corresponding to the digestive tract image.

[0225] The digestive tract mucosal cleanliness identification model is trained based on the sample digestive tract image and the corresponding digestive tract mucosal cleanliness identification result label.

[0226] Here, the mucosal cleanliness of the digestive tract area corresponding to the cleanliness of the image is qualified, that is, the digestive tract image meets the input requirements of the semantic segmentation model and the image classification model.

[0227] It is understandable that after rinsing the digestive tract area corresponding to the digestive tract image, a new digestive tract image of the rinsed digestive tract area is acquired, and then the detection method for digestive tract cancer is re-executed based on the digestive tract image.

[0228] Furthermore, the digestive tract mucosal cleanliness recognition model can share a feature extraction model with the image classification model and the semantic segmentation model. That is, the image features of the digestive tract image are input into the digestive tract mucosal cleanliness recognition model to obtain the digestive tract mucosal cleanliness recognition result output by the digestive tract mucosal cleanliness recognition model.

[0229] The digestive tract cancer detection method provided in this invention inputs a digestive tract image into a digestive tract mucosal cleanliness recognition model to obtain a digestive tract mucosal cleanliness recognition result output by the model. This result indicates whether the digestive tract image is clean enough. If the image is determined to be unclean based on the result, a flushing prompt is output to suggest flushing the corresponding digestive tract area. This ensures that clean images are input into the image classification and semantic segmentation models, guaranteeing the use of clean images for digestive tract cancer detection and improving detection accuracy. It also assists doctors in determining whether flushing is necessary, improving detection efficiency and quality.

[0230] Based on any of the above embodiments, the method further includes:

[0231] The images acquired by the endoscopic examination are input into the in vivo-in vitro recognition model to obtain the in vivo-in vitro recognition result output by the in vivo-in vitro recognition model. This in vivo-in vitro recognition result is used to indicate whether the input image is an in vivo image or an in vitro image.

[0232] Based on the in vivo and in vitro identification results of multiple images, the total examination time of the digestive tract is determined.

[0233] The in vivo and in vitro recognition model is trained based on sample images and their corresponding in vivo and in vitro recognition result labels.

[0234] Here, the total examination time is the time from the moment the endoscope enters the body from outside to the moment it leaves the body.

[0235] In one embodiment, if the endoscope is a gastroscope, the total examination time refers to the time from the moment the endoscope enters the oral cavity from outside the body to the moment it leaves the body. If the endoscope is a colonoscope, the total examination time refers to the time from the moment the endoscope enters the anus from outside the body to the moment it leaves the body.

[0236] Furthermore, the total examination time should be reduced by the time spent on biopsy, staining, or surgery at each digestive tract site. Based on this, it is necessary to determine the biopsy time or staining or surgery time for each digestive tract site.

[0237] After obtaining the total examination time, an examination quality control report can be generated based on at least one of the two results for subsequent reading by doctors or patients.

[0238] The method for detecting digestive tract cancer provided in this embodiment of the invention automatically monitors the total examination time through the above-described manner, thereby improving the detection quality of digestive tract cancer.

[0239] Based on any of the above embodiments, considering that colonoscopy also requires monitoring the reach of the ileocecal junction, when the endoscope is a colonoscope, after inputting the digestive tract image into the digestive tract location recognition model and obtaining the digestive tract location recognition result output by the digestive tract location recognition model, the method further includes:

[0240] Based on the identification results of the digestive tract location, it is determined whether the ileocecal region has been reached.

[0241] It should be understood that determining the location to the terminal ileum, appendix orifice, or ileocecal valve based on the digestive tract location identification results indicates that the ileocecal region has been reached.

[0242] The method for detecting digestive tract cancer provided in this embodiment of the invention automatically monitors whether the ileocecal junction has been reached through the above-described manner, thereby improving the detection quality of digestive tract cancer.

[0243] It should be noted that the above embodiments are based on the assumption that the ideal AI-assisted gastrointestinal tumor examination and diagnosis technology is: (1) following the guidelines and procedures, performing biopsies or other indications on suspected tumor lesions under ordinary white light endoscopy, and staining indications on low-probability tumor lesions, and then using AI to identify suspected tumor lesions after the physician performs chemical staining or electronic staining. This ensures high specificity while improving the sensitivity of detecting tumor lesions, especially early cancer lesions; (2) with the help of intelligent targeted biopsy that locks the boundaries of the lesion area, the positive rate of biopsy of suspected lesions is improved from the biopsy level.

[0244] Based on the above embodiments, an examination quality control report (such as an upper gastrointestinal endoscopic tumor screening quality control report) can be generated for subsequent reading by doctors or patients. For example, when a biopsy prompt is output, the examination quality control report includes an image of the gastrointestinal tract corresponding to the biopsy prompt and a record of whether the doctor performed a biopsy; when a staining prompt is output, the examination quality control report includes an image of the gastrointestinal tract corresponding to the staining prompt and a record of whether the doctor stained it; when a flushing prompt is output, the examination quality control report includes an image of the gastrointestinal tract corresponding to the flushing prompt and a record of whether the doctor flushed it; when a missed detection prompt is output, the examination quality control report includes a record of whether the missed gastrointestinal tract site was re-examined; furthermore, the examination quality control report may also include the total examination time and the time for withdrawal of the endoscope.

[0245] Based on the above embodiments, the principle for selecting hyperparameters during the training phase of each model is to maximize sensitivity while ensuring specificity is not less than 90%. For ease of understanding, assume that the number of positive samples in the test set is P, and the number of negative samples is F; the number of true positive samples determined by the semantic segmentation result of the semantic segmentation model is STP, and the number of false positive samples is SFP; the number of true positive samples determined by the image classification result of the image classification model is CTP, and the number of false positive samples is CFP; based on this, the sensitivity is (STP+CTP) / P, and the specificity is (F-SFP-CFP) / F.

[0246] Based on the above embodiments, the present invention can significantly improve the sensitivity of gastrointestinal cancer identification while ensuring a certain biopsy positivity rate, and can also achieve targeted localization of suspected lesion areas. In addition, it can provide auxiliary diagnostic suggestions of biopsy prompts, staining prompts and no prompts according to different output probabilities of the image classification model, thereby effectively improving the effect of gastrointestinal cancer identification and diagnosis.

[0247] The detection device for digestive tract cancer provided by the present invention will be described below. The detection device for digestive tract cancer described below can be referred to in correspondence with the detection method for digestive tract cancer described above.

[0248] Figure 8 is a schematic diagram of the digestive tract cancer detection device provided by the present invention. As shown in Figure 8, the digestive tract cancer detection device includes an image classification module 810, a semantic segmentation module 820, a mode determination module 830, and a prompt output module 840.

[0249] The image classification module 810 is used to input a digestive tract image into an image classification model and obtain the image classification result output by the image classification model; the image classification result is used to characterize the cancer identification result of the digestive tract image.

[0250] The semantic segmentation module 820 is used to input the digestive tract image into the semantic segmentation model and obtain the semantic segmentation result output by the semantic segmentation model; the semantic segmentation result is used to indicate whether there is a lesion region in the digestive tract image.

[0251] The method determination module 830 is used to determine the detection method of the digestive tract part corresponding to the digestive tract image based on the semantic segmentation result and the image classification result.

[0252] The prompt output module 840 is used to output prompt information corresponding to the detection method when the detection method indicates that the digestive tract part corresponding to the digestive tract image needs to be detected.

[0253] Figure 9 illustrates a schematic diagram of the physical structure of an electronic device. As shown in Figure 9, the electronic device may include: a processor 910, a communication interface 920, a memory 930, and a communication bus 940. The processor 910, communication interface 920, and memory 930 communicate with each other via the communication bus 940. The processor 910 can call logical instructions in the memory 930 to execute a method for detecting gastrointestinal cancer. This method includes: inputting a gastrointestinal image into an image classification model to obtain an image classification result output by the image classification model; the image classification result is used to characterize the cancer identification result of the gastrointestinal image; inputting the gastrointestinal image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model; the semantic segmentation result is used to indicate whether a lesion region exists in the gastrointestinal image; based on the semantic segmentation result and the image classification result, determining the detection method for the corresponding gastrointestinal region in the gastrointestinal image; and, if the detection method indicates that the corresponding gastrointestinal region in the gastrointestinal image needs to be detected, outputting a prompt message corresponding to the detection method.

[0254] Furthermore, the logical instructions in the aforementioned memory 930 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0255] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the digestive tract cancer detection method provided by the above methods. The method includes: inputting a digestive tract image into an image classification model to obtain an image classification result output by the image classification model; the image classification result is used to characterize the cancer identification result of the digestive tract image; inputting the digestive tract image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model; the semantic segmentation result is used to indicate whether there is a lesion area in the digestive tract image; based on the semantic segmentation result and the image classification result, determining the detection method for the digestive tract part corresponding to the digestive tract image; and when the detection method indicates that the digestive tract part corresponding to the digestive tract image needs to be detected, outputting a prompt message corresponding to the detection method.

[0256] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a method for detecting gastrointestinal cancer provided by the methods described above. This method includes: inputting a gastrointestinal image into an image classification model to obtain an image classification result output by the image classification model; the image classification result being used to characterize the cancer identification result of the gastrointestinal image; inputting the gastrointestinal image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model; the semantic segmentation result being used to indicate whether a lesion region exists in the gastrointestinal image; determining a detection method for the gastrointestinal region corresponding to the gastrointestinal image based on the semantic segmentation result and the image classification result; and outputting a prompt message corresponding to the detection method when the detection method indicates that the gastrointestinal region corresponding to the gastrointestinal image needs to be detected.

[0257] Figure 10 is a schematic diagram illustrating the working principle of the early gastric cancer identification model provided by the present invention. As shown in Figure 10, in one embodiment, the early gastric cancer identification model provided by the present invention first preprocesses the acquired gastroscopy image. The preprocessed image is then input into a white light staining image recognition model, which automatically distinguishes the image as either a white light mode or a staining mode (including electronic staining and chemical staining), and accordingly calls the early gastric cancer (white light) or early gastric cancer (staining) identification model for further analysis.

[0258] In white light mode, the image is first segmented using a semantic segmentation model to determine whether there are lesion areas. If there are no lesions, no prompt is given.

[0259] If a lesion is present, the region is input into the image classification model, and a decision is made based on the obtained early cancer probability and a preset threshold. When the early cancer probability is greater than or equal to the biopsy threshold (the lesion edge is outlined with a solid line in Figure 10), a biopsy suggestion is given; when the early cancer probability is between the staining threshold and the biopsy threshold, it is judged as a low-suspected early cancer (the lesion edge is outlined with a dashed line in Figure 10), and a staining suggestion is given; if the early cancer probability is lower than the staining threshold, no suggestion is given.

[0260] In the staining mode, if the semantic segmentation result shows a lesion and the probability of early cancer is greater than or equal to the biopsy threshold, a biopsy suggestion is given; otherwise, no suggestion is given.

[0261] Specifically, based on data from 1721 cases (589 cases of early gastric cancer and 1132 cases of non-early gastric cancer) in a multicenter clinical trial, an early gastric cancer (white light) dataset was constructed, containing 4,854 images of early gastric cancer and 21,492 images of non-early gastric cancer. Based on this dataset, a training set and an internal test set were randomly divided according to the cases, and an early gastric cancer (white light) recognition model was trained using this technical approach. The sensitivity and specificity of the recognition model on the internal test set were 100.00% and 93.58%, respectively. The early gastric cancer (white light) pure classification model trained using the above training set also had a sensitivity of 100.00% on the same internal test set, but its specificity was only 36.92%. Statistical analysis showed that the specificity of the early gastric cancer (white light) recognition model trained using this technical approach was significantly higher than the specificity of the aforementioned early gastric cancer (white light) pure classification model (P<0.001).

[0262] Based on data from 1721 cases (589 cases of early gastric cancer and 1132 cases of non-early gastric cancer) in a multicenter clinical trial, an early gastric cancer (staining) dataset was constructed, containing 11642 images of early gastric cancer and 11488 images of non-early gastric cancer. Based on this dataset, a training set and an internal test set were randomly divided according to the cases, and an early gastric cancer (staining) recognition model was trained using this technical approach. The sensitivity and specificity of the recognition model on the internal test set were 100.00% and 88.34%, respectively. The early gastric cancer (staining) pure classification model trained using the above training set also had a sensitivity of 100.00% on the same internal test set, but its specificity was 79.99%. Statistical analysis showed that the specificity of the early gastric cancer (staining) recognition model trained using this technical approach was significantly higher than the specificity of the aforementioned early gastric cancer (staining) pure classification model (P<0.001).

[0263] The test results of the early gastric cancer recognition model trained based on this technical solution on the internal test set, externally published positive data, and real-world early gastric cancer videos are shown in Table 1 below:

[0264] Table 1

[0265] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0266] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0267] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Industrial applicability

[0268] This invention provides a method, apparatus, device, storage medium, and program product for detecting gastrointestinal cancer, relating to the field of image recognition technology. The method includes: inputting a gastrointestinal image into an image classification model to obtain an image classification result output by the model; the image classification result is used to characterize the cancer identification result of the gastrointestinal image; inputting the gastrointestinal image into a semantic segmentation model to obtain a semantic segmentation result output by the model; the semantic segmentation result is used to indicate whether a lesion region exists in the gastrointestinal image; based on the semantic segmentation result and the image classification result, determining the detection method for the corresponding gastrointestinal region in the image; and outputting a prompt message corresponding to the detection method when the detection method indicates that the corresponding gastrointestinal region in the image needs to be detected. This invention can achieve efficient, highly sensitive, and highly accurate detection of gastrointestinal cancer, possessing good economic value and application prospects.

Claims

A method for detecting a digestive tract cancer, characterized by, The method comprises the following steps: inputting the digestive tract image into an image classification model to obtain an image classification result output by the image classification model; the image classification result is used to represent an identification result of cancer identification of the digestive tract image; inputting the digestive tract image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model; the semantic segmentation result is used to indicate whether there is a lesion area in the digestive tract image; determining a detection mode of a digestive tract part corresponding to the digestive tract image based on the semantic segmentation result and the image classification result; in the case that the detection mode indicates that the digestive tract part corresponding to the digestive tract image needs to be detected, outputting prompt information corresponding to the detection mode.

1. The method for detecting a digestive tract cancer according to claim 1, wherein If the digestive tract image is a stained image, the image classification result comprises a first classification result or a second classification result, and the determination of the detection mode of the digestive tract part corresponding to the digestive tract image based on the semantic segmentation result and the image classification result comprises: if it is determined based on the semantic segmentation result that there is a lesion area in the digestive tract image and the image classification result is the first classification result, it is determined that the detection mode is biopsy; if it is determined based on the semantic segmentation result that there is a lesion area in the digestive tract image and the image classification result is the second classification result, it is determined that the detection mode is no detection; if it is determined based on the semantic segmentation result that there is no lesion area in the digestive tract image, it is determined that the detection mode is no detection; wherein the digestive tract part corresponding to the stained image has been stained; the cancer probability value represented by the first classification result is greater than the cancer probability value represented by the second classification result; If the digestive tract image is an unstained image, the image classification result comprises a third classification result, a fourth classification result or a fifth classification result, and the determination of the detection mode of the digestive tract part corresponding to the digestive tract image based on the semantic segmentation result and the image classification result comprises: if it is determined based on the semantic segmentation result that there is a lesion area in the digestive tract image and the image classification result is the third classification result, it is determined that the detection mode is biopsy; if it is determined based on the semantic segmentation result that there is a lesion area in the digestive tract image and the image classification result is the fourth classification result, it is determined that the detection mode is staining; if it is determined based on the semantic segmentation result that there is a lesion area in the digestive tract image and the image classification result is the fifth classification result, it is determined that the detection mode is no detection; if it is determined based on the semantic segmentation result that there is no lesion area in the digestive tract image, it is determined that the detection mode is no detection; wherein the digestive tract part corresponding to the unstained image has not been stained; the cancer probability value represented by the third classification result is greater than the cancer probability value represented by the fourth classification result, and the cancer probability value represented by the fourth classification result is greater than the cancer probability value represented by the fifth classification result.

1. The method for detecting a digestive tract cancer according to claim 1, wherein If the digestive tract image is a stained image, the step of inputting the digestive tract image into an image classification model to obtain an image classification result output by the image classification model comprises: inputting the digestive tract image into a first image classification model corresponding to a stained image to obtain an image classification result output by the first image classification model; if the digestive tract image is an unstained image, the inputting the digestive tract image into an image classification model to obtain an image classification result output by the image classification model comprises: inputting the digestive tract image into a second image classification model corresponding to an unstained image to obtain an image classification result output by the second image classification model; if the digestive tract image is a stained image, the inputting the digestive tract image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model comprises: inputting the digestive tract image into a first semantic segmentation model corresponding to a stained image to obtain a semantic segmentation result output by the first semantic segmentation model; if the digestive tract image is an unstained image, the inputting the digestive tract image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model comprises: inputting the digestive tract image into a second semantic segmentation model corresponding to an unstained image to obtain a semantic segmentation result output by the second semantic segmentation model; wherein a digestive tract part corresponding to the stained image has been stained, and a digestive tract part corresponding to the unstained image has not been stained.

1. The method for detecting a digestive tract cancer according to claim 1, wherein The semantic segmentation model is an integrated semantic segmentation model, and the integrated semantic segmentation model comprises a plurality of different segmentation models. For any pixel of the digestive tract image, the integrated semantic segmentation model is configured to determine a plurality of class prediction results of the pixel, any of the class prediction results comprising a lesion pixel or a non-lesion pixel, if any of the plurality of class prediction results is a lesion pixel, determining a final class prediction result of the pixel as a lesion pixel, and if all of the plurality of class prediction results are non-lesion pixels, determining the final class prediction result of the pixel as a non-lesion pixel.

1. The method for detecting a digestive tract cancer according to claim 1, wherein Before the inputting the digestive tract image into an image classification model to obtain an image classification result output by the image classification model, the method further comprises: inputting the digestive tract image into a feature extraction model to obtain an image feature output by the feature extraction model; Accordingly, the inputting the digestive tract image into an image classification model to obtain an image classification result output by the image classification model comprises: inputting the image feature into an image classification model to obtain an image classification result output by the image classification model; Accordingly, the inputting the digestive tract image into a semantic segmentation model to obtain a semantic segmentation result comprises: inputting the image feature into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model. The method for detecting gastrointestinal cancer according to any one of claims 1 to 5 is characterized in that, The image classification result comprises a target classification result or a plurality of non-target classification results, and a cancer probability value represented by each of the plurality of non-target classification results is less than a cancer probability value represented by the target classification result; Before the determining a detection mode of a digestive tract part corresponding to the digestive tract image based on the semantic segmentation result and the image classification result, the method further comprises: If it is determined, based on the semantic segmentation result, that there is no lesion region in the digestive tract image, and the image classification result is the target classification result, adjusting a segmentation threshold of the semantic segmentation model to obtain and output a lesion region in the digestive tract image; The semantic segmentation model determines, for any pixel of the digestive tract image, a class prediction result of the pixel based on the segmentation threshold. The class prediction result of the pixel includes a lesion pixel or a non-lesion pixel. The method for detecting a digestive tract cancer according to claim 6, wherein The adjusting the segmentation threshold of the semantic segmentation model includes: determining a lowest segmentation threshold as 0 and a highest segmentation threshold as a current segmentation threshold of the semantic segmentation model; determining a sum value of the lowest segmentation threshold and the highest segmentation threshold, and adjusting the segmentation threshold of the semantic segmentation model to half of the sum value; updating the semantic segmentation result based on the adjusted segmentation threshold; if it is determined, based on the updated semantic segmentation result, that a proportion of the lesion region in the digestive tract image is greater than a first preset proportion, updating the lowest segmentation threshold to the adjusted segmentation threshold, and returning to the step of determining the sum value of the lowest segmentation threshold and the highest segmentation threshold, and adjusting the segmentation threshold of the semantic segmentation model to half of the sum value; if it is determined, based on the updated semantic segmentation result, that the proportion of the lesion region in the digestive tract image is less than a second preset proportion, updating the highest segmentation threshold to the adjusted segmentation threshold, and returning to the step of determining the sum value of the lowest segmentation threshold and the highest segmentation threshold, and adjusting the segmentation threshold of the semantic segmentation model to half of the sum value; The semantic segmentation model outputs, for any pixel of the digestive tract image, a prediction value of the pixel. If the prediction value of the pixel is greater than or equal to the segmentation threshold, the class prediction result of the pixel is determined as a lesion pixel. The proportion of the lesion region is determined based on a ratio of the lesion region in the digestive tract image to an area of the digestive tract image. The first preset proportion is greater than the second preset proportion. The method for detecting gastrointestinal cancer according to any one of claims 1 to 5 is characterized in that, The method further includes: inputting the digestive tract image into a digestive tract site recognition model to obtain a digestive tract site recognition result output by the digestive tract site recognition model; outputting prompt information corresponding to the digestive tract site recognition result to prompt a digestive tract site currently being examined; The digestive tract site recognition model is trained based on sample digestive tract images and digestive tract site recognition result labels corresponding to the sample digestive tract images. The method for detecting a digestive tract cancer according to claim 8, wherein After the digestive tract image is input into the digestive tract site recognition model to obtain the digestive tract site recognition result output by the digestive tract site recognition model, the method further includes: in a case where a digestive tract site indicated by the digestive tract site recognition result is examined, determining, from a target digestive tract site set, a digestive tract site that is not examined; outputting prompt information corresponding to the digestive tract site that is not examined. The method for detecting a digestive tract cancer according to claim 8, wherein After the digestive tract image is input into the digestive tract site recognition model to obtain the digestive site recognition result output by the digestive tract site recognition model, the method further includes: determine, based on the historical digestive tract site recognition results of the plurality of acquired digestive tract images and the acquisition time of the plurality of acquired digestive tract images, a retreat time length of each of the plurality of digestive tract sites; determine, based on the retreat time lengths of the plurality of digestive tract sites, a total retreat time length of the endoscope for acquiring the digestive tract image. The method for detecting gastrointestinal cancer according to any one of claims 1 to 5 is characterized in that, Before the step of inputting the digestive tract image into the image classification model to obtain the image classification result output by the image classification model, the method further comprises: inputting the digestive tract image into a digestive tract mucosa cleanliness recognition model to obtain a digestive tract mucosa cleanliness recognition result output by the digestive tract mucosa cleanliness recognition model; the digestive tract mucosa cleanliness recognition result is used to indicate whether the digestive tract image is a cleanliness qualified image; if it is determined based on the digestive tract mucosa cleanliness recognition result that the digestive tract image is a cleanliness unqualified image, outputting a flushing prompt information to prompt to flush the digestive tract site corresponding to the digestive tract image; wherein the digestive tract mucosa cleanliness recognition model is trained based on sample digestive tract images and digestive tract mucosa cleanliness recognition result labels corresponding to the sample digestive tract images. A device for detecting a digestive tract cancer, characterized by comprise: an image classification module, configured to input a digestive tract image into an image classification model to obtain an image classification result output by the image classification model; the image classification result is used to represent an identification result of cancer identification on the digestive tract image; a semantic segmentation module, configured to input the digestive tract image into a semantic segmentation model to obtain a semantic segmentation result output by the semantic segmentation model; the semantic segmentation result is used to indicate whether there is a lesion area in the digestive tract image; a mode determination module, configured to determine a detection mode of a digestive tract site corresponding to the digestive tract image based on the semantic segmentation result and the image classification result; a prompt output module, configured to output prompt information corresponding to the detection mode in a case where the detection mode indicates that the digestive tract site corresponding to the digestive tract image needs to be detected. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor executes the computer program to realize the digestive tract cancer detection method according to any one of claims 1 to 11. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the digestive tract cancer detection method according to any one of claims 1 to 11. A computer program product comprising a computer program, characterized in that The computer program is executed by the processor to realize the digestive tract cancer detection method according to any one of claims 1 to 11.

Images