System with multi-order classification module

Abstract

A clinical support system and method for real-time activation of a detection and characterization module trained for identification or diagnosis of anomalies within the digestive tract is disclosed. The clinical support computer may be programmed to activate a first mucous membrane identification module and a first plurality of detection and characterization modules based at least in part on detecting a connection with the endoscope, monitor an image stream from the endoscope using the first mucous membrane identification module to identify a mucous membrane tissue type, executing a first detection module of a first plurality of detection and characterization modules based on identifying the mucosal tissue type as a first mucosal tissue type, and processing, using the first detection module, an image from an image stream received from the endoscope to identify a region of interest to be output to a display device, the region of interest identifies potential anomalies within the image.

Description

[0001] Priority claim

[0002] This application claims the benefit of priority to U.S. Provisional Patent Application Serial No. 63 / 363,654 filed on April 27, 2022 and U.S. Provisional Patent Application Serial No. 63 / 387,140 filed on December 13, 2022, the contents of which are incorporated herein by reference. Background Art

[0003] Using an endoscopic instrument to examine a patient's digestive (GI) tract is a fairly routine medical preventive / diagnostic medical procedure. For the upper digestive tract, a doctor (or more generally, a healthcare provider (HCP)) examines the interior of a patient's esophagus, stomach, and duodenum (the first part of the small intestine). The instrument used in such a process is a thin light-emitting tube that includes a camera for viewing the inner surfaces of the organs that make up the upper digestive tract - such an instrument is generally referred to as an endoscope. Therefore, upper endoscopy, also known as upper endoscopy, is a process for visually examining a patient's digestive system. Each part of the upper digestive tract has a distinctly different tissue, called a mucosa, which can provide an indication of the location within the digestive tract for a trained eye. The visual inspection performed during endoscopy is intended to identify abnormalities that may require additional diagnosis.

[0004] Similarly, endoscopic instruments can be used to visually examine a patient's lower gastrointestinal tract (GI tract). Lower GI endoscopy, also known as colonoscopy or sigmoidoscopy, allows an HCP to view the mucosal lining of a patient's lower GI tract. The procedure is often used as a screening test for asymptomatic individuals, or to help diagnose unexplained abdominal pain, rectal bleeding, or changes in bowel habits. Routine colonoscopies are recommended after a certain age to enable early detection of cancer or other GI problems before they progress. Identification of abnormalities within the GI tract requires a well-trained HCP and attentive viewing of the camera image during the procedure. Endoscope manufacturers have developed some image processing techniques to help identify potential abnormalities, but these techniques may be difficult for HCPs to use in practice. Summary of the invention

[0005] Advances in artificial intelligence (AI) capabilities have sparked great interest in exploring healthcare use cases. From a practical perspective, multiple different use cases may be applicable to various healthcare settings. One such healthcare setting is an endoscopy suite, in which an HCP can use an endoscope to examine multiple discrete internal anatomical structures. The inspection of each discrete internal anatomical structure may correspond to one or more dedicated AI use cases (e.g., algorithms, trained AI models), and these dedicated AI use cases may not be applicable to other internal anatomical structures. Consider, for example, an esophagogastroduodenoscopy (also known as upper gastrointestinal endoscopy), during which the HCP can begin the process by inserting an endoscope into the patient's esophagus to examine the patient's esophageal mucosa before advancing the endoscope into the patient's stomach and finally into the patient's duodenum. Due to significant differences between the physiological structure of the mucosa within each of the esophagus, stomach, and duodenum, a trained AI model that reliably identifies and / or provides a diagnosis for abnormalities (e.g., cancerous lesions) within one of these regions may not be able to identify and / or diagnose abnormalities within other regions that may be examined during a single procedure.

[0006] It is challenging to train HCPs on the evolving nature of AI tools, and therefore, healthcare systems that require manual changes to intra-procedural settings to switch between available AI tools create a highly error-prone environment. Furthermore, the need to make manual changes to intra-procedural settings increases overall procedure time, which reduces the number of cases that can be supported per day in a single healthcare setting. Techniques are needed to improve the quality and increase the amount of healthcare provided in a single healthcare setting where multiple AI tools can be deployed.

[0007] This is an emerging technology area, and existing systems are essentially independent AI models trained to perform specific AI functions in discrete anatomical regions. These AI models may also be developed by multiple entities, making AI capabilities more fragmented and difficult to integrate.

[0008] To address these challenges, the inventors developed an end-to-end system that intelligently and seamlessly switches between discrete AI models during the procedure based on endoscope selection and characteristics of the tissue currently being viewed. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In the drawings, which are not necessarily drawn to scale, similar reference numerals may describe similar components in different views. Similar reference numerals with different letter suffixes may represent different instances of similar components. The drawings generally illustrate various embodiments discussed in this document by way of example and not by way of limitation.

[0010] Figure 1 An example software architecture diagram is shown in accordance with at least one example of the present disclosure.

[0011] FIG. 2A to FIG. 2E An example intra-procedural process flow layered on a software architecture is shown in accordance with at least one example of the present disclosure.

[0012] Figure 3A A machine learning model training diagram according to at least one example of the present disclosure is shown.

[0013] Figure 3B A machine learning model inference graph according to at least one example of the present disclosure is shown.

[0014] Figure 4 A machine learning training technique according to at least one example of the present disclosure is shown.

[0015] Figure 5 Clinical decision support techniques for automated selection of detection and / or classification modules during a procedure are shown in accordance with at least one example of the present disclosure.

[0016] Figure 6 A block diagram of an example machine on which any one or more of the techniques discussed herein may be performed is shown in accordance with at least one example of the present disclosure. DETAILED DESCRIPTION

[0017] In the example, the software architecture includes a multi-order hierarchy of modules, in which the output of the higher-level modules is used as a trigger to activate / deactivate the lower-level modules. In this way, the higher-level modules monitor the current conditions of the operating environment and selectively activate the lower-level modules designed to operate under the current conditions based on these current conditions. For example, the high-level module can determine the type of instrument connected to the endoscopy workstation, and initialize (e.g., put it into operation) a set of lower-level modules designed to be used with the instrument type (e.g., colonoscope and esophagoscope) or even more sophisticated instrument models (e.g., esophagoscope A and esophagoscope B). In addition or alternatively, the high-level module can determine the type of tissue currently being imaged by the endoscope, and activate the CADe module and / or CADx module that uniquely corresponds to the tissue type. The CADe module / CADx module is an artificial intelligence computer module for the detection and / or classification of abnormal tissue. In this context, "CAD" is an abbreviation for computer-aided detection, where the CADe version indicates a version of computer-aided detection configured to identify abnormal tissue. CADx version refers to a computer-aided detection system configured to identify detected abnormal tissue and classify the detected abnormal tissue. In addition, in the context of the present application, the term "higher-level module" refers to a software component that generates outputs for determining operating parameters corresponding to other software components, which are properly referred to as "lower-level modules" relative to the above-mentioned "higher-level modules". In addition, the term "module" is used in the text to define a functionally discrete segment of software code designed to perform a specific function or provide a specific service. A "module" can be executed in a suitable computing system to produce a defined output based on a defined input or a set of inputs. In some examples, a "module" can include a complete application, such as an application executable within a clinical decision support system or a mobile computing device.

[0018] The software architecture discussed herein may also allow for an "app store" approach to CADe modules and CADx modules. The software architecture allows the HCP to select from a variety of CADe modules / CADx modules that are compatible with the selected instrument for diagnosis within various portions of the digestive tract. An interface may be provided to allow the HCP to associate the selected CADe module / CADx module with different segments of the digestive tract (which are detected by the mucosal tissue recognition module during the procedure, as discussed in detail below).

[0019] Figure 1An example software architecture diagram according to at least one example of the present disclosure is shown. The example software architecture 100 includes multiple hierarchical levels, starting from the first level 102 including the instrument identification module 104. The second level 110 in this example may include multiple mucosal identification modules 112A to 112N, including the first mucosal identification module 112A to the Nth mucosal identification module 112N-the present disclosure may refer to the mucosal identification module 112 as a shorthand for referring to one or more modules in the multiple mucosal identification modules 112A to 112N. In addition, the multiple mucosal identification modules 112A to 112N may be referred to as multiple mucosal identification modules 112. In this example, the second level 110 may also include one or more anatomical identification modules (anatomy identification module) 114 for the lower digestive tract. The anatomical identification module 114 may also operate based on mucosal tissue identification. As described in detail below, both the mucosal identification module 112 and the anatomical identification module 114 can use a trained machine learning model to identify the corresponding tissue type. In addition, in some examples, the second level 110 can utilize a single mucosal recognition module (e.g., mucosal recognition module 112) that is programmed / trained to recognize a variety of different mucosal tissue types. In other words, in some examples, multiple mucosal recognition modules 112A to 112N are packaged into a single mucosal recognition module.

[0020] In this example, the software architecture 100 may include a third level 120 that includes CADe modules for specific areas of the upper or lower digestive tract. In this example, the third level 120 is shown to include an esophageal CADe module 122 and a stomach CADe module 124, as well as a CADe module 126 for identification of abnormalities in the lower digestive tract. The software architecture 100 may include any number of levels, which are represented by the Nth level 130 in the present disclosure. The Nth level 130 is shown to include an esophageal CADx module 132 and a stomach CADx module 134, as well as a CADx module 138 for characterization / classification of abnormalities in the lower digestive tract (e.g., characterization of polyps). As described above, the various levels of the software architecture 100 provide a structure that enables a clinical decision support system (CDSS) to seamlessly switch between using multiple different AI tools within a single medical examination based on the current situation (e.g., the current position of the endoscopic camera device within the patient's digestive tract as determined by the mucosal recognition module).

[0021] An exemplary endoscopy software architecture, software architecture 100, may include a first level that includes one or more instrument identification modules (e.g., instrument identification module 104) and one or more anatomically specific AI suites. As illustrated below, the exemplary endoscopy software architecture includes an upper gastrointestinal AI suite (e.g., an esophagogastroduodenoscopy (EGD) AI suite) and a lower gastrointestinal AI suite (e.g., a colonoscopy AI suite). Each of the AI ​​suites includes multiple modules designed to perform specific AI functions by analyzing image streams captured by an endoscope. In addition, the outputs of higher-level modules within the AI ​​suite can be used as a determining factor as to which lower-level modules are operating at a given point in time. For example, a determination of the type and / or model of an instrument selected by a user (or inserted into an endoscopy tower) can cause the CDSS to initialize a specific AI suite corresponding to the instrument type and / or model. As another example, the latest output of a second-level module can indicate which third-level modules are currently operating, and so on.

[0022] In the illustrated example, the second level 110 of the software architecture 100 includes various modules designed to determine the type of tissue currently being imaged by the endoscope. Specifically, there are mucosal recognition modules (e.g., multiple mucosal recognition modules 112) each designed to determine whether a specific type of mucosa is currently being imaged within the upper gastrointestinal AI suite. In addition or alternatively, a single mucosal recognition module (e.g., a single AI-based tissue classification model) can be configured to determine which of a plurality of different types of mucosa is currently being imaged. Although shown at the second level 110, in an alternative implementation, these mucosal recognition modules 112 (or more broadly, anatomical structure recognition modules if the target anatomical structure is not a mucosa) can be the first level. In other words, the instrument recognition within the illustrated first level is an optional part of the software architecture 100.

[0023] In addition, below the anatomy recognition module, there are various AI models that are specifically developed and trained to analyze the type of tissue currently being viewed as determined by the anatomy recognition module (which may also be an AI model). For example, where the most recent image analyzed at the second level 110 indicates that esophageal tissue / mucosa is being imaged endoscopically, then at the third level 120, an AI model configured to analyze esophageal tissue / mucosa (e.g., esophageal CADe module 122) is used. Such AI models may include CADe and / or CADx type models, which in some implementations are divided into one or more levels as shown.

[0024] In a practical sense, this type of layered software architecture enables an AI-enabled healthcare system to intelligently switch between multiple different AI models, each of which is developed and trained to operate in a specific operating environment, in a seamless and dynamic nature throughout the process as operating conditions fluctuate. As a specific example, which will be described in more detail below, as an endoscope passes from a first anatomical structure to a second anatomical structure, the AI-enabled healthcare system can automatically switch from a first AI model configured to analyze the first anatomical structure type to a second AI model configured to analyze the second anatomical structure type.

[0025] In this way, the concept adds significant value by creating an operating environment that seamlessly operates to facilitate a single goal (e.g., identifying, diagnosing, and ultimately treating diseased tissue) from the perspective of an end user (e.g., HCP) by dynamically switching between various functionalities and / or AI tools in response to changing operating conditions. At the same time, while appearing to be a single tool from the end user's perspective, the concept retains modularity from a developer's perspective, which enables functionality to be easily added, modified, and continuously improved (e.g., without having to perform a complete software overhaul) as the underlying technology advances. In addition, the concept facilitates providing different degrees of functionality to end users based on subscription level (HCP A can subscribe to fewer or different functionality than HCP B).

[0026] To further convey these concepts, an exemplary intra-process system flow is described below in conjunction with a series of figures. FIG. 2A to FIG. 2E An example intra-procedural process flow layered on a software architecture is shown in accordance with at least one example of the present disclosure. FIG. 2A to FIG. 2E Each graph in the diagram represents a different operation in the technique and is represented by a different timestamp (T 1 、T 2 etc.) logo.

[0027] Figure 2A – Time = T 1 In this example, technique 200 may begin at operation 202, which occurs at a first time T 1, and involves a HCP connecting an esophagoscope to a system operating an exemplary endoscopy software architecture (e.g., software architecture 100). The instrument identification module 104 recognizes that the instrument is an esophagoscope and responds by registering the intended anatomical structure, such as by initializing the upper gastrointestinal AI suite (e.g., because the most likely use of the esophagoscope is to examine the upper gastrointestinal space). As illustrated below, because the instrument has just been connected and the examination has not yet been conducted (e.g., images of the upper gastrointestinal space have not yet been captured), the output of each module in the upper gastrointestinal AI suite can be N / A (e.g., empty) because images of the anatomical structure to be examined have not yet been captured.

[0028] Figure 2B – Time = T 2 At the second moment T 2 At 204, the technique 200 proceeds to operation 204, which involves the HCP inserting an esophagoscope into the patient's esophagus, and the mucosal identification module (e.g., multiple mucosal identification modules 112) operating to continuously (or periodically at an appropriate sampling rate) determine the type of tissue currently being imaged. As shown, the mucosal identification module 112 generates an output that indicates that the esophageal mucosa is currently being imaged with a high degree of confidence (e.g., measured by an appropriate threshold level, such as 0.95 or higher - different confidence levels are within the scope of the present disclosure). In some examples, if it is determined that the identified tissue type is inconsistent with the registered anatomical structure, the system can generate a warning or other prompt to display to the HCP. Note that the software can provide settings that allow the HCP to set a threshold confidence level for the mucosal identification module 112. Based on this determination at the second level 110 of the software architecture 100, the technique 200 continues at operation 206, where the system executes the esophageal CADe module 122 to continuously analyze the images being generated during the process. At time T 2 At , the output of the esophageal CADe module 122 indicates that the tissue being observed is normal (e.g., non-abnormal). Therefore, the system continues in the first imaging modality (e.g., visible light imaging) and continues to run the esophageal CADe module 122, while the mucosal recognition module 112 belonging to the upper gastrointestinal AI suite also continues to run (or runs at an appropriate sampling frequency). In this example, multiple mucosal recognition modules 112A to 112N continue to execute in the background to detect changes in the imaged mucosal tissue that indicate a transition to a different segment of the gastrointestinal tract being examined.

[0029] In an alternative example (not shown), at time T 2 At 4 pm, the HCP can activate the NBI module to enable certain CADx modules to operate. In non-precancerous lesions and depending on the size and location within the colon, a polypectomy may be performed to remove the lesion for subsequent evaluation by a pathologist or discard it.

[0030] Figure 2C – Time = T 3 At the third moment T 3 At 208, the technique 200 can continue, which includes the HCP manipulating the esophagoscope in a manner to image the abnormal tissue. Thus, the esophageal CADe module 122 generates an output indicating the presence of an abnormality within the space being imaged. In an example, a user interface dialog box can be output to a display device. In addition, an audible sound (alarm) can be generated to ensure that the HCP is aware of the abnormal tissue detected. For example, it should be understood that various CADe models are designed to detect abnormalities such as polyps, but do not characterize these polyps (e.g., benign, malignant, etc.). In some examples, a graphic such as a region of interest bounding box is displayed superimposed on the endoscopic image to alert the HCP that abnormal tissue has been detected.

[0031] Based on the output from the esophageal CADe module 122, the system may respond by taking one or more predetermined actions. For example, the system may respond to the output from the esophageal CADe module 122 indicating that an abnormality has been detected by automatically activating a function to provide additional information about the abnormal tissue being imaged. In some embodiments, in response to the output from the esophageal CADe module 122, the system may automatically deactivate a first imaging modality (e.g., full spectrum or white light imaging) and activate a second imaging modality (e.g., narrow band imaging (NBI) of blue or red light) in conjunction with an esophageal CADx module 132 (within the Nth level 130) that utilizes images captured via the second imaging modality to characterize the abnormal esophageal tissue. In other examples, the esophageal CADx module 132 may be designed to operate using white light, so the system does not activate the second imaging modality. In the immediately preceding example, the esophageal CADe module 122 may be appropriately described as a “higher-level module” relative to the esophageal CADx module 132 because the outputs of the esophageal CADe module 122 cause the system to trigger or activate functionality of the “lower-level” esophageal CADx module 132 .

[0032] In some embodiments, the system can respond to the output of the esophageal CADe module 122 indicating that an abnormality is being imaged by generating a UI element (e.g., visual, auditory, tactile) to prompt the HCP as to whether images captured via a second imaging modality are needed. If the HCP provides input indicating the need for such a second modality image, the system can respond by switching from a first imaging modality (e.g., visible light) to a second imaging modality (e.g., NBI). Then, in response to capturing images via the second modality, the system can automatically utilize the esophageal CADx module 132 that can analyze these images to generate an output indicating tissue classification and corresponding confidence scores. In some examples, the HCP can manually turn on NBI or other modalities. In these examples, information about the new modality can be captured without the need to detect different types of images.

[0033] Figure 2D – Time = T 4 In this example, technique 200 may continue at operation 210, which occurs at a fourth time T 4 At operation 210, the second imaging modality is turned on (either automatically or in response to input from the HCP). In response to the second imaging modality being activated, the system begins providing captured images to the esophageal CADx module 132, which classifies the abnormality identified via CADe (e.g., via the esophageal CADe module 122 executed in operation 206). For example, as illustrated, the esophageal CADx module 132 provides an output indicating that the abnormality is malignant stage 1 and that the classification corresponds to a confidence level of 0.87.

[0034] Figure 2E – Time = T 5 In this example, the technique 200 may be performed at a fifth time T 5 The process ends with operations 212 and 214. In these operations (212, 214), the HCP advances the esophagoscope through the esophagus and into the stomach. As a result, the esophagoscope begins to capture images of the gastric mucosa. Since the mucosal recognition modules (e.g., multiple mucosal recognition modules 112A to 112N) continue to operate throughout the technique 200, the system seamlessly determines to switch from operating an AI module configured to analyze the esophageal mucosa to an AI module configured to analyze the gastric mucosa based on the output of these mucosal recognition modules (112A to 112N). Therefore, as Figure 2E As shown, the output of the esophageal CADe module 122 becomes inapplicable (or empty, as it may cease operation altogether), and the stomach CADe module 124 begins outputting an indication of whether the gastric mucosa currently being viewed is normal or abnormal. In practice, the technique 200 continues to operate throughout an endoscopic procedure, of which the above discussion illustrates a portion of such a procedure.

[0035] Alternative use cases: landmark recognition and automatic report generation

[0036] During a colonoscopy, an important consideration is that a complete colonoscopy has been performed, meaning that the HCP has advanced the colonoscope from the anus all the way to the end of the colon. The cecum is located at the end of the colon and marks the distinct transition from the colonic space to the small intestine. Current best practice is for the HCP to advance the colonoscope all the way to the end of the colon and then begin withdrawing the colonoscope while inspecting the colonic mucosa as it is withdrawn.

[0037] The software architecture 100 can also be used to examine the colon space and can automatically populate various aspects of the process report based on feature point identification. For example, an anatomical structure recognition module, such as the anatomical structure recognition module 114, can be configured to analyze the image to specifically identify the cecum. Once identified, the system can automatically populate the report with instructions including:

[0038] Time to reach the cecum

[0039] Duration of colonoscope advancement

[0040] o This can be calculated by recording the time when the anatomy recognition module first identifies the colonic mucosa (indicating the moment the colonoscope enters the anus) and recording the time when the cecum is identified, and then calculating the advancement time as the difference between these times.

[0041] Duration of colonoscope exit (calculated in the reverse manner as above—i.e., the time when the cecum is identified when the colonoscope is removed from the anus minus the time when the colonic mucosa is no longer identified in the image)

[0042] Figure 3A A machine learning model training diagram 300A is shown according to at least one example of the present disclosure. Diagram 300A shows components and inputs for training a model 302 using machine learning.

[0043] Machine learning (ML) is the application of providing computer systems with the ability to perform tasks by inferring based on patterns found in data analysis, without explicit programming. Machine learning explores the study and construction of algorithms (also referred to herein as tools) that can learn from existing data and make predictions about new data. Although examples are presented with respect to some machine learning tools, the principles presented herein can be applied to other machine learning tools.

[0044] Machine learning (ML) algorithms use data (e.g., action primitives and / or interaction primitives, target vectors, rewards, etc.) to find correlations between identified features that affect outcomes. Features are individual measurable properties of observed phenomena. Example features for model 302 may include diagnostic data (e.g., from a physician), reported patient outcome data, labeled mucosal tissue endoscopic images, and corresponding endoscopic positions. Features that may include and / or be referred to as clinical data may be compared to input data (e.g., endoscopic tissue images).

[0045] The concept of features is related to the concept of explanatory variables used in statistical techniques such as linear regression. Selecting features that are informative, discriminative, and independent is important for the effective operation of ML in pattern recognition, classification, and regression. Features can be of different types, such as numeric features, strings, and graphs.

[0046] During training, the ML algorithm analyzes the input data based on the identified features and optionally the configuration parameters defined for the training (e.g., environmental data, state data, patient data such as demographics and / or comorbidities, etc.). The result of the training is a model 302 that is capable of accepting input to generate complex tasks. In this example, the model 302 will be trained to recognize different types of mucosa from input endoscopic images.

[0047] In an example, the input data may be labeled (e.g., for use as features in a training phase). Labeling may include identifying mucosal tissue and the location of the tissue within the digestive tract. The labeled training images may be weighted and / or may be used to generate different versions of the model 302.

[0048] Input training data for model 302 may include clinical data, which may include patient data such as weight, height, and any other patient data that may affect risk factors associated with the target diagnosis.

[0049] A neural network - sometimes also called an artificial neural network - is a computing system based on the biological neural networks of animal brains. Such systems gradually improve in performance of tasks - this is called learning - usually without the need for task-specific programming. For example, in image recognition, a neural network can be taught to recognize images that include an object by analyzing example images that have been labeled with the object's name and learning the object and the name, and the neural network can use the results of the analysis to recognize and / or classify objects in unlabeled images. For example, in Figure 3A In the example of FIG. 3 , model 302 may be trained to recognize mucosal tissue types and / or locations within the digestive tract, and / or percentage likelihood and / or confidence of tissue locations.

[0050] Neural networks are based on a collection of connected units called neurons, where each connection between neurons (called a synapse) can transmit a unidirectional signal with an activation strength that varies with the strength of the connection. A receiving neuron can activate and propagate this signal to downstream neurons connected to it, typically based on whether the combined input signal from potentially many sending neurons has sufficient strength, where strength is a parameter.

[0051] A deep neural network (DNN) is a stacked neural network consisting of multiple layers. The layers consist of nodes that are loosely modeled after neurons in the human brain, where computation occurs and neurons fire when they encounter enough stimulation. The nodes combine inputs from the data with a set of coefficients and / or weights that amplify or suppress the inputs, assigning importance to the inputs for the task the algorithm is trying to learn. These input-weight products are summed and passed through a so-called node activation function to determine whether and to what extent the signal is further passed through the network to affect the final result. DNNs use a cascade of multiple layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. Higher-level features are derived from lower-level features to form a hierarchical representation. The layer after the input layer can be a convolutional layer that produces a feature map, which is the filtered result of the input and is used by the next convolutional layer.

[0052] The DNN can be a specific type of DNN, such as a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory (LSTM), etc. In some examples, other artificial neural networks can be used. In some examples, a classifier can be used instead of a neural network. The classifier may not include a hidden layer, but can classify a specific input as corresponding to a specific output. For example, for mucosal image input data, a location recognition within the digestive tract can be generated by a classifier.

[0053] The input data for training the model 302 may include data captured from the endoscope and labeled data from medical practitioners. The model 302 may be used in the inference phase (described below with respect to Figure 3B ), to determine the location of a specific input mucosal tissue image within the digestive tract.

[0054] like Figure 3AAs shown, the training data may include signal training data including endoscopic images of mucosal tissue and other process-related data such as endoscopic instrument type and illumination modality. In some examples, the signal training data includes annotations or labeling data (e.g., as image labeling data) provided by a medical practitioner. For example, a medical practitioner may annotate each input endoscopic mucosal tissue image (e.g., each image data N). In some examples, patients and / or medical practitioners may also provide clinical data for training model 302. Clinical data may include diagnostic data and patient data.

[0055] Based on the signal training data and / or the annotation training data, the model 302 can generate output weights corresponding to various processing nodes distributed across the input layer, the output layer, and one or more hidden layers. The model 302 and the trained weights can later be used to infer an indication of the digestive tract location based on an endoscopic input image of mucosal tissue.

[0056] Figure 3B A machine learning model inference graph 300B is shown according to at least one example of the present disclosure. In the inference graph 300B, a model 304 (e.g., model 302 after training and / or updating, etc.) can be used to output predictions, such as a location within the digestive tract or a mucosal tissue type, etc. A confidence level and / or weighting can be output as a prediction, or in addition to other predictions discussed above. The machine learning model inference graph 300B can represent an exemplary computer-based clinical decision support system (CDSS) configured to assist in predicting the location of an endoscopic instrument within a patient's digestive tract.

[0057] As shown in Figure 3, model 304 can receive signals from an endoscope as input, such as image data of a patient's digestive tract. The input data can also include endoscopic instrument type and illumination modality, etc. Model 304 can generate outputs (e.g., inferences) including locations within the digestive tract or mucosal tissue types, etc. Model 304 can have a run time that occurs while the patient is undergoing an endoscopic procedure. Model 304 can provide real-time or near real-time information to the physician about the location within the digestive tract. Digestive tract location information can also be used by CDSS to activate or deactivate certain other AI / ML modules (e.g., CADe modules and / or CADx modules) that contribute to the detection and / or characterization of tissue abnormalities.

[0058] Figure 4A machine learning training technique 400 according to at least one example of the present disclosure is shown. In this example, the technique 400 outlines the basic steps for training a machine learning model based on an input endoscopic image to detect different mucosal tissue types and / or digestive tract locations. The technique 400 may include operations such as receiving mucosal endoscopic image data at 402, labeling the image data at 404, training a machine learning model at 406, and outputting the machine learning model at 408.

[0059] In this example, technique 400 can be performed at 402 with a system (such as the following Figure 6 The computer system described in detail in ) starts with receiving multiple mucosal tissue images from the endoscope. The mucosal tissue images can originate from the upper digestive tract and / or the lower digestive tract. The images from the upper digestive tract can include images of mucosa including esophageal mucosa, gastric mucosa and duodenal mucosa, each with visually different aspects. The image data from the lower digestive tract can include images from the cecum, ascending colon, right flexure of the colon, transverse colon, left flexure of the colon, descending colon, sigmoid colon, rectum and anal canal, and other different areas of the lower digestive tract.

[0060] Technique 400 can continue at 404, where the system receives label data for each of the training mucosal endoscopic images received in operation 402. The labeling operation at 404 can also include weighting the training images according to the strength of representation of a particular mucosal tissue type. An output of operation 404 can include a set of training data including annotations associated with each image of the received image data. At 406, technique 400 can continue, where the training data is used to train a machine learning model, which can then be used to predict the location of an input endoscopic image within the digestive tract. The above Figure 3A Additional details regarding training a machine learning model that may be implemented within the context of technique 400 are discussed. Once trained, technique 400 may end at operation 408 with the system outputting a machine learning model for use in predicting digestive tract location and / or mucosal tissue type. In an example, the trained machine learning model output by technique 400 may be referenced below. Figure 5 The technology discussed is used within 500.

[0061] Figure 5A clinical decision support technique 500 for automatic selection of detection and / or classification modules during a procedure according to at least one example of the present disclosure is shown. In this example, the technique 500 may include operations such as detecting an endoscope at 502, activating a mucosal recognition module at 504, activating a first plurality of detection and characterization modules at 506, monitoring an image stream at 508, identifying a mucosal tissue type at 510, executing a first detection and / or characterization module at 512, alternatively executing a second detection and / or characterization module at 514, and outputting detection and / or characterization data at 516. The technique 500 discusses detection and characterization modules that may be CADe and CADx modules, respectively.

[0062] In this example, technique 500 may optionally begin at 502 with the system (e.g., CDSS) detecting the type of endoscope. For example, the system may detect a connection to the endoscope and determine that the endoscope belongs to the category of "esophagoscope" instruments. In addition or alternatively, the system may more finely determine the specific model of the detected endoscope (e.g., the system may identify the endoscope as a GIF-XP190N version of the EVIS EXERA III sold by Olympus Corporation of the Americas). In this operation, detection of the endoscope type may be triggered by connecting the instrument to the CDSS. In another example, detection of the endoscope type may be triggered by initializing the instrument after connection to the CDSS, or at some other logical point in the process before the instrument is used.

[0063] At 504, technology 500 continues, wherein the system activates the first mucosa recognition module. The activation of the mucosa recognition module can be based on the type and / or model of the endoscopic instrument connected to the system. For example, based on the system having determined at 502 that an instrument belonging to the "esophagoscope" instrument category is connected, the system can start running a mucosa recognition module, which is designed to utilize one or more classification type ML models to analyze endoscopic images and output images depicting squamous mucosa, gastric mucosa or indications of villous mucosa. As discussed above, for different processes, such as upper digestive tract processes and lower digestive tract processes, different types of endoscopes are used. Accordingly, the mucosa recognition module can be customized for the upper digestive tract and the lower digestive tract (or based on other specific process features). In some examples, the mucosal tissue recognition module can be trained to span the entire digestive tract.

[0064] At 506, the technique 500 continues by activating a first plurality of detection and characterization modules. In this example, the activation of the detection and characterization modules is also based on the type of endoscopic instrument detected. For example, in response to determining the type (e.g., esophagoscope) and / or model (e.g., GIF-XP190N) of the endoscope, the system may begin running Figure 1. Thus, it should be understood that aspects of each of operations 504 and 506 may be performed based on the results and / or determinations of operation 502. In another example, operations 504 and 506 may be based on another system input that indicates where within the patient's digestive tract the process is being performed. Another factor in determining which mucosal recognition module to activate at 504 and which of the multiple detection and characterization modules to activate at 506 may involve the available light mode within the connected instrument (e.g., normal white light versus NBI or other lighting type).

[0065] At 508, the technique 500 may continue, where the system monitors the image stream generated by the endoscope. Operation 508 may be operated continuously throughout the use of the endoscopic instrument. At 510, the technique 500 may continue, where the system uses an active mucosal identification module (e.g., a first mucosal identification module) to continuously (or periodically at an appropriate sampling rate) identify mucosal tissue types within each image processed from the image stream monitored in operation 508. Operation 510 may be run on each image within the monitored image stream, or it may be performed periodically on images extracted from the monitored image stream. In this example, when the first mucosal tissue type is identified at 510, the technique 500 may continue at operation 512, where the system executes a first detection module and / or a first characterization module. For example, based on an indication that squamous mucosa is currently being imaged at operation 510, the system may perform operation 512, in which the esophageal CADe module 122 and / or the esophageal CADx module 132 may be executed, because squamous mucosa is located in most of the esophagus. Executing a detection or characterization module at operation 512 may include selecting an illumination modality supported by the endoscopic instrument in use. At 516, the technique 500 continues with the system outputting detection and / or characterization data to assist the HCP in performing the ongoing procedure. Outputting the detection data at 516 may include outputting a region of interest superimposed on an endoscopic image to the HCP to identify abnormal tissue. Operation 516 may also include displaying a user prompt for providing analysis options (e.g., switching illumination modalities and / or executing a characterization module).

[0066] In some examples, the technique 500 loops through operations 508 to 516 in real time or near real time to continuously update a graphical display of detection and / or characterization data on a display device displaying an endoscopic image stream. In an example, after the first detection module generates a region of interest identifying potentially abnormal tissue, the technique 500 may loop back to operation 512 to execute the first characterization module. At operation 516, the first characterization module may then output a tissue classification. In some examples, the characterization module may also output a confidence score or indicator separately from the tissue classification.

[0067] Returning to operation 510 (which, as described above, is performed continuously or periodically simultaneously with operations 512 and 516), when the mucosal identification module identifies a second mucosal tissue type, the technique 500 can continue at operation 514, where the system executes a second detection module and / or a second characterization module. For example, based on an indication that the gastric mucosa is currently being imaged, the system can automatically switch to executing operation 514, where the gastric CADe module 124 and / or the gastric CADx module 134 can be executed because the gastric mucosa is located in the stomach. The technique 500 can end at operation 516 with the system outputting detection and / or characterization data to assist the HCP in performing the ongoing process. In this example, the detection of the second mucosal tissue type at operation 510 indicates that the process has been converted to a different region of the digestive tract. In the example, when the process is detected to be converted to a new region of the digestive tract, an indication of the event can be output to a display device coupled to the system to further assist the HCP.

[0068] The above invention is discussed with reference to endoscopic applications, but the same concepts can be applied to endoscopic, ultrasound and microscopy diagnostic applications. The system described herein can switch between AI modules within the process and will attach metadata that identifies which AI modules are operating on different parts of the video stream. The system supports modular insertion of AI modules when adding or updating functionality. The system can generate reports that detail which algorithms (modules) were selected throughout the process and why.

[0069] Figure 6 A block diagram of an example machine 600 on which any one or more of the techniques (processing) discussed herein can be executed according to some embodiments is shown. In an alternative embodiment, the machine 600 can be operated as an independent device and / or can be connected (e.g., networked) to other machines. In a networked deployment, the machine 600 can operate as a server machine, a client machine, or both in a server-client network environment. In an example, the machine 600 can act as a peer machine in a peer-to-peer (P2P) (or other distributed) network environment. The machine 600 can be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, a web device, a network router, a switch or a bridge, or any machine capable of executing instructions (sequentially or otherwise) specifying the actions to be taken by the machine. In addition, although only a single machine is illustrated, the term "machine" should also be understood to include any collection of machines that individually or collectively execute a set (or multiple sets) of instructions to execute any one or more of the methods discussed herein, such as cloud computing, software as a service (SaaS), and other computer cluster configurations.

[0070] The machine (e.g., computer system) 600 may include a hardware processor 602 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a hardware processor core, or any combination thereof), a main memory 604, and a static memory 606, some or all of which may communicate with each other via an interconnection link (e.g., a bus) 608. The machine 600 may also include a display unit 610, an alphanumeric input device 612 (e.g., a keyboard), and a user interface (UI) navigation device 614 (e.g., a mouse). In an example, the display unit 610, the input device 612, and the UI navigation device 614 may be a touch screen display. The machine 600 may additionally include a storage device (e.g., a drive unit) 616, a signal generating device 618 (e.g., a speaker), a network interface device 620, and one or more sensors 621, such as a global positioning system (GPS) sensor, a compass, an accelerometer, or other sensors. The machine 600 may include an output controller 628, such as a serial (e.g., universal serial bus (USB), parallel or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection to communicate with and / or control one or more peripheral devices (e.g., printer, card reader, etc.).

[0071] The storage device 616 may include a machine-readable medium 622 on which is stored one or more sets of data structures or instructions 624 (e.g., software) that implement or are utilized by any one or more of the techniques or functions described herein. The instructions 624 may also reside, completely or at least partially, within the main memory 604, within the static memory 606, or within the hardware processor 602 during execution of the instructions 624 by the machine 600. In an example, one or any combination of the hardware processor 602, the main memory 604, the static memory 606, or the storage device 616 may constitute a machine-readable medium.

[0072] Although machine-readable medium 622 is illustrated as a single medium, the term "machine-readable medium" may include a single medium or multiple media (e.g., a centralized or distributed database and / or associated caches and servers) configured to store one or more instructions 624. The term "machine-readable medium" may include any medium capable of storing, encoding, or carrying instructions for execution by machine 600 and causing machine 600 to perform any one or more of the techniques of the present disclosure, or capable of storing, encoding, or carrying data structures used by such instructions or associated with such instructions. Non-limiting examples of machine-readable media may include solid-state memory and optical and magnetic media.

[0073] The instructions 624 may also be sent or received over a communication network 626 using a transmission medium via the network interface device 620 using any of a number of transmission protocols (e.g., frame relay, Internet Protocol (IP), transmission control protocol (TCP), user datagram protocol (UDP), hypertext transfer protocol (HTTP), etc.). Example communication networks may include a local area network (LAN), a wide area network (WAN), a packet data network (e.g., the Internet), a mobile telephone network (e.g., a cellular network), a plain old telephone (POTS) network, and a wireless data network (e.g., the Institute of Electrical and Electronics Engineers (IEEE) 802.11 family of standards, known as The IEEE802.16 family of standards is known as IEEE802.15.4 family of standards, peer-to-peer (P2P) networks, and the like. In an example, the network interface device 620 may include one or more physical jacks (e.g., Ethernet, coaxial, or telephone jacks) or one or more antennas to connect to the communication network 626. In an example, the network interface device 620 may include multiple antennas to communicate wirelessly using at least one of single-input multiple-output (SIMO) technology, multiple-input multiple-output (MIMO) technology, or multiple-input single-output (MISO) technology. The term "transmission medium" should be deemed to include any intangible medium that can store, encode, or carry instructions for execution by the machine 600, and includes digital communication signals or analog communication signals or other intangible media to facilitate the communication of such software.

[0074] The method examples described herein may be implemented at least in part by a machine or computer. Some examples may include a computer-readable medium or machine-readable medium encoded with instructions, which can be operated to configure an electronic device to perform the method described in the above example. The implementation of such a method may include code, such as microcode, assembly language code, high-level language code, etc. Such code may include computer-readable instructions for performing various methods. The code may form a part of a computer program product. In addition, in an example, such as during execution or at other times, the code may be tangibly stored on one or more volatile, non-transient or non-volatile tangible computer-readable media. Examples of these tangible computer-readable media may include, but are not limited to, hard disks, removable disks, removable optical disks (e.g., compact disks and digital video disks), cassettes, memory cards or memory sticks, random access memory (RAM), read-only memory (ROM), etc.

[0075] Example

[0076] The following non-limiting examples describe in detail certain aspects of the subject matter to address challenges and provide the benefits discussed herein, among other things.

[0077] Example 1 is a digestive (GI) tract mucosal recognition system for training a machine learning model for use in a computer-based clinical decision support system to assist in the real-time selection of a computer-aided detection and characterization module trained for a specific anatomical region. In this example, the digestive tract mucosal recognition system may include data of mucosal images and a computing system having a processing circuit and a memory device. The memory device may include instructions that, when executed by the processing circuit, cause the processing circuit to perform operations including: marking, training a machine learning model, and storing the machine learning model. The marking may include: marking multiple mucosal images from a database based on the position within the digestive tract to generate training data. Training the machine learning model may include: using the training data to predict the position of the endoscope within the digestive tract based on real-time mucosal imaging from the endoscope.

[0078] Example 2 is a method for training a machine learning model for a computer-based clinical decision support system that provides real-time selection of computer-aided detection and characterization modules trained for specialized identification or diagnosis of abnormalities within a specific anatomical region. In this example, the method may include the following operations: receiving data, labeling the received data, training a machine learning model, and outputting the machine learning model. The received data may include: data captured by an endoscope including mucosal images from various parts of the digestive tract. Labeling the received data may include: labeling based on the position within the digestive tract to generate training data. Training the machine learning model may be accomplished using the training data to train the model to predict the position of the endoscope within the digestive tract based on real-time mucosal imaging from the endoscope.

[0079] In Example 3, the subject matter of Example 2 may optionally include receiving the data by receiving an endoscopic image including esophageal mucosa, gastric mucosa, and duodenal mucosa.

[0080] In Example 4, the subject matter of any of Examples 2 and 3 can optionally include training the machine learning model by training an upper gastrointestinal tract model and a lower gastrointestinal tract model.

[0081] In Example 5, the subject matter of Example 4 can optionally include training the lower gastrointestinal tract model by using training data specific to the lower gastrointestinal tract.

[0082] In Example 6, the subject matter of Example 5 may optionally include: the training data specific to the lower gastrointestinal tract includes labeled endoscopic images from an image group selected from one or more of the following anatomical regions: cecum, ascending colon, right flexure of the colon, transverse colon, left flexure of the colon, descending colon, sigmoid colon, rectum, and anal canal.

[0083] Example 7 is an endoscope clinical support system for real-time activation of a computer-aided detection and characterization module that is trained for specialized identification or diagnosis of abnormalities within a patient's digestive (GI) tract. In this example, the system may include an endoscope and a clinical support computing device. The endoscope may include an illumination component and a sensor configured to generate medical image data. The clinical support computing device is communicatively coupled to the endoscope. The clinical support computing device also includes: a processing circuit; a display device configured to display medical image data and related user interface graphics; and a memory that includes instructions that, when executed by the processing circuit, cause the processing circuit to perform operations that include the following. A first mucosal recognition module and a first plurality of detection and characterization modules are activated based at least in part on detecting a connection to the endoscope. An image stream from the endoscope is monitored using the first mucosal recognition module to identify a mucosal tissue type. Based on identifying the mucosal tissue type as a first mucosal tissue type, a first detection module in the first plurality of detection and characterization modules is executed. and processing an image from an image stream received from the endoscope using a first detection module to identify a region of interest to be output to a display device, wherein the region of interest identifies a potential abnormality within the image.

[0084] In Example 8, the subject matter of Example 7 can optionally include activating the first mucosal recognition module by receiving endoscope type or model information identifying the endoscope.

[0085] In Example 9, the subject matter of Example 8 can optionally include receiving endoscope type or model information comprising receiving illumination information identifying different types of illumination that the illumination component is capable of generating.

[0086] In Example 10, the subject matter of any one of Examples 7 to 9 may optionally include: the memory also includes instructions that, when executed by the processing circuit, cause the processing circuit to perform operations, such as: continuously monitoring images extracted from an image stream received from an endoscope using a first mucosal identification module; and executing a second detection module among the first plurality of detection and characterization modules based on identifying the mucosal tissue type as a second mucosal tissue type by the first mucosal identification module.

[0087] In Example 11, the subject matter of Example 10 can optionally include executing the second detection module by outputting an indication to a display device that the endoscope has begun imaging a different region of the digestive tract.

[0088] In Example 12, the subject matter of any of Examples 7 to 11 may optionally include that monitoring an image stream from an endoscope using a first mucosal recognition module may include: implementing a machine learning model to predict a position of the endoscope within the digestive tract based on real-time mucosal imaging from the endoscope, the machine learning model being trained at least in part based on training data comprising a plurality of labeled mucosal images from different locations within the digestive tract.

[0089] In Example 13, the subject matter of any one of Examples 7 to 12 may optionally include that monitoring the image stream using the first mucosa recognition module may include periodically processing images extracted from the image stream using the first mucosa recognition module.

[0090] In Example 14, the subject matter of any one of Examples 7 to 13 may optionally include that executing the first detection module may include: selecting a first imaging modality from a plurality of imaging modalities supported by the endoscope.

[0091] In Example 15, the subject matter of Example 14 may optionally include identifying the region of interest by selecting a second imaging modality from a plurality of imaging modalities supported by the endoscope.

[0092] In Example 16, the subject matter of Example 15 can optionally include the memory further comprising instructions that, when executed by the processing circuit, cause the processing circuit to perform operations comprising: executing a first characterization module of the plurality of detection and characterization modules based at least in part on identifying the region of interest.

[0093] In Example 17, the subject matter of Example 16 may optionally include executing the first characterization module by outputting an indication of a tissue classification and a corresponding confidence score associated with the region of interest to a display device.

[0094] In Example 18, the subject matter of any one of Examples 14 to 17 may optionally include that identifying the region of interest comprises: displaying a user prompt on a display device, the user prompt providing an option to activate a second imaging modality of a plurality of imaging modalities supported by the endoscope.

[0095] Example 19 is a system for real-time activation of a computer-aided detection and characterization module that is trained for specialized identification or diagnosis of abnormalities within a patient's digestive (GI) tract. The system may include one or more processing units and a computer-readable medium having computer-executable instructions encoded thereon that cause the one or more processing units to perform operations such as the following. Image data generated by a medical imaging device is received during a medical examination. At a first time during the medical examination, a first portion of the image data is provided to one or more anatomical structure recognition modules, the one or more anatomical structure recognition modules being configured to output respective indications of whether respective portions of the image data depict a first anatomical structure type or a second anatomical structure type. A first indication that the first portion of the image data depicts a first anatomical structure type is received from one or more anatomical structure recognition modules. In response to the first indication, a first CAD model is deployed to analyze the first portion of the image data, wherein the first CAD model is configured to generate annotations associated with abnormalities depicted in the image of the first anatomical structure type. At a second time after the first time during the medical examination, a second portion of the image data is provided to the one or more anatomical structure recognition modules. Receiving a second indication from one or more anatomical structure identification modules that a second portion of the image data depicts a second anatomical structure type, and, in response to the second indication, deploying a second CAD model to analyze the second portion of the image data, wherein the second CAD model is configured to generate an annotation associated with an abnormality depicted in the image of the second anatomical structure type.

[0096] In Example 20, the subject matter of Example 19 may optionally include computer executable instructions that further cause one or more processing units to: receive an output from the first CAD model indicating that an anomaly is detected within a first portion of the image data; and in response to the output from the first CAD model, deploy a third CAD model to analyze at least some of the first portion of the image data, wherein the third CAD model is configured to generate an output that classifies the anomaly detected by the first CAD model.

[0097] In Example 21, the subject matter of Example 20 may optionally include deploying the third CAD model by activating an alternative illumination modality on the medical imaging device.

[0098] In Example 22, the subject matter of Example 21 can optionally include a narrow-band imaging modality where the alternative illumination modality is blue light or red light.

[0099] In Example 23, the subject matter of any one of Examples 20 to 22 may optionally include that the third CAD model is configured to output a tissue classification and a confidence score associated with the tissue classification.

[0100] In Example 24, the subject matter of any one of Examples 19 to 23 may optionally include that the first anatomical structure type is squamous mucosa corresponding to the esophageal anatomical region, and the second anatomical structure type is gastric mucosa corresponding to the gastric anatomical region.

[0101] In Example 25, the subject matter of any one of Examples 19 to 24 may optionally include that the image data comprises a sequence of a plurality of image frames generated by an endoscope during a medical examination.

[0102] In Example 26, the subject matter of any one of Examples 19 to 25 may optionally include deploying the first CAD module by outputting an indication of the first anatomical structure type.

[0103] In Example 27, the subject matter of any one of Examples 19 to 26 may optionally include that deploying the second CAD model comprises: outputting an indication of the second anatomical structure type.

Claims

1. A digestive (GI) tract mucosa recognition system for training a machine learning model for use in a computer-based clinical decision support system to assist in the real-time selection of a computer-aided detection and characterization module trained for a specific anatomical region, the digestive tract mucosa recognition system comprising: a database of mucosal images captured from various parts of the digestive tract; Processing circuit; as well as a memory comprising instructions that, when executed by the processing circuit, cause the processing circuit to perform operations comprising: labeling a plurality of mucosal images from the database based on locations within the digestive tract to generate training data; training a machine learning model based at least in part on the training data to predict a position of the endoscope within the digestive tract based on real-time mucosal imaging from the endoscope; and The machine learning model is stored.

2. A method for training a machine learning model for use in a computer-based clinical decision support system that provides real-time selection of a computer-aided detection and characterization module trained for specialized identification or diagnosis of abnormalities within a specific anatomical region, the method comprising: receiving data captured by an endoscope including images of mucosa from various portions of the digestive tract; labeling the received data based on the location within the digestive tract to generate training data; training the machine learning model based at least in part on the training data to predict a position of the endoscope within the digestive tract based on real-time mucosal imaging from the endoscope; as well as Output the machine learning model.

3. The method according to claim 2, wherein: Receiving the data includes: receiving an endoscopic image including esophageal mucosa, gastric mucosa, and duodenal mucosa.

4. The method according to claim 2, wherein: The training of the machine learning model includes: training an upper digestive tract model and a lower digestive tract model.

5. The method according to claim 4, wherein: The training of the lower gastrointestinal tract model includes using training data specific to the lower gastrointestinal tract.

6. The method according to claim 5, wherein: The training data specific to the lower digestive tract includes labeled endoscopic images from a set of images selected from one or more of the following anatomical regions: Cecum; Ascending colon; Right flexure of colon; transverse colon; left flexure of colon; Descending colon; Sigmoid colon; rectum; and Anal canal.

7. An endoscopy clinical support system for real-time activation of a computer-aided detection and characterization module trained for specialized identification or diagnosis of abnormalities within a patient's digestive (GI) tract, the system comprising: an endoscope comprising an illumination component and a sensor configured to generate medical image data; a clinical support computing device communicatively coupled to the endoscope, the clinical support computing device comprising: Processing circuit; a display device configured to display the medical image data and associated user interface graphics; and a memory comprising instructions that, when executed by the processing circuit, cause the processing circuit to perform operations comprising: activating a first mucosal identification module and a first plurality of detection and characterization modules based at least in part on detecting the connection with the endoscope; monitoring the image stream from the endoscope using the first mucosal identification module to identify mucosal tissue types; and Based on identifying the mucosal tissue type as a first mucosal tissue type, executing a first detection module of the first plurality of detection and characterization modules; and Images from the image stream received from the endoscope are processed using the first detection module to identify a region of interest to be output to the display device, wherein the region of interest identifies potential abnormalities within the image.

8. The system according to claim 7, wherein: Activating the first mucosal recognition module is performed based on receiving endoscope type or model information identifying the endoscope.

9. The system according to claim 8, wherein: Receiving endoscope type or model information includes receiving illumination information identifying different types of illumination that the illumination component is capable of generating.

10. The system according to claim 7, wherein: The memory also includes instructions that, when executed by the processing circuit, cause the processing circuit to perform operations comprising: continuously monitoring images extracted from the image stream received from the endoscope using the first mucosal recognition module; and Based on the first mucosal identification module identifying the mucosal tissue type as a second mucosal tissue type, a second detection module of the first plurality of detection and characterization modules is executed.

11. The system according to claim 10, wherein: Executing the second detection module includes outputting to the display device an indication that the endoscope has begun imaging different regions of the digestive tract.

12. The system according to claim 7, wherein: Monitoring the image stream from the endoscope using the first mucosal recognition module includes: implementing a machine learning model to predict the position of the endoscope within the digestive tract based on real-time mucosal imaging from the endoscope, the machine learning model being trained at least in part based on training data, the training data including multiple labeled mucosal images from different locations within the digestive tract.

13. The system according to claim 7, wherein: Monitoring the image stream using the first mucosal recognition module includes periodically processing images extracted from the image stream using the first mucosal recognition module.

14. The system according to claim 7, wherein: Executing the first detection module includes selecting a first imaging modality from a plurality of imaging modalities supported by the endoscope.

15. The system of claim 14, wherein: Identifying the region of interest includes selecting a second imaging modality from the plurality of imaging modalities supported by the endoscope.

16. The system of claim 15, wherein: The memory also includes instructions that, when executed by the processing circuit, cause the processing circuit to perform operations including executing a first characterization module of the plurality of detection and characterization modules based at least in part on identifying the region of interest.

17. The system of claim 16, wherein: Executing the first characterization module includes outputting an indication of a tissue classification and a corresponding confidence score associated with the region of interest to the display device.

18. The system of claim 14, wherein: Identifying the region of interest includes displaying a user prompt on the display device, the user prompt providing an option to activate a second imaging modality of the plurality of imaging modalities supported by the endoscope.

19. A system comprising: one or more processing units; as well as A computer-readable medium having computer-executable instructions encoded thereon, the computer-executable instructions causing the one or more processing units to: receiving image data generated by a medical imaging device during a medical examination; providing a first portion of the image data at a first time during the medical examination to one or more anatomy identification modules, the one or more anatomy identification modules configured to output respective indications of whether respective portions of the image data depict a first anatomical structure type or a second anatomical structure type; receiving, from the one or more anatomical structure identification modules, a first indication that the first portion of the image data depicts the first anatomical structure type; In response to the first indication, deploying a first CAD model to analyze the first portion of the image data, wherein the first CAD model is configured to generate annotations associated with abnormalities depicted in the image of the first anatomical structure type; providing a second portion of the image data to the one or more anatomical structure identification modules at a second time after the first time during the medical examination; receiving from the one or more anatomical structure identification modules a second indication that the second portion of the image data depicts the second anatomical structure type; and In response to the second indication, a second CAD model is deployed to analyze the second portion of the image data, wherein the second CAD model is configured to generate annotations associated with abnormalities depicted in the image of the second anatomical structure type.

20. The system of claim 19, wherein: The computer executable instructions further cause the one or more processing units to: receiving an output from the first CAD model, the output indicating that an anomaly was detected within the first portion of the image data; as well as In response to the output from the first CAD model, a third CAD model is deployed to analyze at least some of the first portion of the image data, wherein the third CAD model is configured to generate an output that classifies the anomalies detected by the first CAD model.

21. The system of claim 20, wherein: Deploying the third CAD model includes activating an alternative illumination modality on the medical imaging device.

22. The system of claim 21, wherein: The alternative illumination modality is a narrow-band imaging modality of blue or red light.

23. The system of claim 20, wherein: The third CAD model is also configured to output a tissue classification and a confidence score associated with the tissue classification.

24. The system of claim 19, wherein: The first anatomical structure type is squamous mucosa corresponding to the esophageal anatomical region, and the second anatomical structure type is gastric mucosa corresponding to the gastric anatomical region.

25. The system of claim 19, wherein: The image data comprises a sequence of a plurality of image frames generated by an endoscope during the medical examination.

26. The system of claim 19, wherein: Deploying the first CAD module includes outputting an indication of the first anatomical structure type.

27. The system of claim 19, wherein: Deploying the second CAD model includes outputting an indication of the second anatomical structure type.

Images