Graphic user interface having dynamic spatial indicators of images in a sequence

Abstract

A system for presenting in-vivo studies includes processor(s) and memory storing instructions. The instructions, when executed, cause the system to perform: displaying a graphical user interface (GUI) configured to present at least a portion of an in-vivo study, where the in-vivo study includes a sequence of in-vivo images of a gastrointestinal tract, and where the GUI includes: a user interface element configured to be engaged by a user to adjust a setting for an artificial intelligence (AI) detector, and a graphical spatial representation of an entirety of the sequence of in-vivo images; receiving a setting for the AI detector based on a user engagement of the user interface element; and displaying a plurality of indications in spatial relation to the graphical spatial representation, where the spatial relation corresponds to locations of images, of the sequence of in-vivo images, which are selected based on the setting for the AI detector.

Description

GRAPHIC USER INTERFACE HAVINGDYNAMIC SPATIAL INDICATORS OF IMAGES IN A SEQUENCECROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to and the benefit of U.S. Provisional Application No. 63 / 593,051, filed October 25, 2023, which is hereby incorporated by reference herein in its entirety.TECHNICAL FIELD

[0002] The present disclosure relates to endoscopy images, and more particularly, to a graphic user interface for viewing endoscopy images.BACKGROUND

[0003] Endoscopy allows examining of a gastrointestinal tract (GIT) endoscopically. There are capsule endoscopy systems and methods that are aimed at examining a specific portion of the GIT, such as the small bowel (SB) or the colon. CE is a non-invasive procedure which does not require the patient to be admitted to a hospital, and the patient can continue most daily activities while the capsule is in his body.

[0004] For a typical CE procedure, the patient is referred to a procedure by a physician. The patient then arrives at a medical facility (e.g., a clinic or a hospital), to perform the procedure. The capsule, which is about the size of a multi-vitamin, is swallowed by the patient under the supervision of a healthcare professional (e.g., a nurse or a physician) at the medical facility and the patient is provided with a wearable device (e.g., a belt having a recorder in a pouch and a strap to be placed around the patient’s shoulder). The wearable device typically includes a storage device. The patient may be given guidance and / or instructions and then is released to his or her daily activities.

[0005] The capsule captures images as it travels naturally through the GIT. Images and additional data (e.g., metadata) are then transmitted to the recorder that is worn by the patient. The capsule is typically disposable and passes naturally with a bowel movement. The procedure data(e.g., the captured images or a portion of them and additional metadata) is stored in the storage device of the wearable device.

[0006] The procedure data is uploaded from the wearable device to a computing system, which has a software engine stored thereon. The procedure data is then processed by the software engine to generate a compiled study. Typically, the number of images in the procedure data to be processed is of the order of tens of thousands, and the generated study typically includes thousands of images.

[0007] A reader (which may be the procedure supervising physician, a dedicated physician, or the referring physician) may access the study via a reader application. The reader then reviews the study, evaluates the procedure, and provides input via the reader application. Since the reader needs to review thousands of images, the reading time of a study may usually take between half an hour to an hour on average and the reading task may be tiresome. A report is then generated by the reader application based on the compiled study and the reader’s input. On average, it may take an hour to generate a report. The report may include, for example, images of interest (e.g., images which are identified as including pathologies) selected by the reader; evaluation or diagnosis of the patient’s medical condition based on the procedure’s data (i.e., the study), and / or recommendations for follow up and / or treatment provided by the reader. The report may then be forwarded to a referring physician. The referring physician may decide on a required follow up or treatment based on the report.SUMMARY

[0008] The present disclosure relates to a graphic user interface for viewing endoscopy images.

[0009] In accordance with aspects of the present disclosure, a system for presenting in-vivo studies includes: at least one processor and at least one memory storing instructions. The instructions, when executed by the at least one processor, cause the system at least to perform: displaying, on a screen, a graphical user interface configured to present at least a portion of an in- vivo study, where the in-vivo study includes a sequence of in-vivo images of at least a portion of a gastrointestinal tract (GIT) of a person, where the graphical user interface includes: a user interface element configured to be engaged by a user to adjust a setting for an artificial intelligence detector, and a graphical spatial representation of an entirety of the sequence of in-vivo images; receiving a setting for the artificial intelligence detector based on a user engagement of the userinterface element; and displaying a plurality of indications in spatial relation to the graphical spatial representation, where the spatial relation corresponds to locations of images, of the sequence of in-vivo images, which are selected based on the setting for the artificial intelligence detector.

[0010] In various embodiments of the system, the user interface element includes at least three selectable settings, and the setting received for the artificial intelligence detector is one of the at least three selectable settings.

[0011] In various embodiments of the system, the at least three selectable settings include a first selectable setting and a second selectable setting. The first selectable setting sets a lower threshold than the second selectable setting for classifying an object in an image as an indicator of a pathology, resulting in a greater number of indications in the plurality of indications. The second selectable setting sets a higher threshold than the first selectable setting for classifying an object in an image as an indicator of a pathology, resulting in a smaller number of indications in the plurality of indications.

[0012] In various embodiments of the system, the artificial intelligence detector classifies an object in an image as an indicator of a pathology based on at least one classification score.

[0013] In various embodiments of the system, the graphical spatial representation of the entirety of the sequence of in-vivo images is a bar, and the plurality of indications are lines in the bar, where the lines are brighter than a brightness of the bar.

[0014] In various embodiments of the system, the instructions, when executed by the at least one processor, further cause the system at least to perform: transmitting, to a server, the setting received for the artificial intelligence detector, where the server is configured to execute the artificial intelligence detector; and receiving, from the server, information for generating the plurality of indications.

[0015] In various embodiments of the system, the instructions, when executed by the at least one processor, further cause the system at least to perform: in a case the user interface element is set to a highest certainty setting for the artificial intelligence detector, displaying surrounding images for the images, of the sequence of in-vivo images, which are selected based on the setting for the artificial intelligence detector.

[0016] In accordance with aspects of the present disclosure, a method for presenting in-vivo studies includes: displaying, on a screen, a graphical user interface configured to present at least aportion of an in-vivo study, where the in-vivo study includes a sequence of in-vivo images of at least a portion of a gastrointestinal tract (GIT) of a person, where the graphical user interface includes: a user interface element configured to be engaged by a user to adjust a setting for an artificial intelligence detector, and a graphical spatial representation of an entirety of the sequence of in-vivo images; receiving a setting for the artificial intelligence detector based on a user engagement of the user interface element; and displaying a plurality of indications in spatial relation to the graphical spatial representation, where the spatial relation corresponds to locations of images, of the sequence of in-vivo images, which are selected based on the setting for the artificial intelligence detector.

[0017] In various embodiments of the method, the user interface element includes at least three selectable settings, and the setting received for the artificial intelligence detector is one of the at least three selectable settings.

[0018] In various embodiments of the method, the at least three selectable settings include a first selectable setting and a second selectable setting. The first selectable setting sets a lower threshold than the second selectable setting for classifying an object in an image as an indicator of a pathology, resulting in a greater number of indications in the plurality of indications. The second selectable setting sets a higher threshold than the first selectable setting for classifying an object in an image as an indicator of a pathology, resulting in a smaller number of indications in the plurality of indications.

[0019] In various embodiments of the method, the artificial intelligence detector classifies an object in an image as an indicator of a pathology based on at least one classification score.

[0020] In various embodiments of the method, the graphical spatial representation of the entirety of the sequence of in-vivo images is a bar, and the plurality of indications are lines in the bar, where the lines are brighter than a brightness of the bar.

[0021] In various embodiments of the method, the method further includes: transmitting, to a server, the setting received for the artificial intelligence detector, where the server is configured to execute the artificial intelligence detector; and receiving, from the server, information for generating the plurality of indications.

[0022] In various embodiments of the method, the method further includes: in a case the user interface element is set to a highest certainty setting for the artificial intelligence detector,displaying surrounding images for the images, of the sequence of in-vivo images, which are selected based on the setting for the artificial intelligence detector.

[0023] In accordance with aspects of the present disclosure, a processor-readable medium stores instructions which, when executed by at least one processor of a system, cause the system at least to perform: displaying, on a screen, a graphical user interface configured to present at least a portion of an in-vivo study, where the in-vivo study includes a sequence of in-vivo images of at least a portion of a gastrointestinal tract (GIT) of a person, where the graphical user interface includes: a user interface element configured to be engaged by a user to adjust a setting for an artificial intelligence detector, and a graphical spatial representation of an entirety of the sequence of in-vivo images; receiving a setting for the artificial intelligence detector based on a user engagement of the user interface element; and displaying a plurality of indications in spatial relation to the graphical spatial representation, where the spatial relation corresponds to images, of the sequence of in-vivo images, which are selected based on the setting for the artificial intelligence detector.

[0024] In various embodiments of the processor-readable medium, the user interface element includes at least three selectable settings, and the setting received for the artificial intelligence detector is one of the at least three selectable settings.

[0025] In various embodiments of the processor-readable medium, the at least three selectable settings include a first selectable setting and a second selectable setting. The first selectable setting sets a lower threshold than the second selectable setting for classifying an object in an image as an indicator of a pathology, resulting in a greater number of indications in the plurality of indications. The second selectable setting sets a higher threshold than the first selectable setting for classifying an object in an image as an indicator of a pathology, resulting in a smaller number of indications in the plurality of indications.

[0026] In various embodiments of the processor-readable medium, the artificial intelligence detector classifies an object in an image as an indicator of a pathology based on at least one classification score.

[0027] In various embodiments of the processor-readable medium, the graphical spatial representation of the entirety of the sequence of in-vivo images is a bar, and the plurality of indications are lines in the bar, where the lines are brighter than a brightness of the bar.

[0028] In various embodiments of the processor-readable medium, the instructions, when executed by the at least one processor, further cause the system at least to perform: transmitting, to a server, the setting received for the artificial intelligence detector, where the server is configured to execute the artificial intelligence detector; and receiving, from the server, information for generating the plurality of indications.

[0029] In various embodiments of the processor-readable medium, the instructions, when executed by the at least one processor, further cause the system at least to perform: in a case the user interface element is set to a highest certainty setting for the artificial intelligence detector, displaying surrounding images for the images, of the sequence of in-vivo images, which are selected based on the setting for the artificial intelligence detector.

[0030] The details of one or more embodiments of the disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the techniques described in this disclosure will be apparent from the description and drawings, and from the claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0031] A detailed description of embodiments of the disclosure will be made with reference to the accompanying drawings, wherein like numerals designate corresponding parts in the figures:

[0032] FIG. l is a diagram of a gastrointestinal tract (GIT), in accordance with aspects of the present disclosure;

[0033] FIG. 2 is a block diagram of an example of a system for analyzing medical images captured in-vivo via a capsule endoscopy (CE) procedure and for providing the images to a user system or device, in accordance with aspects of the disclosure;

[0034] FIG. 3 is a block diagram of an example of components of a computing system or of a user system, in accordance with aspects of the disclosure;

[0035] FIG. 4 is a diagram of an example of a graphical user interface having one setting of a user interface element, in accordance with aspects of the disclosure;

[0036] FIG. 5 is a diagram of the graphical user interface having another setting of the user interface element, in accordance with aspects of the disclosure;

[0037] FIG. 6 is a diagram of the graphical user interface having yet another setting of the user interface element, in accordance with aspects of the disclosure;

[0038] FIG. 7 is a diagram of another example of a graphical user interface having a setting of the user interface element, in accordance with aspects of the disclosure;

[0039] FIG. 8 is a diagram of the graphical user interface of FIG. 8 having another setting of the user interface element, in accordance with aspects of the disclosure; and

[0040] FIG. 9 is a flow diagram of an example of an operation, in accordance with aspects of the disclosure.DETAILED DESCRIPTION

[0041] The present disclosure relates to a graphic user interface (GUI) for viewing endoscopy images. A healthcare profession reads an in-vivo study to look for indications of a pathology, such as bleeding, polyps, inflammation, Crohn’s disease, Celiac disease, and / or diverticulitis, among other pathologies. Machine learning has come a long way in being able to detect one or more such pathologies. For example, detection of indicators of Celiac disease using machine learning is disclosed in International Publication No. WO2022107124, which is hereby incorporated by reference herein in its entirety.

[0042] Aspects of the present disclosure relate to a GUI that allows a user to adjust one or more settings of an artificial intelligence detector and that graphically indicates which images of an in-vivo study are selected based on the settings. As a user changes the one or more settings, the graphical indications of which images are selected based on the settings will also change correspondingly. Generally, the artificial intelligence settings may be changed to set a lower threshold or a higher threshold for detecting pathologies. A lower threshold for detecting pathologies may yield a greater number of images that may contain pathologies (and have possibly higher false positives), and a higher threshold for detecting pathologies may yield a smaller number of images (and have possibly lower false positives). Accordingly, in aspects, a user may adjust the user interface element to also control the number of images that are provided for review.

[0043] In the following detailed description, specific details are set forth in order to provide a thorough understanding of the disclosure. However, it will be understood by those skilled in the art that the disclosure may be practiced without these specific details. In other instances, well- known methods, procedures, and components have not been described in detail so as not to obscure the present disclosure.

[0044] To the extent consistent, any or all of the aspects, embodiments, and examples detailed herein may be used in conjunction with any or all of the other aspects or embodiments detailed herein.

[0045] Reference throughout this specification to “one aspect” or “an aspect” means that a particular feature, structure, or characteristic described in connection with the aspect is included in at least one aspect. Thus, the appearances of the phrases “in one aspect” or “in an aspect” in various places throughout this specification are not necessarily all referring to the same aspect. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more aspects.

[0046] Although the disclosure is not limited in this regard, discussions utilizing terms such as, for example, “processing,” “computing,” “calculating,” “determining,” “establishing,” “analyzing,” “checking,” or the like, may refer to operation(s) and / or process(es) of a processor, a computing platform, a computing system, or other electronic computing device, that manipulates and / or transforms data represented as physical (e.g., electronic) quantities within computing registers and / or memories into other data similarly represented as physical quantities within the computing registers and / or memories or other non-transitory information storage medium that may store instructions to perform operations and / or processes.

[0047] Although the disclosure is not limited in this regard, the terms “plurality” and “a plurality” as used herein may include, for example, “multiple” or “two or more.” The terms “plurality” or “a plurality” may be used throughout the specification to describe two or more components, devices, elements, units, parameters, or the like.

[0048] As used herein, the term “exemplary” means “an example” and is not intended to mean preferred. Unless explicitly stated, the methods described herein are not constrained to a particular order or sequence. Additionally, some of the described methods or elements thereof can occur or be performed simultaneously, at the same point in time, or concurrently.

[0049] Depending on the context, the term “GIT” may mean a portion of the gastrointestinal tract and / or the entirety of a gastrointestinal tract. Thus, disclosures relating to a GIT may apply to a portion of the GIT and / or the entirety of a GIT.

[0050] The terms “image” and “frame” may each refer to or include the other and may be used interchangeably in the present disclosure to refer to a single capture by an imaging device. For convenience, the term “image” may be used more frequently in the present disclosure, but it willbe understood that references to an image shall apply to a frame as well. The term “in-vivo study” means and includes a compilation of medical images that are captured in-vivo by an imaging device for a single in-vivo procedure. An in-vivo study may include any number of such medical images for a single in-vivo procedure, which may be a capsule endoscopy procedure or may be another type of in-vivo procedure.

[0051] The present disclosure may refer to in-vivo images or studies that have an “indication” of one or more GIT conditions. Such descriptions do not mean and are not intended to mean that such images or studies definitely show the GIT condition(s). Additionally, the present disclosure may refer to in-vivo images or studies that have an “indication” of no GIT conditions. Such descriptions do not mean and are not intended to mean that such images or studies definitely do not show the GIT condition(s). Rather, such descriptions mean and are intended to mean that the images or studies merely have such indications.

[0052] The term “machine learning” means and includes any technique which analyzes existing data to learn a model between inputs and outputs in the existing data. The terms “artificial intelligence” and “machine learning model” mean and include any implementation of the learned model, in software and / or hardware, that can receive new input data and that can predict / infer output data by applying the learned model to the new input data. Machine learning may include supervised learning and unsupervised learning, among other things. Examples of machine learning models include, without limitation, deep learning neural networks and support vector machines, among other things. The terms “artificial intelligence” and “machine learning” will be used interchangeably herein.

[0053] As used herein, the term “device” means and refers to any physical product that can provide an associated function and may include a physical product implemented in a single housing or include a physical product implemented in more than one housing.

[0054] The following description refers to images captured by a capsule endoscopy device. However, it is intended and shall be understood that the following description applies to other manners of obtaining images of a GIT or portion of a GIT, as well.

[0055] Referring to FIG. 1, an illustration of a gastrointestinal tract (GIT) 100 in a human is shown. The GIT 100 is an organ system within humans and animals. The GIT 100 generally includes a mouth 102 for taking in sustenance, salivary glands 104 for producing saliva, an esophagus 106 through which food passes aided by contractions, a stomach 108 to secret enzymesand stomach acid to aid in digesting food, a liver 110, a gall bladder 112, a pancreas 114, a small intestine / small bowel 116 (“SB”) for the absorption of nutrients, and a colon 40 (e.g., large intestine) for storing water and waste material as feces prior to defecation. The colon 40 generally includes an appendix 42, a rectum 48, and an anus 43. Food taken in through the mouth is digested by the GIT to take in nutrients and the remaining waste is expelled as feces through the anus 43.

[0056] The type of procedure performed may determine which portion of the GIT 100 is the portion of interest. Examples of types of procedures performed include, without limitation, a procedure aimed to specifically exhibit or check the small bowel, a procedure aimed to specifically exhibit or check the colon, a procedure aimed to specifically exhibit or check the colon and the small bowel, or a procedure to exhibit or check the entire GIT: esophagus, stomach, SB, and colon, among other possibilities.

[0057] FIG. 2 shows a block diagram of a system for analyzing medical images captured in- vivo via a capsule endoscopy (“CE”) procedure and providing the images to a user system or device. The system generally includes a capsule system 210 configured to capture images of the GIT and a computing system 230 (e.g., local system and / or cloud system) configured to process the captured images.

[0058] The capsule system 210 includes a swallowable CE imaging device 212 (e.g., a capsule) configured to capture images of the GIT as the CE imaging device 212 travels through the GIT. The images may be stored on the CE imaging device 212 and / or transmitted to a receiving device 214, typically via an antenna. In some capsule systems 210, the receiving device 214 may be located on the patient who swallowed the CE imaging device 212 and may, for example, take the form of a belt worn by the patient or a patch secured to the patient.

[0059] The capsule system 210 may be communicatively coupled with the computing system 230 and can communicate captured images to the computing system 230. The computing system 230 processes the received images using image processing technologies, machine learning technologies, and / or signal processing technologies, among other technologies. The computing system 230 may include local computing devices that are local to the patient and / or local to the patient’s treatment facility, a cloud computing platform that is provided by cloud services, or a combination of local computing devices and a cloud computing platform.

[0060] In the case where the computing system 230 includes a cloud computing platform, the images captured by the capsule system 210 may be transmitted to the cloud computing platform.In various embodiments, the images can be transmitted by or via the receiving device 214 worn or carried by the patient. In various embodiments, the images can be transmitted via the patient’s smartphone or via any other device which is connected to the Internet and which may be coupled with the CE imaging device 212 or the receiving device 214.

[0061] The images processed by the computing system 230 and / or the processing results may be communicated to a user system or user device 250 for a healthcare professional to read. In various embodiments, the computing system 230 may host a downloadable application and may provide the downloadable application to the user system / device 250. The user system / device 250 may download, install, and execute the application. In such embodiments, the application executing on the user system / device 250 may communicate with the computing system 230 to obtain study images and study information and then display such images and information locally on the user system / device 250 in a graphical user interface (GUI). In various embodiments, the computing system 230 may provide a web service, and the user system / device 250 may access the web service using a web browser. In such embodiments, the computing system 230 provides the study images and study information to the user system / device 250 in a GUI as website content. Such examples are merely illustrative and do not limit how a computing system may provide study images and study information to a user system / device.

[0062] FIG. 3 shows a block diagram of example components of the computing system 230 of FIG. 2 and / or the user system / device 250 of FIG. 3. The system 300 includes a processor 305, an operating system 315, a memory 320, a communication device 322, a storage 330, input devices 335, and output devices 340. The communication device 322 of the system 300 may allow communications with other systems or devices via a wired network (e.g., Ethernet) and / or a wireless network (e.g., Wi-Fi, cellular network, etc.).

[0063] The processor 305 may be or may include one or more central processing units (CPU), graphics processing unit (GPU), controllers, microcontrollers, microprocessors, and / or other computational devices. The operating system 315 may be or may include any code segment designed and / or configured to perform tasks involving coordination, scheduling, arbitration, supervising, controlling or otherwise managing operation of system 300, for example, scheduling execution of programs. Memory 320 may be or may include, for example, a Random Access Memory (RAM), a read-only memory (ROM), a Dynamic RAM (DRAM), a Synchronous DRAM (SD-RAM), a double data rate (DDR) memory chip, a Flash memory, a volatile memory, a non-volatile memory, a cache memory, a buffer, a short term memory, a long term memory, and / or other memory devices. The memory 320 stores executable code 325 that implements the data and operations of the present disclosure, which will be described later herein. Executable code 325 may be any executable code, e.g., an application, a program, a process, task, or script. Executable code 325 may be executed by the processor 305 possibly under control of operating system 315.

[0064] Storage 330 may be or may include, for example, a hard disk drive, a solid-state drive (SSD), a digital versatile disc (DVD), a universal serial bus (USB) device, and / or other removable and / or fixed device for storing electronic data. Instruct! ons / code and data (e.g., images) may be stored in the storage 330 and may be loaded from the storage 330 into the memory 320, where it may be processed by processor 305.

[0065] Input devices 335 may include, for example, a mouse, a keyboard, a touch screen, and / or any other device that can receive an input. Output devices 340 may include one or more monitors, screens, displays, speakers, and / or any other device that can provide an output.

[0066] Other aspects of the system 300 and the capsule system (210, FIG. 2) are described in International Publication No. WO2020236683A1, entitled “Systems and Methods For Capsule Endoscopy Procedure,” which is hereby incorporated by reference in its entirety. Generally, the technology of the present disclosure may be utilized by capsule endoscopy systems or methods and may be presented in a user interface, such as the example user interfaces described in International Publication No. W02020079696, entitled “Systems and Methods for Generating and Displaying a Study of a Stream of In-Vivo Images,” which is hereby incorporated by reference herein in its entirety.

[0067] As mentioned above, aspects of the present disclosure relate to a GUI that allows a user to adjust one or more artificial intelligence settings and that graphically indicates which images of an in-vivo study are selected based on the settings. As described above, in various embodiments, an application may execute on a user system / device and may display study images and study information locally on the user system / device 250 in a GUI. In various embodiments, a user system / device may access a web service using a web browser, and the web service may provide the study images and study information to the user system / device in a GUI as website content. Such examples are merely illustrative, and other implementations are contemplated to be within the scope of the present disclosure.

[0068] FIG. 4 is a diagram of an example of a GUI. The GUI 400 includes a region 410 for displaying images of an in-vivo study, a user interface element 420 for adjusting a setting for an artificial intelligence detector, and a graphical spatial representation 430 of the entirety of the images in the in-vivo study. The arrangement, shapes, and dimensions of the GUI 400 and the various components of the GUI 400 are merely examples. The region 410 for displaying images may include one or more bounding boxes 412 that indicate the location(s) of one or more pathologies detected by the artificial intelligence detector (not shown). Referring also to FIG. 2, the artificial intelligence detector may execute in the computing system 230, which may be a server, such as a proprietary server, a cloud server, and / or an on-site server, among other possibilities.

[0069] In the illustrated embodiment of FIG. 4, the user interface element 420 is a slider that has multiple positions corresponding to different settings for the artificial intelligence detector. The slider is merely an example. In various embodiments, the user interface element 420 may be any interface element that has multiple selections or positions (e.g., lists, buttons, spin wheels, etc.) and that enables a user to choose a setting. The user interface element 420 may have any number of positions or selections, such as two positions or selections, three positions or selections, or more than three positions or selections. In various embodiments, the GUI 400 may include multiple user interface elements (not shown) corresponding to multiple settings for the artificial intelligence detector.

[0070] In FIGS. 4-8, selections made in the user interface element 420 determine the extent of images that a user can review. Using the slider as an example, the user interface element 420 has one end labeled as “Details” and the other end labeled as “Overview.” When the “Details” end of the user interface element 420 is selected, the entirety of the images in the in-vivo study is available for the user to review. This, in various embodiments, the “Details” end may be labeled as “Full Video” instead of “Details.” When the “Overview” end of the user interface element 420 is selected, generally much fewer images will be available for the user to review. As explained in more detail below, the fewer images for review may result from setting the artificial intelligence detector to a high certainty threshold for selecting images of interest so that only high certainty images are available for a user to review.

[0071] In the illustrated embodiment of FIG. 4, the graphical spatial representation 430 of the entirety of the images in the in-vivo study is in the form of a bar, where the beginning of the bar(e.g., left end) corresponds to the first image of the in-vivo study and the end of the bar (e.g., right end) correspond to the last image of the in-vivo study. Indications 432 (e.g., lines) are shown in spatial relation to the graphical spatial representation 430 to indicate the locations of images in the in-vivo study which have resulted from the artificial intelligence detector setting corresponding to position / selection 422 of the user interface element 420. To clearly display the indications 432, the indications 432 may be brighter than the brightness of the graphical spatial representation 430. For example, in FIG. 4, the bar may have a dark gray color, while the lines may be brightly colored. In various embodiments, the indications 432 may create a “color bar” by, for example, using a representation of average colors in the respective images as the indication 432. An example of creating a color bar using average color of images is described in U.S. Patent No. 7,215,338, which is incorporated by reference herein in its entirety. A bar is just one example of a graphical spatial representation 430, and lines are just one example of indications 432. In various embodiments, the graphical spatial representation 430 and the indications 432 may have other shapes.

[0072] With continuing reference to FIG. 4, the user interface element 420 has the illustrated position / selection 422, which corresponds to a setting for an artificial intelligence detector. Accordingly, the position / selection 422 may also be characterized as a setting 422 for the artificial intelligence detector. The artificial intelligence detector may execute in the computing system 230. Meanwhile, the GUI may be implemented or may execute in a user system or device (e.g., 250, FIG. 2). In aspects of the present disclosure, when a user changes the setting 422 for the artificial intelligence detector (by engaging the user interface element 420), the user system or device receives the setting 422 from the user input and transmits the setting 422 to the computing system 230. In various embodiments, the computing system 230 may execute the artificial intelligence detector on the images in the in-vivo study using the setting 422 to determine which images of the in-vivo study are selected based on setting 422. In various embodiments, the computing system 230 may be able to determine which images of the in-vivo study are selected based on the setting 422 without re-executing the artificial intelligence detector on all images of the in-vivo study (an example is provided below). The computing system 230 transmits, to the user system or device 250, information about which images are selected based on the setting 422, and the user system or device 250 receives the information to generate the indications 432. In the example of FIG. 4, indications 432 occupy almost the entirety of the graphical spatial representation 430. Therefore,the computing system 230 has determined that almost all of the images of the in-vivo study are selected based on the artificial intelligence detector setting 422 of the user interface element 420.

[0073] In contrast, FIG. 5 and FIG. 6 show different settings 424, 426 of the user interface element 420 and show different indications 434, 436 in the graphical spatial representation 430 corresponding to the settings. In FIG. 5, the setting 424 corresponds to fewer indications 434, and in FIG. 6, the setting 426 corresponds to even fewer indications 436.

[0074] When a setting of the user interface element 420 results in indications that are spaced apart, such as the indications 436 of FIG. 6, a user reviewing the images corresponding to the spaced apart indications may see “skipping” when reviewing the images. Such visual skipping may be visually disorienting to the user. In accordance with aspects of the present disclosure, when displaying a target image, a number of images before the target image and / or after the target image may be displayed, as well, so that the user’s review of the target image is smoother and the “skipping” effect is reduced. In various embodiments, a number of images surrounding a target image may be displayed when the user interface element 420 is set to the “Overview” setting. In various embodiments, a number n of images surrounding a target image may be displayed whenever two target images to be displayed are separated by more than a threshold number of images. Such and other variations are contemplated to be within the scope of the present disclosure.

[0075] The differences in the quantity of indications 432, 434, 436 in the graphic spatial representation 430, among FIGS. 4-6, are attributable to the different settings 422, 424, 426 for the artificial intelligence detector, as set by the user interface element 420. In accordance with aspects of the present disclosure, the different settings 422, 424, 426 correspond to different thresholds used by the artificial intelligence detector to classify an object in an image as an indicator of a pathology. Such an object may be, for example, blood or a polyp. Machine learning models for detecting and classifying objects that are indicators of a pathology will be understood by persons skilled in the art. For example, a machine learning model for detecting polyps is disclosed in International Publication No. W02022054046A2, which is hereby incorporated by reference herein in its entirety.

[0076] An artificial intelligence detector may produce one or more classification scores and may classify an object in an image as an indicator of a pathology based on the classification score(s). Persons skilled in the art will understand classification scores and will understand that the classification score(s) may be compared to one or more thresholds to determine how an objectin an image should be classified. In aspects of the present disclosure, a lower threshold for classifying an object in an image as an indicator of a pathology would result in a greater number of images detected as having an indicator of a pathology, and a higher threshold for classifying an object in an image as an indicator of a pathology would result in a smaller number of images detected as having an indicator of a pathology. Accordingly, in aspects of the present disclosure, the settings 422, 424, 426 may be used to adjust a number of images to be reviewed. In various embodiments, changes to the user interface element correspond to changing the threshold(s) for classifying an object in an image. The classification scores produced by the artificial intelligence detector may be stored (e.g., in the user system / device and / or in the computing system) and may be compared to the changed threshold(s) when the user interface element is adjusted. In this manner, the artificial intelligence detector may not need to be re-executed when the user interface element is adjusted. Rather, the new threshold(s) may be compared to the stored classification scores (e.g., by the user system / device and / or by the computing system) to determine which images (and their corresponding locations) are selected as a result of the new threshold corresponding to the adjusted user interface element.

[0077] FIG. 7 shows another example of a graphical user interface (GUI) that includes the user interface element 420 and the graphical spatial representation 430. In the GUI of FIG. 7, the indications 730 (e.g., lines) shown in spatial relation to the graphical spatial representation 430 are not within the graphical spatial representation 430 itself. Rather, the indications 730 (e.g., lines) are shown adjacent to and outside of the graphical spatial representation 430, but still shown in spatial relation to the graphical spatial representation 430 to indicate the locations of images in the in-vivo study which are selected based on the artificial intelligence detector setting corresponding to position / selection 712 of the user interface element 420.

[0078] FIG. 8 shows the GUI of FIG. 7 with a different position / selection 812 of the user interface element 420. The number of indications 830 in FIG. 8 is less than the number of indications 730 in FIG. 7 due to the different position / selection of the user interface element 420.

[0079] The GUI of FIG. 7 and FIG. 8 includes another user interface element 720 (e.g., a checkbox, a toggle, etc.) that allows a user to turn on or turn off display of the indications 730, 830. When the user interface element 720 is set to turn off display of the indications 730, 830, the indications 730, 830 would not be shown. In the GUI of FIG. 7 and FIG. 8, the graphical spatialrepresentation 430 does not change and may be a “color bar,” as described above herein, that represents the entirety of images in the in-vivo study.

[0080] In various embodiments of FIG. 7 and FIG. 8, the user interface element 420 and the user interface element 720 may be unavailable and non-selectable by a user. For example, the functionality provided by the user interface element 420 and the user interface element 720 may be enhanced features that are not available in every implementation. In such situations, the user interface element 420 and the user interface element 720 may both be shown in gray color or in another visualization to indicate they are not available to be used, and they can be disabled.

[0081] In various embodiments of FIG. 7 and FIG. 8, the user interface element 720 can be used to enable or disable the artificial intelligence detector. When the user interface element 720 is set to disable the artificial intelligence detector, the user interface element 420 becomes disabled and can be shown in gray color or another visualization to indicate that it is not available to be used. When the user interface element 720 is set to enable the artificial intelligence detector, the user interface element 420 becomes enabled and can be shown normally to indicate that it is available to be used.

[0082] The arrangement, shapes, and dimensions of the GUI and the various components of the GUI in FIGS. 4-8 are merely examples. Other embodiments are contemplated to be within the scope of the present disclosure.

[0083] Accordingly, GUIs and various operations have been described above. FIG. 9 shows an example of an operation from the perspective of the user system or device.

[0084] At block 910, the operation involves displaying, on a screen, a graphical user interface configured to present at least a portion of an in-vivo study, where the in-vivo study includes a sequence of in-vivo images of at least a portion of a gastrointestinal tract (GIT) of a person, and where the graphical user interface includes: a user interface element configured to be engaged by a user to adjust a setting for an artificial intelligence detector, and a graphical spatial representation of an entirety of the sequence of in-vivo images. In various embodiments, the GUI may have a user interface element and a graphical spatial representation as shown in FIGS. 4-6.

[0085] At block 920, the operation involves receiving a setting for the artificial intelligence detector based on a user engagement of the user interface element. The setting may be, for example, a setting for a threshold used by the artificial intelligence detector to classify an object in an image as an indicator of a pathology.

[0086] At block 930, the operation involves displaying a plurality of indications in spatial relation to the graphical spatial representation, where the spatial relation corresponds to locations of images, of the sequence of in-vivo images, which are selected based on the setting for the artificial intelligence detector. The graphical spatial representation may be a bar, and the plurality of indications may be lines, as described in connection with FIGS. 4-8. The lines are positioned in or adjacent to the bar according to locations of images of an in-vivo study which are selected based on the setting for the artificial intelligence detector.

[0087] Although not shown in FIG. 9, between blocks 920 and 930, the operation may involve transmitting, to a server, the setting received for the artificial intelligence detector (where the server is configured to execute the artificial intelligence detector), and receiving, from the server, information for generating the plurality of indications.

[0088] FIG. 9 is merely an example. In various embodiments, the operations may include other blocks not shown in FIG. 9. In various embodiments, the operations may not include every block shown in FIG. 9. Such and other embodiments are contemplated to be within the scope of the present disclosure.

[0089] The embodiments disclosed herein are examples of the disclosure and may be embodied in various forms. For instance, although certain embodiments herein are described as separate embodiments, each of the embodiments herein may be combined with one or more of the other embodiments herein. Specific structural and functional details disclosed herein are not to be interpreted as limiting, but as a basis for the claims and as a representative basis for teaching one skilled in the art to variously employ the present disclosure in virtually any appropriately detailed structure. Like reference numerals may refer to similar or identical elements throughout the description of the figures.

[0090] The phrases “in an embodiment,” “in embodiments,” “in various embodiments,” “in some embodiments,” or “in other embodiments” may each refer to one or more of the same or different embodiments in accordance with the present disclosure. A phrase in the form “A or B” means “(A), (B), or (A and B) .” A phrase in the form “at least one of A, B, or C” means “(A); (B); (C); (A and B); (A and C); (B and C); or (A, B, and C) .”

[0091] The systems, devices, and / or servers described herein may utilize one or more processors to receive various information and transform the received information to generate an output. The processors may include any type of computing device, computational circuit, or anytype of controller or processing circuit capable of executing a series of instructions that are stored in a memory. The processor may include multiple processors and / or multicore central processing units (CPUs) and may include any type of device, such as a microprocessor, graphics processing unit (GPU), digital signal processor, microcontroller, programmable logic device (PLD), field programmable gate array (FPGA), or the like. The processor may also include a memory to store data and / or instructions that, when executed by the one or more processors, causes the one or more processors to perform one or more methods and / or algorithms.

[0092] Any of the herein described methods, programs, algorithms or codes may be converted to, or expressed in, a programming language or computer program. The terms “programming language” and “computer program,” as used herein, each include any language used to specify instructions to a computer, and include (but is not limited to) the following languages and their derivatives: Assembler, Basic, Batch files, BCPL, C, C+, C++, Delphi, Fortran, Java, JavaScript, machine code, operating system command languages, Pascal, Perl, PL1, Python, scripting languages, Visual Basic, metalanguages which themselves specify programs, and all first, second, third, fourth, fifth, or further generation computer languages. Also included are database and other data schemas, and any other meta-languages. No distinction is made between languages which are interpreted, compiled, or use both compiled and interpreted approaches. No distinction is made between compiled and source versions of a program. Thus, reference to a program, where the programming language could exist in more than one state (such as source, compiled, object, or linked) is a reference to any and all such states. Reference to a program may encompass the actual instructions and / or the intent of those instructions.

[0093] It should be understood that the foregoing description is only illustrative of the present disclosure. Various alternatives and modifications can be devised by those skilled in the art without departing from the disclosure. Accordingly, the present disclosure is intended to embrace all such alternatives, modifications, and variances. The embodiments described with reference to the attached drawing figures are presented only to demonstrate certain examples of the disclosure. Other elements, steps, methods, and techniques that are insubstantially different from those described above and / or in the appended claims are also intended to be within the scope of the disclosure.

Claims

What is Claimed:

1. A system for presenting in-vivo studies, the system comprising: at least one processor; and at least one memory storing instructions which, when executed by the at least one processor, cause the system at least to perform: displaying, on a screen, a graphical user interface configured to present at least a portion of an in-vivo study, the in-vivo study comprising a sequence of in-vivo images of at least a portion of a gastrointestinal tract (GIT) of a person, the graphical user interface comprising: a user interface element configured to be engaged by a user to adjust a setting for an artificial intelligence detector, and a graphical spatial representation of an entirety of the sequence of in-vivo images; receiving a setting for the artificial intelligence detector based on a user engagement of the user interface element; and displaying a plurality of indications in spatial relation to the graphical spatial representation, the spatial relation corresponding to locations of images, of the sequence of in-vivo images, which are selected based on the setting for the artificial intelligence detector.

2. The system of claim 1, wherein the user interface element comprises at least three selectable settings, wherein the setting received for the artificial intelligence detector is one of the at least three selectable settings.

3. The system of claim 2, wherein the at least three selectable settings comprise a first selectable setting and a second selectable setting, wherein the first selectable setting sets a lower threshold than the second selectable setting for classifying an object in an image as an indicator of a pathology, resulting in a greater number of indications in the plurality of indications, andwherein the second selectable setting sets a higher threshold than the first selectable setting for classifying an object in an image as an indicator of a pathology, resulting in a smaller number of indications in the plurality of indications.

4. The system of any one of the preceding claims, wherein the artificial intelligence detector classifies an object in an image as an indicator of a pathology based on at least one classification score.

5. The system of any one of the preceding claims, wherein the graphical spatial representation of the entirety of the sequence of in-vivo images is a bar, and wherein the plurality of indications are lines in the bar, wherein the lines are brighter than a brightness of the bar.

6. The system of any one of the preceding claims, wherein the instructions, when executed by the at least one processor, further cause the system at least to perform: transmitting, to a server, the setting received for the artificial intelligence detector, the server configured to execute the artificial intelligence detector; and receiving, from the server, information for generating the plurality of indications.

7. A method for presenting in-vivo studies, the method comprising: displaying, on a screen, a graphical user interface configured to present at least a portion of an in-vivo study, the in-vivo study comprising a sequence of in-vivo images of at least a portion of a gastrointestinal tract (GIT) of a person, the graphical user interface comprising: a user interface element configured to be engaged by a user to adjust a setting for an artificial intelligence detector, and a graphical spatial representation of an entirety of the sequence of in-vivo images; receiving a setting for the artificial intelligence detector based on a user engagement of the user interface element; and displaying a plurality of indications in spatial relation to the graphical spatial representation, the spatial relation corresponding to locations of images, of the sequence of in- vivo images, which are selected based on the setting for the artificial intelligence detector.

8. The method of claim 7, wherein the user interface element comprises at least three selectable settings, wherein the setting received for the artificial intelligence detector is one of the at least three selectable settings.

9. The method of claim 8, wherein the at least three selectable settings comprise a first selectable setting and a second selectable setting, wherein the first selectable setting sets a lower threshold than the second selectable setting for classifying an object in an image as an indicator of a pathology, resulting in a greater number of indications in the plurality of indications, and wherein the second selectable setting sets a higher threshold than the first selectable setting for classifying an object in an image as an indicator of a pathology, resulting in a smaller number of indications in the plurality of indications.

10. The method of any one of claims 7-9, wherein the artificial intelligence detector classifies an object in an image as an indicator of a pathology based on at least one classification score.

11. The method of any one of claims 7-10, wherein the graphical spatial representation of the entirety of the sequence of in-vivo images is a bar, and wherein the plurality of indications are lines in the bar, wherein the lines are brighter than a brightness of the bar.

12. The method of any one of claims 7-11, further comprising: transmitting, to a server, the setting received for the artificial intelligence detector, the server configured to execute the artificial intelligence detector; and receiving, from the server, information for generating the plurality of indications.

13. A processor-readable medium storing instructions which, when executed by at least one processor of a system, cause the system at least to perform:displaying, on a screen, a graphical user interface configured to present at least a portion of an in-vivo study, the in-vivo study comprising a sequence of in-vivo images of at least a portion of a gastrointestinal tract (GIT) of a person, the graphical user interface comprising: a user interface element configured to be engaged by a user to adjust a setting for an artificial intelligence detector, and a graphical spatial representation of an entirety of the sequence of in-vivo images; receiving a setting for the artificial intelligence detector based on a user engagement of the user interface element; and displaying a plurality of indications in spatial relation to the graphical spatial representation, the spatial relation corresponding to locations of images, of the sequence of in- vivo images, which are selected based on the setting for the artificial intelligence detector.

14. The processor-readable medium of claim 13, wherein the user interface element comprises at least three selectable settings, wherein the setting received for the artificial intelligence detector is one of the at least three selectable settings.

15. The processor-readable medium of claim 14, wherein the at least three selectable settings comprise a first selectable setting and a second selectable setting, wherein the first selectable setting sets a lower threshold than the second selectable setting for classifying an object in an image as an indicator of a pathology, resulting in a greater number of indications in the plurality of indications, and wherein the second selectable setting sets a higher threshold than the first selectable setting for classifying an object in an image as an indicator of a pathology, resulting in a smaller number of indications in the plurality of indications.

16. The processor-readable medium of any one of claims 13-15, wherein the artificial intelligence detector classifies an object in an image as an indicator of a pathology based on at least one classification score.

17. The processor-readable medium of any one of claims 13-16, wherein the graphical spatial representation of the entirety of the sequence of in-vivo images is a bar, andwherein the plurality of indications are lines in the bar, wherein the lines are brighter than a brightness of the bar.

18. The processor-readable medium of any one of claims 13-17, wherein the instructions, when executed by the at least one processor, further cause the system at least to perform: transmitting, to a server, the setting received for the artificial intelligence detector, the server configured to execute the artificial intelligence detector; and receiving, from the server, information for generating the plurality of indications.

19. The processor-readable medium of any one of claims 13-18, wherein the instructions, when executed by the at least one processor, further cause the system at least to perform: in a case the user interface element is set to a highest certainty setting for the artificial intelligence detector, displaying surrounding images for the images, of the sequence of in-vivo images, which are selected based on the setting for the artificial intelligence detector.

Images