System and method for generating and displaying information for review of an in vivo image stream

Abstract

The system and method may display and / or provide analysis of several selected images of the patient's gastrointestinal tract collected in vivo by the swallowable capsule. The images may be displayed for review (e.g., as information for review) and / or further analysis by the user. A subset of images may be displayed representing the image stream and automatically selected according to a first selection method. Upon user input, additional images corresponding to the currently displayed images may be displayed, the additional images being automatically selected according to a second selection method. The second selection method may be based on a relationship between the images of the in-vivo image stream and the currently displayed image.

Description

[Technical Field]

[0001] The present disclosure relates to methods, systems, and computer program products for displaying and / or analyzing and / or reporting medical images from a series or stream of images captured in vivo. In particular, the present disclosure relates to methods, systems, and computer program products for generating and displaying studies of in vivo images captured via a Capsule Endoscopy (CE) procedure. [Background technology]

[0002] The capsule system may include a swallowable capsule that captures images of the gastrointestinal tract (gastrointestinal tract, "GI" tract or "GIT"). The images may be stored on the capsule and / or transmitted to a receiving device, which typically includes an antenna. The receiving device may receive and store the images (e.g., in a memory device within the receiving device). The swallowable capsule may include one or more cameras or imaging devices, a power source(s), a processor(s), and a transmitter(s).

[0003] The capsule, receiving device, or removable storage device of the receiving device may be coupled to a computing device, such as a server. The computing device may process the received images and may provide an interface for displaying the images, typically as video or video (e.g., a series of moving images). Such processing may also be performed "locally" on a workstation operated by a medical professional after receiving the images transferred from the receiver or recorder. In some embodiments, a physical receiving device located at the patient may transfer or download the images to a computer for processing and viewing. The workstation used to process or view the images may be a local computer, tablet, or workstation.

[0004] A medical professional, such as a doctor, may review the images as they are displayed, for example, on a workstation, terminal, or remotely, for example, via a web browser, such that the medical professional typically views the images as a video or image stream (e.g., typically including fewer than all captured images due to pre-processing).

[0005] It is desirable to present the thousands of images collected by the capsule so that a medical professional can accurately review the images and identify important images or features in a reasonable amount of time.

[0006] A typical prior art view of a procedure result or procedure review information seen by a physician includes thousands of images viewed as a video or movie (e.g., sequentially displaying one image at a time for each image stream captured by a single camera). Viewing thousands of images can be a tedious task, taking up valuable time. Also, viewing thousands of images as a movie can increase the likelihood of missing an image of interest. It is desirable to reduce the time required for medical personnel to view images. It is also desirable to make the viewing task more efficient and more informative. Summary of the Invention [Means for solving the problem]

[0007] Embodiments of the present invention may include a capsule device and, optionally, a receiving device. The capsule device may be a swallowable capsule that captures images while traversing the gastrointestinal tract and transmits the images to a receiving device that typically includes an antenna. The receiving device may receive the images and, for example, store the images in a storage device within the receiving device. The receiving device may transmit the images in real time (e.g., during the procedure) to a “cloud”-based server or another remote device. The receiving device may be, for example, a belt or patch, or any other item worn by the patient. The receiving device may forward the received images to the remote device for display, including real-time display, storage, processing, and / or analysis. The receiving device may transmit the images to the remote device, for example, via the patient's mobile phone. A local or remote (e.g., “cloud”) server or any other suitable computing device may process and analyze the received images and other procedure-related data (e.g., image capture time and patient data) to generate review information for the CE procedure. The computing device may provide an interface, locally or remotely, to display the review information for review by a user (e.g., a healthcare professional) for review. Other methods of storing, receiving, processing, and displaying images may also be used.

[0008] Embodiments of the present invention include systems, methods, and computer program products for displaying and / or providing analysis of several selected images for review and / or further analysis (e.g., for review information) by an observer or user. The images are selected from an image stream captured in vivo by a capsule endoscope traversing a portion of the gastrointestinal tract, such as the small intestine and / or colon. The disclosed methods, systems, tools, and / or computer program products may be used, inter alia, for the diagnosis, treatment decisions, monitoring, and / or screening of various conditions, disorders, abnormalities, and diseases of the GIT. For example, the present application provides the following: (Item 1) 1. A method for displaying review information of a capsule endoscopy procedure for user review, the capsule endoscopy procedure including capturing an in vivo image stream of at least a portion of interest of a subject's GIT, the method comprising: displaying, on a display device, a subset of images of the in-vivo image stream for review by the user, the subset of images representing the captured in-vivo image stream, the subset of images being automatically selected from the in-vivo image stream according to a first selection method; upon receiving user input, displaying on the display device one or more additional images corresponding to the currently displayed image of the image subset, wherein the one or more additional images are automatically selected from the in-vivo image stream according to a second selection method, the second selection method being based on a relationship between an image of the in-vivo image stream and the currently displayed image; and generating a report, the report including an image selected by the user from the displayed images. (Item 2) Item 10. The method of item 1, wherein the image subset is displayed in a static manner in a default display of the review information. (Item 3) said displaying said image subset; 2. The method of claim 1, further comprising simultaneously displaying a plurality of images of the subset, the plurality of images being captured by a single imaging device of a capsule device used in the capsule endoscopy procedure. (Item 4) Item 4. The method of item 3, wherein the simultaneously displayed images of the subset are displayed in a carousel or matrix layout. (Item 5) Item 4. The method of item 3, wherein the simultaneously displayed images of the subset are displayed in a horizontal carousel layout and such that no image covers another image. (Item 6) Item 3. The method of item 3, wherein the simultaneously displayed images of the subset are displayed in a horizontal carousel layout with a central image being a larger image size than any of the other simultaneously displayed images. (Item 7) Item 1. The method of item 1, wherein the relationship on which the second selection method is based is selected from the group consisting of images identified as including at least a portion of the same features, images identified as including at least a portion of the same type of features, images captured closely in time, images localized adjacently along the at least a portion of the subject's GIT, and combinations thereof. (Item 8) 8. The method of claim 7, wherein the feature is a feature of interest in the digestive (GI) tract. (Item 9) selecting the image subset according to the first selection method; 2. The method of claim 1, further comprising: using the one or more hardware processors to: for each image of at least a portion of the image subset, select the one or more corresponding additional images from the in-vivo image stream according to the second selection method. (Item 10) Item 10. The method of item 1, wherein the one or more additional images include at least one image of the image stream that is not included in the selected image subset. (Item 11) Item 10. The method of item 1, wherein the one or more additional images include only images of the image stream that are not included in the selected image subset. (Item 12) Item 10. The method of item 1, wherein the relationship on which the second selection method is based generates, for different images in the image subset, one or more corresponding different additional images. (Item 13) Item 10. The method of item 1, wherein the one or more additional images are displayed in a static manner in the default display of the review information. (Item 14) Item 10. The method of item 1, wherein the one or more additional images are displayed in the same window as the corresponding currently displayed image. (Item 15) 2. The method of claim 1, further comprising using the one or more hardware processors to display a map representing at least the portion of interest of the GIT of the object and corresponding to at least a selected one of the image subsets, the map including localized image indications representing the images in the image subset according to their estimated positions along at least the portion of interest of the GIT of the object. (Item 16) Item 16. The method of item 15, wherein the indication of the image of the image subset that is currently displayed and in focus is highlighted. (Item 17) receiving a user selection of a graphical indication within the map; 16. The method of claim 15, further comprising using the one or more hardware processors to: (Item 18) Item 10. The method of item 1, wherein the image subset is displayed in a matrix layout in a default display of the review information. (Item 19) Item 10. The method of item 1, further comprising using the one or more hardware processors to display, upon user input, a predefined number of images automatically selected from the image subset, wherein the predefined number of images is selected from the image subset according to a third selection method. (Item 20) 20. The method of claim 19, wherein the images of the image subset estimated to be of greatest interest according to the third selection method are selected. (Item 21) Item 10. The method of item 1, wherein at least the first selection method is based on deep learning techniques. (Item 22) Item 10. The method of item 1, further comprising using the one or more hardware processors to display an indication of a location within the displayed image suspected of containing at least a portion of a feature of interest. (Item 23) said at least part of interest of said GIT of said subject being divided into sections; Images of the image subset are localized along at least the portion of interest of the GIT of the subject according to the partition; 2. The method of claim 1, further comprising using the one or more hardware processors to display an indication of the section in which the currently displayed image of the image subset is located. (Item 24) 24. The method of claim 23, further comprising using the one or more hardware processors to display the image set according to the partition. (Item 25) Item 10. The method of item 1, further comprising using the one or more hardware processors to display an indication of an estimated cleaning level of an image of the captured image stream. (Item 26) Estimating the size of the identified feature in the displayed image; 2. The method of claim 1, further comprising using the one or more hardware processors to: (Item 27) estimating the extent of features identified in the image subset along at least the portion of interest of the GIT; 2. The method of claim 1, further comprising using the one or more hardware processors to: (Item 28) 2. The method of claim 1, further comprising using the one or more hardware processors to display, upon user input, a plurality of images selected from the in vivo image stream representing a transition from one anatomical fragment to another anatomical fragment of the at least one portion of interest of the GIT. (Item 29) 2. The method of claim 1, further comprising displaying the review information, wherein the review information does not include any images of the captured in-vivo image stream other than the image subset and the additional images corresponding to images in the subset. (Item 30) Item 10. The method of item 1, further comprising using the one or more hardware processors to filter the display of the image subset based on user input. (Item 31) Item 10. The method of item 1, wherein images identified as containing features of interest according to the first selection method are selected. (Item 32) 32. The method of claim 31, wherein the feature of interest is selected from the group consisting of a pathology, a foreign body, an abnormality, an anatomical feature, and combinations thereof. (Item 33) 1. A method comprising: receiving an automatically selected image subset from an in-vivo image stream of at least a portion of interest of a subject's GIT captured via a capsule endoscopy procedure, wherein the images of the image subset are selected based on a first selection method; displaying the automatically selected image subset in a static manner on a display device; upon receiving user input, displaying additional information about a currently displayed image of the automatically selected image subset; and generating a report, the report including an image selected by the user from the displayed images. (Item 34) Item 34. The method of item 33, wherein the additional information includes one or more additional images automatically selected from the in-vivo image stream according to a second selection method. (Item 35) Item 34. The method of item 33, wherein the display of the image subset includes simultaneously displaying multiple images of the subset, the multiple images being captured by a single imaging device of a capsule device used in the capsule endoscopy procedure. (Item 36) Item 35. The method of item 34, wherein the one or more additional images include at least one image of the image stream that is not included in the selected image subset. (Item 37) Item 34. The method of item 33, wherein the one or more additional images are displayed in the same window as the corresponding currently displayed image. (Item 38) Item 34. The method of item 33, comprising using the one or more hardware processors to display a map representing at least the portion of interest of the GIT of the object and corresponding to at least a selected image subset, the map including localized image indications representing the images in the image subset according to their estimated positions along the at least the portion of interest of the GIT of the object. (Item 39) the at least the portion of interest of the GIT of the object is divided into sections, and images of the image subset are localized along at least the portion of interest of the GIT of the object according to the sections; 34. The method of claim 33, further comprising using the one or more hardware processors to display an indication of the section in which the currently displayed image of the image subset is located. (Item 40) 34. The method of claim 33, further comprising using the one or more hardware processors to display an indication of the cleaning level on an image of the captured in-vivo image stream. (Item 41) Item 34. The method of item 33, wherein images identified as containing features of interest according to the first selection method are selected. (Item 42) 1. A method for generating review information of a capsule endoscopy procedure for user review, the capsule endoscopy procedure including capturing an in vivo image stream of at least a portion of interest of a subject's GIT, the method comprising: selecting an image subset from the captured in-vivo image stream according to a first selection method, the selected image subset representing the captured in-vivo image stream; for each image in at least a portion of the image subset, selecting one or more corresponding additional images from the in-vivo image stream according to a second selection method; and generating the review information for the capsule endoscopy procedure, the review information including the selected image subset and additional information, the additional information including the one or more selected additional images corresponding to each image of at least a portion of the image subset. (Item 43) 43. The method of claim 42, further comprising using the one or more hardware processors to display the review information on a display device for user review. (Item 44) The display of the review information comprises: displaying the selected image subset on a display device; and upon user input, displaying on the display device one or more additional images that correspond to a currently displayed image of the image subset. (Item 45) 43. The method of claim 42, wherein the generating of the review information comprises generating additional information in addition to the one or more additional images corresponding to images from the image subset selected from the group consisting of: a division of at least the portion of interest of the GIT into sections; a position of images from the image stream along at least the portion of interest of the GIT; a map representing at least the portion of interest of the GIT, the map including an indication of images from the image subset along at least the portion of interest of the GIT; a plurality of images selected from the image subset according to a third selection method; an indication of a feature of interest within an image; an indication of a cleansing level; an indication of an estimated size of a feature of interest; an indication of an extent of a feature of interest; a selection of images from the in vivo image stream representing a transition between anatomical fragments of at least the portion of interest of the GIT; and combinations thereof. (Item 46) Item 43. The method of item 42, wherein the relationship on which the second selection method is based is selected from the group consisting of images identified as including at least a portion of the same features, images identified as including at least a portion of the same type of features, images captured closely in time, images localized adjacently along the at least a portion of the GIT of the subject, and combinations thereof. (Item 47) Item 43. The method of item 42, wherein the one or more additional images include at least one image of the image stream that is not included in the selected image subset. (Item 48) Item 43. The method of item 42, wherein the one or more additional images include only images of the image stream that are not included in the selected image subset. (Item 49) Item 43. The method of item 42, wherein the relationship on which the second selection method is based generates, for different images in the image subset, one or more corresponding different additional images. (Item 50) Item 43. The method of item 42, wherein at least the first selection method is based on deep learning techniques. (Item 51) Item 43. The method of item 42, wherein images identified as containing features of interest according to the first selection method are selected. (Item 52) Item 43. The method according to item 42, wherein the first selection method and the second selection method each include one or more selection rules. (Item 53) 1. A system for displaying review information of a capsule endoscopy procedure for a user's review, the capsule endoscopy procedure including capturing an in vivo image stream of at least a portion of interest of a subject's GIT, the system comprising: one or more processors; a non-transitory storage device storing instructions that, when executed by the processor, cause the processor to perform the method of any one of items 1 to 41; a display device configured to display the review information. (Item 54) 1. A system for generating review information of a capsule endoscopy procedure for user review, the capsule endoscopy procedure including capturing an in vivo image stream of at least a portion of interest of a subject's GIT, the system comprising: one or more processors; A non-transitory storage device storing instructions that, when executed by the processor, cause the processor to perform the method of any one of items 42 to 52. (Item 55) 55. The system of claim 54, further configured to perform the capsule endoscopy procedure, the system further comprising a capsule device including at least one imaging device configured to capture the in-vivo image stream while the capsule device traverses the at least the portion of interest of the patient's GIT, the captured in-vivo image stream being non-transiently provided to the non-transient storage device, and the non-transient storage device further configured to receive and store the provided in-vivo image stream. [Brief explanation of the drawings]

[0009] Embodiments of the present invention are illustrated by way of example and not limitation in the figures of the accompanying drawings in which like reference numerals indicate corresponding like or similar elements and in which: [Figure 1] FIG. 1 is a high-level block diagram of an exemplary computing device that may be used in embodiments of the present invention. [Figure 2] 1 illustrates a first or default screen that is initially presented to a viewer or user, according to some embodiments. [Figure 3] 1 illustrates an exemplary screen, according to some embodiments. [Figure 4] 1 illustrates an exemplary screen or display displaying additional information, e.g., review information, according to some embodiments. [Figure 5] 5 illustrates the example screen of FIG. 4 after a zoom operation, according to some embodiments. [Figure 6A] 1 shows a schematic diagram of a review information view mode according to one embodiment of the present invention; [Figure 6B]FIG. 6B is a schematic diagram of a map (eg, a bar) in the default view mode of FIG. 6A, according to one embodiment. [Figure 7] 6B shows an example diagram of a display or view of a second level of information for an image selected according to the display illustrated in FIG. 6A, according to some embodiments. [Figure 8A] 1 illustrates a default review information view according to one embodiment. [Figure 8B] 8B illustrates a matrix layout of the image shown in FIG. 8A according to some embodiments. [Figure 9A] 1 is an exemplary screenshot of a default view mode, according to one embodiment of the present disclosure. [Figure 9B] 9B is an exemplary screenshot of a display or view of a second level of information for a selected image in the display of FIG. 9A. [Figure 10] 1 shows a sample display according to one embodiment of the present invention. [Figure 11] 1 is a flowchart of a method according to an embodiment of the present invention.

[0010] It will be understood that for simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn accurately or to scale. For example, the dimensions of some elements may be exaggerated or emphasized relative to other elements for clarity, or several physical components may be included in a single functional block or element. Furthermore, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous elements. DETAILED DESCRIPTION OF THE INVENTION

[0011] The images may be displayed for review (e.g., as review information) and / or further analysis by the user. A subset of images may be displayed that represent the image stream and are automatically selected according to a first selection method. Upon user input, additional images corresponding to the currently displayed images may be displayed, the additional images being automatically selected according to a second selection method. The second selection method may be based on a relationship between the images of the in-vivo image stream and the currently displayed image. Detailed Description

[0012] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, those skilled in the art will understand that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the present invention. Some features or elements described with respect to one embodiment may be combined with features or elements described with respect to other embodiments. For clarity, descriptions of the same or similar features or elements may not be repeated.

[0013] Although embodiments of the invention are not limited in this respect, for example, descriptions utilizing terms such as “processing,” “calculating,” “computing,” “determining,” “establishing,” “analyzing,” and “ascertaining” may refer to operation(s) and / or process(es) of a computer, computing platform, computing system, or other electronic computing device that manipulates and / or transforms data represented as physical (e.g., electronic) quantities in the computer's registers and / or memory into other data also represented as physical quantities in the computer's registers and / or memory, or other information in a non-transitory storage medium that may store instructions for performing the operation and / or process. Although embodiments of the invention are not limited in this respect, as used herein, “plurality” and “a plurality” may include, for example, “multiple” or “two or more.” The terms “plurality” or “aplurality” may be used throughout the specification to describe two or more components, devices, elements, units, parameters, etc. As used herein, a term set may include one or more items. Unless otherwise specified, the method embodiments described herein are not limited to a particular order or sequence. In addition, some of the method embodiments described or elements thereof may occur or be performed simultaneously, at the same time, or together.

[0014] The term "position" and its derivatives, as referred to herein with respect to an image, may refer to the estimated position of the capsule along the GIT while the image is being captured, or the estimated position of a portion of the GIT scattered across the image along the GIT.

[0015] The type of capsule endoscopy procedure may be determined based on, among other things, the portion of the GIT that is of interest and to be imaged (e.g., the colon or small bowel (SB)), or based on the particular use (e.g., to confirm the status of a G1 disease such as Crohn's disease, or for screening for colon cancer).

[0016] In this specification, the terms screen / screens, view / views, and display / displays may be used interchangeably and may be understood according to the particular context.

[0017] The terms "default view," "default display," and "default screen," as referred to herein, may refer accordingly to the "view," "display," and "screen" that is first displayed at the start of each review of procedural review information by the present systems and methods, and this first view, display, or screen is predefined and is not the result of user selection.

[0018] The terms "default view" and "default display," as referred to herein with respect to a view or display of particular information, may accordingly refer to the first "view" and "display" of the particular information at the start of each review of procedural information by the present systems and methods, this first view or display of the particular information being predefined and not the result of a user selection.

[0019] The terms "by default" or "as a default," as used herein with reference to a particular display or view of information, may refer to information displayed within a particular view, display, or screen frame that does not require user action or input to upload, initiate, or trigger the display.

[0020] The terms "surrounding" or "adjacent," as used herein with respect to images (e.g., images surrounding or adjacent to another image / images), may refer to spatial and / or temporal characteristics unless otherwise indicated. For example, images surrounding or adjacent to another image / images may be images estimated to be located near the other image / images along the GIT and / or images captured within a certain threshold, e.g., within 1 cm, or 2 cm, or within 1 second, 5 seconds, or 10 seconds of the capture time of the other image.

[0021] The terms "GIT" and "portion of GIT" may refer to or include the other depending on their context. Thus, the term "portion of GIT" may also refer to the entire GIT, and the term "GIT" may also refer to only a portion of the GIT.

[0022] Embodiments of the present invention may differ from prior art CE procedure viewing applications in that prior art applications typically present, as a first or default view or display, a video or image stream, e.g., a series of images presented as a time series. In embodiments of the present invention, a set of images collected from the GIT, e.g., via a swallowable capsule, may be processed and then initially presented, e.g., as a set of still images, e.g., as a default or first view, as images are uploaded or processed, e.g., at the start of each review of procedural review information after capsule capture according to embodiments of the described systems and methods. Each image presented in the default view may be of a feature of interest (e.g., a particular instance of a pathology) and may be associated with or correspond to other images of that same feature (e.g., the same instance of a pathology) that are not shown in the default view. Pointing to or selecting (e.g., clicking using an input device) a single still image displayed to represent an instance of the feature of interest may present a view of second level or additional information, which may, for example, be a video or scrollable series of images also showing instances of the feature.

[0023] Images of different portions of the GIT (e.g., SB), large intestine, or colon, esophagus, stomach, etc. may be presented, and the user interface may change according to the different portions displayed. The type of procedure performed, e.g., SB procedure, colon procedure, or both (e.g., a procedure aimed at specifically illustrating or identifying the SB, or a procedure aimed at specifically illustrating or identifying the colon, or both), may determine which portion of the GIT is the portion of interest. Typically, the majority of the image displayed is the portion of interest. Each portion of interest may be divided into sections. Each section (e.g., an image showing the portion of the GIT estimated to be located within that section) may be displayed in different views, e.g., via different tabs. For example, if the portion of interest is the colon (e.g., the procedure is a colon procedure), it may be sectioned into five anatomical sections (cecum, ascending colon, transverse colon, descending colon, and rectum), each with a separate view accessible via a tab. Thus, the user may be presented with tabs labeled cecum, ascending colon, transverse colon, descending colon, and rectum, and clicking on cecum may reveal a first level collection that is estimated to best represent images in the received image stream that are estimated to be located within the cecum.

[0024] The second-level images may be displayed on the same screen as the first-level images (e.g., a first-level image may be transitioned to display as a moving or scrollable image to display its corresponding second-level image, while other still first-level images remain on the display), or may be displayed in a separate or pop-up window. The second-level images may precede and / or follow or succeed the corresponding first-level image according to image capture time. The second-level images may include the corresponding first-level image (seed image), or the display of the second-level image may include the corresponding first-level image. Thus, second-level images that include or supplement the corresponding first-level image may be displayed according to their capture time. The second-level images may then be numbered such that the first-level image (e.g., the "best" representative image) is numbered zero, and the second-level image or other second-level images are numbered by their distance within the image from image zero.

[0025] The user may click a checkbox or otherwise select an image displayed in the first level to place that image in a report or final report (typically displayed or delivered after the user has reviewed the images). The user may filter the first level images to display only "selected" images. Various tools or image analysis capabilities may be presented, such as automatic pathology size estimation tools (e.g., for polyps) and pathology size estimation tools. Images may also be filtered and / or selectively displayed based on user input in other ways.

[0026] Different types of image displays or views may be selected by the user, such as a matrix or grid view, a carousel view (e.g., displaying images as if seen edge-on in a ring of images, with one "current" image displayed full-sized in the foreground and the others at reduced sizes around the "ring"). A map or "heat map" may associate a particular image with a spatial orientation along the GIT, with a particular feature or combination of features having their "spread" or length indicated along the heat map, for example, using color. A map may represent a portion of the GIT or a portion of interest in the GIT. A map may correspond to first-level images in that it may represent the portion of the GIT from which the image was taken and / or may include a marker corresponding to the estimated location of the image. Thus, the map may include localized image indications or markers that represent the image according to its location or estimated location within the GIT.

[0027] As used herein, processor-derived descriptions such as location, size, or cleanup level may be estimated or approximated based on information available to the processor and algorithmic methods.

[0028] A user may add information to an image, and / or information may be automatically added to the image. For example, a user may label an image with a selectable pathology type. The process may automatically draw a box or boundary around the location of automatically identified pathologies within the image.

[0029] According to some embodiments, creating or generating the review information may include generating additional information or a second level of information in addition to one or more additional images corresponding to the images in the image subset. For example, the additional information may include a map representing at least the portion of interest of the GIT, including a division of at least the portion of interest of the GIT into sections, a location of the images in the image stream along at least the portion of interest of the GIT, an indication of the images in the image subset along at least the portion of interest of the GIT, a predefined number of images selected from the image subset according to a third selection method, an indication of a feature of interest within the images, an indication of a cleaning level determined for each image, section, and / or procedure, an indication of an estimated size of the feature of interest, an indication of the extent of the feature of interest, a selection of images from the in-vivo image stream representing transitions between anatomical fragments of at least the portion of interest of the GIT, and combinations thereof.

[0030] In some aspects, the system or method may display capsule endoscopy procedure review information, including images captured by a capsule (e.g., swallowed by a patient) during the capsule endoscopy procedure, for user review. The system may receive a selected image subset (e.g., first-level images) from a stream or in-vivo image set of at least a portion of a patient's or subject's GIT captured via the capsule endoscopy procedure. According to some aspects, the system may receive a stream or in-vivo image set of at least a portion of a patient's or subject's GIT. The system may then automatically choose or select an image subset from the received stream or in-vivo image set according to a first selection method, the image subset "representing" the received stream or in-vivo image set. By "representing," we mean representing a portion of interest in the patient's GIT from the perspective of making a medical diagnosis or for the benefit of making a medical diagnosis and making a treatment decision. For example, the system may identify one or more features of interest (e.g., one or more types of pathologies) within the received images. The one or more features of interest may be predetermined based on a particular type of capsule endoscopy procedure. Based on the identification of the features of interest, an image subset may be selected to represent, among other things, the identified pathology.

[0031] For each image of at least a portion of the image subset, the system may receive, select, or identify one or more images from the image stream or the in-vivo image set that correspond to the image in the image subset (e.g., second-level images). The one or more corresponding images may be selected according to a second selection method. According to some aspects, for each image in the image subset, one or more corresponding images are selected and received. According to some aspects, the selected corresponding one or more images may include the corresponding image in the image subset. According to some aspects, the corresponding one or more images may not include the image in the image subset. According to some aspects, the system may select the corresponding one or more images according to the second selection method.

[0032] The second selection method may be based on a filter or relationship between images in the in-vivo image stream. Such a filter or relationship between images may be images identified as containing at least a portion of the same feature (e.g., the same specific ulcer) or the same type of feature (e.g., images identified as containing an ulcer), images captured closely in time along the GIT, or images located adjacently along the GIT, or a combination thereof. According to some aspects, a predefined number of images selected from the image subset may be received and displayed. The predefined number of images (e.g., 5 images, 10 images, or any number greater than 10 images) may be displayed upon receiving user input. The predefined number of images may be selected according to a third selection method to summarize the image subset. According to some aspects, the third selection method may select images from the image subset that are determined or estimated to be of greatest interest. For example, in a colon procedure, images from a received image stream or image set that are identified with a degree of certainty as containing a polyp may be selected according to the first image subset selection method.

[0033] The five images from the image subset identified as containing the five largest polyps or the five images identified as containing polyps with the highest degree of certainty may be selected according to a third selection method. Each image selection method described herein may include one or more filters or selection or detection rules, and selection by each method may be performed in one or more stages. The selection or detection rules may be applied by utilizing algorithms, such as machine learning algorithms and deep learning algorithms. For example, when selecting an image subset from a received image stream, detection of one or more features of interest may be performed in a first stage. Different rules (e.g., algorithms) may be used to detect different types of features, such as different pathologies or passages between different anatomical parts of the GIT. Typically, the result of the detection algorithm may be a score assigned to an image indicating a level of certainty that the image contains the detected feature. The relation or filter on which the second selection method is based may generate corresponding different additional images for different images from the first level or images from the image subset. At least the first selection method may be based on deep learning techniques, such as neural network techniques. The first selection method and the second selection method may each include one or more selection rules.

[0034] Images for the second stage may be selected from the image stream based on the detection results of the first stage. This selection may include different selection rules for different types of features, such as different types of pathologies and pathways between anatomical parts. The selection rules in the second stage may be based, among other things, on a threshold determined for the detection score. For example, in an SB procedure, different rules may be determined for selecting images for the image subset in the second stage based on the type of pathology or feature identified in the image, the type of spread of the pathology identified along the Gl tract (e.g., focal, diffuse, or "carpet-like" distribution), and the severity or location of the identified pathology. Rules may also be based on the maximum number of images to be displayed per particular feature, per type of feature, and / or the entire image set selected. For example, in the case of a focal pathology, the system may select only one image (e.g., one rule) representing the pathology to appear in the image subset. In the case of a diffuse pathology, the system may select several images (e.g., another rule) to represent the pathology in the image subset.

[0035] As another example, in a colon procedure used as a cancer screening tool, the features of interest may include only one pathology: polyps. The system may then include a rule that selects one image for the image subset for each identified polyp. The identified polyps may be determined according to a threshold polyp detection score for the images. Alternatively, or in addition, another rule may be determined that the first subset includes a maximum number of images (e.g., 50, 100, 160, 200, or a maximum of approximately tens of images, a maximum of approximately 100 images, or a maximum of approximately hundreds of images). If two rules are applied and the number of images selected according to the first rule exceeds the maximum number determined according to the second rule, a third rule may be applied. Such a third rule may determine that the maximum number of images according to the second rule among the images selected according to the first rule that have the highest detection score are selected for the image subset. Images in which polyps are identified with a high detection score may be considered to be more important images, for example, because such images have a higher probability of showing polyps.

[0036] According to some embodiments, rules based on tracking features, such as polyp ulcers, may be included. Such rules may be applied in additional stages. Such rules may enable identification of images capturing the same specific feature (e.g., the same polyp). Such rules may be used in a second selection method to determine one or more images showing at least a portion of the same feature to be shown in the images of the image subset. Such feature tracking may be performed, for example, based on methods such as those disclosed in U.S. Pat. No. 9,430,706, which is incorporated herein by reference in its entirety, or optical flow techniques.

[0037] The system may create or generate review information for a capsule endoscopy procedure, for example, by utilizing the methods described herein. This review information may include at least a selected image subset (e.g., Level 1 views) and, optionally, additional information. The additional information may include, for example, one or more additional images selected (e.g., by the system) corresponding to each image of at least a portion of the “Level 1” image subset. The additional information may further include information such as: location of the displayed image along the GIT; an indication of the location and / or type of identified or suspected feature within the displayed image; an indication of the extent or type of identified or suspected feature along the GIT (e.g., the extent of a “carpet-like” pathology or diffuse disease); an indication of a segmentation of the GIT portion of interest; an indication of the anatomical portions of the GIT; and a display of an image depicting the anatomical pathways between different anatomical portions of the GIT. According to some aspects, some or all of the additional information may be displayed only upon user input, e.g., upon user request via an action (e.g., by activating a control or hovering over a particular display area). According to some embodiments, the level 1 images and / or the level 2 images may be displayed in a static manner. Typically, the level 2 images are not displayed only when the user selects a level 1 image for further review. Typically, the second level image collection includes one or more images that are not included or displayed as first level images. The review information images may be displayed in the same view as the first level images (e.g., when displayed in the same window) or in a different view (e.g., when displayed in a pop-up window).

[0038] The information for review may be displayed in one or more views for user review. Each information for review view may include at least a display of the image subset. Different information for review views may feature, for example, image display layouts (e.g., matrix or carousel), image display browsing (e.g., static or video), which display additional information of the information for review (in all cases), and in which the additional information is displayed. For example, according to some embodiments, two information for review views may be displayed. One view may include a display of a matrix layout in static format along with additional information (e.g., GIT segmentation and segmentation cleanup level) of the image subset (e.g., “first level” or “level 1” images), but does not include a display of one or more images corresponding to images in the image subset (e.g., “second level” or “level 2” images). A second view may include a display of the image subset in a carousel layout and a display of all of the additional information included in the information for review, including the second level images. In the case of multiple review information views, one view may be defined as the default review information view. The default review information view, in contrast to other review information views, is not selected by the user for display. The default review information view is displayed when a specific action is selected for a review information view. Thus, the default review information view is the first review information view displayed. In the case of a single review information view, the single review information view is the default review information view.

[0039] The review information view may include one or more views. The image subset is displayed in a main or default view of the review information view. The main or default view of the review information view may further include the display of additional information, such as a map identifying the location of the images in the image subset along the GIT, an indication of the cleaning level for each image in the image subset, a segmentation of the GIT portion of interest, the capture time of the images in the image subset, and / or an estimated transit time through the GIT portion of interest (e.g., an estimate of the time it took the capsule device to traverse the GIT portion of interest). Other additional information may be displayed only upon receipt of user input, such as the display of one or more additional images corresponding to the images in the displayed image subset. Such “optional” information display (e.g., for the user to determine whether this additional information is displayed) may be in the main view or a separate view. For example, one or more additional images corresponding to the images in the image subset may be displayed in the main view, e.g., in the same image window in which the corresponding image in the image subset is displayed, or in a different dedicated area of ​​the main view of the review information view. In another example, one or more additional images corresponding to images in the image subset may be displayed in a separate view, such as a pop-up window.

[0040] A review information display or view may include multiple features. Review information display features may include a manner in which the image subset is displayed, a layout for displaying the images (e.g., carousel or matrix), including the number of images displayed simultaneously, a manner for browsing between images (if not all images are displayed simultaneously), and the display of additional information. For example, features may include which additional information may be displayed in a particular display or view and which additional information may not be displayed. Characteristics related to the display of additional information may include determining which information is displayed by default in a particular view and which information is displayed in a particular view only upon user request (e.g., displaying instructions for image features). Review information display features may specifically include a layout in which the second level images are displayed one at a time or several simultaneously, for example, in a carousel or as video clips, in the same window as the first level images, or in a pop-up or separate window or screen. According to some aspects, browsing among images in a displayed image collection may be static (e.g., when browsing among images or pages of images is driven solely by user input), automatically (e.g., when a user activates a browsing control and browsing is automatically driven among images or pages of a displayed image collection), image-by-image (e.g., each browsing action replaces only one currently displayed image), or page-by-screen (e.g., when multiple images are displayed simultaneously, each browsing action replaces all currently displayed images). According to some aspects, the display of images, e.g., a selected image subset, is by and based on a segmentation of a portion of interest in the GIT. For example, in a review information view, the image collection may be displayed segment by segment, with each segment displayed by activating a tab. According to some aspects, some display or view characteristics may be predefined by the user or may change upon user input (e.g., the number of images displayed simultaneously in a view).In some embodiments, a user may browse or cause a display to be displayed to move between images or "pages" of a first level image collection (e.g., if the display contains x images, then browsing the next page will result in the next x images). Automated browsing (as opposed to a video display) may automatically move to the next image or next "page."

[0041] In one embodiment, in a view or display of second level images, only one image is displayed at a time (as opposed to a first level image where multiple first level images are displayed simultaneously), and the second level images may be displayed as video clips, for example using an autoplay mode.

[0042] The review information may include all data related to the case or procedure presented to the physician, including all of the images and information the system provides to the user, such as the time of image capture, the images (first and second level), and additional data such as the results of the system's analysis of the data related to the case or procedure. The review information may also include patient information (e.g., details, background, history).

[0043] According to some aspects, system embodiments may display a subset of images for user review on a display device (e.g., a monitor), typically with one or more representative images per group among the displayed images, such that each different group may have one or more representative images displayed in a "first level" display. The first level display may, for example, display these images by "default" without user input or selection, and may display images in a static format rather than as moving or animated images.

[0044] First level or "representative" images captured by a single imaging device of a capsule device used in the procedure may be displayed simultaneously. The number of images or maximum number of images that may be displayed simultaneously in a view or display may be predefined, e.g., the number or maximum number of first level images.

[0045] According to some aspects, the review information includes first level information displayed in a default view of the review information. Displaying the first level information is therefore mandatory, and the user does not decide whether to display this type of information. According to some aspects, the first level information consists of only a selected image subset representing a stream or image set received from a capsule device or a receiving device. The review information may also include second level information. The second level information may be displayed only upon user input (e.g., upon user request) and therefore its display may be optional. The second level information may provide additional information to the first level information. The second level information, or at least a portion thereof, is associated with or corresponds to an image in the image subset.

[0046] Such selection and display of treatment-related information may make the user's review process more efficient, shortening review time and resulting in a better diagnosis. According to the present methods and systems, users may review the treatment review information in a manner that suits their skills and preferences and / or the needs derived from a particular patient, treatment, and / or treatment review information. In this manner, the disclosed systems and methods may enable personalized review of treatment review information. For example, an experienced user may need less information to make a diagnosis than a novice user. Thus, a novice or less experienced user may choose to view and review additional information that is not necessary for an experienced user. As another example, a review of treatment review information for a subject suspected of and / or known to be ill may include the display of more additional information than a review of treatment review information for a subject suspected of and / or known to be healthy.

[0047] Furthermore, additional information that is not typical or has not previously been displayed or provided to a user within the framework of review information for a CE procedure may be provided in accordance with embodiments of the described systems and methods. Such information may aid and assist the user in making diagnostic and / or treatment decisions, resulting in faster and better diagnoses and treatments. Additional information includes, among other things, information generated or achieved by employing cutting-edge computer-aided decision-making techniques, such as deep learning techniques.

[0048] According to some aspects, a user may select an image in the first level display (e.g., by providing user input to a graphical user interface (GUI)) to display a second level of information, where the image corresponds to a group of images and, therefore, may correspond to other images not yet displayed in the group. According to some aspects, all images in the group may contain or capture the same automatically discovered feature of interest, and therefore, images in the group other than the single image displayed in the first level (images corresponding to the first level image) may show the same feature as shown in the image displayed for that group in the first level. Upon receiving user input regarding the first level images, the system may display one or more images of the currently displayed image subset that correspond to the selected first level image. This may result in a second level display of additional information. For example, the selected group of images may be displayed in a static manner, and the user may then scroll through the group of images, displaying the images in the group in order, or another display method may be used, such as displaying the group of images as a video clip. The user may add information or annotate the displayed images while the images are being displayed. After the user has finished reviewing the images, the system may create or generate a report that includes the images selected by the user from the displayed images, but not other images. In some embodiments, this report may include only the images viewed by the user and possibly user-entered information, but not other images from the captured in-vivo image stream. Typically, only the selected image subset, and images corresponding to images in the subset, are available for display in the report for the user's review.

[0049] According to some aspects, intermediate review information is displayed and may optionally be generated by the disclosed systems and methods. The intermediate review information may be generated based on a partial stream or set of images received from the capsule device or from a receiving device in communication with the capsule device. The partial stream of images may include only images captured up to a specific point relative to the start of the procedure (e.g., the subject swallows the capsule) or until the capsule reaches a specific location within the GIT portion of interest and before the end of the procedure. The partial stream of images is received, the intermediate review information is generated, and the intermediate review information is reviewed, typically by a medical professional during the procedure. According to some aspects, the intermediate review information is generated and displayed before the capsule device reaches the end of the GIT portion of interest. The generation and display of the intermediate review information for user review may be performed by the disclosed systems and, mutatis mutandis, according to the disclosed methods. The intermediate review information may include a subset of images selected from the received partial stream of images. The intermediate workup information may typically include less additional information compared to the full workup information (e.g., the workup information). For example, in some embodiments, the intermediate workup information does not include second-level images. The user may then generate an interim report based on the intermediate workup information by using the systems and methods disclosed herein. The intermediate workup information may be used to provide necessary medical diagnosis and treatment, including referring the patient for a medical procedure that occurs the same day or shortly after the CE procedure is completed.

[0050] For example, if a polyp to be removed is identified while a healthcare professional is reviewing the intermediate review information, the patient may be referred for a same-day colonoscopy. Same-day colonoscopy may have advantages, including eliminating another colon preparation for the patient. According to some aspects, a user may determine parameters, such as a specific time point or a specific location within the portion of interest of the GIT, that determine the partial stream of images from which the intermediate review information will be generated. According to some aspects, a user may select a time period for receiving the intermediate review information.

[0051] A last image may be displayed. The last image may be the last image captured by the capsule or an image identified as being located at the last position captured by the capsule (note that the capsule moves back and forth). This may be displayed, for example, to inform the physician how far the capsule has captured images or whether it has reached the area of ​​interest. The last image may be part of an image subset or may be part of additional information.

[0052] According to some embodiments, images captured in vivo may be received and a plurality of these images may be automatically selected for display (e.g., by the processor shown in FIG. 1). In some embodiments, these selected image subsets may be identified automatically and / or by a user, and a case report or report may be generated that includes only images from the identified subset of images (e.g., one or more or all of the images in the subset).

[0053] The images selected for display may be, for example, suspicious images containing a particular pathology, abnormality, or anomaly, and / or images selected to represent or illustrate a particular portion of the GIT or a particular anatomical area of ​​the GIT.

[0054] The disclosed systems, methods, and computer program products may be implemented or performed via one or more software applications that may receive or access selected images and / or the entire and / or portion of an in vivo captured image stream. Such applications may, for example, display selected images to a user, interact with the user (e.g., via a GUI), facilitate analysis of the user-displayed images, provide automated analysis of the displayed images, and / or generate reports. According to some embodiments of the present disclosure, such applications may also perform the selection of images to be displayed. Systems and computer program products according to the present disclosure may include one or more non-transitory computer-readable storage media (which may also store data such as images, image streams, etc.). Systems according to the present disclosure may include one or more hardware processors, one or more display devices, one or more I / O devices, and / or one or more communication devices utilizing one or more communication technologies. The one or more software applications may include program code stored on one or more non-transitory computer-readable storage media and executable by the one or more hardware processors. The images and / or any additional data may be displayed to a user via one or more display devices, and the user may interact with one or more applications via one or more I / O devices. The image stream, portions of the images, or selected subsets of the image stream may be received or remotely accessed (e.g., via the Internet) by one or more communication devices.

[0055] FIG. 1 illustrates a high-level block diagram of an exemplary computing device that may be used in embodiments of the present invention. Computing device 100 may include a controller or processor 105, which may be or include, for example, one or more central processing unit(s) (CPU), one or more graphics processing unit(s) (GPU or GPGPU), chip, or any suitable computing or computer device, operating system 115, memory 120, storage device 130, input device(s) 135, and output device(s) 140. Modules or devices that collect, receive (e.g., a patient-worn receiver), display, or select medical images collected by the swallowable endoscopic capsule for display may be, include, or be executed by a computing device such as that included in FIG. 1. Communication element 122 may enable communication with remote or external devices, for example, via the Internet or another network, wirelessly, or the like.

[0056] Operating system 115 may be or include any code segment designed and / or configured to perform tasks of coordinating, scheduling, arbitrating, supervising, controlling, or otherwise managing the operation of computing device 100, such as scheduling program execution. Memory 120 may be or include, for example, random access memory (RAM), read-only memory (ROM), dynamic RAM (DRAM), synchronous DRAM (SD RAM), double data rate (DDR) memory chips, flash memory, volatile memory, non-volatile memory, cache memory, buffers, short-term memory devices, long-term memory devices, or other suitable memory or storage devices. Memory 120 may be or include multiple, possibly different, memory devices. Memory 120 may store, for example, instructions for executing methods (e.g., code 125) and / or data such as user responses, interrupts, etc.

[0057] Executable code 125 may be any executable code, such as an application, program, process, task, or script. Executable code 125 may be executed by controller 105, possibly under control of operating system 115. For example, when executable code 125 is executed, it may cause a medical image to be displayed or selected for display as described herein. In some embodiments, more than one computing device 100 or device 100 components may be used for multiple functions described herein. One or more computing devices 100 or computing device 100 components may be used for the various modules and functions described herein. Devices including components similar to or different from those included in computing device 100 may also be used, and may be connected to a network and used as a system. One or more processor(s) 105 may be configured to perform embodiments of the present invention, for example, by executing software or code. Storage device 130 may be or include, for example, a hard disk drive, a floppy disk drive, a compact disk (CD) drive, a writable CD (CD-Recordable (CD-R)) drive, a universal serial bus (USB) device, or other suitable removable and / or fixed storage device. Data, such as instructions, code, medical images, image streams, etc., may be stored in storage device 130, loaded from storage device 130 into memory 120, and processed by controller 105. In some embodiments, some of the components shown in FIG. 1 may be omitted.

[0058] Input device(s) 135 may be or may include, for example, a mouse, a keyboard, a touchscreen or pad, or any suitable input device. It will be appreciated that any suitable number of input devices may be operably connected to computing device 100, as indicated by block 135. Output device(s) 140 may include one or more monitors, screens, displays, speakers, and / or any other suitable output device. It will be appreciated that any suitable number of output devices may be operably connected to computing device 100, as indicated by block 140. Any applicable input / output (I / O) device may be connected to computing device 100, for example, a wired or wireless network interface card (NIC), a modem, a printer, or a facsimile machine; a universal serial bus (USB) device, or an external hard drive may be included in input device(s) 135 and / or output device(s) 140.

[0059] Embodiments of the invention may include one or more articles such as computer or processor non-transitory readable media (e.g., memory 120 or storage device 130), or computer or processor non-transitory storage media such as memory, disk drive, or USB flash memory that encode, include, or store instructions, e.g., computer-executable instructions that, when executed by a processor or controller, perform the methods disclosed herein.

[0060] Multiple computer systems including processors such as those of Figure 1 may be used; for example, the capsule, receiver, cloud-based system, and / or workstation or handheld computing device for displaying images may include the components of Figure 1. A cloud platform (e.g., a remote server) including the components of Figure 1 may receive treatment data such as images and metadata, process and generate review information, and display it for physician review (e.g., on a web browser running on a workstation or handheld computer). An "on-premise" option may use a workstation or local server at the medical facility to store, process, and display images.

[0061] According to some embodiments of the present disclosure, a user, e.g., a physician, may build an understanding of a case by reviewing a display of still images (e.g., captured by a swallowable capsule) that have been automatically selected as potentially interesting images. The case may be, for example, images captured during the passage of a capsule endoscope through a patient's gastrointestinal tract (GIT) or a portion thereof (e.g., the small intestine and / or colon). The still images may be displayed in one or more screens and arranged in various layouts (e.g., in a matrix, row, column, and / or within or on an illustration of the GIT or a portion thereof). According to some embodiments, the display of still images may be a default review information display or review information mode or view. According to some embodiments, images are displayed only in a static format (as opposed to an image stream or video). A user may browse among still images (e.g., images displayed in a static format), for example, by scrolling through one or more images (e.g., a group of images). In some embodiments, the still images may be displayed horizontally to simulate the elongated structure of the GIT. Such a display may facilitate the physician's review task. The number of still images displayed simultaneously may be predetermined or preselected by the user. In some embodiments, the number of images displayed simultaneously may be user-configurable. In some embodiments, the number of images displayed simultaneously may be between 3 and 7 images.

[0062] According to some embodiments of the present disclosure, a relatively small number of images from the captured images are displayed per case for user review. By "relatively small number," we mean at most or at least on average approximately a few hundred, as opposed to current methods of displaying video streams of images that typically include thousands of images per case (e.g., approximately 6,000 images). In some embodiments, only up to a few hundred images are displayed for user review. In some embodiments, the number of images displayed for user review is up to approximately 1,000 (i.e., less than 2,000 per case).

[0063] Browsing still images (e.g., images displayed in a static fashion) allows a user more control over the time and manner of display of each image and / or each group (or sequence) of images, as opposed to viewing or reviewing a video stream of images (e.g., a moving image). Browsing a relatively small number of still images, as opposed to browsing or reviewing a video stream of thousands of images, can greatly ease the user's review process, reducing reading time per case and potentially resulting in a better diagnosis.

[0064] Selecting only a relatively small number of images per case or review for display and review may be possible by, among other things, utilizing selection or decision-making methods that provide high sensitivity (e.g., high probability of identifying images of interest) and high specificity (e.g., high probability of identifying images of no interest) per case. According to some embodiments, image selection may be performed by utilizing state-of-the-art methods, such as deep learning methods.

[0065] In some embodiments according to the present disclosure, additional information about the capsule endoscopy procedure (e.g., aside from the selected image) may be displayed. The additional information may be displayed by default (e.g., within a default screen), or may be displayed upon user request or after user input.

[0066] In some embodiments, the additional information may refer to the currently displayed image or group of currently displayed images. For example, a user may indicate a currently displayed still image or images to receive additional information about the indicated image / images. The additional information may include, for example, one or more images surrounding or adjacent to the indicated image / images, an indication of an area of ​​interest within the indicated image / images, and / or the type and / or size of the pathology / pathologies, and / or any other structure or feature of interest suspected or identified (e.g., with a degree of certainty) in the indicated images.

[0067] In some embodiments, a display of a case (e.g., a patient or a set of images taken from a patient during a single pass of a capsule through a procedure) may include two levels or categories of information that may be displayed separately or together depending on user input and settings: a first level (e.g., "Level 1") of information that includes a default display in which selected still images are displayed by default for user review (e.g., a main view of the default view of review information), and a second optional level (e.g., "Level 2") of information that includes a display of additional information and may be displayed upon user request or following user action.

[0068] The default, first, or main screen or display (e.g., in a default review information view) may include “first-level” images of the case, e.g., images that show the pathology (with some degree of certainty), which may be automatically selected as images of interest. In some embodiments, if multiple images are identified as capturing a particular pathology (e.g., an ulcer), only one of these images may be selected to be presented on the default or main screen (e.g., a first-level image). Other features or structures may be identified as structures of interest separate from the pathology (e.g., a particular anatomical structure or foreign body, abnormality, combinations thereof, etc.). A user may select and activate an image from the first level and view second-level images in relation to the selected image in the first level. For example, if a user selects (e.g., clicks, hovers over, or otherwise indicates) an image of an ulcer from the first-level display, in the second level, for example, the user may scroll through other images of this ulcer (which may precede and / or follow the selected image).

[0069] In some embodiments, selection for display (e.g., via a control) or display of second level or category information for a first level in-focus image (e.g., a user-selected image) may be made via a separate window or screen (e.g., a pop-up window). Typically, such a separate window or screen appears upon user request or action, e.g., upon receipt of user input regarding the display of additional levels or categories. In some embodiments, a first level in-focus image is displayed along with its associated second level image in a separate view, e.g., a separate window or screen. The separate window or screen may provide more screen area for displaying the in-focus image and its associated second level information.

[0070] In some embodiments, the second level information and / or controls for displaying the second level information may be presented on the same initial or default screen; for example, a GUI item may allow user input to request the second level display or to control the second level display after it is displayed, and the second level display may appear alongside the first level display, for example, on the same screen or display area or a dedicated screen or display area. In such embodiments, a separate window or screen is not needed or displayed to present the second level information. Providing the second level information within the same screen or window with the first level or default information may allow the user to stay in context, thus facilitating and improving the review process.

[0071] In some embodiments, the second level of information may include additional images. The additional images may also be selected from the received series of images or the set of captured images. In some embodiments, the second level of information may include multiple images displayed in the first level and surrounding one or more still images designated by the user. The surrounding images may be selected according to their capture time relative to the capture time of the designated image(s) and / or their position along the imaged GIT portion relative to the location of the designated image(s). In some embodiments, a predetermined number of images closest to the designated image(s) in the first level may be displayed in the second level. In some embodiments, the second level display may include a display of the designated first level image(s) and images surrounding the designated first level image(s) or captured around the capture time. In some embodiments, the second level display may be combined with two or more images in the first level, for example, when two first level images are estimated to be adjacently located within the imaged GIT portion. In some embodiments, the additional images and the indicated images may be displayed at the second level as still images and / or as moving images (eg, as video clips, video sequences, or videos).

[0072] In some embodiments according to the present disclosure, still images may be displayed locally, along with location information providing, for example, the location of the image capture site within the G1 tract. In some embodiments, all of the images selected for display may be displayed locally. In some embodiments, image localization may be provided only upon user request (e.g., as part of the second level of information). Typically, a user request, or user selection, or user change of mode, or other input, is received as input to the computing device (e.g., via an input device, as shown in FIG. 1 ). In some embodiments, only images selected by the user as images of interest, e.g., images identified by a physician to illustrate a polyp in a colon procedure, are displayed locally. Localization of findings or pathologies may provide important information that may, for example, affect treatment decisions or provided treatment. Thus, for example, the location of an identified polyp may determine whether a sigmoidoscopy procedure or colonoscopy should be performed to remove the polyp and may guide the physician during the procedure.

[0073] The imaged GIT portion may be divided into segments. For example, the small intestine may be divided into three equal-length segments, e.g., according to the method disclosed in U.S. Patent Publication No. 2019 / 0244351, incorporated herein by reference in its entirety, or other known methods. The colon may be divided into five anatomical segments, e.g., the cecum, ascending or right colon, transverse colon, descending or left colon or left colon-sigmoid colon, and rectum, or two anatomical segments separated by the splenic flexure, e.g., according to the method disclosed in http: / / www.cs.technion.ac.il / ~cs234313 / projects sites / S18 / 12 / site / , incorporated herein by reference in its entirety. Display of the images (e.g., first-level images, second-level images, or both) may include an indication of the segment in which each image is located. In some embodiments, the location of each image within the segment may be indicated. In some embodiments, the identified area of ​​interest (e.g., an identified feature of interest in the image) or its location (e.g., location within the image) may be indicated within the image. For example, the identified area may be contoured or the location of the identified area within the image may be indicated with an arrow. The location of the image along the imaged GIT portion may be determined by known methods of position estimation. For example, the location of the image along the imaged GIT portion may be determined based on a computerized evaluation of the progress of the capsule device through the subject's GIT. Such a computerized evaluation may be based, for example, on the method and system disclosed in U.S. Patent No. 8,792,691, entitled "System and method for detecting motion pattern of in vivo imaging device," to Krupnik et al., and assigned to the common assignee of the present application, or other known methods.

[0074] In general, the division of the GIT into anatomical segments may be performed, for example, based on the identification of capsule endoscope pathways between different anatomical segments, which may be performed, for example, based on machine learning techniques.

[0075] In some embodiments, a map representing at least the entire imaged GIT portion or a fragment thereof may be displayed. The map may include an indication of the selected images (e.g., of the first-level images and / or the second-level images). Thus, the map may provide the user with an overview of the imaged GIT portion or fragment thereof in terms of suspicious or potentially interesting areas, their extent (e.g., their occurrence along the length of the GIT), and / or their distribution. The extent may be estimated or generated automatically by the system and presented to the user visually (e.g., an indication of the length on the map) or numerically (e.g., a measurement of the extent, such as by percentage). The map may be used by the user to navigate between the selected images. In some embodiments, the map may be in the form of a bar or a similar form to that of the GIT. In some embodiments, the indication of the selected images may be in the form of a heat map, which is an abstracted representation of all or a portion of the GIT, such as a bar displaying selected images according to their position along the GIT. "Heat" may refer to the color used to convey information on the map, such as the color assigned to a line indicating a particular image along a bar. Different colors may represent different types of images, such as, for example, first or second level images. Such heat maps or bars may include lines (e.g., colored differently) indicating segmentation of the GI portion, etc. Colored segments of the map may provide specific information about those segments. The system or GUI may receive a user selection of an image designation or location within the map and, in response, display additional information about the image represented by the user-selected image or location designation, or the image indicated in the map.

[0076] Indication of selected images in the map may be according to their estimated position along the imaged GIT portion. In some embodiments, a section of the imaged GIT portion may be indicated in the map. In some embodiments, the currently displayed section (i.e., the section in which the currently displayed image is located) may be indicated in the map (e.g., by coloring or highlighting the section). Selected images may be indicated in the map based on their chronological order. In some embodiments, the in-focus image may be indicated in the map. The user may select an image to be displayed or an image to be in focus from the map.

[0077] In some embodiments, the map may include an indication of the type of feature or structure of interest identified along the GIT portion that the map represents, which may be, for example, an icon representing the type of feature (e.g., bleeding, ulcer, or polyp).

[0078] In some embodiments, a map and / or graphical representation may be displayed above the selected image and / or at the top of the screen. Such a top display may make it easier to navigate the selected image via the map (e.g., the user does not have to move their eyes, e.g., look down, to see the map).

[0079] The cleansing or cleanliness levels that may be provided or displayed by embodiments of the present invention may describe the level of cleansing of the intestines from contents, feces, and food residues, typically after "preparation" prior to an examination procedure, such as a capsule endoscopy procedure, which preparation is intended to cleanse such visual obstructions from the G1 tract. Known standards for colon cleansing levels include four levels (although other numbers of levels or standards may also be used). Several methods also exist for assessing small intestinal cleansing, and in one embodiment, a colon cleansing standard may be used to describe the level of small intestinal cleansing.

[0080] In some embodiments, the additional information may include an indication or a graphical representation of the cleanliness level of the image, providing an indication of the cleanliness level of the imaged portion of the GIT. The cleanliness level may be estimated or determined (e.g., automatically by a system such as that of FIG. 1 ) for each image in the sequence of images captured, or for each image only in selected images, and / or for each section of the imaged GIT portion, and / or for the entire imaged GIT portion. Generally, the imaged GIT portion may contain material such as bubbles that may obscure the image and / or contents (e.g., naturally occurring substances in the GIT that may obscure the image) and / or the tissue of the GIT. The images may then be analyzed to detect such obstructions. A score may be assigned to each image indicating the cleanliness level detected in the image. In some embodiments, the cleanliness level may be presented in the form of a graph or bar, e.g., a color bar. In some embodiments, the graphical representation of the cleanliness level may be displayed below the map or above the map and in a corresponding manner. In some embodiments, several categories, such as low cleanliness level, medium cleanliness level, and high cleanliness level, may be determined, and the cleanliness level of the image may be determined according to these categories.

[0081] In some embodiments, a score may be determined automatically (e.g., by the system shown in FIG. 1 ) and assigned to each image or each selected image and / or section and / or treatment, which may reflect mucosal coverage of the imaged segment of the GIT (e.g., colon). The treatment for this material may be the set of images captured by a particular capsule swallowed once by a particular patient at a particular time instance. For example, such a score may reflect how much (e.g., percentage) of the tissue (mucosa) of the GIT segment the images captured. Such a score may be based, for example, on estimated or determined cleansing level, contents, air bubbles, capsule transit time, capsule velocity, and / or capsule dynamics. The score may provide information to the physician or observer regarding the adequacy of treatment. This score may be provided in addition to or instead of the cleansing level and may be displayed in a manner similar to that described above for displaying the cleansing level. The score may be provided to the user by default or upon request, before, during, and / or after review of the selected images.

[0082] In some embodiments, the additional information may include a series of images in the captured series of images that have been identified as containing passages between anatomical divisions or fragments of the imaged GIT portion. Such passages may be, for example, passages from the stomach to the small intestine, passages from the small intestine to the colon, or passages from the colon to the outside of the patient's body. In some embodiments, the additional information may include only a representation of such a series of images (e.g., an image overview). In some embodiments, the representation of this passage sequence of images may include a color bar. Such a passage representation may enable a user to more closely review the passages between different anatomical fragments of the imaged GIT portion. In some embodiments, the passage sequence of images or a representation thereof may be displayed upon user request or after user action, e.g., by hovering over an indication of such passages, e.g., within a map representing the imaged GIT portion. In some embodiments, such anatomical passages may be features of interest identified and represented within the first-level images or a selected image subset. For example, a single image may be selected to represent a passageway within the image subset (e.g., a first level image), and one or more images associated with the selected image in the image subset (e.g., second level images) may be additional images selected to represent the passageway.

[0083] In some embodiments, an automatic size estimate may be provided for a feature, pathology, or any other finding automatically identified in an image and / or indicated in the image by a user, such as a polyp. The automatic size estimate may be provided as additional information, e.g., as a second level of information, and / or upon user request, rather than by default. In some embodiments, an indication of an area of ​​interest and its size estimate may be provided by default. According to some embodiments, the automatic size estimate may be based on known size estimation methods, for example, providing a size estimate based on user input. The user input may indicate at least one dimension of a feature to be measured as shown in the image, e.g., a length or diameter represented by two or more image pixels indicated by the user. Such methods may be found in U.S. Pat. Nos. 9,412,054 and 9,911,203, which are incorporated herein by reference in their entireties, or other known methods may be used. To perform a fully automatic size estimate, the automatic estimation of the user input may then be performed. For example, by segmenting an identified feature of interest in an image and selecting two edge points or two peripheral points of the segmented feature that are furthest apart from each other, in some embodiments, a user may modify or replace the two automatically selected points and receive a size estimate based on user-provided input.

[0084] In some embodiments, an automatic interim report may be generated during a capsule endoscopy procedure. Such an automatic interim report may be based, for example, on images captured up to a specific time and / or on images of a specified anatomical region of the imaged portion of the GIT, and / or on images of a specific finding, such as significant bleeding or a polyp that needs to be removed. An interim report may be generated, for example, after a predetermined period of time has elapsed since the patient swallowed the capsule endoscopy device or since the capsule endoscopy device reached an anatomical landmark (e.g., the beginning of the small intestine or the beginning of the colon). As another example, an interim report may be generated when the capsule endoscopy device reaches a specific anatomical landmark or when a specific condition or event, such as bleeding or a polyp, is identified. The interim report may then be provided to the assigned physician.

[0085] The automated interim report may include the received image stream set and / or images selected to represent specific findings. In some embodiments, the automated interim report may also include additional information as disclosed herein. In some embodiments, the automated interim report may include indications and suggestions for diagnosis for specific findings and / or suggestions or recommendations for treatment or a course of treatment action.

[0086] In some embodiments, the generation of an automated interim report or interim workup information for review by a medical professional according to different embodiments, along with real-time indication of identified pathologies or any other findings that may require medical intervention (such as surgery or minimally invasive procedures), may enable the performance of such intervention without delay, for example, as soon as the capsule endoscopy procedure is completed or immediately thereafter. In some cases, for example, in the case of severe bleeding, immediate medical intervention may be required. However, even when the required medical intervention is less urgent (e.g., when a polyp that needs to be removed is identified), the interim report may enable the patient to utilize the fasting and / or bowel preparation performed for the capsule endoscopy procedure for the required medical intervention, for example, by performing the colonoscopy on the same day. Same-day colonoscopy may be more convenient and less burdensome for the patient, for example, by avoiding the need for a second preparation.

[0087] Real-time and effective automated interim reports or interim review information during treatment may be facilitated by utilizing technologies such as, for example, distributed, decentralized, and / or cloud computing. In some embodiments, the automated interim reports or review information may include an estimated or determined level of cleansing or a score regarding the adequacy of treatment to this stage, as described herein. Such information may be used by a physician to provide and / or modify instructions provided to the patient regarding pre-treatment and / or dietary restrictions if a decision is made to perform a subsequent medical procedure (e.g., a same-day colonoscopy procedure).

[0088] Embodiments may apply mutatis mutandis to the display of still and / or moving images (e.g., video). In some embodiments, the additional information included in the information for review by the systems and methods disclosed herein may include diagnostic suggestions and / or treatment suggestions and recommendations or courses of treatment action. Such suggestions may be based on the processing and analysis performed by and in accordance with the systems and methods disclosed herein and may be based on relevant standard of care.

[0089] In one embodiment, selection of images of interest to be displayed for user review occurs in two stages. In a first stage, multiple images of interest from a captured series of images are automatically selected and displayed to the user (e.g., by the computing device shown in FIG. 1 ), e.g., in a matrix layout, grid layout, or other layout. The selected images may also be displayed locally, e.g., with location information providing the location of the image capture site within the GIT. In a second stage, the user may select an image from the automatically selected images (e.g., by providing user input) and receive a display of additional information about the selected image (e.g., the image in focus). The additional information may include the automatically selected additional images. The user's image selection may then be used to generate or add to an overall case map or report.

[0090] FIG. 2 illustrates a first screen, e.g., a default screen, initially presented to the observer or user as a default screen for the default display of review information. The observer may navigate between various regions or segments of the imaged GIT portion, which in the example of FIG. 2 are three tertiles of the small intestine. Each tertile may be represented as one of three tabs 210 on this exemplary screen. For example, when a user selects a segment or tertile of the small intestine by clicking the respective tab 210 in the GUI displayed in FIG. 2, the images 206 associated with that automatically selected tertile are displayed to the user in a matrix layout 220 in the central area of ​​the screen, for example. In this example, each tertile represents one-third of the capsule's transit time through the small intestine. In some embodiments, each tertile may represent an estimate of one-third of the small intestine's length. In some embodiments, the tertiles may be analyzed based on the detection of landmarks along the small intestine. A button 202 may be clicked or otherwise indicated by the user to toggle between the report screen and the first view. Estimated entry and exit times 204 into the small intestine or other GIT segment may be displayed.

[0091] In some embodiments, a user may switch between a Level 1 view and a Level 2 view. For example, a user may switch to the Level 2 view by clicking on an image shown in Level 1. When a user selects a particular image, for example, by clicking on the image when displayed on the GUI, a larger version of that image is displayed in the “additional information” (e.g., second-level) area 230 of the screen. Each image in the additional information display may correspond to one first-level image in the default view, for example, by showing the same feature or pathology indicated by that first-level image. A user may view additional information about a selected one of the images 206 in the “additional information” or second-level area, such as additional still images captured before and after the selected image in chronological order, or a short clip (e.g., a moving image sequence or video) containing images surrounding the selected image (e.g., several images before and after, or images captured several seconds before and after). In FIG. 2, the second-level images are displayed in the same view as the display of the first-level images. The user can also identify a selected image as an image of interest or otherwise mark it for inclusion in the case report, for example, by checking a dedicated checkbox 208 that may be located on the selected image in the main display area. Additional functionality may be available to the user on this screen, such as the option to mark (e.g., via the user entering information into the GUI) or add comments to the selected image.

[0092] Such a screen or display, such as the example of FIG. 2, may include a heat map or other information presentation 240, which may include, for example, displaying information in a graphical or graph form, a graphical representation of each segment or tertile. When a user selects an image of interest, an indication (such as a vertical line 242) may be added to the respective segment's graphical representation or map at a location corresponding to the selected image's location within that segment. Indications such as 242, and similar indications shown in other figures, may be local image indications or markers that represent the image according to its location or estimated location within the GIT. This second stage of user selection may be separate from or integrated with the user's selection of images for a report, as described above. Thus, in some embodiments, after a user selects and adds images to a map, the user may then make another selection of images to be included in the report. In other embodiments, the images the user selects and adds to the map may also be images to be included in the report.

[0093] In the example display of Figure 2, if the number of images automatically selected for a category or tertile is greater than the number of images that can fit within the central display area, the user may be able to scroll up or down. In the example of Figure 2, images selected by the user for display in the "Additional Information" area may be highlighted (e.g., with a blue frame), and their respective marks on the heat map area may also be highlighted simultaneously.

[0094] FIG. 3 shows an exemplary screenshot. In some embodiments, a map or case map 310 generated based on a user's image selection may be displayed, for example, in a separate screen, as shown in the example of FIG. 3. In the example of FIG. 3, the case map 310 is a thin bar with various markings, time stamps, and other symbols 312 illustrating in graphical form the GIT of a case subject (e.g., a person whose GIT was imaged by a capsule). The map may include an exemplary representation of substantially the entire case, including the entire GIT portion imaged (which in FIG. 3 is only a portion of the entire GIT). In the example of FIG. 3, a graphical representation of the entire small intestine is displayed, although other exemplary graphical representations of the entire case are possible. In some embodiments, the exemplary graphical representation may include elements 314 that resemble or suggest anatomical features of the associated segments. For example, the representation in FIG. 3 includes indications of anterior and posterior organs along the digestive tract (e.g., images of the stomach and colon displayed at opposite ends of the small intestine representation).

[0095] Alternatively, the screen may display a map that more generally represents the division.

[0096] Separate markings may be used to illustrate separate segments, such as the three tertiles, of the small intestine. In the example of Figure 3, these markings are vertical dashed lines 316 at the beginning and end of, and evenly spaced along, the graphical representation of the small intestine.

[0097] The images selected by the user may be represented by marks on the map, where, for example, these marks are solid vertical lines 312, and the user may access the respective images by, for example, clicking on or hovering over these marks.

[0098] The map screen or display of FIG. 3 may also include an estimated or determined cleansing level, e.g., a graphical representation or indication of the automatically determined cleansing level score for each segment and / or an indication of the image quality during capsule passage through that segment. Similar indications of cleansing level are shown in other figures herein. The cleansing score may be calculated, for example, based on an automated estimation of the portion or percentage of the image frame that is covered with content or otherwise does not show the tissue wall of the respective organ. In this screenshot, the cleansing level is represented as a bar 320 containing a graph. The cleansing level bar further includes a horizontal dashed line 322, which may represent a threshold level to assist the user in assessing whether the cleansing level at any given time is above or below that threshold. The threshold may be, for example, an average or limit value of a suitable value.

[0099] The map shown in Figure 3 may be displayed on a screen separate from the screen or display of the Level 1 images (as shown in Figure 2) or together with the selected images.

[0100] In one embodiment, an image from a series of captured images may be automatically selected. A map containing an indication of the automatically selected image is initially displayed to the user, or displayed by default (e.g., a pre-added map). The map includes markings of automatically identified regions or sections of the imaged organ. The user can then access more detailed presentation information for the region of the map that they wish to explore.

[0101] In one embodiment, the user is presented with a graphical representation (e.g., a map) of the entire case, e.g., the entire GIT portion of interest (e.g., the entire SB or the entire colon). For example, the user may be presented with a graphical representation of the extent of the entire small intestine divided into three tertiles, e.g., based on automatic identification of capsule endoscope entry into the small intestine, capsule endoscope exit from the small intestine, and measurements of the transit time therebetween. In some embodiments, the division of the GIT portion may be based on an estimation of the length traversed by the capsule endoscope device. Marks (e.g., in the form of vertical solid lines) on such a graphical representation may indicate automatically selected images, e.g., Level 1 images, or images from a selected image subset that are likely to be of interest or likely to be of interest. Diffusion features (e.g., regions of an organ that may include an image of interest or a grouping of multiple adjacent images of interest) may be identified on the map. In such an embodiment, the diffusion features may be displayed as bars that have a lighter color tone than the marks for the images of interest and extend from the respective starting positions of the images to the respective ending positions of the images.

[0102] FIG. 4 shows an exemplary screen or display on which additional information is displayed. On the same screen, the user may view a selected image by providing input, for example, by clicking or hovering over the respective markings 510 on the map 502. When the user selects an image, for example, by clicking the respective markings 510 or thumbnail 512 displayed in the map's display area, a larger version 530 of the same image may be displayed in an "additional information" area 540 (e.g., in the lower left corner of the screen, as used herein). The user may then view additional images from each section of the organ, or a short clip containing images captured before and after the selected image (e.g., images surrounding the selected image). The user may be able to utilize other functions, such as adding notes or marks to the displayed images or selecting images for inclusion in a report. Markers of the division of the organ into sections (e.g., tertile division marks) 504 may be displayed at their respective locations on the graphical representation of the section.

[0103] FIG. 5 shows the example screen of FIG. 5 after a zoom operation. In some embodiments, a user may input the command "zoom in" to cause the system, e.g., a workstation, to zoom in on an entire area or segment of map 502 to view a group of selected images from that respective segment. The selected images of (e.g., corresponding to) each segment may be displayed, for example, as a series of grids, lists, etc. ordered by time 520 of thumbnails 522 on a graphical representation of the respective area. The user may also navigate between segments or sub-areas of the map (e.g., by providing input to a GUI) using a reduced representation of map 540, for example, as shown in the example of FIG. 5 in the upper right corner of the screen.

[0104] In some embodiments, a user may edit the map, for example, by providing input via a GUI, to cause a process to be performed to remove markings on selected images, for example, if the user determines that the respective image is not an image of interest.

[0105] After completing the review, the user may proceed to display a report generation screen, where the user may review the images selected for the report, e.g., the images the user selected as images of interest.

[0106] In one embodiment, in the default view mode, automatically selected images may be displayed in a static and columnar layout on one side of the screen, e.g., in a vertical column on the right side of the screen. The user may select images to be displayed in the image display area, e.g., on the left side of the screen. Each selected image may be displayed with an indication of the identified area of ​​interest within the image, e.g., by coloring the area of ​​interest a particular color. The display may include check boxes indicating the identification of the area of ​​interest. The user may then identify the area of ​​interest and check the appropriate check box accordingly (e.g., bleeding, ulcer, or other). A map or heat map of the entire GIT portion of interest or a fragment thereof may also be displayed, e.g., below the image display area. The map may include an indication of the selected image and may be used by the user to navigate between these images.

[0107] In one exemplary embodiment, image review may be based on two levels of information. A first level of information may be presented in a default view mode, and a second level of information may be displayed upon user request (e.g., user input to a GUI received in a process performed by a system such as that shown in FIG. 1 ) or following a user action. The default view mode may include displaying an image automatically selected (e.g., by a system such as that shown in FIG. 1 ) from a series of images captured within the GIT (e.g., small intestine) or a portion thereof. The imaged portion of the GIT may be divided into multiple segments (e.g., the SB may be divided into three segments / tertiles, or the colon may be divided into five anatomical parts or segments). The selected images are displayed in a static fashion according to the section in which they are located. The user may browse through the static images. The default view mode may also include a display of a map of the entire imaged GIT portion, indicating the selected images according to the section and their estimated position within the section. In some embodiments, the default display of the map may include only the section / sections in which the displayed image is located. In some embodiments, the user may enter a process that causes the map representing the section of the imaged GIT portion to zoom in on the section (e.g., increase the size of the displayed image).

[0108] The second level of information may include a display of additional images surrounding the selected image and may be displayed separately from the first level of image (e.g., in a pop-up window or screen). The default view or display (e.g., first level of information) may also include an indication of the fragment of the GIT portion that was imaged displayed in the second level of information. The default view may further include an indication of the cleaning level of the sequence of images captured within the GIT portion. The surrounding images displayed in the second level of information may be displayed in a static manner and / or as a video clip, short video, or moving image.

[0109] 6A and 6B provide an example of the display and analysis of the small intestine.

[0110] Figure 6A shows a schematic diagram of a default view mode according to a disclosed embodiment. Figure 6B shows a schematic diagram of a map (e.g., bar) in the default view mode of Figure 6A according to one embodiment.

[0111] According to one embodiment shown in FIGS. 6A and 6B, the default view 700 may include the simultaneous display of a set of still images 712 containing only a few images (e.g., four images), allowing the user to carefully and clearly review the displayed images. The image sets may be displayed, for example, in a row 710 layout simulating the layout of a GIT, or in another layout. Images may be displayed substantially centered in the view. The user may move the focus (e.g., primary or maximum display) between images in the displayed image sets or scroll through the image sets, for example, via user input. Selected images within each section (e.g., each tertile) may be divided into such image sets and displayed according to their chronological order (e.g., according to their capture time). The tertile to which the currently displayed image set belongs may be indicated in the view.

[0112] The default view may include a map 740 located at the bottom of the screen, for example, as shown in FIGS. 6A and 6B. The map 740 may be in bar or row form or in a linear format and may include sub-rows (e.g., three rows) representing types of information, and in one embodiment, may include three types of information, indicated as 741, 742, and 743 in FIG. 6B. An exemplary map is a map of the entire small intestine. The map may be divided into segments, such as three tertiles 745, by vertical dashed lines 747. The map may also include a portion of the colon. The in-focus image may be indicated within the map by a different colored vertical line 760 and / or cursor 762 (e.g., by a blue line 760 and cursor 762), as shown in FIGS. 6A and 6B. The frame within the in-focus image may also be a different color (e.g., blue, as shown in FIGS. 6A and 6B). The user may also receive an indication of whether the frame has already been reviewed with the second level of information. Thus, for example, a frame of an image that has already been reviewed at the second level may also have its color changed, for example from black to gray.

[0113] The first subrow 741 of FIG. 6B includes a heat map indicating selected images located along the imaged GIT portion, which includes the small intestine herein. The selected images are indicated as vertical lines 770 of a particular color (e.g., gray) and are positioned along the bar according to their estimated location along the small intestine. The user may provide input (e.g., mouse over) to the line representing the image (e.g., a miniature image) to receive a thumbnail display of the image along with an indication of its capture time. Portions of the bar may be colored differently to indicate anatomical areas of interest, as shown in FIGS. 6A and 6B (e.g., provided as light blue areas indicating estimated proximal and distal areas of the small intestine). Upon completion of the heat map, at least one image of the next anatomical portion of the GIT, herein the colon, may be displayed to indicate, among other things, that the entire small intestine or GIT portion of interest has been imaged and the procedure is complete. The line indications on the images may change color (e.g., to orange) as the user reviews them.

[0114] The second sub-row 742 may include indications of segments of the imaged GIT portion, here the small intestine, whose images are displayed at the second level of information. These segments may be indicated by indications 780, such as gray-colored rectangles. Each rectangle, e.g., a second-level segment, may include and / or relate to one or more selected images from the first level, as may be seen in Figures 6A and 6B. This is the case when two or more selected first-level images are relatively close to each other.

[0115] A third sub-row 743 may, for example, include a graph indicating the cleansing level of images captured along the small intestine. The graph may be colored according to the level value, for example, red when the (e.g., cleansing) value is relatively low.

[0116] Pathways between different anatomical segments of the imaged GIT portion, for example, the stomach to SB (e.g., stomach to small intestine) and / or the SB to colon, may also be indicated as shown in FIGS. 6A and 6B. Pathways may be indicated above an indication (e.g., a dashed line) of the associated segment in the map, and may include transit times. Optionally, a small color bar representing a series of images captured within the identified transit area may be displayed above the indication of the associated segment. The user may provide input (e.g., movement) to the color bar and review the transit images. The color bar may represent only a sample of images in the identified transit area.

[0117] The images shown in FIG. 6A may be different; for example, selected images may be displayed in a matrix format, so that more images are displayed simultaneously. The user may, for example, request the use of another display mode of view instead of the view mode of FIG. 6A. According to such an alternative view, the images for each tertile may be displayed in a separate matrix, e.g., a separate tab. The user may scroll between the images within each tertile.

[0118] FIG. 7 shows an exemplary diagram of a display or view of a second level of information (eg, review information) for an image selected according to the display illustrated in FIG. 6A.

[0119] A second level of information may be provided in accordance with this embodiment by a pop-up window 910 that displays a selected image 912 by default. The window / display / view may display a series of sub-images surrounding the selected image (e.g., several images before and after the selected image, or images captured a few seconds before and after the selected image). A user may use an input device such as a mouse scroll wheel to cause the images to scroll in a static manner (e.g., move forward and backward in sequence or time) or to display the series of sub-images as a video clip. A map 920 may be displayed below the image display area.

[0120] According to the example of FIG. 7, the map may include a portion of the map of the first sub-row of the first-level default view, representing the relevant section of the imaged GIT portion. The portion representing the series of second-level sub-images may be indicated in color, for example, orange. An additional second-level map may be presented above the portion of the first-level map, including an indication of a selected image of the first-level view included in the series of second-level sub-images. When such an image, for example, a selected first-level image, is displayed in the image display area of ​​this second-level data pop-up window, it is indicated in this second-level map (for example, a square in the illustrated example, with the relevant square colored blue and a blue cursor positioned above its marking).

[0121] A portion of the first level map representing a fragment containing the second level image may be displayed below the image display area, and selected images may be indicated within this map, with selected first level images in review being indicated, for example, by coloring the image line indications in different colors.

[0122] In one embodiment, one image at a time may be selected from a group or "pile" of selected images by browsing the images horizontally (e.g., in a carousel fashion). A heat map representing the imaged GIT portion and indicating the selected image may be displayed, for example, above the image display area. Each displayed image may be referenced to its indication in the heat map, for example, by a line connecting the image to its point or heat map or representation on the map.

[0123] In one embodiment, one image at a time from a group or pile of selected images may be displayed. A user may view a single image by using an input control (e.g., a mouse, scroll wheel, etc.) to provide input to the GUI to scroll the images vertically and select one image at a time. A heat map or other map representing the imaged GIT portion and indicating the selected image may be displayed, for example, adjacent to the image display area. Each displayed image may be referenced to its indication in the heat map, for example, by a connecting line linking the image to the associated heat map or map location or image indication.

[0124] The main view of the default view mode, e.g., the default view of the review information, may include a display of a heat map representing the imaged GIT portion and indicating a selected image. The image may be located in the heat map according to its estimated location within the imaged GIT portion. The selected image may be displayed in an image display area. The selected image may be displayed in a static manner or as part of a video clip with additional related images upon user request or after user action. The user may, for example, hover over or click on an image indication on the heat map to receive, for example, a thumbnail and / or a static representation and / or clip of the image.

[0125] 8A shows a screenshot or display of any view mode, according to one embodiment. In one embodiment shown in FIG. 8A, the first level images, or images of a selected image subset, may be displayed in a slide or “carousel” style 1300, such that one image 1302 is displayed more prominently in the center, while other images 1304 are displayed to the side at reduced size with distance from the main, central image 1302, so that user input (e.g., using arrows 1310 on the edge of the screen or a mouse scroll bar) “rotates” another of the other images 1304 in view so that it is centered over the central image 1302. A map 1320 with washes 1326 and other additional information may be displayed in (typically above) the carousel display 1300 of first level images.

[0126] The second level images may be displayed in the same window as the first level images, thus eliminating the need to activate a separate window. A user may, for example, move over or click on an image, after which the image becomes "activated" or "in focus," as shown in FIG. 8A. For example, a user may provide input to the GUI to switch to Level 2, and the central image 1302 may be viewed as a video or image stream, or as a "scrollable" sequence of images displayed in a static fashion of all images related to or associated with the particular instance (e.g., second level images) represented by a single central image in Level 1.

[0127] In this embodiment, still images (e.g., automatically preselected images) are displayed horizontally in a carousel configuration 1300. In the particular example of FIG. 8A , five images are displayed simultaneously. The center image 1302 is the image in focus. The number of images typically displayed simultaneously thereafter is generally an odd number of images. In some embodiments, various odd numbers of images may be displayed simultaneously. The images may be displayed in a carousel format that allows the user to browse horizontally between images and ensures that images do not obscure other images. The sizes of the images vary according to their position within the display carousel 1300. The center image 1302, being the image in focus, is the largest. The sizes of the other images 1304 vary with respect to their proximity to the center image. The closer an image is to the center image 1302, the larger its size. Such a layout may allow the user to closely and more clearly review one image while simultaneously reviewing and receiving information about adjacent or surrounding images. Such a view may allow comparison between adjacent images while still displaying the in-focus image in detail.

[0128] A bar or map 1320 (e.g., a heat map) may illustrate a representation of the entire imaged GIT portion or a fragment thereof. The exemplary display displayed in FIG. 8A is a display of the small intestine. The map 1320 is displayed above the display of the image and on the upper area of ​​the screen. The map thus represents the small intestine and is divided into three tertiles (e.g., by vertical dashed lines 1322). The map may include information regarding the position of the displayed image along the imaged GIT portion and / or information regarding the cleansing level of the displayed image and / or the captured image and / or the imaged GIT portion. The map may include two or more sub-rows presenting different information. The image in focus may be indicated in the map by, for example, a connecting line 1324 as shown in FIG. 8A. In some embodiments, all currently displayed images may be indicated in the map (e.g., by a symbol or line, or a connecting line).

[0129] In this embodiment, the map 1320 includes two sub-columns 1326 and 1328. The first sub-column 1326 may include information about the cleaning level of the captured images. In this embodiment, the cleaning level is presented in the form of a graph, and may be presented in the form of a color bar (as in FIG. 10 ). Different colors indicate different cleaning level categories, such as low, medium, and high. The second sub-column 1328 may include a bar where selected images are indicated (e.g., by a vertical line) and according to their estimated position along the imaged portion of the GIT.

[0130] Further, according to this embodiment, the second level of information is presented on the same initial or default screen (e.g., the main or default screen of the review information view). The in-focus image (e.g., center image 1302) is shown with additional controls 1308 that allow the user to, for example, scroll between surrounding images of the selected image as a second level display. The scroll bar may indicate the numbering of the surrounding images relative to the in-focus image. For example, the in-focus image may be designated "0," the image immediately to the right "1," the image immediately to the right of that "2," etc. The image immediately before the in-focus image (i.e., the image immediately to the left) may be designated "-1," the image immediately before that "-2," etc. Thus, the user may be oriented in terms of distance and direction (in images and time) relative to the selected original image 1302. Additionally, a text box may be displayed below the in-focus image, allowing the user to enter information such as the type of pathology identified and comments. In some embodiments, the second level of controls and / or data may appear (e.g., below the image) when an image is selected as the focused image and / or when the user positions the image as the center image 1302 (e.g., by scrolling), or when the user moves over the focused image and clicks on it, or upon any other user action.

[0131] FIG. 8B shows the matrix layout of image 1300 shown in FIG. 8A.

[0132] Figure 9A is an example screen of a default view mode according to one embodiment of the present disclosure, and Figure 9B is an example screenshot of a second level information display or view of a selected image within the display of Figure 9A.

[0133] In this embodiment, the case displayed for review is a colon case. In a default screen (e.g., the default screen for the review information view), images selected from the image stream captured during the procedure (e.g., automatically by an image selection process) are displayed in a matrix 1400, as shown in FIG. 9A . The colon may be divided into five anatomical segments: the cecum, ascending colon, transverse colon, descending colon-sigmoid colon, and rectum. The displayed selected images may be localized (e.g., displayed with localization information) according to the five segments and displayed by segment; for example, images may be localized by being designated as being within one of the five colon segments or another GIT segment or category rather than a specific location. Images for each segment may be displayed in a dedicated display area, and the user may switch between segment views via tabs. The particular screen shown in FIG. 9A shows images identified as being captured in the descending colon-sigmoid segment. Images selected by the user for inclusion in the report may be indicated, for example, by a blue frame 1410. Additional information 1420 may be displayed to the user. Such additional information may be (FIG. 9B), i.e., a diagram of the colon indicating which section is being displayed, patient-related details, procedure date, procedure time for the relevant region of the GIT (here, the pre-colonic region and the colon), and whether the review information is complete (in the illustrated example, whether the capsule has reached the end of the colon). Such procedure time and completeness indication may be determined based on identifying relevant landmarks, such as transit or entry through the colon and exiting the body.

[0134] Additional information, e.g., second-level information, may be displayed upon user request. For example, a user may click on a displayed image to open a pop-up window 1450 that displays the image by default and allows review of additional information, as shown in FIG. 9B . Additional information that is displayed or may be displayed (e.g., upon user request) according to this embodiment may include automatic indication of areas of interest within the image, e.g., polyps, or, if the capsule endoscope includes two or more cameras, the head of the capsule that captured the image. Indication of areas of interest may be, for example, by segmenting the area and / or by an arrow pointing to such area. Automatic identification and indication of areas of interest (e.g., suspected polyps) may provide guidance to the physician and improve diagnosis, e.g., polyp detection.

[0135] 10 shows a sample display with icon 1010 (which in one embodiment depicts a pile of images with the number "3" near it, corresponding to three images 1032 selected by the user). The selected images 1032 may be contained within a window or dialog box 1030. The user may click or select on icon 1010 (e.g., using a mouse to enter into a GUI), and the system may cause a window 1030 to appear containing the images the user checked or selected. The user may then review all of the checked images 1032 at once, in a side-by-side or row-by-row fashion.

[0136] In another embodiment, the system may display a certain number of the most relevant or suspicious images (e.g., "Top X Most Suspicious Images," e.g., the 5 or 10 most relevant images) in a separate view or display. Such a view provides a first quick indication to the user and may be useful, for example, for colon screening. In the context of colon screening, the user may determine whether polyps are present (which may indicate the possibility of cancer and require a colonoscopy). The view may be added in a separate tab or window.

[0137] The automatic polyp (or other feature) size estimation function may calculate and provide an indication of the polyp's diameter, length, or other estimated size in response to a user request. The size of the polyp or feature may be automatically calculated, for example, by measuring the maximum distance between two points around the identified polyp's perimeter. The user may select other points for measuring the polyp's size, and the process may calculate the size according to the user's selection. For example, the display of the feature or polyp may include a bounding box for the polyp, with the polyp located within this box. A line within the bounding box may be displayed representing the measured length, and the size may be displayed or suggested, for example, near the line. The user may change the position of each of the two endpoints of the line if the user believes the automatically selected points are inaccurate. One advantage of automatic polyp size estimation, or any other feature of interest, is standardization. For example, current methods determine polyp size based on manual user input (e.g., points defining the length of the polyp) or according to manual estimation, and different users or different user inputs may provide or result in different size estimates for the same polyp.

[0138] In some embodiments, cleaning levels or other data may be displayed per image, per section, and / or per treatment.

[0139] In some embodiments, in each section, an option may be provided to present the most suspicious images first (eg, in a matrix view) or in another way, eg, with an asterisk.

[0140] In some embodiments, the representation of the GIT portion of interest in the map may refer not only to first level images, but also to second level images, so that, for example, a range of diseases and pathologies may also be represented.

[0141] In some embodiments, to provide personalization of the remediation information, the parameters of the remediation information (e.g., how many images are included in the first / second level, how many remediation information images are selected to represent a particular section of the remediation GIT portion, thresholds for selecting images to be included in the remediation information, etc.) may be adjusted or determined for each patient. In some embodiments, such adjustments may be made manually, for example, by a user performing the procedure or a referring physician entering the data. In some embodiments, such parameter selection may be automatic, or at least in part (e.g., automatic adjustment of some of the determined parameters), based on patient information or details, for example, including the patient's medical history. The automatic adjustment may utilize natural language processing (NLP) algorithms or machine learning-based tools, such as the Comprehend Medical system offered by Amazon. Such tools may enable interoperability of free text descriptions and make them searchable. The system may then use the patient's medical background to adjust the remediation information. For example, for a patient (e.g., a case), if the system recognizes in the free text that the patient has a family history of polyps, the system may modify the review information or presentation information to include more images (e.g., to increase the sensitivity of the review information) as opposed to a healthy patient with no history of polyps.

[0142] In some embodiments, integration of speech-to-text or automatic speech recognition systems (e.g., "Transcribe" by Amazon) may significantly reduce report creation time in the case of free text, for example, by allowing users to begin "writing" their report using speech-to-text as they review images. The use of speech-to-text may reduce the need for clicks or other manual user input. For example, a user may say, "Show me pictures of ulcers from the atlas," and an action may be taken based on this, which can be initiated by the system without clicks or manual user input (e.g., to search by key descriptors and display an atlas of common disease images that can be used as a review aid to compare review information images with reference images). Speech-to-text may also enable "chatbots" to assist the system's GUI, allowing users to ask the system questions and have the system check whether similar questions exist in a "FAQ" or other repository and provide an answer.

[0143] In some embodiments, the system may display transition segments within the GIT portion of interest. For example, instead of a color bar, images representing transition segments from each side (e.g., SB stomach and colon) may be presented or available in the first-level view or default view of the information for review (and indicated in the map), e.g., with a second-level display including transition images (between segments). In some embodiments, transition images (e.g., images indicating transitions between GIT segments) may be displayed at each end of the displayed map in response to user input, e.g., after the user clicks on an icon indicating the stomach or colon, accordingly. Such anatomical transition images may include several images identified as being located adjacent to an anatomical transition (e.g., from SB to colon). One option for displaying transition images may include one image for each transition in the image subset (e.g., first-level images), and then include multiple images representing the transition as one or more corresponding images (e.g., second-level images). Images from the image subset may then appear in the map outside (e.g., at both ends of) the portion of interest. Another option is to represent such an image as a small colored bar above the map at either end of the area of ​​interest, or for the user to receive a window or screen containing a display of the image by clicking on an icon at the end of the area (e.g., the SB icon and toilet icon in the colon scan information).

[0144] In some embodiments, the default view of the information for review may show a matrix display of the image subset to allow the user to quickly review the first level of images first. The user may then switch to another view of the information for review, such as a "carousel view" as described elsewhere. For example, clicking on an image in the matrix display initiates the display of another view of the information for review, including a carousel display of the image subset, with the clicked image becoming the center image of the carousel (e.g., a type of zoomed view). This may be most beneficial when the information for review includes a relatively large number of images (e.g., several hundred images). In some embodiments, the matrix view may also include a display of second level images and / or other additional information.

[0145] In some embodiments, machine learning, artificial intelligence, or AI may be used to automatically identify, localize, and size precancerous colon polyps and small intestinal pathologies or other features. In embodiments, modules may be used or implemented that act as "detectors" for specific features or pathologies, such as nodular lymphoid hyperplasia (NLH), ulcers, vascular ectasias, erosions, blood, lesions, or inflammatory strictures.

[0146] A report may be generated after the user's review based on the user's input during the review.

[0147] Figure 11 shows a flowchart of a method according to an embodiment of the present invention. The operations of Figure 11 may use a system such as that shown in Figure 1, but may also use other systems.

[0148] In operation 1100, a subject or patient may swallow or ingest a swallowable capsule that includes, for example, one or more imagers or imaging devices (e.g., digital imagers or cameras), a transmitter, and other units such as a power source and an antenna.

[0149] In operation 1110, the capsule may traverse the subject's GIT, or a portion of the GIT (e.g., by the subject's natural peristaltic movements) and transmit images (e.g., via a wireless transmitter) to an external receiver.

[0150] In operation 1120, the images transmitted by the capsule may be received, recorded, and processed by an external receiver (although other methods of transmitting images may be used. Various combinations of devices may be used to receive and store the images.

[0151] In operation 1130, in one embodiment, a set of images of interest may be identified or selected automatically, e.g., by one or more processors, using, e.g., a first selection method. Each image of interest may exhibit a different automatically identified feature or pathology. Groups of images may be created, each group including one first-level image of interest and a corresponding set of images that are not in the first-level image of interest but that exhibit the same features as the corresponding one first-level image for the group. Each first-level image may be identified as the “most representative” image in the group.

[0152] In operation 1140, the first level images may be displayed on a monitor or display device as still (e.g., non-video) images in a main view or default view of the default view of the review information (e.g., without any user input or command to initiate such display). The first level images may be an image subset of the in-vivo image stream. A user may review the first level images. The first level image subset may represent the received image stream.

[0153] In operation 1150, the user may provide input to display a second level of images, and the system or GUI may receive the input and display on the display device or monitor one or more additional images that correspond to the currently displayed image of the image subset. For example, an image in the group that corresponds to the selected or clicked first level image may be displayed. The one or more additional images in the group may be automatically selected from the received in-vivo image stream according to a second selection method. The second selection method may be based on a relationship between the images in the in-vivo image stream and the currently displayed image.

[0154] In operation 1160, in one embodiment, the user may change the display of the second level images, for example, to display the images as a moving image or to move forward and backward through the group of images in a static fashion, for example, using a scroll wheel. The review information may be displayed, for example, in a carousel mode, a matrix mode, or another mode.

[0155] A report may be generated or displayed to the user in operation 1170. The report may include images selected by the user from the displayed images, and possibly other information such as comments inserted by the user.

[0156] Other or different actions may be used.

[0157] In embodiments, the technology of capsule endoscopy procedures, as well as the generation of information for review, medical image display, and particularly medical diagnosis, may be improved, thus improving the ability of medical personnel to diagnose information and provide better and more efficient treatment. For example, medical image processing and display technology may be improved by providing a system that allows physicians to more quickly find, view, and analyze regions of interest within the GIT, thereby providing automated analysis that may support and improve the physician's judgment when compared to conventional medical techniques. For example, with conventional GIT imaging systems, physicians may be forced to view long footage. Additionally, automatic identification, location, and / or sizing of features of interest are not provided.

[0158] Although several embodiments of the present disclosure have been shown in the above drawings, it is not intended that the disclosure be limited thereby, but rather that the disclosure be given the broadest possible scope within the art, and that the specification be interpreted accordingly. Accordingly, the above description should be interpreted merely as an exemplification of particular embodiments, and not as limiting.

[0159] It should be clear that most of the principles, elements, and features described above with respect to the embodiments according to the present disclosure and / or shown in the drawings are not limited to the embodiments in which they are described / shown, but may be applied independently, mutatis mutandis, to each other or to any other embodiment according to the present disclosure, in any combination deemed appropriate by those skilled in the art.

Claims

1. 1. A system for displaying review information of a capsule endoscopy procedure for user review, the capsule endoscopy procedure including capturing an in vivo image stream of at least a portion of interest of a subject's GIT, the system comprising: one or more processors; a display device configured to display the review information; a non-transitory storage device having instructions stored thereon, the instructions, when executed by the one or more processors, displaying on the display device an image subset of an image stream captured via the capsule endoscopy procedure for review by the user, the image subset representing the image stream, the image subset being automatically selected from the image stream according to a first selection method, the first selection method selecting as the image subset images that are identified as containing features of interest with a degree of certainty based on a machine learning algorithm; upon receiving user input, displaying on the display device one or more additional images corresponding to a currently displayed image of the image subset, the one or more additional images being automatically selected from the in-vivo image stream according to a second selection method, the second selection method being based on a relationship between an image of the in-vivo image stream and the currently displayed image; generating a report, the report including images selected by the user from the displayed images; upon receiving user input, displaying on the display device a predefined number of summary images corresponding to summaries of the image subset, the predefined number of summary images being automatically selected from the image subset according to a third selection method, the third selection method selecting as the predefined number of summary images those images from the image subset that are identified with the highest degree of certainty as containing the feature of interest; a non-transitory storage device that causes the one or more processors to A system comprising:

2. The system of claim 1 , wherein the image subset is displayed in a static manner in a default display of the review information.

3. 2. The system of claim 1, wherein displaying the image subset includes simultaneously displaying multiple images of the image subset, the multiple images being captured by a single imaging device of a capsule device used in the capsule endoscopy procedure.

4. The system of claim 3 , wherein the simultaneously displayed images of the image subset are displayed in a carousel or matrix layout.

5. 2. The system of claim 1, wherein the relationship on which the second selection method is based is selected from the group consisting of images identified as including at least a portion of the same features, images identified as including at least a portion of the same type of features, images captured closely in time, images localized adjacently along the at least a portion of the subject's GIT, and combinations thereof.

6. The system of claim 1 , wherein the one or more additional images comprises at least one image of the image stream that is not included in the selected image subset.

7. The system of claim 1 , wherein the one or more additional images include only images of the image stream that are not included in the selected image subset.

8. 2. The system of claim 1, further comprising: using the one or more processors to display a map representing at least the portion of interest of the subject's GIT and corresponding to at least the selected image subset, the map including local image indications representing the images in the image subset according to their estimated positions along the at least the portion of interest of the subject's GIT.

9. The system of claim 1 , wherein the image subsets are displayed in a matrix layout in a default display of the review information.

10. the at least the portion of interest of the subject's GIT is divided into a plurality of sections; images of the image subset are localized along at least the portion of interest of the subject's GIT according to the plurality of sections; 2. The system of claim 1, wherein the method further comprises using the one or more processors to display an indication of the plurality of sections in which the currently displayed image of the image subset is located.

11. The system of claim 10 , further comprising using the one or more processors to display the image subsets according to the plurality of segments.

12. The system of claim 1 , further comprising using the one or more processors to display an indication of an estimated cleaning level of an image of the captured image stream.

13. The system of claim 1 , further comprising using the one or more processors to display the review information, wherein the review information does not include any images of the captured in-vivo image stream other than the image subset and the additional images corresponding to images within the image subset.

14. The system of claim 1 , further comprising using the one or more processors to filter the display of the image subset based on user input.

15. The system of claim 1, wherein the feature of interest is selected from the group consisting of a pathology, a foreign body, an abnormality, an anatomical feature, and combinations thereof.

Images