Systems and methods for selecting in-vivo captured images for display

Abstract

A method performed by a system for selecting an image from a plurality of image groups from a plurality of imagers sourced from an in-vivo device includes: calculating or otherwise associating a gross score (GS) of the images in each image group to indicate the probability that each image contains at least one pathology; dividing each image group into image subgroups; identifying a set of maximum gross scores (MGS) Set(i), where each image subgroup in each image group has an MGS; and selecting an image for processing by identifying the MGS|max in each set of MGS S(i), identifying the maximum MGS|max, and selecting the image associated with the maximum MGS|max. The method further includes modifying the set of MGS Set(i) associated with the selected image and repeating the steps described above until a predetermined criterion is met that is selected from the group consisting of the number of images N and a score threshold.

Description

[0001] This application is a divisional application of a patent application with an application date of May 18, 2017, an application number of 201780038002.1, and an invention title of "Systems and Methods for Selecting Images Captured In Vivo for Display". Technical Field

[0002] The present invention relates to a method and system for selecting images from a series of images captured in vivo for display. More specifically, the present invention relates to a system and method for selecting and displaying multiple images of interest captured in vivo according to a predefined "budget" of the images set in advance. Background Art

[0003] Capsule endoscopy ("CE") provides thousands of images of the gastrointestinal tract according to an imaging procedure, such as 100,000 to 300,000 images. Reviewing a large number of images is time-consuming and inefficient, and in addition, having to examine so many images may prevent some doctors from even attempting such image review. This is because, when viewing a movie, such as a moving image stream that can be used for medical diagnosis, the viewer may want to view only certain frames, or may wish to view only a short preview, thus summarizing only the specific frames of great concern (e.g., according to predefined criteria), while skipping other images that are unimportant or less important. For example, an in vivo imager may capture, during a procedure, for example, 150,000 images, only some of which may contain polyps and / or blood spots and / or other pathologies within the gastrointestinal ("GI") tract. For example, polyps, ulcers, diverticula, and bleeding are focal pathologies that may only appear in the images, and thus, if "hidden" among a very large number of images, they will be missed / omitted by the doctor.

[0004] It would be beneficial to highlight for a user, such as a doctor, a relatively small set of images containing the most pathologically prominent images, thereby saving a significant amount of review time and effort. It would also be beneficial to reduce the number of images, but at the same time ensure that all pathologies in the gastrointestinal tract or a segment thereof (with at least one representative image) are represented in the smaller set of images. Summary of the Invention

[0005] In some embodiments, a method for selecting images for processing from images captured by an in-vivo device may include performing the following operations on a group of connected images captured in a body cavity (each image being associated with an overall score GS indicative of the probability that the image contains at least one type of pathology): dividing the group of connected images into image subgroups according to a predetermined criterion (e.g., similarity between complete consecutive images, the same 'object' appearing in consecutive images, etc.), selecting from a particular image subgroup the image whose overall score GS is the maximum in the image subgroup, invalidating the GS associated with the images close to the selected image, and repeating the selection and invalidation process for the remaining non-invalid GSs until a number N of images are selected or until the value of the identified maximum GS is below a scoring threshold. (The image subgroups may be formed or created, for example, by analyzing the similarity between images, and the maximum GS in the subgroup may be identified and the image thereof selected, or the maximum GS may be found and the images 'around' the image associated with the maximum GS may be grouped.) The connected images may be, for example, images that are consecutive to each other in a stream when ordered chronologically, for example, according to their capture time.

[0006] In some embodiments, a method for selecting images for processing may include: (i) identifying the maximum GS among the GSs of the images, selecting the image associated with the maximum GS and invalidating the maximum GS associated with the selected image; (ii) identifying an object in the selected image; (iii) searching for the same object as the identified object in images close to the selected image; (iv) and invalidating the GS associated with the images close to the selected image, and repeating or iterating steps (i) to (iv) for the non-invalid GSs until a number N of images are selected or until the value of the identified maximum GS is below a scoring threshold.

[0007] In these embodiments, the object in the selected image may optionally be selected from the group consisting of: (1) the entire image, in which case it may be determined (e.g., by a processor) that the selected image and images proximate to the selected image form an image subgroup if there is a similarity (likeness) between these images; and (2) an object (e.g., a pathology), in which case it may be determined (e.g., by a processor) that the selected image and images proximate to the selected image form an image subgroup if these images contain the same or identical imaging object (e.g., the same pathology, such as for a polyp). In these embodiments, the step of searching for the same object in images proximate to the selected image may include tracing the object backward (in images before the selected image) and forward (in images after the selected image). In one embodiment, 'backward' (and before) and 'forward' (and after) refer to the order of the images in a portion of the connected image group that forms the connected image group based on, for example, the capture time of the images or other ordering. Thus, 'backward' may be, for example, viewing an image before the current image in terms of time or capture time or the ordering in the image stream, and forward may be the opposite.

[0008] In some embodiments, a method applied to a connected image group captured in a body cavity by using an imager, where each image may be associated with an overall score ("GS") indicating the probability that the image contains at least one type of pathology and a distance d indicating the distance between images, may include: partitioning or separating the connected image group or a portion thereof into image subgroups, where each image subgroup contains a base image and images similar to the base image; identifying the maximum overall score ("MGS") for each image subgroup and selecting images for processing, the selection may include: (i) identifying the maximum MGS (MGS|max) among the MGSs identified for the image subgroup and selecting the image associated with or having the maximum MGS; (ii) invalidating the MGS associated with or having the selected image; (iii) modifying each MGS based on the distance d between the image associated with or having the particular MGS and each selected image; (iv) identifying the modified MGS having the maximum value (MGS|max) in the modified MGSs. The embodiments may further include selecting the image associated with or having the modified MGS; and repeating steps (ii) to (iv) until a predetermined criterion selected from the group consisting of the number of images N and a scoring threshold is satisfied. The N selected images or a subset of the N selected images may be, for example, displayed on a display device of a user. The MGS in each image subgroup may be a local maximum overall score in the image subgroup.

[0009] Modifying each specific GS (e.g., MGS) can be based on the distance d between an image associated with or having the specific GS (or MGS) and the closest selected image. Identify the minimum distance Dmin between any one of the images associated with or having the specific GS (or MGS) and the selected image and use the minimum distance Dmin to modify the specific GS (or MGS) to perform the modification of the specific GS (or MGS). (The smaller the Dmin, the greater the decrease in the value of the specific GS or MGS.) The distance d between the image associated with the GS (or MGS) and the selected image can be calculated or measured in the following units: (i) time or (ii) the number of intervening images, or (iii) the number of subgroups of intervening images, or (iv) the distance traveled by the in-vivo device in the body cavity when taking the image, or (v) the distance of the in-vivo device in the body cavity relative to the connected in-vivo image group or the markers in the body cavity. Each unit instance can be calculated or measured as: (i) an absolute value calculated or measured relative to the start of the connected image group or relative to the start of the body cavity or relative to a marker in the connected image group or in the body cavity, or (ii) a percentage or fraction of the connected image group or the body cavity, or (iii) a percentage or fraction of a video clip containing all or most of the connected image group, or (iv) a percentage or fraction of the distance traveled by the in-vivo device, or (v) a percentage or fraction of the time traveled by the in-vivo device.

[0010] Modifying the GS (e.g., MGS) can include applying a modification function to the GS or MGS relative to the selected image. The modification function can be selected from the group consisting of: a symmetric function, an asymmetric function, a linear function, a non-linear function, a Gaussian exponent, an exponent with an absolute distance value, a step function, and a triangular function whose vertex is at the selected frame and decreases linearly with distance. The modification function can be selected according to parameters selected from the group consisting of: the type of pathology, the location in the gastrointestinal tract, and the speed of the in-vivo device in the body cavity.

[0011] An embodiment can include applying a modification function to the MGS based on the location in the body cavity where the image associated with the GS or MGS is captured. The body cavity is the gastrointestinal tract, and a first modification function can be applied to the GS or MGS associated with an image captured in the small intestine, and a second modification function can be applied to the GS or MGS associated with an image captured in the colon.

[0012] An embodiment may include setting the total score of the image to zero (e.g., set to zero) in the case where the image is noisy. The image may be considered noisy if the image contains bubbles and / or gastric contents and / or if the image provides tissue coverage below a tissue coverage threshold and / or the image meets a darkness criterion.

[0013] The total score of the image may be calculated by using a number m of score values S1, …, Sm, which are respectively output by m pathology detectors, and the m score values indicate the probability that the image contains m or k (k < m) types of pathologies. The pathologies may be selected from the group consisting of: polyps, bleeding, diverticula, ulcers, lesions, and red pathologies.

[0014] In other embodiments, a method applied to a group of q connected images respectively captured by q imagers in a body cavity, where each image in each group is associated with a total score (GS) indicating the probability that the image contains at least one type of pathology and a distance d indicating the distance between the images in each group, may include: for each group of the q connected image groups, performing: dividing or separating the image group or a part thereof into image subgroups, each image subgroup including a base image and an image similar to the base image; and (B) identifying a set of maximum total scores (MGS) Set(i) (i = 1, 2, 3, …), the set of maximum total scores Set(i) including the maximum total score (MGS) of each image subgroup in the image group; and selecting images for processing by performing: (i) identifying the maximum MGS (MGS|max) in each set of MGS S(i); (ii) identifying the maximum MGS|max among all the MGS|max and selecting the image associated with the maximum MGS|max; (iii) invalidating the maximum MGS|max associated with the selected image; (iv) modifying the specific set of MGS Set(i) associated with the selected image based on the distance d between the image associated with the specific set of MGS Set(i) in step (i) and each selected image; and repeating steps (i) to (iv) until a predetermined criterion selected from the group consisting of the number of images N and a score threshold is met.

[0015] The in-vivo device may include several imagers selected from one and two imagers, and modifying the specific set of MGS Set(i) associated with a selected image from the first imager may include also modifying the MGS in the specific set Set(i) based on the image from the second imager.

[0016] A system for selecting an image from images captured by q imagers of an in-vivo device may include a storage unit for storing q groups of connected images respectively captured by the q imagers in a body cavity, and for storing a total score (GS) indicating the probability that the image includes at least one type of pathology, associated with each image in each associated group, and a distance d indicating the distance between the images in each group, and a data processor. The data processor may be configured for each of the q groups of connected images to: (A) divide or separate the group of images or a part thereof into image subgroups, each image subgroup including a base image and an image similar to the base image; and (B) identify a set of maximum total scores (MGS) Set(i) (i = 1, 2, 3,...), the set of maximum total scores Set(i) may include the maximum total score (MGS) of each image subgroup in the group of images. The data processor may also be configured to select an image for processing by performing the following operations: (i) identify the maximum MGS (MGS|max) in each set of MGS S(i); (ii) identify the maximum MGS|max among all the maximum MGSs and select the image associated with the maximum MGS|max; (iii) invalidate the maximum MGS|max associated with the selected image; and (iv) modify the specific set of MGS Set(i) associated with the selected image based on the distance d between the image associated with the MGS in the specific set of MGS Set(i) in step (i) and each selected image, and repeat steps (i) to (iv) until a predetermined criterion selected from the group consisting of the number of images N and a score threshold is satisfied. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The principles and operations of the system and method according to the present invention can be better understood with reference to the drawings and the following description, and it should be understood that these drawings are given for illustrative purposes only and are not intended to be restrictive, where:

[0018] Figure 1 shows an in-vivo imaging system according to an embodiment of the present invention;

[0019] Figure 2 schematically illustrates an image scoring scheme according to an embodiment of the present invention;

[0020] Figure 3 schematically illustrates an image grouping scheme according to an embodiment of the present invention;

[0021] Figure 4A and 4B shows an example scoring graph according to an embodiment of the present invention;

[0022] Figure 4C andFigure 4D Separate illustrations of the manipulation of scores in score graphics according to embodiments of the present invention Figure 4A and Figure 4B ; the manipulation of scores in score graphics according to embodiments of the present invention

[0023] Figure 5 Illustrating a score modification process according to an exemplary embodiment of the present invention

[0024] Figure 6A Illustrating a method for selecting images originating from one imager according to an exemplary embodiment

[0025] Figure 6B Illustrating a method for selecting images originating from one imager according to another exemplary embodiment

[0026] Figure 6C Illustrating the calculation of the distance between images according to an exemplary embodiment

[0027] Figure 7 Illustrating a method for selecting images originating from two imagers according to another exemplary embodiment

[0028] Figure 8 Schematically illustrating an exemplary implementation of an image selection method according to an exemplary embodiment

[0029] Figure 9 Illustrating a method for selecting images originating from one imager according to another exemplary embodiment of the present invention; and

[0030] Figure 10 Illustrating Figure 9 the score modification process according to the embodiment illustrated in

[0031] It should be understood that, for simplicity and clarity of illustration, the elements shown in the figures are not necessarily drawn to scale. For example, for clarity, the dimensions and / or aspect ratios of some of the elements may be exaggerated relative to other elements. Additionally, where considered appropriate, the reference numerals of the figure elements may be repeated between the figures to represent corresponding, equivalent, or similar elements. Detailed Description

[0032] In the following description, various aspects of the present invention will be described. For purposes of explanation, specific configurations and details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced without the specific details presented herein. Additionally, well-known features may be omitted or simplified so as not to obscure the present invention.

[0033] Discussions using terms such as "process," "compute," "calculate," "determine," "establish," "analyze," "examine," etc. can refer to operations and / or processes of a computer, computing platform, computing system, or other electronic computing device, which manipulates and / or transforms data representing physical (e.g., electronic) quantities within registers and / or memories of the computer into other data similarly representing physical quantities within registers and / or memories of the computer or other information non-transitory storage media storing instructions for performing the operations and / or processes.

[0034] Unless explicitly stated, the operations described herein are not subject to a particular order or sequence. For example, some method steps can occur or be performed at the same point in time or simultaneously, and some method steps can occur or be performed in the reverse order.

[0035] Embodiments of the systems and methods of the present invention can be used in conjunction with an imaging system or device capable of obtaining images of an in-vivo object. Any imaging device or system that can be incorporated into an in-vivo device can be used, and embodiments of the present invention are not limited by the type, nature, or other aspects of the imaging system, device, or unit used. Some embodiments of the present invention relate to ingestible in-vivo devices, such as self-ingestible capsules. However, the in-vivo device does not need to be ingestible or autonomous and can have other shapes or configurations.

[0036] The in-vivo imaging device can capture a series of images as it traverses the gastrointestinal system and typically transmits images by emitting one image frame at a time. The images can later be compiled at or by a receiver to produce a displayable video clip or image stream or series of images. The images transmitted from the in-vivo imaging device to the receiver can be color images. In one example, each image can include 256 rows, with 256 pixels per row, where each pixel can represent or be associated with binary data (e.g., bytes) that can represent, for example, the color and brightness of the pixel.

[0037] Detecting an abnormality in tissue imaged by the imaging device can include measuring one or more pixel parameters (e.g., color parameters) of one or more pixels in the image. The detected abnormality for display can be any abnormality, such as a polyp, lesion, tumor, ulcer, blood spot, vascular dysplasia, cyst, embryonal cell tumor, hamartoma, tissue malformation, nodule, to name just a few.

[0038] Embodiments of the present invention relate to methods and processes for selecting, for example, for further processing and / or for analysis and / or display, a relatively small set of N images (e.g., 100 images), which is referred to herein as an 'image budget' or limit, such that each image in the selected group of images contains or represents at least one pathology, and the probability that the selected group of images will contain multiple images of the same pathology will be minimized so as not to waste the image budget on similar or redundant images. In some embodiments of the present invention, the selected images may be displayed to the user 'as is' (e.g., without further processing). In some other embodiments of the present invention, the selected images may be further processed, where the processing or further processing of the selected images may include analyzing the selected images, for example, to further investigate the pathology using more stable software tools, for example, to confirm or refute the initial determination that the image contains a pathology or to assess the severity of the pathology. The selected images may be used in combination with other unselected images belonging to the same subgroup as the selected images in the process of preparing a three-dimensional rendering of the pathology. The further processing of the selected images may include emphasizing a specific region in the image in a manner suitable for the pathology identified in the image, for example, to emphasize the pathology so that the viewer can immediately see it without having to waste time searching anywhere in the image.

[0039] Embodiments of the method and process for selecting images may include a stage of applying one or more pathology detectors (D1, D2,...) to the images to score them according to the probability that the images contain one or more pathologies, and another stage that involves using an 'image budget' to iteratively or repeatedly select images for display, one image at a time, or a pathology-related score threshold, or a combination of the image budget and a pathology-related scoring threshold, or any other suitable criterion. The use of a pathology detector may be optional because the pathology scoring may be calculated by some other system (e.g., by a remote computer), and the subject matter of the systems and methods of the present invention may receive the scoring as input from other systems and perform the image selection method based on the received scoring. Each time an image is selected for, e.g., display or further processing, the image score is modified ('suppressed', value decreased) in a manner that increases the probability that other images that also contain clinically important information but belong to other pathologies or other locations or parts in other body cavities will also be selected for display. 'Suppressing the score' means decreasing the value of the score by an amount calculated by using a score modification function.

[0040] Some embodiments disclosed in the present invention are beneficial for rectal imaging because, for example, they present (e.g., to a doctor) a small number of images that contain only the clinically most important images. Additionally, some embodiments disclosed herein enable a user (e.g., a doctor) to select any number of important images (e.g., 20 images or 50 images, etc.) that the user may want to review, or to apply an image selection process to a specific type of pathology (e.g., polyps) or to a specific location (e.g., the colon) in the gastrointestinal tract, while limiting the total number of images containing all pathologies at the specific location with the specific pathology to a'reasonable' or 'convenient' number. That is, rather than having to review thousands of images indiscriminately, a user can use the embodiments disclosed herein to review as many images as desired, each image showing a pathology, and to direct the image selection process to any desired pathology and / or any desired segment or portion of the gastrointestinal tract. Thus, the embodiments disclosed herein can, for example, provide the user (e.g., a doctor) with flexibility in terms of: (1) setting the number of images to be displayed or further processed to a desired value, (2) selecting the type of one or more pathologies that the selected images will show, and (3) selecting one or more locations in the gastrointestinal tract from which to select images for display or for further processing.

[0041] As used herein, the term "object" can, in some embodiments, refer to the entire image, e.g., it can refer to image properties or image features that characterize the image as a whole. In other embodiments, the term "object" can, for example, refer to an imaging object (i.e., the object whose image appears in the image). The imaging object can be, but is not limited to, for example, a polyp, a lesion, a blood spot, a diverticulum, an ulcer, a pathology, etc.

[0042] The imaging object (e.g., a polyp) contained in a selected image can be small, medium, or large relative to the size of the image, and it can be located at any position in the image and have any rotation angle relative to the image. If the imaging object ('original' object) contained in the selected image is also contained in (also shown in) a neighboring image, then the processor can determine (e.g., by using any suitable object characterization algorithm) that the imaging object in the neighboring image and the original object in the selected image are the same or equivalent imaging objects, even if the size and / or position and / or rotation angle of the imaging object contained in the neighboring image are respectively different from the size and / or position and / or rotation angle of the original object. That is, even if the imaging object in one image is translated or rotated or has a different size relative to the imaging object in another image, they can be considered similar by or due to their imaging objects.

[0043] As used herein, the term "image subgroup" or simply "subgroup" refers in some embodiments to a series of adjacent images whose image characteristics are similar to a selected image with which they are associated and form a group, and thus these images can be similar to each other by "similarity". In other embodiments, the term "subgroup" refers to a series of adjacent images that form a group because they contain the same imaging object (or simply the same 'object'). Thus, after an image is selected (using any of the image selection methods described herein), according to some embodiments, an image subgroup can be defined relative to the selected image by searching for and finding adjacent or neighboring images that are similar to the selected image (e.g., that have similar image characteristics to the selected image) or, in other embodiments, by searching for and finding adjacent or neighboring images of the same (imaging) object as the (imaging) object contained in the selected image.

[0044] That is, if the selected image contains an object (e.g., a polyp), then this object is 'tracked' backward in the images prior to the selected image and 'tracked' forward in the images after the selected image. The same specific object (e.g., a specific polyp) found in the selected image can be found in one or more previous images and / or in one or more subsequent images, or it can occur only in (be contained in) the selected image. (An image subgroup can contain only one image; e.g., the selected image.)

[0045] Figure 1 A block diagram of an in - vivo imaging system according to an embodiment of the present invention is shown. The system can include an in - vivo imaging device (e.g., an imaging capsule) 40. Device 40 can be implemented using a swallowable capsule, but other types of devices or suitable implementations can be used. Device 40 can include one or more imagers 46 (e.g., two imagers - one on each side of device 40) for capturing images in a body cavity (e.g., in the gastrointestinal tract). Device 40 can also include one or more illumination sources 42 for illuminating the body cavity, e.g., in an individual's gastrointestinal tract, and a transmitter / receiver 41 for transmitting the images (e.g., in the form of data frames) captured in the body by the imagers 46. Device 40 can also include an optical system that includes, for example, lenses to focus the reflected light onto the imagers 46.

[0046] Preferably, the receiver 12 is positioned outside the patient's body at one or more locations to receive images transmitted, for example, by the device 40. The receiver 12 may include an antenna or antenna array that can be releasably attached to the patient. The receiver 12 may include a storage unit 16 (e.g., a computer memory or a hard disk drive) to store all or selected images received by the receiver 12 from the device 40, for example. The receiver 12 may also store metadata associated with the images received from the device 40, such as time information transmitted by the device 40, and indicate the time when the device 40 captured each image and / or the time when each image was received at the receiver 12. (As described below, for example, in connection with Figure 6C , this time information can be used by the processor to determine (e.g., calculate) the distance between images at this instance time as a prerequisite for the modification (suppression) of the image total score.) Preferably, the receiver 12 is small and portable and can be worn on the patient's body during image reception (and recording).

[0047] The in-vivo device system may also include a workstation 13 to which the receiver 12 can transfer images originating from the device 40. The workstation 13 may include or be connected to a display device 18 for displaying, in particular, images originating from the device 40. The workstation 13 can receive image data and optionally other types of data from the receiver 12 and process the image data to automatically display, for example, the images captured by the device 40 (and / or a video clip containing a series of images) on the display device 18. The workstation 13 can enable a user (e.g., a doctor) to select, for example, images for display or enhance the displayed images. The workstation 13 can enable a user to start the image selection process disclosed herein, for example, by clicking a mouse, pressing, or tapping a button, such that a relatively small, predetermined, or configurable number N of images (e.g., N = 100 images) containing the most clinically important or prominent images will be automatically displayed on the doctor's display device 18, rather than expecting the doctor to review hundreds or thousands of images while trying to find the important images alone. The workstation 13 can enable the user to select the value of N. The workstation 13 can enable the user to limit the image selection process to a specific part of the body cavity; that is, only select images captured in a specific part of the body cavity, e.g., in a specific part of the gastrointestinal tract (e.g., in the small intestine). In some embodiments, the number (N) of selected images is not preset in advance but is determined or can be derived from the result of using another criterion. For example, a scoring threshold mechanism can be used for image selection as described, for example, in connection with Figure 7 , which is described below. In some embodiments, the number (N) of selected images can be used in combination with a scoring threshold or mechanism. For example, the scoring threshold or mechanism can be used as the'main' criterion, and the number N can be used as an upper limit on the number of images finally selected.

[0048] The workstation 13 may include a storage device 19 and a data processor 14. The storage device 19 may store q groups of connected images respectively captured by q imagers (46) in a body cavity. The storage device 19 may also store, in association with each image in each group, a total score (GS) indicating the probability that the image contains at least one type of pathology and a distance d indicating the distance between the images in each group.

[0049] The storage device 19 may include an image storage device 20 for storing image data, a score storage device 22 for storing pathology scores and total scores calculated for the images, and a modification function storage device 24 for storing one or more modification functions for modifying the scores. The score storage device 22 may also store modified scores (scores modified by the modification functions). The workstation 13 may include a number m of pathology detectors 15 (e.g., designated as D1, D2, ..., Dm) to respectively detect m different, same or similar pathologies / anomalies (e.g., focal pathologies). The pathology detector Di (i = 1, 2, …, m), which may be or include, for example, a pattern recognition classifier applied to the image data, may output a (pathology) score indicating the probability that the image contains (shows) the corresponding type of pathology. It may not be necessary to have m pathology detectors (15) because the image pathology scores calculated for the images may be calculated elsewhere (e.g., by a remote computer) and transmitted to, for example, the workstation 13. The pathology detector may analyze various parameters and / or features associated with the image (e.g., pixel intensity levels, hue, amplifier gain, exposure time, etc.) and identify representative or characteristic features involved in the pathology type, and a classifier, which may be part of the pathology detector, may use these features to output a value (e.g., a score) indicating the probability that the image contains pathology.

[0050] The image total score may be calculated by using a number m of score values S1, …, Sm respectively output by the m pathology detectors, where the m score values indicate the probability that the image contains m or k (k < m) types of pathology. The pathologies may be polyps, bleeding, diverticula, ulcers, injuries, red pathologies, etc. (Other pathology types may be detected or recognized in the image and similarly treated.) The image may contain more than one pathology.

[0051] The data processor 14 may be configured to implement embodiments of the present invention, for example, by executing software or code stored, for example, in the storage device 19. In some embodiments, the detector 15 or other classifiers, modules, etc. described herein may be or may be executed by the data processor 14 executing this code.

[0052] In some embodiments, data processor 14 may apply different modification functions to pathology scores associated with images captured in different parts of a body cavity. For example, modification or suppression of a pathology score associated with an image captured at a first location (e.g., small intestine) in a body cavity may be performed by using a first modification function, while modification or suppression of a pathology score associated with an image captured at a second location (e.g., colon) in the body cavity may be performed by using a second modification function, and so on. The data processor 14 may enable a user (e.g., a doctor) to select a modification function based on the part of the body cavity from which the image is taken, and this may be beneficial such that the user can 'shift the focus' to (i.e., bias the image selection process towards) a particular part of the body cavity that the user is more interested in. Shifting the focus to (i.e., biasing the image selection process towards) a particular part or region of the body cavity may be implemented, for example, by substantially reducing the value of the pathology score associated with the luminal part while strictly reducing the value of the pathology score associated with other parts of the lumen.

[0053] The data processor 14 may process image data stored in the image storage device 20 and, in some embodiments, use m detectors to calculate various types of pathology scores for each image. The processor 14 may store the various scores in the score storage device 22 and update the score content of the score storage device 22 from time to time. Updating the scores in the score storage device 22 by the processor 14 may include, for example, adding new scores, zeroing scores, scaling scores, and updating scores by using modification functions stored in the modification function storage device 24. The processor 14 may use a score calculation and modification (suppression) process to select, for example, a predetermined number N of the most clinically important images captured by the in-vivo device 40 for display. The processor 14 may select one such image iteratively or repeatedly at a time, and whenever the processor 14 selects an image, the processor 14 may, for example, store the selected image in the image storage device 20 or refer to reference information (e.g., a pointer) of the selected image. The manner in which the processor 14 calculates and modifies or suppresses pathology scores and uses the pathology scores to select N clinically most important images is described and illustrated below.

[0054] As an example, the data processor 14 may be particularly configured to perform the following operations as described below for each of q groups of contiguous images:

[0055] Partition or separate the group of images or a part thereof into image subgroups, each image subgroup including a base image and images similar to the base image, and

[0056] Identify a set of maximum total scores (MGS) Set(i) (i = 1, 2, 3, …), where the set of maximum total scores Set(i) includes the maximum total scores (MGS) of each image subgroup in the image group; and select an image for processing by performing the following operations: (i) identify the maximum MGS (MGS|max) in each set of MGS S(i); (ii) identify the maximum MGS|max among all the maximum MGSs, and select the image associated with the maximum MGS|max; (iii) invalidate the maximum MGS|max associated with the selected image; and (iv) modify the set of MGS Set(i) associated with the selected image based on the distance d between the image associated with or having an MGS in the specific set of MGS Set(i) and each selected image. The data processor 14 can repeat steps (i) to (iv) until a predetermined number N of images have been selected for display or further processing or until the value of the identified (modified) maximum MGS (MGS|max) is below the score threshold Sth.

[0057] The data processor 14 can be or include any standard data processor, such as a microprocessor, a multiprocessor, an accelerator board, or any other serial or parallel high-performance data processor. The display device 18 can be a computer screen, a conventional video display, or any other device capable of providing images and / or other data.

[0058] Preferably, the imager 46 can be or include a CMOS camera or other device, such as a CCD camera. The illumination source 42 can include, for example, one or more light emitting diodes (LEDs), or another suitable light source.

[0059] During operation, the imager 46 captures an image and sends data representing the image to the transmitter 41, which uses, for example, electromagnetic radio waves to transmit the image to the receiver 12. The receiver 12 transfers the image data to the workstation 13 for storage, processing, and display. The in vivo imager can capture a series of static images as it traverses the gastrointestinal tract. The images can later be presented by the workstation 13 as, for example, an image stream or a video clip of the traversal of the gastrointestinal tract. The in vivo imaging system can collect a large amount of data because the device 40 may take several hours to traverse the gastrointestinal tract and can record images at a rate of, for example, two images per second or twenty-four images per second, resulting in a record of thousands of images. The image capture and transmission rate (or frame capture rate or 'frames per second' rate) can vary.

[0060] Preferably, the image data recorded and transmitted by the device 40 is digital color image data, but other image formats can be used. As an example, according to a known method, each image frame contains 256 rows, each row contains 256 pixels, and each pixel contains a digital byte whose value represents, for example, color and brightness or is represented by the digital byte.

[0061] Figure 2 Schematically illustrates an image scoring scheme according to an embodiment of the present invention. Associated with Figure 1 is described Figure 2 . Assume that the in-vivo device 40 includes two imagers ("Imager-1" and "Imager-2"), and each imager captures a number k of images by the time the device 40 is excreted by the patient. (The embodiments described herein can equally apply to any sequence of images, regardless of the number of imagers involved in their capture. For example, the embodiments described herein can equally apply to images captured by an in-vivo device including one imager. Sequence) Also assume that a number m of pathology detectors are used to analyze each image, where each detector is configured to detect different or similar pathologies in the image by calculating a score Si for each image, such that the score calculated for a particular image indicates the probability that the gastrointestinal site captured by the particular image contains a particular pathology. As an example, the first pathology detector can be configured to detect polyps in the colon; the second pathology detector can be configured to detect bleeding in the gastrointestinal tract; the third pathology detector can be configured to detect ulcers in the gastrointestinal tract; the fourth pathology detector can be configured to detect diverticula in the gastrointestinal tract, etc. Alternative or additional pathology detectors can be used. A particular pathology can be detected by using more than one detector. For example, one polyp detector can detect polyps by detecting their contours, and another polyp detector can detect polyps by detecting the color pattern characterizing the polyps, etc.

[0062] Referring to Figure 2 , Imager-1 captures a set of k consecutive images designated as Image-1 / 1 (image #1 of Imager-1), Image-2 / 1 (image #2 of Imager-1), …, Image-k / 1 (image #k of Imager-1). Similarly, Imager-2 captures a set of k consecutive images designated as Image-1 / 2 (image #1 of Imager-2), Image-2 / 2 (image #2 of Imager-2), …, Image-k / 2 (image #k of Imager-2). The consecutive images can be, for example, images that are immediately before or after each other in an image stream.

[0063] Apply m detectors to Image-1 / 1 to produce m scores (one score per detector) designated as S1-1 / 1 (the score of detector 1 calculated for image 1 of Imager-1), S2-1 / 1 (the score of detector 2 calculated for image 1 of Imager-1), …, Sm-1 / 1 (the score of detector m calculated for image 1 of Imager-1). Similarly, apply m detectors to Image-2 / 1 to produce m scores (one score per detector) designated as S1-2 / 1 (the score of detector 1 calculated for image 1 of Imager-1), S2-2 / 1 (the score of detector 2 calculated for image 1 of Imager-1), …, Sm-2 / 1 (the score of detector m calculated for image 1 of Imager-1), and so on.

[0064] Similarly, apply m detectors to Image-1 / 2 to produce m scores (one score per detector) designated as S1-1 / 2 (the score of detector 1 calculated for image 1 of Imager-2), S2-1 / 2 (the score of detector 2 calculated for image 1 of Imager-2), …, Sm-1 / 2 (the score of detector m calculated for image 1 of Imager-2). Similarly, apply m detectors to Image-2 / 2 to produce m scores (one score per detector) designated as S1-2 / 2 (the score of detector 1 calculated for image 2 of Imager-2), S2-2 / 2 (the score of detector 2 calculated for image 2 of Imager-2), …, Sm-2 / 2 (the score of detector m calculated for image 2 of Imager-2). Thus, each image can be associated with m scores that respectively indicate the probability that the image contains various (imaged) pathologies.

[0065] Figure 3 An example of image grouping according to an embodiment of the present invention is schematically illustrated. For simplicity, Figure 3 a group of connected images 300 captured by one imager is shown.

[0066] An image subgroup (SGi) contains temporally consecutive images that are similar or analogous to each other (e.g., based on visual features; e.g., configurational features or computed features; e.g., ratios between image parameters, e.g., light intensity, color grade, etc.), and thus, they may depict or contain the same pathology (e.g., polyp, hemorrhage, ulcer, lesion, diverticulum, etc.). However, because the image 'budget' (e.g., limit or maximum number N) of the images to be selected for (e.g.) display is extremely small (compared to the total number of thousands or tens of thousands of captured images), images that are similar to each other within the same image subgroup can be subtracted or removed (e.g., subtracted or removed from the image selection process), so that only the representative images representing the corresponding image subgroups from each image subgroup remain.

[0067] To begin grouping images (to begin segmenting the image group) into subgroups, the similarity level between the first image designated as 'Image-1' and the temporally adjacent image designated as 'Image-2' is examined (310). In Figure 3 the instance, it is assumed that Image-1 is similar to Image-2. Similarly, the similarity between Image-1 and Image-3 is examined (320). In Figure 3 the instance, it is also assumed that Image-1 is similar to Image-3. Similarly, the similarity between Image-1 and Image-4 is examined. In Figure 3 the instance, it is also assumed that Image-1 is similar to Image-4. Similarly, the similarity between Image-1 and Image-5 is examined. In Figure 3 the instance, it is assumed that Image-1 is dissimilar to Image-5. Thus, continuing the instance, the first image subgroup (SG1) contains four images: Image-1 (image 330, which can be regarded as the base image in image subgroup SG1), Image-2, Image-3, and Image-4. The same image grouping process is continued for the next image subgroup (SG2) by using the image Image-5 (image 340) as the base image in the new subgroup. Image 340 is the first image that does not belong to image subgroup SG1 because it is dissimilar 330 to the base image in image subgroup SG1. Thus, image 340 serves as the base image in image subgroup SG2, and as an example, subgroup SG2 contains three images: Image-5, Image-6, and Image-7. Image subgroup SG3 contains the image Image-8 as the base image (350), and Figure 3Additional images not shown. Additional subgroups of images can be identified in a similar manner. The similarity between images can be determined (e.g., calculated) based on any suitable criterion or by using other image grouping methods. For example, the similarity between images can be determined based on a comparison of pre-features or parameters (e.g., light intensity, color, etc.), or the similarity between images can be determined in reverse order or “backward”; that is, from a particular image to a previously captured image. Suitable clustering methods can be used to group the images.

[0068] Other methods can be used to segment the image group. For example, the image group can be segmented into subgroups of images such that each subgroup of images has the same number of images. In some embodiments, the number of images in a subgroup of images or included in a subgroup of images can depend on the part of the body cavity or organ where the in-vivo device captures the images (e.g., in the small intestine or in the colon). For this purpose, the position of the in-vivo device in the body cavity can be determined or detected from, for example, one or more images or by using an external positioning system. Subsequently or simultaneously, the position of the in-vivo device can be transmitted to, for example, workstation 13 ( Figure 1 ), and stored, for example, in storage device 19 in association with the images captured at this position. As an example, data processor 14 can segment or partition the images captured in the small intestine into small subgroups of images and the images captured in the colon into large subgroups of images, or vice versa. (The number of images in a'small subgroup of images' contains a smaller number of images compared to a 'large subgroup of images'.)

[0069] Figure 4A and 4B show an example scoring graph according to an embodiment of the present invention. Figure 4A and Figure 4B associated with Figure 3 are described, and which respectively show example scores calculated for the subgroup of images SG1 by referring to two pathology detectors D1 and D2 for each image in the subgroup SG1. (As described herein, for example, in connection with Figure 1 and Figure 2 , a number m of detectors (D1, D2,..., Dm) can be applied to each image in the image group in order to calculate a number m of scores (S1, S2,..., Sm) respectively indicating the probabilities that the image shows or contains m pathologies of types respectively detectable by the m detectors for the image.

[0070] Figure 4A shows an example score (S1) obtained for the subgroup of images SG1 by using the pathology detector D1. For example, the subgroup of images SG1 contains four images, which are designated as '1', '2', '3', and '4' on the 'image number' axis in Figure 4A . (In Figure 4BThe same number of images is also used. ) For example, assume that the scores calculated from the four images of detector D1 are: for image number 1, S1 = 90, for image number 2, S1 = 105, for image number 3, S1 = 150, and for image number 4, S1 = 124. Figure 4B Shows the example scores (S2) obtained for subgroup SG1 by applying pathology detector D2. For example, assume that the scores calculated from the four images of detector D1 are: for image number 1, S2 = 95, for image number 2, S2 = 125, for image number 3, S2 = 80, and for image number 4, S2 = 75.

[0071] After calculating various scores (e.g., one score per detector per image) for each image in each image subgroup SG1, SG2, SG3, etc., the local maximum score values are identified for each score type and for each image subgroup. As an example, the score value S1 = 150 shown at 410 ( Figure 4A ) is the local maximum of the scores calculated by pathology detector D1 for image subgroup SG1, and the score value S2 = 125 shown at 420 ( Figure 4A ) is the local maximum of the scores calculated by pathology detector D2 for image subgroup SG1. An image subgroup can contain more than one local maximum score value for a particular pathology detector, and different pathology detectors can produce different numbers of local maximum score values for the same image subgroup.

[0072] The local maximum score values identified in each image subgroup can be further processed during the image selection process, while other scores (scores that are 'non - prominent' scores) can be excluded from this process. The exclusion of the scores can be implemented, for example, by setting the 'non - prominent' scores (scores that are not local maximum scores) to zero or by reducing their values to any value below the lowest pre - suppression score value, in order to ensure the exclusion of the scores from the selection process. Figures 4A to 4B The zeroed - out score values S1 and S2 in Figures 4C to 4D are shown as S1' and S2' in Figure 4C . For example, the zeroed - out version of the score value S1 related to image number 1 is shown as S1’ (at 430) in Figures 4A to 4D . (In the example shown in Figure 4C , the score values S1' related to images '1', '2', and '4' - see Figure 4D-associated score value S2'). Non-prominent scores can be excluded from the image selection process because they may represent images that are potentially less clinically important than images with local maximum score values (based on the pathology associated with the type of these scores). In some embodiments, an image can be selected from a subgroup of images even if other images in the subgroup of images may be as important as the selected image in terms of appearance. After non-prominent score values associated with all score types and images are set to zero, a total image score can be calculated for an image by using the local maximum score values. (Setting the score values to zero is inconsequential for determining the total score of an image.)

[0073] As an example, Figures 4A to 4B each of which shows a subgroup of images that contains only a few images and only one local maximum score value for each type of score. However, this is only an example - a subgroup of images can contain many images (the number depending on the similarity between the images) and can additionally contain several local maximum score values. After all local maximum score values are identified in each subgroup of images, the non-maximum score values (other score values) in each subgroup of images are set to zero, as illustrated by Figures 4C to 4D for example, in order to exclude from the image selection process the images that produced the score values (associated with the score values). As described herein, the images in a subgroup of images are characterized by a high degree of similarity (e.g., above a predetermined similarity threshold), and thus, only the more prominent images can be selected for display, while many other unimportant or redundant images in each subgroup of images can be removed from the image selection process without risking loss of clinically important images. Since all original images are saved (e.g., in Figure 1 the image storage device 20), if a particular image is ultimately selected (and e.g., displayed to a user such as), then the system (e.g., Figure 1 the workstation 13) can enable the user to also display the images before or after the image (e.g., the images whose scores have been set to zero). Embodiments of the present invention can enable the user to use the selected image (e.g., when the selected image is being displayed) to, for example, fully or partially display the associated subgroup of images by clicking or tapping on the selected image when the selected image is being displayed.

[0074] Calculate the total score (GS) of the image

[0075] If all the scores for a particular image are zero, which means that no pathology in the image is detected by any of the pathology detectors D1, D2, …, Dm, then the image is excluded from the image selection process categorically. However, since these detectors (e.g., classifiers) calculate the probability (of the presence of pathology), it is possible that the detectors will output some low non - zero score values in such situations. In other words, the pathology detectors can output low non - zero score values for an image even if the image does not contain pathology. Additionally, an image can have more than one low non - zero score value (the score value for each pathology detector). However, as described herein, if a particular pathology score output by a corresponding pathology detector for a particular image is not the local maximum score value within the image subgroup containing the particular image, then the particular pathology score is set to zero; for example, set to zero. In some embodiments, setting all score values that are not local maximum score values to zero ensures that only local maximum score values are considered in the image selection process described herein because they are more likely to indicate pathology.

[0076] If an image has one local maximum score value, then the local maximum score value can be further used as the image total score (GS) associated with the image. However, as described herein, an image can have or be associated with more than one local maximum score value. (For more than one pathology detector, some images may have at least one local maximum score value.) Thus, an overall score GS can be derived (e.g., based on a calculation) for each image from all the local maximum score values calculated for or associated with the image. For example, Equation (1) or Equation (2) can be used to calculate the image total (pathology) score GS:

[0077] GS = f(S1, S2,…, Sm) (1)

[0078] GS = Max{f(S1), f (S2), …, f (Sm)} (2)

[0079] where Si (i = 1, 2, …, m) is the score value output from the pathology detector number i, and the pathology score values S1, S2, …, Sm are pathology scores respectively related to m types of pathology. Each of the pathology score values S1, S2, …, Sm can have a value that is zero (after undergoing the zero - setting process, or in the absence of such a process) or a local maximum score value. For example, if m = 3 (if three pathology detectors are used to analyze an image), then the image total score GS can be calculated using Equation (3) for example:

[0080] GS ＝ Max{W1xS1, W2xS2, W3xS3} (3)

[0081] Where S1, S2, and S3 are pathology scores associated with three pathology detectors respectively, and W1, W2, and W3 are 'weights' associated with three types of pathology respectively.

[0082] An image can have more than one non-zero pathology score, and the various scores are associated with more than one type of pathology. In this situation, the image can contain a 'dominant' pathology; that is, a pathology that is visually or otherwise more prominent than other pathologies contained in the image. Thus, given a strict image budget, only the clinically most significant images are selected for display, which are typically the images with the highest pathology scores. (The selection of images can be performed regardless of which type of pathology in the image has the highest pathology score.) Thus, each image is given or associated with a total pathology score (GS) that represents or is more 'biased' towards the more visually or otherwise prominent pathologies in order to increase the probability that the image will be selected for display. (Equations (1) to (3) can be modified to achieve the goal.) Whether a particular image is selected for display may not be important because it has a higher pathology score for one pathology compared to the pathology score of some other pathology, because once selected for display, the image will show every pathology contained in the image, rather than only showing the pathology with the highest pathology score.

[0083] In some embodiments, knowing the type of pathology score that caused the image selection (knowing the pathology type) can be used to enhance the displayed image (e.g., highlighting the pathology). For example, if the image is selected for display because it contains a focal pathology (e.g., a polyp), then enhancing the image display can include, for example, identifying the center of the polyp and overlaying a visually prominent cross ('x') and / or a visually prominent circle (e.g., a red circle) over the center of the polyp in order to draw the viewer's (e.g., a physician's) attention to the polyp, rather than having the viewer spend time searching for the pathology in the image. In another example, if the image is selected for display because it contains a segmented or distributed pathology, then the enhanced image can include overlaying visually prominent boundary lines (e.g., circles, boxes, etc.) over the segmented or distributed pathology and / or highlighting the entire image without focusing on the exact location of the pathology in the image in order to draw the viewer's (e.g., a physician's) attention to the polyp, rather than having the viewer spend time searching for the pathology in the image.

[0084] Figure 5 Steps A to 5E illustrate the image selection process according to an example embodiment. Figure 5Step A shows an example image group 500 that includes twenty-eight connected images captured by an in-vivo device that includes an imager and its corresponding total scores. (The twenty-eight images are shown as being temporally ordered according to their capture times, with image number '1' being captured first and image number '28' being captured last.) In practice, the number of images in an image group can be very large (e.g., tens of thousands), however, a small number is used for illustrative purposes only Figure 5 The images shown in Step A. Figure 5 Step A also shows the image group 500 being segmented into three image subgroups (SGs) designated as SG1, SG2, and SG3. The images in the image group can be captured at a rate as low as, for example, two images per second or at a much higher rate such as 74 images per second. The image selection method disclosed herein is not limited to any particular image capture rate; the image selection method disclosed herein is applicable to any image capture rate.

[0085] Assume that the images in the image group 500 are analyzed by two pathology detectors (e.g., a polyp detector and a bleeding detector), and any one of equations (1) to (3) is used to calculate the total (pathology) score GS for each image based on the two cut scores output by the two pathology detectors. It is also assumed that a filtering detector (e.g., a content detector, a bubble detector, a tissue coverage detector, etc.) is used to detect images that show contents and / or bubbles, etc. at some sites of, for example, the gastrointestinal tract, and it is assumed that a score value of zero is assigned to such images (see Figure 5 image numbers 8, 12, 19, and 23 in Step A) so as to exclude such score values from the image selection process.

[0086] After processing the image group 500 in the manner described herein, e.g., in combination with Figure 3 and Figures 4A to 4D , in this example, the images are segmented or partitioned or separated into three image subgroups SG1, SG2, and SG3 based on the similarity between the images. (For simplicity, the image group 500 includes twenty-eight images and three image subgroups, and the image budget (the number of images to be selected for further analysis and / or display, for example) is three.)

[0087] After the images are segmented or partitioned or separated into image subgroups, the local maximum total score (MGS) value in each image subgroup can be identified. (MGS is in Figure 5In step A, it is shown as circled. For example, in image subgroup SG1, there are two MGS values (MGS values 80 and 90 shown at 530 and 540 respectively, which are associated with images 4 and 7 respectively), in image subgroup SG2, there are three MGS values (MGS values 71, 79 and 82 shown at 550, 560 and 570 respectively, which are associated with images 16, 18 and 20 respectively), and in image subgroup SG3, there is one MGS value (MGS value 78 shown at 580, which is associated with image 25).

[0088] In some embodiments, each MGS in each image subgroup can be used (participated in) in the image selection process, and at the same time, other scoring values can be 'inhibited' by, for example, setting its value to zero; for example, setting it to zero. In Figure 5 step B, each MGS in each of image subgroups SG1 to SG3 is used in the image selection process, and Figure 5 the values of other (non-local maximum) scores in step A are set to zero. In other embodiments, only the MGS with the maximum value among the MGSs in each image subgroup is used in the image selection process, while other scores (other MGSs and non-local maximum score values, etc.) are set to zero.

[0089] Figure 5Step C shows the scoring vector 502 for each MGS that contains each image subgroup: two MGSs associated with image subgroup SG1, three MGSs associated with image subgroup SG2, and one MGS associated with image subgroup SG3. ('Vector' herein refers to an ordered set of values.) The scoring vector may contain only the largest MGS in the corresponding image subgroup, e.g., only MGS = 90 from image subgroup SG1, only MGS = 82 from image subgroup SG2, or only one MGS from image subgroup SG3 (MGS = 78). The scoring vector 502 may also contain, or associate information about the relative position of the images (in image group 500) involved (resulting in) the MGSs stored in the scoring vector 502 with its MGS content. (Each image position may be associated with the relevant MGS.) As an example, two MGS values 90 and 71 (shown at 504 and 506 respectively) are adjacent in the scoring vector 502. However, the distance between MGSs does not necessarily translate into the distance between the images associated with these MGSs. For example, although two MGSs may be adjacent in the scoring vector 502, their associated images may be far apart (e.g., a larger number, e.g., 35 images, may be inserted (e.g., positioned in between in an ordered image stream) between the images associated with these MGSs). Thus, each MGS may be associated with position information that can: (1) indicate the position of the associated image and (2) enable the calculation of a distance d, which can be measured as, for example; time (e.g., measured as the difference between image capture times), intervening images, or any other measure of the distance between any two images. An image subgroup may contain more than one maximum (pathological) score or be associated with or result in such a score, as each pathological detector may produce one maximum score for each image subgroup. In some embodiments, if an image subgroup has two or more maximum scores, then only the highest maximum score (i.e., the maximum score with the highest value among the two or more maximum scores) is used as the image total score (GS) in the image selection process. Thus, only one image can be selected from each image subgroup. For example, (again referring to Figures 4A to 4B ), image subgroup SG1 contains two maximum scores - maximum score 410 and maximum score 420, and since its value (150) is higher than the value of maximum score 420 (~125), only maximum score 410 can be selected for the image selection process.

[0090] In some embodiments, the scoring vector 502 may have a length corresponding to the number of images in the image group (according to the number of vector elements). (The scoring vector 502 may contain the score values for each image in the image group, but for example due to what is described herein and as shown in Figure 5The process of resetting the score values illustrated in step B, or in cases where the images associated with these score values are detected as containing, for example, contents or bubbles or are detected as not showing sufficient tissue, etc., some score values can be local maximum score values while other score values can be zero.)

[0091] In some embodiments, the scoring vector 502 can contain all the MGSs identified in each image subgroup or only the maximum MGS from each image subgroup. Since an MGS can be identified one at a time, the length of the scoring vector 502 can change (increase) by one vector element each time a new MGS is identified in each image subgroup. (Depending on the value of the pathology score calculated by the pathology detector for each image, an image subgroup can contain one MGS or multiple MGSs.)

[0092] Each MGS added to the scoring vector 502 can be associated with position information regarding the position of the associated image relative to other images in the image group or relative to a reference point. The reference point can be, for example, time (e.g., the capture time of a particular image) or a physical marker in or associated with the body cavity from which the image was taken. For example, the position of an image relative to other images can be determined (e.g., calculated) based on the time at which each image was captured or based on the serial number of the images captured in sequence, where the first image captured can be assigned (e.g.) serial number one, the second image captured can be assigned (e.g.) serial number two, etc. (Other image identification schemes can be used.)

[0093] The relative positions of the images in the image group, as described herein, can be used to determine (e.g., calculate) the distance between the images (e.g., based on the time difference, based on the number of intervening images, etc.). As described herein, the distance between the images is used to variably (e.g., in a decreasing manner) modify (suppress) the MGSs in order to reduce the probability that an image close to a selected image will be selected for (e.g.) display, and at the same time slightly increase the probability that a'remote' image (relative to the selected image) will still be selected even if its original score value (MGS) may be relatively low. (The term'remote' is used herein relative to a selected image.)

[0094] After an image is selected (e.g., for analysis and / or for display), the MGS of other images can be variably modified or suppressed such that the farther an image is from the selected image, the more moderately its score value is decreased (suppressed), thus the decreasing nature of the score value modification process with respect to each selected image. Although the distance between an image associated with a particular MGS and some selected images can be relatively long, the distance between an image associated with a particular MGS and another selected image can be short (if more than one image is selected). In this situation, the shorter distance can be used to strictly modify the particular MGS rather than moderately using the longer distance to modify or suppress it, because it may be desirable to reduce the chance of selecting an image associated with the particular MGS (where the shorter distance Dmin indicates its similarity to the closest selected image).

[0095] Figure 5 Steps D to 5F illustrate the effect of the example decreasing modification (suppression) process on the MGS content of the example score vector 502 of Figure 5 Step C. Figure 5 Step D shows the original (unmodified or unsuppressed) set of MGSs used in the selection of the first image. (The first image is selected using the original MGS values.) Figure 5 Step E shows the modified (suppressed) MGSs used in the selection of the second image. (The second image is selected after modifying or suppressing (the original) MGS values with respect to the first selected image.) Figure 5 Step F shows the modified (suppressed) MGSs used in the selection of the third image. (The third image is selected after modifying or suppressing (the original) MGS values with respect to the first and second selected images, using the minimum distance Dmin identified between the third image and the closest selected image.)

[0096] Reference Figure 5 Step D, the (original) score value 90 is currently the maximum MGS in the score vector 502 (e.g., MGS = 90 = MGS|max). Thus, the image that produces this score value (the image associated with this MGS) is selected. After the image is selected, this score value is set to zero to ensure that the selected image will not be selected again. Figure 5 Step E shows the maximum score value 90 set to zero at 510. Figure 5 Step E also shows the other MGSs in the score vector 502 that are Figure 5 the modified versions of the original MGSs shown in Step D.

[0097] As described herein, the score value modification (suppression) process variably modifies the score values (MGSs) of images in a decreasing manner. As an example, Figure 5 the score value 80 in vector cell number 1 in Step D is adjacent to the score value 90 in cell number 2, but Figure 5The score value 78 in vector cell number 6 in step D is the farthest from the score value 90 in cell number 2. Assuming that MGS = 80 relates to an image that is much closer to the selected image (e.g., closer to the image related to MGS = 90) than the image related to MGS = 78, vector cell number 1( Figure 5 In step D), MGS = 80 strictly decreases by a relatively large number (the decreased value is shown in Figure 5 step E, cell 1), for example, from 80 to 65. However, MGS = 78 related to the remote image (in vector cell number 6( Figure 5 In step D)) decreases loosely (moderately or slightly) by a relatively large number (the decreased value is shown in Figure 5 step E, cell 6), for example, from 78 to 76. The same principle can be used to modify (decrease) the rest of the MGS.

[0098] After performing the first MGS modification process, the next image (the second image in this case) can be selected based on the modified score values. Refer to Figure 5 step E. Among the modified score values (among the modified MGS), the score value '76' is the largest (MGS = 76 = MGS|max), so the image that produces this score value can be selected, and after selecting the second image, the score value related to it (score value 76) can be set to zero or otherwise manipulated to ensure that the second image will not be selected again. Figure 5 Step F shows the maximum score value 76 set to zero at 520. Figure 5 Step F also shows the other MGS in the scoring vector 502 that is Figure 5 the modified version of the original MGS shown in step D.

[0099] Each original MGS in step D can be modified differently from one image selection to another image selection because the original MGS of the unselected image modified based on Dmin and the distance between the unselected image and the selected image can change as more and more images are selected. Dmin is the distance between its related image and the closest selected image. After performing the second MGS modification process, the next image (the third image at this stage) can be selected based on the (new) modified score values. Refer to Figure 5 step F. Among the modified score values, the score value 68 (cell number 4, Figure 5 Figure 5 Figure 5Step F) is the maximum (MGS = 68 = MGS|max), so the image that yields this score value can be selected. If the image selection budget contains only three images, then the image selection process can be terminated at this stage. However, if the image budget is large and the scoring vector contains more (e.g., hundreds or thousands) of MGSs associated with multiple images respectively, then the image selection process can continue in a similar manner until the image budget is exhausted.

[0100] The image selection method disclosed herein (e.g., in Figures 6A to 6B and in Figure 7 ) guarantees in some embodiments that: (1) the total number of selected images does not exceed a predetermined number of images (e.g., the image budget), (2) the selected images contain the clinically most important pathologies, and (3) the method of image selection disclosed herein can be 'fine-tuned' to select only (3.1) images captured from a specific segment or part of the gastrointestinal system (e.g., for analysis or display) and / or (3.2) images containing a specific type of pathology (e.g., polyps).

[0101] A specific segment or part of the gastrointestinal system can be relatively long (e.g., the small intestine) or relatively short (e.g., the colon). The image selection process can be fine-tuned to select only images taken from a specific segment of the small intestine or from a specific segment of the colon or from any other gastrointestinal segment of interest. For example, the fine-tuning of the image selection process to selectively select images taken from a specific gastrointestinal segment can be performed by setting the foreign score values of all images taken outside the gastrointestinal segment of interest to zero. (An example of this selective image selection process is shown in Figure 8 and will be described below.)

[0102] Figure 6A A method of selecting images for display (or for any other purpose) according to an example embodiment of the present invention is shown. In connection with Figure 1 and Figure 3 is described Figure 6A . Figure 6A The image selection method of Figure 2 is applicable to a group of images obtained in vivo from an imager of an in vivo device (e.g., Imager-1, Figure 6A The embodiment shown in

[0103] At step 610, the data processor 14 may receive a group of connected images (e.g., group 300) continuously captured in vivo by the in vivo device 40 within the body cavity. The data processor 14 may calculate a total score (GS) for each image in the group of images at step 610 or in a separate step, which may indicate the probability that the image contains at least one type of pathology. The data processor 14 may receive the GS value for each image having the image, or it may calculate the GS value 'in place' based on the scores that the pathology detectors D1, …, Dm (pathology detector 15, Figure 1 ) may output for the image. Alternatively, the data processor 14 may not require the images, but instead may only obtain the GS associated with the images, and additionally, the data processor 14 may obtain information about the relative position of each image in the group of images.

[0104] At step 620, the data processor 14 may divide the group of images into image subgroups, such that each image subgroup contains a base image (e.g., Figure 3 base images 330, 340, and 350 therein) and subsequent images that are similar or analogous to the base image. (For example, the division of the group of images may be performed in a manner combined with Figure 3 as described.)

[0105] Since each image subgroup may contain a number of images, each of which is associated with a total score, the data processor 14 identifies the total score having the highest value in each image subgroup at step 630. (The total score having the highest value in the image subgroup is referred to herein as the maximum total score (MGS) of the image subgroup.)

[0106] At step 640, the data processor 14 identifies the MGS| max , which is the MGS having the highest value among all the MGSs of the image subgroups, and at step 650, the data processor 14 may select the image associated with the MGS| max for further processing (e.g., analysis) and / or for display, and assigns a value to a loop counter n (at step 660).

[0107] At step 670, the data processor 14 checks whether the number of selected images is N, where N is the number of images that the predetermined image budget permits to be selected. If the image budget has been exhausted, i.e., if n≥N (this condition is shown as "Yes" at step 670), then the data processor 14 may terminate the image selection process and, for example, only analyze the selected images or, for example, display the selected images in a separate display window. However, if the image budget has not been exhausted, that is, if n<N (this condition is shown as "No" at step 670), then the data processor 14 may, at step 680, invalidate the MGS| max value of the selected images (e.g., associated with them or calculated for them) so as to ensure that currently selected images will not be selected again, thus giving other images a chance to be selected. (In the context of the present invention, 'invalidate' means setting the value of the MGS| max to a value that reduces the probability of the associated image being selected again to zero; for example, setting it to zero or to a negative value.) It should be noted that a scoring threshold may be used instead of or in combination with the predetermined number N of images, as described, for example, in connection with Figure 6B step 616, which is described below.

[0108] At step 682, the data processor 14 may modify the (original) MGS of the image subgroup such that the value of each particular (original) MGS changes based on the distance that exists between the image associated with the particular original MGS and the closest selected image (e.g., by the data processor 14 reducing its value). For example, the closer a candidate image is to the closest selected image, the more the value of the MGS associated with the candidate image is reduced. (As used herein, a 'candidate image' is any image in the image subgroup whose total score has been identified, for example, by the data processor 14 as an MGS and which has not been selected, although it may have been selected in some other iteration 690 or not selected at all, hence the adjective 'candidate' in 'candidate image'.) For example, since only one image has been selected at this stage (n = 1), each MGS is modified depending on the distance between its associated (candidate) image and the one selected image, and the'modifying factor' or process decreases with the distance from the selected image. (Throughout this specification, modifying the MGS means modifying the original MGS, and the'modified' MGS is an MGS that has a reduced or lowered value relative to the corresponding original MGS. The original MGS and the modified version of the original MGS are associated with the same image.)

[0109] At step 684 (after modifying the MGSs), the data processor 14 may identify the modified MGS that currently has the maximum value (i.e., MGS|max) among the modified MGSs. At step 686, the data processor 14 may select the image associated with the newly identified MGS|max. At step 688, the data processor 14 may increment the loop counter n by one in each repetition until all the image budgets are used (i.e., until n = N, for example, N = 100).

[0110] If the value of the loop counter n is still less than N at step 670, then the data processor 14 may invalidate the MGS|max associated with the selected last image at step 680, and then modify each MGS at step 682 based on the distance between the image associated with the MGS and the closest selected image. For example, if two images “A” and “B” have been selected (e.g., from an image budget of, for example, ten images), then each particular MGS is modified according to the distance d between the image associated with the particular MGS and the closest selected image. For example, if the image associated with a particular MGS is closer to, for example, image ‘A’, then the distance d between the image associated with the particular MGS and image ‘A’ is used to modify the particular MGS, and as described herein, the shorter the d, the greater the reduction in the value of the particular MGS. Each time the data processor 14 executes the loop 690 to select an image, the (original) MGSs are modified in a different way because the distance between the candidate image and the selected image that is the basis of the MGS modification process changes each time another image is selected.

[0111] Figure 6A Variations of the embodiment shown in may include steps ordered as Figure 6A shown, and the image selection process may include: (i) identifying the maximum MGS (MGS|max) among the MGSs identified for an image subgroup according to step 640 (or step 684, depending on the stage of the process), and selecting the image associated with the maximum MGS according to step 650 (or step 686, depending on the stage of the process); (ii) invalidating the MGS associated with the selected image identified at step 640 (or at step 684); (iii) modifying each MGS (the original MGS) based on the distance d between the image associated with a particular MGS and each selected image according to step 682; (iv) identifying the modified MGS that has the maximum value (e.g., MGS|max) among the modified MGSs according to step 684. Embodiments of the method may further include selecting the image associated with the modified MGS|max according to step 686 and repeating steps (ii) to (iv) until a predetermined criterion selected from the group consisting of the number of images N and the scoring threshold is satisfied.

[0112] Figure 6B An example implementation of an image selection method is shown. Associated with Figure 6A and Figure 1 and Figure 6A described as follows. ‘Score’ is a scoring vector of the original MGS that first contains multiple image subgroups (e.g., one MGS per image subgroup). The vector ‘Score’ may also contain the modified MGS during the image selection process. Figure 6B

[0113] ‘ScoreOrig’ is a scoring vector that holds the original MGS for each image selection repeat loop (e.g., repeat loops 690 and 632). Whenever an image is selected, the entire content of the Score vector changes due to the modification of the MGS. On the other hand, the content of the ScoreOrig vector changes one MGS at a time due to the invalidation of one MGS each time an image associated with the specific MGS is selected. (In each repeat, another MGS currently identified as MGS|max is invalidated until N MGSs in the vector ScoreOrig are invalidated.)

[0114] At step 612, the data processor 14 initializes the vector ScoreOrig with the original MGS first stored / contained in the vector Score. At step 612, the data processor 14 may also reset the loop counter n to or initialize the loop counter n with a reference numeral, e.g., zero.

[0115] ​At step 614, the data processor 14 checks whether N images have been selected (which checks whether the image budget has been exhausted). If N images have been selected (the condition is shown as "Yes" at step 614), then the data processor 14 may terminate the image selection process. However, if fewer than N images have been selected (the condition is shown as "No" at step 614), then the data processor 14 may identify the MGS that currently has the maximum value in the Score vector at step 616 (i.e., MGS|max). If MGS|max has been found but its value is lower than a predetermined score threshold Sth (e.g., if MGS|max is equal to zero or if it is negative), the condition is shown as "No" at step 616, then the data processor 14 may terminate the image selection process. However, if the identified MGS|max is greater than the scoring threshold Sth (the condition is shown as "Yes" at step 616), then the data processor 14 may select the image associated with the MGS that is currently identified as MGS|max at step 618. In some embodiments, the iteration / repeat loop 632 may not include step 614 (e.g., this step may be omitted, removed, or ignored), and the number of finally selected images may depend on the value of Sth. (The lower Sth is, the greater the number of selected images.) Thus, by using only the scoring threshold Sth or by using the image count limit N (at step 614) and the scoring threshold Sth (at step 616), the number of images finally selected by the process can only be known ex post facto, i.e., only after the image selection process is completed. However, the user can change or set the value of the scoring threshold to decrease or increase the number of images to be selected, and the user can also limit the number of images to a desired number by setting the corresponding value to the parameter N.

[0116] At step 622, the data processor 14 may invalidate the MGS that is currently identified as MGS|max in the vector ScoreOrig, for example, by replacing the value MGS|max in the vector ScoreOrig with, for example, the value (-∞) or the value zero, so as to exclude this image from the image selection process (to ensure that this image will not be selected more than once). Then (still at step 622 or in a separate step), the data processor 14 retrieves the MGS from the vector ScoreOrig and modifies the retrieved MGS by using the modification function f(d). Finally, the data processor 14 stores the modified MGS in the Score vector, thus updating the content of the Score vector with the new modified MGS value whenever a new image is selected at step 618.

[0117] If the allocated image budget has not been exhausted (per step 614), then at step 616, identify the next largest MGS (MGS|max) among the modified MGSs, and at step 618, select the image associated with MGS|max (among the modified MGSs), and so on.

[0118] Modify the original MGS and GS

[0119] Modify the function f(d) to take into account the distance between the image just selected for display and the candidate images in the image group (e.g., based on the number of intervening or temporally interpolated images). In some embodiments, the function f(d) can be designed such that the closer a candidate image is to the currently selected image, the greater the output value of the function, and thus the lower the value of the MGS topic of the modification process. For example, assume that the images are numbered in the order in which they were captured, and instance image number 250 has just been selected for display. Continuing with the example, instance image number 290 in the instance image group is 40 images away from image number 250 (d = 290 - 250 = 40 images), and instance image number 100 in a series of images is 150 images away from image number 250 (d = |100 - 250| = 150 images). The function f(d) can be selected (configured) such that a relatively high number (e.g., 45) will be output by the function f(d) for image 290 to strictly suppress (substantially reduce) the score value of the closer image (the score value of image number 290 in this example). In contrast, the score value of instance image 100 will only be loosely (less strictly, or slightly)'suppressed' by the function f(d), e.g., by the function f(d) outputting a relatively low number (e.g., 12) for image 100. (As Figure 6B shown, for example, the modified score can be calculated by using the formula Score = ScorOrig - f(d), see step 622. Thus, the greater the value of f(d), the smaller the value of Score, and thus the greater the modification to the original score.)

[0120] The basic principle behind the distance-dependent MGS modification process is that if a candidate image close to a selected frame is clinically extremely important (indicated or inferred by its relatively high MGS value), then there is still a good chance of selecting a close image in the next iteration or repetition, even after strictly making its MGS value smaller. However, if a candidate image close to a selected image has low clinical importance, which may be reflected in its relatively low score value, or if the candidate image has the same clinical importance as the selected image, then by reducing its score value so that a candidate image relatively far from the selected image may have a better chance of being selected even when the far candidate image and the close candidate image may be similar in terms of clinical importance, "skipping" the candidate image (skipping the close candidate image by reducing its chance of being selected) can be beneficial. In other words, only when the original MGS value of a close image is relatively extremely high, which indicates or infers that the close image is clinically extremely important, can it be beneficial to select an image close to the selected image for display so that even significantly reducing the high score value of the close image will still give the chance of selecting the image in the next image selection iteration / repetition in this situation. Another reason is that the closer an image is to a selected image, the greater the probability that the selected image and the images near the selected image show or contain the same pathology. Therefore, it may be beneficial to finally select only one of the two images that is more conducive to showing other pathologies elsewhere in the body cavity.

[0121] The selected image can be chosen because its associated polyp score has a high value, and the images close to the selected image have high score values due to bleeding. In this situation, in order to ensure the selection of two images (related to different pathologies), the score suppression process can be selectively applied to the scores; that is, the score suppression process can be applied only to the scores related to the same type of pathology (e.g., polyp, or bleeding, etc.).

[0122] The modification function f(d) can be any function that reduces the value of MGS based on the distance d between the image related to MGS and the closest (previously) selected image. The modification function f(d) can be designed such that the shorter the distance d between two instances (the image related to MGS and the selected image closest to it), the greater (more strictly) the reduction in the value of MGS. The function f(d) can be linear (e.g., f(d) = a * d, where 'a' is a coefficient), or non-linear (e.g., f(d) = a * d 2 + b * d + c, where 'a' and 'b' are coefficients and 'c' is a constant). The function f(d) can be a Gaussian exponent, or an exponent with an absolute distance value, or a step function, or a triangular function whose vertex or peak is located at the selected frame / image and decreases linearly with distance.

[0123] The distance d between images (e.g., between an image associated with the MGS and a selected image) can be calculated or measured in the following units: (i) time, or (ii) the number of intervening images, or (iii) the number of subgroups of intervening images, or (iv) the distance traveled by the in-vivo device in the body cavity when the images are taken, or (vi) the percentage of a video clip that contains all or most of the groups of connected images, or (v) the distance of the in-vivo device in the body cavity relative to a group of connected in-vivo images or a marker in the body cavity, or (vii) an estimated distance (e.g., by linear progression or a similar technique).

[0124] Each of the unit examples described above can be calculated or measured as: (i) an absolute value, e.g., a value that can be calculated or measured relative to the start of a group of consecutive images, or relative to the start of the body cavity, or relative to any marker in the group of consecutive images or in the body cavity, or (ii) a percentage, e.g., as a percentage of the entire group of consecutive images, or a percentage of the body cavity or a part of the body cavity, or (iii) the percentage of a video clip that contains all or most of the groups of connected images, or (iv) the percentage of the distance traveled by the in-vivo device, or (v) the percentage of the time traveled by the in-vivo device, e.g., the percentage of the time taken by the in-vivo device to traverse the distance captured in a certain series of images). The body cavity can be an organ of the gastrointestinal or rectal system.

[0125] The data processor 14 can enable a user (e.g., a doctor) to select a modification function based on the part of the body cavity from which the images are taken, and this can be beneficial because by selecting an appropriate modification function, the user can bias (with 'preference' or 'preference') the image selection process towards a particular part of the body cavity that the user is more interested in, e.g., by loosely (slightly) reducing the value of the MGS associated with (derived from) the focused lumen part while strictly reducing the MGS values derived from other lumen parts.

[0126] Figure 6C An example group 652 of connected images according to an example embodiment is shown. And Figure 6B is described in connection with Figure 6C . Assume that the image group 652 contains a large number (e.g., 100,000) of connected images. The large number of images can be normalized such that each image has a normalized image number. For example, the first image captured (the image shown at 642) can be assigned the value 0.00, and the first image captured (the image shown at 644) can be assigned the value 1.00. Also assume that a scoring vector similar to Figure 6B Score contains eight MGSs corresponding to eight images designated as MGS1, MGS2, …, MGS8 in Figure 6C , and the number of images to be selected (e.g., for further analysis and / or for display) is three, i.e., assume N = 3)

[0127] according to Figure 6B At step 616, it is assumed that the value of MGS6 is currently the highest among MGS1, MGS2, ..., MGS8 (e.g., MGS6=MGS|max), and therefore, at step 618, it is assumed that the image associated with MGS6 is the first selected image. At step 622, the value of MGS6 is invalidated (in the associated score vector ScoreOrg.), and based on the image associated with MGS6, the image is selected. i Each MGS in MGS1, MGS2, ..., MGS8 is modified by the distance d between the relevant image and the closest selected image. i . (At this stage, only one image is selected (image 0.63 corresponding to MGS6), so all MGSs are modified relative to this image.) For example, the distance d1 between the image associated with MGS1 and the selected image associated with MGS6 is 0.61 (d1=0.63-0.02=0.61). Similarly, the distance d2 between the image associated with MGS5 and the selected image associated with MGS6 is 1.016 (d2=0.63-0.47=0.16). Similarly, the distance d3 between the image associated with MGS7 and the selected image associated with MGS6 is 0.11 (d3=0.74-0.63=0.11), the distance d4 between the image associated with MGS3 and the selected image associated with MGS6 is 0.45 (d4=0.63-0.18=0.45), and so on. (The distances of other MGSs can be found in the same way.) Next, each MGS is modified according to the distance of its associated image from the selected image. If the image budget has not been exhausted (at step 614, Figure 6B ), then after all MGS are modified in step 622 to produce modified values ​​MGS'1, MGS'2, MGS'3, MGS'4, MGS'5, MGS'6 (MGS6 is invalid), MGS'7, MGS'8, the maximum MGS (i.e., MGS|max) among the modified MGS is identified at step 616, and the image (the second image at this stage) associated with the identified MGS|max among the modified MGS is selected at step 618. Assume that the modified MGS'2 having the maximum value at present is (i.e., MGS'2=MGS|max), and therefore, the second selected image is the image associated with MGS'2.

[0128] After selecting the second image (the image associated with MGS'2 and MGS2) (at step 618), the original MGS2 is invalidated (in the score vector ScoreOrg, at step 622), and the distance between each image associated with each other (non-invalid) MGS and each selected image is calculated to identify the closest selected image for each candidate image. Then, each original MGS is modified again this time according to the distance between the specific image associated with each modified MGS and the selected image closest to the specific image. In other words, when using the image selection process to select more images, the distance based on which the MGS is modified can be changed. For example, after selecting the image (normalized) number 0.63 as the first image (because its MGS6 is the largest), the image number 0.28 is at a distance d4 from the image number 0.63, so MGS3 (the MGS of image number 0.28) is modified according to the distance d4. However, after selecting the image number 0.18 corresponding to MGS2 as the second image (this is because MGS'2 is the new largest MGS in the second selection iteration), the distance d5 between the image number 0.28 and the image number 0.18 is shorter than the distance d4 between the image number 0.28 and the image number 0.63. (Distance d5 = 0.10, distance d4 = 0.35.) Therefore, after selecting the second image, the value of the original MGS3 is modified according to the distance d5, which is the distance from the image number 0.28 to the closest selected image. Similarly, after selecting the second image (image number 0.18), the image number 0.02 is closer to the selected image 0.18 than to the selected image 0.63 (d6 < d1). Therefore, after selecting the second image, the distance d6 is used to modify the original MGS1 associated with the image number 0.02. (After selecting the first image, the distance d1 was used to modify the original MGS1.)

[0129] In one embodiment, the same distance calculation process applies to all other MGSs and applies each time a new image is selected. (As described herein, the distance between candidate images can be calculated or measured in the following units: time (e.g., the capture time of the images being compared), the number of intervening images, the number of image subgroups, and virtually any suitable technique.)

[0130] Reference Figures 6A to 6B , Equation (4) is an example scoring (MGS) modification function, and Equation (5) is an example equation showing an example way of using f(d) to modify the (original) MGS:

[0131] f(d = Dmin(i)) = (-30)*exp.{-(1,000*Dmin(i)) 2} (4)

[0132] Score(i) = ScoreOrig(i) - 30 * exp{-(1,000 * Dmin(i)) 2} (5)

[0133] where ScoreOrig(i) (i = 1, 2, 3, …) is the original MGS(i) value in the scoring vector ScoreOrig, Dmin(i) is the distance d between the image associated with MGS(i) and the closest selected image, and Score(i) is the new score generated from the modified corresponding (original) MGS. (For example, Dmin(i) can have values in the range of 0.0, 1.0], which is based on Figure 6C .) As described herein, after two or more images are selected, a candidate image can be closest to a particular selected image, and Dmin(i) specifies the distance between the candidate image and the closest selected image. Referring to Equation (5), the shorter the d (the smaller the value of Dmin(i)), the larger the value subtracted or removed from ScoreOrig(i). (The more strictly the relevant MGS is modified.)

[0134] Although some of the methods described herein, for example, in connection with Figures 6A to 6B are applicable to in vivo devices that include one imager, embodiments can be generalized to in vivo devices that include any number q of imagers, as described below. Briefly, while an in vivo device with one imager produces a group of images that generates one MGS|max value each time (e.g., in each image selection repetition 690, Figure 6A ), an in vivo device with many imagers produces as many groups of images as possible that generate as many MGS|max values as possible each time, and each time (e.g., during each image selection repetition 770, Figure 7 ), the largest MGS|max among all MGS|max values is identified, and the image associated with the largest MGS|max is selected. As an example, a method for selecting an image from a number q of contiguous groups of images can include performing the following operations for q contiguous groups of images respectively captured by q imagers in a body cavity, where each image in each group can be associated with a total score (GS) indicating the probability that the image contains at least one type of pathology and a distance d indicating the distance between the images in each group:

[0135] Partition or separate the group of images or a portion thereof into image subgroups, each image subgroup including a base image and images similar to the base image; and

[0136] Identify a set of Maximum Gross Scores (MGS) Set(i) (i = 1, 2, 3, …), where the set of Maximum Gross Scores Set(i) contains the Maximum Gross Score (MGS) of each image subgroup in the image group; and select images for processing, where image selection may include: (i) identifying the maximum MGS (MGS|max) in each set of MGS S(i); (ii) identifying the maximum MGS|max among all MGS|max and selecting the image associated with the maximum MGS|max; (iii) invalidating the maximum MGS|max associated with the selected image; (iv) modifying the specific set of MGS Set(i) associated with the selected image based on the distance d between the image associated with the specific set of MGS Set(i) and each selected image; and repeating steps (i) to (iv) until a predetermined criterion selected from the group consisting of the maximum number of images N and the scoring threshold Sth is satisfied. In combination Figure 7 An example method describing a situation where the number q of imagers and thus the number of connected image groups is equal to two is described below.

[0137] Figure 7 Shows a method of selecting images for display according to another embodiment of the present invention. Associated with Figure 1 is described Figure 7 . Figure 7 The image selection method described is applied to two image groups obtained respectively from within the body of two imagers of an in-vivo device (e.g., Imager-1 and Imager-2, Figure 2 ).( Figure 7 refers to a system configuration where the in-vivo device includes two imagers.) Figure 7 The embodiment shown in may include an iterative or repeating loop (770) to select images for display (or for other purposes; e.g., for further analysis), one image per iteration / repeat. Each iteration / repeat may result in the selection of an image from one of the two imagers. One imager may ultimately provide more (selected) images than the other imager, and in some embodiments the image selection method ensures that the total number of images selected from the two imagers does not exceed the allocated image budget. As described in step 616 associated with Figure 6B , the use of an 'image budget' or limit may include completing the image selection process when the number of selected images is equal to a predetermined number of images N, or it may include using a scoring threshold Sth, or it may include using both the predetermined number of images N and the scoring threshold in combination. In some embodiments, using the scoring threshold (Sth) ensures that a certain finite number of images are selected (although unknown in advance) because the value of the MGS decreases from one image selection cycle to another such that at some point, after a finite number of iterations 770, the remaining maximum MGS will necessarily be less than the scoring threshold.

[0138] After the images from the two imagers have been pathologically processed (e.g., by data processor 14 or by a remote computer) in the manner described herein, each image group is independently segmented into image subgroups. The data processor 14 can identify one or more local maximum scores MGS in each image subgroup of each image group, and the data processor 14 can generate two score vectors for separately adapting the two sets of MGS: (1) a score vector Score(1) for accommodating the original MGSs from the image group (image subgroups) associated with the first imager and (2) a score vector Score(2) for accommodating the original MGSs from the image group (image subgroups) associated with the second imager. Each score vector Score(j) can contain a plurality of local maximum score values MGS, (where j = 1, 2). In some embodiments, each local maximum MGS in each image subgroup (in the two image groups) is stored in the corresponding vector Score(j). (As used herein, the index 'i' indicates a single vector value; e.g., Score(j) is an individual score value. The index 'j' indicates the complete vector; e.g., score(j) is a vector containing a plurality of score values.) In other embodiments, only the maximum local maximum score value MGS in each image subgroup is stored in the corresponding vector Score(j). In other embodiments, the maximum score value in each image subgroup is selected for the corresponding vector Score(j), regardless of whether it is a local maximum score value.

[0139] At step 710, the data processor 14 can initialize the score vector ScoreOrig(1) associated with the first imager with the (original) MGS values first stored in the score vector Score(1), and similarly, it can also initialize the score vector ScoreOrig(2) associated with the second imager with the (original) MGS values first stored in the score vector Score(2). The MGSs first stored in the score vector ScoreOrig(1) and the score vector ScoreOrig(2) are referred to herein as 'original MGSs'. The modified MGSs are stored in the score vector Score(1) and the score vector Score(2). (The (original) MGS value stored in the score vector Score(1) changes each time an image is selected from an image associated with the MGS stored in this vector. Similarly, the (original) MGS value stored in the score vector Score(2) changes each time an image is selected from an image associated with the MGS stored in this vector.) At step 710, the data processor 14 can also reset the loop counter n to an initial value; e.g., reset to one.

[0140] At step 720, data processor 14 may check whether N images have been selected (the processor may check whether the image budget has been exhausted). If N images have been selected (the condition is shown as "Yes" at step 720), then data processor 14 may terminate the image selection process. However, if fewer than N images have been selected (the condition is shown as "Yes" at step 720), then data processor 14 may, at step 730, identify, in each scoring vector Score(j), the MGS that currently has the maximum value (this MGS is referred to herein as 'MGS|max'). That is, data processor 14 may identify MGS|max(1) of score Score(1) and MGS|max(2) in score Score(2). Next, data processor 14 may select, from MGS|max(1) and MGS|max(2), the MGS|max(i) having the highest value. If MGS|max(1) is equal to or greater than MGS|max(2) (the condition is shown as "Yes" at step 730; j = 1), then the MGS modification process executed at step 760 will be applied to the original MGS associated with the first imager (applied to the original MGS stored in scoring vector ScoreOrig(1)), and if MGS|max(2) is greater than MGS|max(1) (the condition is shown as "No" at step 730; j = 2), then the MGS modification process will be applied to the original MGS associated with the second imager (applied to the original MGS stored in scoring vector ScoreOrig(2)).

[0141] At step 740, if the value of the selected MGS|max(i) is equal to zero or it is negative, where i = 1 or 2 (depending on which MGS|max(i) is larger: MGS|max(1) or MGS|max(2)), or in the general case where it is below a predetermined positive scoring value threshold (the condition is shown as "No" at step 740), then data processor 14 may terminate the image selection process. (Depending on the value of the scoring threshold, the image selection process may be terminated before N images are selected.) However, if the selected MGS|max(i) is greater than zero (the condition is shown as "Yes" at step 740) or if it is greater than the scoring threshold, then data processor 14 may, at step 750, select the image associated with the MGS|max(i) that is currently identified as the maximum MGS|max among the current MGS|max(1) and MGS|max(2). (The values of MGS|max(1) and MGS|max(2) may change from one iteration of loop 770 to another.)

[0142] At step 760, the data processor 14 may invalidate the MGS currently identified as the maximum MGS|max(i) in two score vectors Score(1) and Score(2) in the score vector ScoreOrig(j) to exclude the selected image from the image selection process (to ensure that this image will not be selected more than once). The data processor 14 may invalidate the MGS by, for example, replacing the value MGS|max(i) in the associated score vector ScoreOrig(j) with the value (-∞) or the value zero. Then (still at step 760, or in a separate step), the data processor 14 may retrieve the original MGS from the relevant score vector ScoreOrig(j) (the score vector ScoreOrig(j) is associated with the MGS|max(i) selected at step 730), then modify the retrieved MGS by using the modification function f(d), and store the modified MGS in the corresponding score vector Score(j), so as to update the score content of the score vector Score(j) with the new modified MGS value each time a new image is selected from the corresponding imager at step 750.

[0143] When step 730 is first executed, both score vectors Score(1) and Score(2) contain (store) the original MGS. Thus, the first execution of step 730 involves identifying and comparing the original MGS|max(i) in score vector Score(1) with the original MGS|max(i) in score vector Score(2). When the first image is selected at step 750, the relevant score vector Score(j) identifying the relevant MGS|max is modified, which process results in a modified MGS in the score vector Score(j). Thus, the second execution of step 730 involves comparing the original MGS|max stored in the other and unmodified score vector Score(j) with the modified MGS|max from the modified score vector Score(j). However, assuming that at some point in the image selection process, at least one image has been selected from each image group, then further execution of step 730 involves comparing the modified MGS|max values, but different score vectors Score(j) are modified in each iteration loop (770), and each modification of the score vector Score(j) is completed with respect to the corresponding original MGS.

[0144] The image selection method described herein can be similarly applied to any number q of imagers that respectively capture q groups of images, including application to one imager. For the general case where there are q groups of images, each group of images can produce one or more image subgroups and associated original MGSs, and additionally, the number q of MGS|max values (one MGS|max value for each image group) can be compared (in a step similar to step 730) in each image selection iteration or repetition loop in order to select, in each image selection iteration loop, an image associated with the MGS|max having the maximum value among all q MGS|max values.

[0145] Figure 8 An example implementation of an image selection method according to an example embodiment is schematically illustrated. A human gastrointestinal tract (schematically shown at 800) particularly includes a small intestine (SB) segment 810 and a colon 820 segment. An in vivo device similar to in vivo device 40 can capture images in the gastrointestinal tract 800 while moving in direction 830.

[0146] Graph 850 is a time graph showing images that are temporally ordered and associated with portions of the gastrointestinal tract from which the images were captured. For example, the first image captured in SB (810) is image 812, which was captured at time t1, and the last image captured in SB (810) is image 814, which was captured at time t2 (t2 > t1). Similarly, the first image captured in the colon (810) is image 822, which was captured at time t3, and the last image captured in the colon (810) is image 824, which was captured at time t4 (t4 > t3 > t2 > t1). According to graph 850, images are captured using a constant imaging rate. However, the imaging rate can vary, for example, according to a change in the movement of the in vivo device or according to a change in the remaining battery power, etc., and the image selection method disclosed herein can also be applicable to embodiments where the imaging rate varies.

[0147] Suppose a user (e.g., a doctor) wants to select only images from a specific segment of the gastrointestinal tract 800. Then the user can indicate this to the workstation (e.g., workstation 13), for example, by mouse - clicking on a dedicated input box on a computer monitor (e.g., display device 18), and the data processor (e.g., data processor 14) that adheres to the user - input data can exclude all foreign parts of the gastrointestinal tract (800) by setting all total scores related to the foreign rectal part to zero. For example (referring to chart 860), suppose the user inputs an instruction to the workstation (for the data processor) to select only those 'pathological' images that originate from the small intestine (810). In response to the user's instruction, the data processor can set the total score (GS) related to the images before the small - intestine images to zero, as shown at 862 (image numbers 123 to 124), and also set the total score related to the images after the small - intestine images to zero, as shown at 864 (image numbers 150 to 163). By setting the total scores related to the images to be excluded to zero (e.g., setting to zero), the images finally selected by the data processor only originate from the small intestine. ('Before a specific image' and 'after a specific image' respectively mean the images captured before and after the specific image.)

[0148] For example, in the sense that the image contains air bubbles and / or gastric contents, and / or if the image provides tissue coverage below a tissue - coverage threshold and / or the image meets a darkness criterion, etc., then the image can be considered noisy. In such cases, it can be beneficial to exclude such images from the image - selection process. For this, the image - selection process can additionally include, for example, an air - bubble detector and / or a gastric - contents detector and / or a tissue - coverage detector, etc., to detect such images, and the method described herein can include setting the total score of the image to zero (e.g., setting to zero) when the image is noisy. As an example, image numbers 132 and 139 are example images that contain air bubbles and gastric contents, so the total scores related to them are shown as set to zero at 866 and 868, respectively.

[0149] In another example (referring to chart 870), suppose the user inputs an instruction to the workstation (for the data processor) to select only images that originate from the colon (820). In response to the user's instruction, the data processor can set the total score related to the images before the colon images to zero, as shown at 872 (image numbers 123 to 149), and also set the total score related to the images after the colon images to zero, as shown at 874 (image numbers 162 to 163). By setting the total scores related to the images to be excluded from the image - selection process to zero, the images finally selected by the data processor only originate from the colon.

[0150] After excluding images associated with foreign segments of the gastrointestinal tract 800 from the image selection process (e.g., by setting their total scores to zero), the unexcluded images (the remaining images captured in the region of interest of the rectum) can be segmented into image subgroups. Then, within each image subgroup, one or more MGSs can be identified, and then the MGSs of the image subgroups can be processed to select images from the gastrointestinal segment of interest (e.g., the small intestine segment 810, or the colon 820 or any other segment of the gastrointestinal tract 800).

[0151] In some embodiments described herein, for example, in combination with Figure 6A the GS of images close to the selected image is variably suppressed (e.g., its value is decreased), thus still giving the close images a chance to be selected as well. Generally, the number of images close to the selected image and containing the same object is only known afterwards, i.e., after the tracking process of searching for a certain object in the close images for the relevant selected image is completed. (The tracking process can be considered completed after the processor performing the tracking process finds the last image containing the same object as in the selected image. (The object contained in the selected image is referred to as the 'original object' herein.) That is, as long as the processor finds the original object in these images, the processor can 'track' the tracking process from the selected image to image after image in an ordered manner (in both directions), and the processor can terminate the tracking when the next image does not contain the original object.) Additionally, if many images on one side (within the ordering of the image stream) or both sides of the selected image contain the original object, then such images can be relatively far from the selected image in chronological order (e.g., 50 images away from the selected image, 45 images in front of the selected image, etc.) and such images can still be considered 'close' images. (If the tracking process indicates that the remote image and all the images intervening between the remote image and the selected image all contain the same object, then the remote image can still be considered a close image.) Thus, using this embodiment can cause redundant selection of additional images containing (showing) the same object (e.g., a pathology), such as the same polyp, the same lesion, etc. However, in some situations, it may be necessary to ensure that only one image showing a specific object (e.g., a pathology) is selected, in order to avoid selecting redundant images and generally make better use of a given (e.g., predetermined) image budget. Using Figure 9 the embodiment shown in can ensure that only one image is selected for each object (e.g., for each specific pathology). After a specific image is selected for display or another purpose, the images close to the selected image can still be made available to the user (e.g., a doctor), and the user can also select these images if necessary, for example, for review (e.g., as a video clip, as a 'collage', etc.), or the processor can manipulate the entire image subgroup associated with the selected image, for example.

[0152] Using image characteristics as an object can enable the definition of multiple image subgroups first, and then the selection of images by, for example, manipulating their total scores in a manner described in conjunction with Figures 6A to 6B and Figure 7 As described, for example, in conjunction with Figure 9 Using the imaging object as an object can enable the selection of images first, and then the use of the selected images to periodically 'its' subgroups to invalidate the entire subgroup before selecting the next image, etc. Whether the subgroup of images is found first and then the image is or can be selected from the subgroup, or whether the image is selected first and then its subgroup is formed, the total score (GS) of each subgroup is manipulated (e.g., invalidated or modified) before selecting the next image.

[0153] Figure 9 A method of selecting images for display (or for any other purpose) according to another exemplary embodiment of the present invention is shown. At step 910, the value of the loop index n is initially set to zero (n = 0). At step 920, the processor may receive or retrieve a set (series) of consecutive images from an individual's gastrointestinal tract and may calculate the total score (GS) of each image. The processor may use any one or other methods described herein to calculate the total score (GS) of the image.

[0154] At step 930, the processor may identify the maximum GS (MGS) among the GSs, and at step 940, the processor may select (or mark) the image whose GS is the MGS, for example, for display, and set the value of n to 1 (n = n + 1 = 1) to indicate that one image has been selected at this stage.

[0155] At step 950, the processor may check whether the value of n is equal to N (N is any preferred maximum number of images allowed to be selected; for example, N = 10, 25, 100, etc.). If the number of selected images is equal to N (this condition is shown as "yes" at step 950), then the image selection process is terminated. However, if the number of selected images is (still) below N (this condition is shown as "no" at step 950), then at step 960, the processor invalidates (e.g., sets to zero) the value of the MGS of the selected image (and its associated), and additionally, the processor may identify objects of clinical importance (e.g., polyps) in the selected image. (It is highly likely that the image contains this object because its GS is currently the largest.)

[0156] At step 970, the processor may search for the same objects of clinical significance (objects that are the same or equivalent to the objects identified in the selected image) in images that are close to the currently selected image. For example, the processor may apply a tracking mechanism to attempt to identify the images that are close to both sides of the selected image (the previous image and the subsequent image), where the images contain the same object (e.g., the same pathology) for which the selected image was likely chosen.

[0157] If the processor finds an image that is close to the currently selected image and contains the same object as the currently selected image (this condition is shown as "Yes" at step 980), then at step 990, the processor invalidates the GS of each image that contains the object. Invalidating the GS of all the close images (images that are close to the current selected image) excludes the close images from the image selection process, thus ensuring that only the object identified (detected) in the selected image will be shown (or for any other reason, e.g., for further processing) once, that is, only one representative image will be shown. However, if the processor does not find an image of the currently selected image that 'only' contains the same object as the currently selected image (this condition is shown as "No" at step 980), then at step 992, the processor may modify all the GSs that have not been invalidated. The rationale for modifying the non-invalidated GSs at step 992 is that if the images that are close to the currently selected image do not contain the object for which the current imager was selected, then the images that are close to the selected image ('close images') may contain another object that could warrant the selection of this image. Therefore, the GSs of the close images may be modified (not invalidated) to give the close images or some of them an opportunity to be selected as well. Modifying the non-invalidated GSs at step 992 can be implemented by using any of the modification methods described herein or any other suitable modification method. Step 992 (modifying the non-invalidated GSs) may be optional. That is, the processor of the Figure 9 embodiment may skip step 992 and proceed to step 930 to start another iteration or repeat 994. In other words, in some embodiments, the processor may not modify the total score, but instead identify the next (next) largest GS, select the relevant image, invalidate its 'adjacent' (close) GSs (if the close images contain the same object as the relevant selected image), and then again (at step 930) identify the next currently largest GS among the non-invalidated GSs and so on (continue steps 940, 950, etc.).

[0158] At each iteration or repeat (994), step 970 results in an image subgroup that includes the selected image as the base image and images that are close to the base image and contain the same object. The image subgroup may sometimes include one or more images that are the selected image.

[0159] After invalidating (at step 990) the images that contain the same object as the currently selected image, or after modifying all non-invalid GSs (if step 992 is not omitted by the processor), another iteration or loop 994 can start at step 930 to select another image. That is, at step 930, the processor can identify the current (new) maximum GS (current / new MGS) among the GSs or among the modified GSs, then at step 940 select the image associated with the MGS, then at step 960 invalidate the current MGS and identify a new object (an object different from the object contained in the previously selected image), then at step 970 search for the same new object in the images close to the newly selected image, and so on.

[0160] Images that contain the same object as the selected image do not necessarily mean that the object is in the same position or has the same orientation in these images. For example, the same object can be shifted between adjacent images, and / or the object in one image can be rotated relative to the object image in an adjacent image. The processor can use any known image processing method to determine whether two images show or contain the same object (e.g., the same polyp). Such techniques are well known in the art.

[0161] Figure 10 Illustrates the image selection process according to an example embodiment. Figure 10 Step A shows an example image group 1000 that includes twenty-eight consecutive images captured by an in-vivo device that includes one imager and its corresponding total scores. (According to their capture times, the 28 images are shown as being temporally ordered, with the image numbered '1' being captured first and the image numbered '28' being captured last.) In practice, the number of images in an image group can be extremely large (e.g., tens of thousands), however, as in other diagrams, a small number is used only for illustrative purposes. Figure 10 The images shown in step A.

[0162] Will be described Figure 9 in association with Figure 10 Step A. After the processor calculates the total scores for the twenty-eight images (at step 920), for example, the processor can identify the current maximum GS (current MGS) at step 930. Referring to Figure 10 Step A, the current (at this stage the first) maximum GS or MGS is the GS of image number ten. (The value of the GS of image number '10' is 93, as shown at 1010.) Therefore, the processor selects (at step 940) image number '10' because it is associated with the current one. Assume that more images (e.g., 35 images) want to be selected. Therefore, the processor invalidates the MGS (e.g., by setting its value to zero, as Figure 10as shown at 1020 in step B) and continues the image selection process by identifying the objects in the selected image (also at step 960).

[0163] Next, at step 970, the processor searches for the same objects in the images close to the selected image. Referring to Figure 10 step A, as an example, assume that image numbers '8' and '9' (the two images before image number '10', the currently selected image) and image numbers '11', '12', '13', and '14' (the four images after image number '10') contain the same objects as image number '10'. (In other words, all the images that make up or form the image subgroup 1030 contain the same objects.) Thus, at step 990, the processor invalidates the GSs of all the images in the image subgroup 1030 because they all contain the same objects and thus only one representative image should be selected. The invalidation of these GSs is shown at Figure 10 step B. For example, although the value of the GS of image number '8' is 70 (at Figure 10 step A), its value at Figure 10 step B (i.e., after invalidating it at step 990) is zero. Except for image number "10", invalidating the GS of image number "10" and the GSs of the images near image number "10" excludes the entire image subgroup 1030 from the image selection process, and image number '10' is selected as the representative image of this image subgroup because at this stage (where this algorithm repeats), it has the largest GS.

[0164] After excluding the first image group (image subgroup 1030) from the image selection process, the processor again identifies at step 930 the next MGS that has a value of 82 in the Figure 10 example of step B (this GS is calculated for image number '20' and is shown at 1040). As with the previous MGS, the processor selects image number '20' at step 940 and invalidates the value of its GS at step 960. The invalid MGS value 1040 of image number '20' (the second selected image) is shown at 1050. As an example, assume that only two images (image numbers '21' and '22') close to the currently selected image (image number '20') contain the same object as image number '20'. Thus, the entire image subgroup 1060 is excluded from the image selection process - since its GS is the second largest GS and image number '20' has been selected, and since image numbers '21' to '22' contain the same object as image number '20'.

[0165] Figure 10 Indicating at Figure 9In each iteration or repetition 994 of the embodiments, another subgroup of images can be subtracted or removed from the image selection process. The third iteration or repetition 944( Figure 9 ) can be applied to Figure 10 the remaining GS in step C; for example, applied to the GS remaining after excluding image subgroups 1030 and 1060. If the selected image does not have any neighboring images containing the same object, then the next image can be selected in a similar manner without modifying the non-invalid GS (by skipping or ignoring step 992) or by modifying the non-invalid GS according to step 992.

[0166] The image selection method disclosed herein can be implemented in various ways, such as by using various scoring modification mechanisms. For example, the modification function for modifying the score can be selected from the group consisting of: symmetric function or asymmetric function ('symmetric' and 'asymmetric' are relative to the selected image), linear function, non-linear function, Gaussian exponent, exponent with absolute value of distance, step function, and triangular function with its vertex at the selected image and decreasing linearly with distance. The modification function can be selected according to parameters selected from the group consisting of: pathological type, location in the gastrointestinal tract, and speed of the in-vivo device in the body cavity. For example, the image selection process can be applied to all images captured by the in-vivo device, and the process can include applying the same modification function to the relevant MGS; the image selection process can be applied to all images captured by the in-vivo device, but the process can include applying different modification functions to the MGSs related to images captured in different parts or segments of the body cavity; it can be applied only to images captured by the in-vivo device in different parts or segments of the body cavity, and the process can include applying the same modification function to the relevant MGS or different modification functions; in the sense that when the in-vivo device moves slowly in the gastrointestinal tract, it can be applied only to or more strictly applied to the MGSs related to images relatively close to the selected image, and when the in-vivo device moves at high speed in the gastrointestinal tract, it can be applied only to or more strictly applied to the MGSs related to images relatively far from the selected image, the modification function can be speed-dependent; when the in-vivo device generates, for example, two series of images (one series for each imager), if one imager takes pictures of a pathology (e.g., a polyp), then the images from the other imager that are temporally close to the said image can be significantly suppressed because it makes no sense to select additional images of the same pathology. (For example, when the in-vivo device moves slowly, or when the image capture rate is high compared to the movement of the in-vivo device, and / or when two (or more) imagers of the in-vivo device take pictures of the same pathology, images of the same pathology can be taken.)

[0167] In some embodiments, the MGS calculated for images associated with one image group may also be suppressed relative to images for which the MGS is calculated for images involved in another image group. For example, if there are two imagers, one imager may observe in the forward direction (the direction of movement of the in-vivo device), while the other imager may observe in the opposite direction. Images captured by the two imagers may be "staged" between consecutive images to determine which imager is "zooming in" and which imager is "zooming out". Then, the asymmetric modification function may reject images taken by the other head at the same rectal site.

[0168] Although data collection, storage, and processing are performed by certain units, the systems and methods of the present invention may be practiced with alternative configurations. For example, the components that collect and transmit image data need not be contained within the capsule and may be contained within any other device suitable for traversing a lumen in the human body, such as an endoscope, a vascular stent, a catheter, a needle, and the like.

[0169] For purposes of illustration and description, the foregoing description of embodiments of the invention has been presented. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Those skilled in the art will appreciate that many modifications, variations, substitutions, alterations, and equivalents are possible in light of the above teachings. Accordingly, it is understood that the appended claims are intended to cover all such modifications and alterations as fall within the true spirit of the invention.

Claims

1. A computer-implemented method for selecting images comprising clinically important information from images captured by an in-vivo device, the method comprising: Receiving a set of consecutive images captured in a body cavity, wherein each image is associated with an overall score GS indicative of the probability that the image includes at least one type of pathology; And Upon user operation: (i) Iteratively selecting images from the images captured by the in-vivo device based on the associated GS of the images until a predetermined criterion is met, wherein one image is selected at a time; (ii) Each time an image is selected, suppressing the GS of the images in the set of consecutive images, wherein performing the suppression increases the probability that other images include clinically important information, wherein the clinically important information in the other images is related to other pathologies or to other locations in the body cavity relative to the pathology or location shown in the selected image; and (iii) Displaying or further processing at least a subset of the selected images, Wherein for each image selection iteration, the selection of an image comprises: Identifying the maximum GS in the GS; and Selecting the image associated with the maximum GS, and Wherein the criterion is the number of images or a GS threshold or a combination thereof.

2. The method according to claim 1, wherein suppressing the GS of an image comprises invalidating the GS associated with the selected image.

3. The method according to claim 1, wherein suppressing the GS comprises suppressing the GS based on a distance d between the image associated with the GS and the selected image.

4. The method according to claim 3, wherein the smaller the distance d, the greater the reduction in the value of the GS.

5. The method according to claim 3, wherein the distance d between the image associated with the GS and the selected image is calculated or measured in the following units: (i) time, or (ii) the number of intervening images, or (iii) the number of subgroups of intervening images, or (iv) the distance traveled by the in-vivo device in the body cavity when capturing the image, or (v) the distance traveled by the in-vivo device in the body cavity relative to the set of images or a marker in the body cavity, or (vi) the percentage of a video clip comprising all or most of the set of consecutive images.

6. The method according to claim 5, wherein each unit instance is calculated or measured as an absolute value relative to the start of the set of consecutive images, or relative to the start of the body cavity, or relative to a marker in the set of consecutive images or in the body cavity, or as (ii) a percentage or fraction of the set of consecutive images or the body cavity, or (iii) a percentage or fraction of a video clip comprising all or most of the set of consecutive images, or (iv) a percentage or fraction of the distance traveled by the in-vivo device, or (v) a percentage or fraction of the time traveled by the in-vivo device.

7. The method according to claim 1, further comprising, for each of a plurality of images in the set of consecutive images: Identifying an object in the image; and Searching for the same object as the identified object in an image adjacent to the image, Among them, The image and the adjacent image comprising the object form a subgroup.

8. The method according to claim 7, wherein: the plurality of images are selected images, for each selected image, suppressing the GS of the images in the set of consecutive images includes: invalidating the sub - group of images associated with the selected image, and the selection of the images includes: identifying the maximum GS in the GS; and selecting the image associated with the maximum GS.

9. The method according to claim 7, wherein: the object is an image feature, dividing at least a portion of a set of consecutive images including a plurality of images into sub - groups, and the selection of the images includes: identifying the maximum total score MGS of each sub - group; selecting the image associated with the maximum MGS among the MGSs, i.e., MGS|max; and invalidating the maximum MGS associated with the selected image.

10. The method according to claim 1, further comprising enabling the user to select one of the following: the pathological type included in the selected image, the position of the selected image in the body cavity, the number of images to be selected, a modification function for suppressing images according to the part of the body cavity from which the images are taken, or a combination of the above.

11. The method according to claim 1, wherein the suppression includes applying a modification function to the GS with respect to the selected image.

12. The method according to claim 11, wherein the modification function is selected according to parameters selected from the group consisting of: pathological type, the position of the in - vivo device in the body cavity, and the speed of the in - vivo device in the body cavity.

13. The method according to claim 11, wherein applying a modification function to a specific GS includes identifying the minimum distance Dmin between the image associated with the specific GS and any selected image, and modifying the specific GS using the minimum distance Dmin.

14. The method according to claim 1, wherein the suppression includes applying a modification function selected from the group consisting of: symmetric function, asymmetric function, linear function, non - linear function, Gaussian exponential function, exponential function with absolute value of distance, step function, and triangular function with a vertex at a selected frame and linearly decreasing with distance.

15. The method according to claim 1, wherein the body cavity is the gastrointestinal tract.

16. The method according to claim 1, further comprising clearing the GS of an image if the image is noisy.

17. A system for selecting images including clinically important information from images captured by an in - vivo device, the system comprising: a data processor configured to, for a set of consecutive images captured in a body cavity, each image having an associated total score GS indicating the probability that the image includes at least one type of pathology, the data processor, when operated by a user, performs: (i) iteratively selecting images from the images captured by the in - vivo device based on the associated GS of the images until a predetermined criterion is met, wherein one image is selected at a time; (ii) Each time an image is selected, the GS of the images in the set of consecutive images is suppressed, and the suppression is performed such that the probability of other images including clinically important information is increased, where the clinically important information in the other images is related to other pathologies or to other locations in the body cavity relative to the pathology and location shown in the selected image; and (iii) Displaying or further processing the selected image or a subset of the selected images on a display, where for each image selection iteration, the selection of an image includes: identifying the maximum GS in the GS; and selecting the image associated with the maximum GS, suppressing the GS of an image includes invalidating the GS associated with the selected image, and the criterion is the number of images or a GS threshold or a combination thereof.

18. The system according to claim 17, wherein suppressing the GS includes suppressing the GS based on the distance d between the image associated with the GS and the selected image.

19. The system according to claim 17, wherein the suppression includes applying a modification function to the GS with respect to the selected image.

Images