Method and system for automatically selecting endoscopic images

Abstract

The invention provides an automatic selection method and system for an endoscope image. The method comprises a plurality of steps executed by an arithmetic device, and the steps comprise: obtaining a plurality of endoscopic images and a plurality of position marks corresponding to the endoscopic images, each position mark being used for instructing an endoscope to shoot a part of a human organ, selecting a plurality of candidate images belonging to the same part from the plurality of endoscopic images according to the position marks, performing image segmentation according to image colors or default symptoms so as to divide each candidate image into a first region and a second region which are not overlapped with each other, and calculating a proportion according to the areas of the first region and the second region, and outputting at least one candidate image of which the proportion is greater than the threshold value as a selection result.

Description

Technical Field

[0001] This application relates to endoscopic images, and more particularly to an automatic method and system for selecting endoscopic images. Background Technology

[0002] Endoscopy is an important diagnostic and treatment technique primarily used to observe the condition of the human digestive tract for purposes including disease diagnosis, treatment, and screening. Through endoscopy, doctors can examine the health of the digestive tract, looking for signs of ulcers, tumors, or other lesions. During the examination, doctors will take photos of any abnormalities to record the results, facilitating subsequent diagnosis and treatment planning. Doctors will examine different sections of the digestive tract organs, such as the greater and lesser walls of the stomach, or the anterior and posterior sections of the intestines, and retain multiple images of each region for reference.

[0003] However, due to differences in examination habits and judgment methods among physicians, endoscopic images vary in terms of shooting angle, target distance, and imaging standards, resulting in significant differences in the image data retained from each examination. When new image analysis technologies are applied to endoscopic images, these inconsistencies make application difficult. Furthermore, the number of images from different sites also varies, making it impossible to determine which images should be used as the basis for analysis when conducting overall disease risk assessments; this requires physician discretion. This situation may lead to each physician's selection preferences influencing the assessment results, thereby reducing the stability and reliability of the assessment. Summary of the Invention

[0004] In view of this, this application proposes an automatic selection method and system for endoscopic images to improve the selection speed of endoscopic images and solve the problem of inconsistent selection criteria.

[0005] An automatic selection method for endoscopic images according to an embodiment of this application includes multiple steps executed by a computing device. These steps include: acquiring multiple endoscopic images and multiple location markers corresponding to these endoscopic images, each location marker indicating a part of a human organ captured by an endoscope; selecting multiple candidate images belonging to the same part from the multiple endoscopic images based on these location markers; performing image segmentation based on image color or default symptoms to distinguish each candidate image into a first region and a second region that do not overlap with each other; calculating a ratio based on the area of ​​the first region and the second region; and outputting at least one candidate image with a ratio greater than a threshold as the selection result.

[0006] An automatic selection system for endoscopic images according to an embodiment of this application includes a storage device and a computing device. The storage device stores multiple endoscopic images and multiple location markers corresponding to these endoscopic images, wherein each location marker indicates a part of a human organ captured endoscopically. The computing device is electrically connected to the storage device. The computing device selects multiple candidate images belonging to the same part from the multiple endoscopic images based on the multiple location markers, performs image segmentation based on image color or default symptom to distinguish each candidate image into a first region and a second region that do not overlap, calculates a ratio based on the area of ​​the first region and the second region, and outputs at least one candidate image with a ratio greater than a threshold as the selection result.

[0007] In summary, the automatic selection method and system for endoscopic images proposed in this application have the following advantages: standardizing the selection process for endoscopic images, improving the efficiency of model development and application based on endoscopic images because manual selection of endoscopic images is not required, and enabling rapid application to different evaluation targets, such as calculating the Corpus-predominant Gastritis Index (CGI) or the Gastric Intestinal Metaplasia Index (GIM) based on endoscopic images.

[0008] The above description of the contents of this application and the following description of the embodiments are used to demonstrate and explain the spirit and principles of this application, and to provide a further explanation of the scope of this patent application. Attached Figure Description

[0009] Figure 1 This is a block diagram of an automatic selection system for endoscopic images according to an embodiment of this application; and

[0010] Figure 2 This is a flowchart of an automatic selection method for endoscopic images according to an embodiment of this application.

[0011] Component designation explanation

[0012] 1. Storage device

[0013] 3. Computing device

[0014] Steps S1-S5 Detailed Implementation

[0015] The following detailed description of the features and advantages of this application in the embodiments is sufficient to enable anyone skilled in the art to understand the technical content of this application and implement it accordingly. Furthermore, based on the content disclosed in this specification, the scope of the patent application, and the drawings, anyone skilled in the art can easily understand the related objectives and advantages of this application. The following embodiments further illustrate the viewpoints of this application in detail, but are not intended to limit the scope of this application in any way.

[0016] Figure 1 This is a block architecture diagram of an automatic selection system for endoscopic images according to an embodiment of this application. Figure 1 As shown, this system includes a storage device 1 and a computing device 3.

[0017] Storage device 1 is used to store multiple endoscopic images and multiple location markers corresponding to these endoscopic images. Each location marker indicates a part of a human organ captured endoscopically, such as the greater wall or lesser wall of the stomach, or the anterior or posterior segment of the intestine. In one embodiment, these endoscopic images are tissue hemoglobin index (IHb) images obtained using a narrow-band image light source. In another embodiment, these endoscopic images are general endoscopic images obtained using white light. This application does not limit the type of light source used when capturing endoscopic images. Furthermore, when capturing different parts of the same human organ, the resulting multiple endoscopic images may contain overlapping portions. Therefore, a single endoscopic image can simultaneously include multiple location markers. The location markers can be manually assigned or automatically generated for each endoscopic image using a deep learning model trained on endoscopic images.

[0018] Storage device 1 may be memory in a computer, a hard disk, or an external storage device connected to the computer. In one embodiment, storage device 1 may be implemented using at least one of the following: flash memory, hard disk (HDD), solid-state drive (SSD), dynamic random access memory (DRAM), static random access memory (SRAM), or other non-volatile memory. However, this application is not limited to the above examples.

[0019] like Figure 1 As shown, the computing device 3 is electrically connected to the storage device 1. The computing device 3 is used to select multiple candidate images belonging to the same location from the multiple endoscopic images based on the multiple location markers, to perform image segmentation based on image color or default symptom to distinguish each candidate image into a first region and a second region that do not overlap with each other, to calculate a ratio based on the area of ​​the first region and the second region, and to output at least one candidate image with a ratio greater than a threshold as the selection result.

[0020] In one embodiment, the computing device 3 may be at least one of the following: a personal computer, a network server, a central processing unit (CPU), a graphics processing unit (GPU), a microcontroller (MCU), an application processor (AP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system-on-a-chip (SOC), a deep learning accelerator, or any electronic device including similar functions. This application does not limit the hardware type of the computing device 3.

[0021] Figure 2 This is a flowchart of an automatic selection method for endoscopic images according to an embodiment of the present application, including steps S1 to S5 executed by the computing device 3.

[0022] In step S1, the computing device 3 acquires multiple endoscopic images and multiple location markers corresponding to these endoscopic images, wherein each location marker is used to indicate a part of a human organ photographed with an endoscope.

[0023] In step S2, the computing device 3 selects multiple candidate images belonging to the same location from the multiple endoscopic images based on the multiple location markers.

[0024] In step S3, the computing device 3 performs image segmentation based on image color or default symptoms to divide each candidate image into a first region and a second region that do not overlap with each other. The number of regions can be determined according to the actual application, and this application does not limit it.

[0025] The first embodiment of image segmentation is based on image color: First, an IHb image is captured using a narrow-band imaging (NBI) light source to show the distribution of red blood cells. In IHb images, red typically represents areas with a high number of red blood cells (healthy areas), while green or blue represents areas with a low number of red blood cells (areas of inflammation or other conditions). Therefore, this embodiment uses different color blocks in the IHb image as the basis for image segmentation, setting specific thresholds to distinguish between reddish and bluish / greenish blocks. In other words, image segmentation involves distinguishing a first region and a second region based on different colors in the IHb image.

[0026] The second embodiment of image segmentation is based on image color: First, multiple endoscopic images are captured using white light. Then, the computing device 3 executes an image processing algorithm to convert these images into IHb images. In this embodiment, multispectral analysis or machine learning techniques can be used to estimate the hemoglobin concentration in the tissue, generating an IHb-like image effect. This allows for the extraction of hemoglobin information from the original image and the generation of an IHb image even without a dedicated light source. The image segmentation formula is the same as in the first embodiment: the first and second regions are distinguished based on different colors in the IHb image.

[0027] The third embodiment of image segmentation is segmentation based on preset symptoms: it is known that deep image segmentation models can be applied to endoscopic images (Nien, Chu-Min, et al. "Criss-cross attention based multi-level fusion network for gastric intestinal metaplasia segmentation." MICCAI Workshop on Imaging Systems for GI Endoscopy. Cham: Springer Nature Switzerland, 2022). By using different data annotations, a model can be trained to segment endoscopic images according to different symptoms. Therefore, in this embodiment, the computing device 3 trains a deep learning model to identify the symptom location from each endoscopic image. Image segmentation includes: for each candidate image, distinguishing a first region and a second region based on the symptom location and parts not belonging to the symptom.

[0028] In step S4, the computing device 3 calculates a ratio based on the areas of the first region and the second region. In one embodiment, the computing device 3 uses the number of pixels in the region as the region area. Regarding the ratio calculation, for example, assuming the area of ​​the healthy region is A and the area of ​​the inflamed region is B, the computing device 3 can calculate A / B or B / A as the ratio. Assuming the segmented region type is greater than two, such as A, B, and C, the computing device 3 can calculate, for example, A / (B+C) or (A+C) / B as the ratio.

[0029] In step S5, the computing device 3 outputs at least one candidate image whose ratio is greater than a threshold as the selection result. The threshold can be the maximum, minimum, average, or a preset value among all ratios calculated in step S4. The phrase "greater than..." in step S5 can also be changed to "less than..." or "between..." depending on the application example. Furthermore, this application does not limit the number of images included in the selection result; it can be more than one image.

[0030] exist Figure 2 The flowchart in steps S3 to S5 demonstrates how to process multiple candidate images of the same human organ site. In practical applications, this process can be performed simultaneously or sequentially on endoscopes of different sites to select representative images for each site.

[0031] Two practical application examples of this application are as follows:

[0032] Example 1: Calculation of the Corpus-predominant Gastritis Index (CGI): This index assesses whether a patient belongs to a high-risk group for gastric cancer using endoscopic images of the stomach from different locations. Currently, physicians manually select images for evaluation, while this application can automatically select images using specific color ratios in IHb images, avoiding human bias and improving efficiency.

[0033] Example 2: Calculation of the Gastric Intestinal Metaplasia (GIM) Index: This index uses NBI images of the stomach from different locations as input to assess the overall severity of gastric mucosal intestinal metaplasia, and thus serves as an assessment standard for high-risk groups for gastric cancer. This application can score based on the proportion of gastric mucosal intestinal metaplasia area in a single image, selecting the image with the highest proportion as the final assessment image, enabling physicians to assess the most severe cases and make appropriate treatments.

[0034] In summary, the automatic selection method and system for endoscopic images proposed in this application have the following advantages: standardizing the selection process of endoscopic images, improving the efficiency of model development and application based on endoscopic images because no manual selection of endoscopic images is required, and enabling rapid application to different evaluation objectives, such as calculating CGI or GIM based on endoscopic images.

[0035] While this application discloses the above-described embodiments, it is not intended to limit the scope of this application. Any modifications and refinements made without departing from the spirit and scope of this application are within the scope of patent protection of this application. For the scope of protection defined in this application, please refer to the appended patent claims.

Claims

1. An automatic selection method for endoscopic images, characterized in that, This includes multiple steps executed by a computing device, said multiple steps including: Acquire multiple endoscopic images and multiple location markers corresponding to the multiple endoscopic images, each of the multiple location markers being used to indicate a part of a human organ as captured by an endoscope; Based on the multiple location markers, select multiple candidate images belonging to the same location from the multiple endoscopic images; Image segmentation is performed based on image color or default symptoms to divide each of the multiple candidate images into a first region and a second region that do not overlap with each other. Calculate a ratio based on the areas of the first region and the second region; and Output at least one of the plurality of candidate images whose proportion is greater than a threshold as the selection result.

2. The automatic selection method for endoscopic images according to claim 1, characterized in that, Also includes: The plurality of endoscopic images are captured using a narrow-band image light source, wherein the plurality of endoscopic images are tissue hemoglobin index images; The image segmentation based on the image color or the default symptom includes: distinguishing the first region and the second region based on different colors in the tissue hemoglobin index image.

3. The automatic selection method for endoscopic images according to claim 1, characterized in that, Also includes: Images of the multiple endoscopes were captured using white light; and An image processing algorithm is executed to convert the types of the plurality of endoscopic images into tissue hemoglobin index images; The image segmentation based on the image color or the default symptom includes: distinguishing the first region and the second region based on different colors in the tissue hemoglobin index image.

4. The automatic selection method for endoscopic images according to claim 1, characterized in that, Also includes: Train a deep learning model to identify a symptom site from each of the plurality of endoscopic images; The image segmentation based on the image color or the default symptom includes: for each of the plurality of candidate images, distinguishing the first region and the second region based on the symptom location and the part that does not belong to the symptom.

5. An automatic selection system for endoscopic images, characterized in that, include: A storage device for storing a plurality of endoscopic images and a plurality of location markers corresponding to the plurality of endoscopic images, wherein each of the plurality of location markers is used to indicate a part of a human organ as captured by an endoscope; and A computing device electrically connected to the storage device is configured to select multiple candidate images belonging to the same location from multiple endoscopic images based on multiple location markers, perform image segmentation based on image color or default symptom to divide each of the multiple candidate images into a first region and a second region that do not overlap with each other, calculate a ratio based on the area of ​​the first region and the second region, and output at least one of the multiple candidate images whose ratio is greater than a threshold as the selection result.

6. The automatic selection system for endoscopic images according to claim 5, characterized in that, The plurality of endoscopic images are tissue hemoglobin index images obtained by narrow-band image light source, and the computing device performs image segmentation based on the image color or the default symptom, distinguishing the first region and the second region based on different colors in the tissue hemoglobin index image.

7. The automatic selection system for endoscopic images according to claim 5, characterized in that, The plurality of endoscopic images are captured with white light. The computing device is also used to execute an image processing algorithm to convert the type of the plurality of images into tissue hemoglobin index images. The computing device performs image segmentation based on the image color or the default symptom and distinguishes the first region and the second region based on different colors in the tissue hemoglobin index images.

8. The automatic selection system for endoscopic images according to claim 5, characterized in that, The storage device is also used to store a deep learning model, and the computing device is also used to run the deep learning model to find a symptom site from each of the plurality of endoscopic images, and the computing device performs the image segmentation based on the image color or the default symptom, distinguishing the first region and the second region for each of the plurality of candidate images based on the symptom site and the part that does not belong to the symptom.

Images