Endoscopic inspection assistance device, endoscopic inspection assistance method, and recording medium

Abstract

In this endoscopic inspection assistance device, an image acquisition means acquires an endoscopic image. A gaze point detection means detects a gaze point of a user with respect to the endoscopic image. A mask image generation means generates a mask image on the basis of the gaze point. A first lesion region estimation means estimates a first lesion region in the endoscopic image on the basis of the endoscopic image and the mask image. A first cut-out means generates a first cut-out image obtained by cutting out the first lesion region from the endoscopic image. The endoscopic inspection assistance device is able to assist decision making of users in the medical field.

Description

Endoscopic examination support device, endoscopic examination support method, and recording medium

[0001] The present disclosure relates to assisting endoscopy.

[0002] Artificial intelligence (AI) that detects and identifies lesions from image data captured during endoscopic examinations is becoming increasingly widespread. Patent Document 1 proposes an examination support device that detects and identifies lesions only in an area where the gaze of the endoscope operator is focused, in order to perform endoscopic examinations efficiently and accurately.

[0003] International Publication No. WO2019-087790

[0004] However, even with Patent Document 1, it is not necessarily possible to accurately diagnose a lesion.

[0005] An object of the present disclosure is to provide an endoscopic examination support device that is capable of accurately diagnosing lesions during endoscopic examination.

[0006] In one aspect of the present disclosure, an endoscopic examination support device comprises: an image acquisition means for acquiring an endoscopic image; a gaze point detection means for detecting a user's gaze point on the endoscopic image; a mask image generation means for generating a mask image based on the gaze point; a first lesion area estimation means for estimating a first lesion area in the endoscopic image based on the endoscopic image and the mask image; and a first cut-out means for generating a first cut-out image by cutting out the first lesion area from the endoscopic image.

[0007] In another aspect of the present disclosure, a method for supporting endoscopic examination includes performing image acquisition to acquire an endoscopic image, performing gaze point detection to detect a user's gaze point on the endoscopic image, performing mask image generation to generate a mask image based on the gaze point, performing first lesion area estimation to estimate a first lesion area in the endoscopic image based on the endoscopic image and the mask image, and performing first cropping to generate a first cropped image by cropping the first lesion area from the endoscopic image.

[0008] In yet another aspect of the present disclosure, a recording medium records a program that causes a computer to perform the following processes: image acquisition to acquire an endoscopic image; gaze point detection to detect a user's gaze point on the endoscopic image; mask image generation to generate a mask image based on the gaze point; first lesion area estimation to estimate a first lesion area in the endoscopic image based on the endoscopic image and the mask image; and first cutting to generate a first cut-out image by cutting out the first lesion area from the endoscopic image.

[0009] According to the present disclosure, lesions can be diagnosed with high accuracy in endoscopic examination.

[0010] 1 is a block diagram showing a schematic configuration of an endoscopic examination system according to the present disclosure. FIG. 2 is a block diagram showing a hardware configuration of an endoscopic examination support device according to the present disclosure. FIG. 3 is a block diagram showing a functional configuration of an endoscopic examination support device according to the present disclosure. FIG. 4 is a flowchart of processing by an endoscopic examination support device according to the present disclosure. FIG. 5 is a diagram showing an example of an evaluation screen for a lesion area. FIG. 6 is a diagram showing an example of an editing screen for a lesion area. FIG. 7 is a diagram showing an example of a confirmation screen and a diagnosis result. FIG. 8 is a block diagram showing a functional configuration of another endoscopic examination support device according to the present disclosure. FIG. 9 is a diagram showing an example of a selection screen for a lesion area. FIG. 10 is a flowchart of processing by another endoscopic examination support device according to the present disclosure. FIG. 11 is a block diagram showing a functional configuration of another endoscopic examination support device according to the present disclosure. FIG. 12 is a flowchart of processing by another endoscopic examination support device according to the present disclosure.

[0011] Preferred embodiments of the present disclosure will now be described with reference to the drawings. <First Embodiment> [Overall Configuration] FIG. 1 shows a schematic configuration of an endoscopic examination system 100. When a lesion area is detected during an examination using an endoscope, the endoscopic examination system 100 diagnoses the lesion area using AI and displays the diagnosis result. In particular, the endoscopic examination system 100 of this embodiment estimates the lesion area based on the line of sight of the doctor directed toward the display device that displays the endoscopic image. The endoscopic examination system 100 then extracts only the lesion area from the endoscopic image and diagnoses the lesion area using AI. In this way, the endoscopic examination system 100 of this embodiment can use the lesion area from which unnecessary background has been removed for AI diagnosis, thereby enabling accurate lesion diagnosis.

[0012] As shown in FIG. 1 , the endoscopic examination system 100 mainly includes an endoscopic examination support device 1, a display device 2, an endoscope 3 connected to the endoscopic examination support device 1, and an eye tracking device 4.

[0013] The endoscopic examination support device 1 acquires from the endoscope 3 an image (i.e., a moving image; hereinafter, also referred to as "endoscopic image Ic") captured by the endoscope scope 3 during an endoscopic examination, and displays on the display device 2 display data for confirmation by the examiner (doctor) performing the endoscopic examination. Specifically, the endoscopic examination support device 1 acquires a moving image of the inside of an organ captured by the endoscope scope 3 during the endoscopic examination as the endoscopic image Ic. Furthermore, if the doctor finds a lesion during the endoscopic examination, he or she operates the endoscope scope 3 to input an instruction to capture the lesion position. The endoscopic examination support device 1 generates an endoscopic image that captures the lesion position based on the doctor's imaging instruction. Specifically, the endoscopic examination support device 1 generates a still endoscopic image from the endoscopic image Ic, which is a moving image, based on the doctor's imaging instruction.

[0014] The display device 2 is a display or the like that displays a predetermined image based on a display signal supplied from the endoscopic examination support device 1 .

[0015] The endoscope 3 mainly comprises an operating unit 36 ​​that allows the doctor to input instructions such as air supply, water supply, angle adjustment, and photography instructions, a flexible shaft 37 that is inserted into the subject's organ to be examined, a tip 38 that incorporates a photography unit such as a miniature imaging element, and a connection unit 39 for connecting to the endoscopic examination support device 1.

[0016] The gaze tracking device 4 includes, for example, a near-infrared LED light source and an infrared camera, and is provided so as to be able to capture an image in front of the display device 2. The near-infrared LED light source is a light source for illuminating the user's eyeball. The infrared camera captures an image of a predetermined range including the user's eyeball at a predetermined number of frames per unit time (for example, 60 frames per second). The gaze tracking device 4 outputs image data (hereinafter also referred to as "eyeball image") captured by the infrared camera to the endoscopic examination support device 1, associating the image data with timestamp information and the like.

[0017] The following explanation will be mainly based on the processing in an endoscopic examination of the large intestine, but the subject of examination is not limited to the large intestine, and may be the digestive tract (digestive organs) such as the stomach, esophagus, small intestine, and duodenum.

[0018] 2 shows the hardware configuration of the endoscopic examination support device 1. The endoscopic examination support device 1 mainly includes a processor 11, a memory 12, an interface 13, an input unit 14, a light source unit 15, a sound output unit 16, and a database (hereinafter referred to as "DB") 17. These elements are connected via a data bus 19.

[0019] The processor 11 performs predetermined processing by executing programs stored in the memory 12. The processor 11 is a processor such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a TPU (Tensor Processing Unit). The processor 11 may be composed of multiple processors. The processor 11 is an example of a computer.

[0020] The memory 12 is composed of various volatile memories used as working memories, such as RAM (Random Access Memory) and ROM (Read Only Memory), and nonvolatile memories that store information necessary for processing by the endoscopic examination support device 1. The memory 12 may include an external storage device such as a hard disk connected to or built into the endoscopic examination support device 1, or may include a storage medium such as a removable flash memory or disk medium. The memory 12 stores programs for the endoscopic examination support device 1 to execute each process in this embodiment.

[0021] The memory 12 also temporarily stores a series of endoscopic videos Ic captured by the endoscope 3 during an endoscopic examination under the control of the processor 11. The memory 12 also temporarily stores endoscopic images captured during an endoscopic examination based on imaging instructions from a doctor. These images are stored in the memory 12 in association with, for example, the subject's identification information (e.g., patient ID), timestamp information, and the like.

[0022] The interface 13 performs an interface operation between the endoscopic examination assistance device 1 and an external device. For example, the interface 13 supplies the display data Id generated by the processor 11 to the display device 2. The interface 13 also supplies illumination light generated by the light source unit 15 to the endoscope 3. The interface 13 also supplies an electrical signal indicating an endoscopic video Ic supplied from the endoscope 3 to the processor 11. The interface 13 also supplies an electrical signal indicating an eyeball image supplied from the gaze tracking device 4 to the processor 11. The interface 13 may be a communication interface such as a network adapter for wired or wireless communication with an external device, or may be a hardware interface compliant with USB (Universal Serial Bus), SATA (Serial AT Attachment), or the like.

[0023] The input unit 14 generates an input signal based on an operation by a doctor. The input unit 14 is, for example, a button, a touch panel, a remote controller, or a voice input device. The light source unit 15 generates light to be supplied to the tip 38 of the endoscope 3. The light source unit 15 may also incorporate a pump or the like for sending water or air to be supplied to the endoscope 3. The sound output unit 16 outputs sound based on the control of the processor 11.

[0024] The DB 17 stores machine learning models such as a segmentation model, a lesion classification model, and a lesion detection model, which will be described later. The DB 17 may include an external storage device such as a hard disk connected to or built into the endoscopic examination support device 1, or may include a storage medium such as a removable flash memory. Instead of providing the DB 17 within the endoscopic examination system 100, the DB 17 may be provided on an external server or the like, and data may be obtained from the server via communication.

[0025] 3 is a block diagram showing the functional configuration of the endoscopic examination support device 1 according to the first embodiment. The endoscopic examination support device 1 functionally includes a gaze position identifying unit 111, a coordinate conversion unit 112, a mask image generating unit 113, a lesion area analyzing unit 114, a background removing unit 115, a lesion classification unit 116, and an output unit 117.

[0026] An endoscopic video Ic is input to the endoscopic examination support device 1 from the endoscope 3. The endoscopic examination support device 1 then generates an endoscopic image from the endoscopic video Ic based on imaging instructions from a doctor. The endoscopic image is input to a gaze position identifying unit 111, a lesion area analyzing unit 114, and a background removing unit 115. An eyeball image is also input to the endoscopic examination support device 1 from the gaze tracking device 4. The eyeball image is input to the gaze position identifying unit 111.

[0027] The gaze position identifying unit 111 identifies the gaze position (hereinafter also referred to as the "gazing point") of the doctor on the display device 2 based on the eyeball image. It is assumed that display data including an endoscopic image is displayed on the display device 2. It is also assumed that the doctor is gazing at a region in the endoscopic image that he wishes to analyze.

[0028] The gaze position identifying unit 111 can identify the gaze point using, for example, a corneal reflex method. Specifically, the gaze position identifying unit 111 recognizes the reflection point of the near-infrared LED on the cornea from the eyeball image. The gaze position identifying unit 111 also recognizes the position of the pupil in the eye from the eyeball image. The gaze position identifying unit 111 then detects the doctor's gaze direction from the positional relationship between the reflection point and the pupil. The gaze position identifying unit 111 identifies the position coordinates (hereinafter also referred to as "gaze point coordinates") of the doctor's gaze point in a coordinate system based on the display device 2 (hereinafter also referred to as "display device coordinate system") based on the gaze direction. The gaze position identifying unit 111 outputs the gaze point coordinates in the display device coordinate system and information regarding the time when the gaze point was measured to the coordinate conversion unit 112.

[0029] The coordinate conversion unit 112 converts the gaze point coordinates in the display device coordinate system into gaze point coordinates in a coordinate system based on the endoscopic image (hereinafter also referred to as the "endoscopic coordinate system"). Note that the relationship between the display device coordinate system and the endoscope coordinate system is predetermined, and mutual coordinate conversion between the display device coordinate system and the endoscope coordinate system is possible. For example, a conversion formula or conversion table from the display device coordinate system to the endoscope coordinate system is prepared in advance, and the coordinate conversion unit 112 performs coordinate conversion using the conversion formula or conversion table. The coordinate conversion unit 112 outputs the gaze point coordinates in the endoscope coordinate system to the mask image generation unit 113.

[0030] The mask image generation unit 113 generates a mask image of the same size as the endoscopic image based on the gaze point coordinates in the endoscopic coordinate system. For example, the mask image generation unit 113 generates a mask image in which the area gazed upon by the doctor is indicated by black pixels and the other areas are indicated by white pixels by pointing each gaze point indicated by the gaze point coordinates on a frame image of the same size as the endoscopic image. Examples of binarization processing by the mask image generation unit 113 are shown below. (Example 1) The mask image generation unit 113 generates a mask image by converting the gaze point with the longest dwell time within a predetermined number of seconds into a black pixel value and converting the other areas into white pixel values. (Example 2) The mask image generation unit 113 generates a mask image by converting the gaze point with a dwell time equal to or longer than a predetermined threshold into a black pixel value and converting the other areas into white pixel values. (Example 3) The mask image generation unit 113 generates a mask image by converting all gaze points that exist within a predetermined number of seconds into black pixel values ​​and converting other areas into white pixel values ​​for each gaze point. (Example 4) The mask image generation unit 113 generates a mask image by converting positions and areas where the gaze point has been measured a predetermined number of times or more into black pixel values ​​and converting other areas into white pixel values ​​for each gaze point.

[0031] The mask image generated by the mask image generating unit 113 is not limited to a binary image, and may be an image (e.g., a grayscale image) that represents the dwell time of the gaze point using shades of a single color. Furthermore, the mask image generating unit 113 may generate a mask image by representing the gaze point estimated in one measurement as a circle of a predetermined size.

[0032] The mask image generating unit 113 outputs the generated mask image to the lesion area analyzing unit 114 .

[0033] The lesion region analysis unit 114 estimates a lesion region in the endoscopic image based on the endoscopic image and the mask image. Specifically, the lesion region analysis unit 114 inputs the endoscopic image and the mask image into a segmentation model, thereby segmenting a region in the endoscopic image identified by the mask image (hereinafter also referred to as a "specific region"). The lesion region analysis unit 114 then estimates the segmented specific region as the lesion region. The segmentation model may be, for example, a model constructed by performing machine learning using training data in which a set of an image and a mask image is used as an input image and the result of segmenting the specific region identified by the mask image is used as the correct answer. Alternatively, a segmentation model such as SAM (Segment Anything Model) published by Meta, Inc. may be used, which learns a large number of general images and medical images and allows input by specifying a target region as a mask image, known as a visual prompt. The lesion region analysis unit 114 outputs the segmentation results to the background removal unit 115 .

[0034] The background remover 115 extracts only the segmented lesion area from the endoscopic image. This generates an image of the lesion area from which unnecessary background information has been removed. The background remover 115 outputs the extracted image of the lesion area to the lesion classification unit 116.

[0035] The lesion classification unit 116 diagnoses the lesion area using an image of the extracted lesion area. Examples of diagnosis include lesion differential diagnosis, such as distinguishing between tumors and non-tumors and determining the depth of invasion, and measuring the size of the lesion. Specifically, the lesion classification unit 116 diagnoses the lesion area using a pre-prepared image recognition model. This image recognition model is a machine learning model that is trained in advance to diagnose the lesion area using an endoscopic image containing the lesion area as input, and is hereinafter also referred to as a "lesion classification model." The internal configuration of the machine learning model is arbitrary, but can be configured, for example, by a convolutional neural network (CNN).

[0036] The lesion classification unit 116 outputs the diagnosis result to the output unit 117. The output unit 117 outputs the diagnosis result to the display device 2.

[0037] In this way, the endoscopic examination support device 1 can generate an image of a lesion area from which unnecessary background information has been removed, thereby improving the accuracy of diagnosis using a lesion classification model.

[0038] In the above configuration, the gaze position identification unit 111 and the coordinate conversion unit 112 are examples of an image acquisition means and a gaze point detection means, the mask image generation unit 113 is an example of a mask image generation means, the lesion area analysis unit 114 is an example of a first lesion area estimation means, the background removal unit 115 is an example of a first cutting means, the lesion classification unit 116 is an example of a diagnostic means, and the output unit 117 is an example of a diagnostic result output means.

[0039] [Endoscopic examination support processing] Next, the endoscopic examination support processing for performing the above-mentioned processing will be described. Fig. 4 is a flowchart of the endoscopic examination support processing by the endoscopic examination support device 1. This processing is realized by the processor 11 shown in Fig. 2 executing a program prepared in advance and operating as each element shown in Fig. 3.

[0040] An endoscopic examination support device 1 receives an endoscopic video Ic from an endoscope 3. The endoscopic examination support device 1 generates an endoscopic image from the endoscopic video Ic based on imaging instructions from a doctor. An eyeball image is also received as input from an eye tracking device 4.

[0041] The endoscopic image is input to the gaze position identifying unit 111, the lesion area analyzing unit 114, and the background removing unit 115, and the eyeball image is input to the gaze position identifying unit 111 (step S111).

[0042] Next, the gaze position identifying unit 111 identifies the doctor's gaze point coordinate in the display device coordinate system based on the eyeball image (step S112). The gaze position identifying unit 111 can identify the doctor's gaze point coordinate in the display device coordinate system by using, for example, a corneal reflex method. Next, the coordinate conversion unit 112 converts the gaze point coordinate in the display device coordinate system into the gaze point coordinate in the endoscope coordinate system (step S113).

[0043] Next, the mask image generator 113 generates a mask image based on the gaze point coordinates in the endoscope coordinate system (step S114). For example, the mask image generator 113 generates a mask image by pointing each gaze point indicated by the gaze point coordinates on a frame image of the same size as the endoscope image, where the area gazed upon by the doctor is displayed as a binary image with black pixels and the other areas as a binary image with white pixels. Note that the mask image is not limited to a binary image, and may be an image that represents the time spent at the gaze point using a single shade of color. Alternatively, the mask image generator 113 may generate a mask image by representing the gaze point estimated in a single measurement as a circle of a predetermined size.

[0044] Next, the lesion region analysis unit 114 estimates a lesion region in the endoscopic image based on the endoscopic image and the mask image (step S115). Specifically, the lesion region analysis unit 114 segments a specific region in the endoscopic image by inputting the endoscopic image and the mask image into a segmentation model. The lesion region analysis unit 114 then estimates the segmented specific region as a lesion region. Next, the background removal unit 115 cuts out the lesion region from the endoscopic image based on the segmentation result (step S116).

[0045] Next, lesion classification unit 116 diagnoses the lesion area based on the image of the extracted lesion area (step S117). Specifically, lesion classification unit 116 diagnoses the lesion area using a lesion classification model. Next, output unit 117 outputs the diagnosis result to display device 2 (step S118). Then, the process ends.

[0046] [Modifications] Next, a description will be given of modifications of the first embodiment. The following modifications can be applied to the first embodiment in appropriate combinations.

[0047] (Modification 1) The endoscopic examination support device 1 may present an evaluation screen for evaluating the estimated lesion area to the doctor.

[0048] FIG. 5 shows an example of a lesion area evaluation screen. In FIG. 5, the display area of ​​the display device 2 includes an endoscopic image 21, a lesion area 22, a cropped image 23, and a mask image 24. The endoscopic image 21 is an endoscopic image generated based on an imaging instruction from a doctor. The lesion area 22 indicates a lesion area estimated by the lesion area analysis unit 114. The cropped image 23 indicates an image of the lesion area cropped by the background removal unit 115. In FIG. 5, the cropped image 23 is an image cropped from the lesion area 22. The mask image 24 indicates a mask image generated by the mask image generation unit 113. In FIG. 5, the mask image generation unit 113 generates the mask image by representing the doctor's gaze point with a gray circle. Note that the endoscopic examination support device 1 does not necessarily include the mask image 24 in the lesion area evaluation screen.

[0049] If two or more lesion areas are estimated for one endoscopic image, two or more lesion areas and their cut-out images may be displayed simultaneously depending on the display space of the display device 2. If two or more lesion areas are estimated for one endoscopic image and it is not possible to display all of the lesion areas and cut-out images, the lesion areas to be displayed may be determined depending on the importance of the lesion areas.

[0050] The doctor looks at the display as shown in Figure 5 and evaluates whether the lesion area 22 and the extracted image 23 are appropriate. Then, the doctor inputs the evaluation into the endoscopic examination support device 1. For example, the doctor can input the evaluation by voice into the endoscopic examination support device 1 via the input unit 14. In this case, the endoscopic examination support device 1 has a voice recognition function and acquires the evaluation by voice recognition of the doctor's input voice.

[0051] The endoscopic examination support device 1 can determine whether or not to diagnose the lesion area based on the doctor's evaluation. If the doctor evaluates the estimated lesion area as appropriate, the endoscopic examination support device 1 performs a diagnosis on the estimated lesion area. On the other hand, if the doctor evaluates the estimated lesion area as inappropriate, the endoscopic examination support device 1 does not perform a diagnosis on the estimated lesion area and terminates the process. This allows the endoscopic examination support device 1 to use only lesion areas evaluated as appropriate by the doctor for diagnosis, thereby improving the accuracy of the diagnosis.

[0052] The process of generating the evaluation screen and the process of determining a diagnosis based on the doctor's evaluation can be executed by, for example, the lesion classification unit 116 of the endoscopic examination support device 1. In this case, the lesion classification unit 116 is an example of an evaluation screen data output means and an evaluation acquisition means.

[0053] (Variation 2) The endoscopic examination support device 1 may present the doctor with an editing screen for editing the estimated lesion area. The lesion area detected by the endoscopic examination support device 1 may contain errors. For example, the size of the lesion area detected by the endoscopic examination support device 1 may differ from the size of the lesion shown in the endoscopic image. Furthermore, the position of the lesion area detected by the endoscopic examination support device 1 may deviate from the position of the lesion shown in the endoscopic image. In such cases, it is preferable that the doctor be able to correct the size and position of the lesion area before diagnosing the lesion area. From this perspective, Variation 2 enables the doctor to edit the lesion area.

[0054] Fig. 6 shows an example of a lesion area editing screen. In Fig. 6, the display area of ​​display device 2 includes endoscopic image 25, lesion area 26, and cropped image 27. Endoscopic image 25 is an endoscopic image generated based on imaging instructions from a doctor. Lesion area 26 indicates a lesion area estimated by lesion area analysis unit 114. Cropped image 27 is an image cropped from lesion area 26.

[0055] The endoscopic examination support device 1 accepts vocal instructions from the doctor via the input unit 14. The endoscopic examination support device 1 then edits the lesion area 26 in accordance with the doctor's instructions. For example, when the doctor gives the vocal instruction "make it smaller," the endoscopic examination support device 1 reduces the lesion area 26. Lesion area 26a indicates the reduced lesion area 26. Furthermore, when the doctor gives the vocal instruction "to the right," the endoscopic examination support device 1 translates the lesion area 26 to the right. Lesion area 26b indicates the lesion area 26 translated to the right. Note that the extracted image 27 displays an image of the edited lesion area. This allows the endoscopic examination support device 1 to use the lesion area edited by the doctor for diagnosis, thereby improving the accuracy of diagnosis.

[0056] The process of generating the edit screen and the edit process based on instructions from a doctor can be executed by, for example, the background removal unit 115 of the endoscopic examination support device 1. In this case, the background removal unit 115 is an example of an edit screen data output means and an edit operation receiving means.

[0057] (Variation 3) In the first embodiment, once an endoscopic image is generated based on an imaging instruction from a doctor, the endoscopic examination support process is executed to the end. Alternatively, the endoscopic examination support device 1 may stop the endoscopic examination support process midway and ask the doctor whether or not to continue the endoscopic examination support process.

[0058] For example, the endoscopic examination support device 1 may present the doctor with a confirmation screen for confirming whether or not to execute diagnosis by the lesion classification unit 116 .

[0059] Fig. 7(A) shows an example of the confirmation screen. In Fig. 7(A), the display area of ​​the display device 2 includes an endoscopic image 31, a lesion area 32, a cropped image 33, and a reject area 34. The endoscopic image 31 is an endoscopic image generated based on an imaging instruction from a doctor. The lesion area 32 indicates the lesion area estimated by the lesion area analysis unit 114. The cropped image 33 indicates an image of the lesion area cropped by the background removal unit 115.

[0060] The reject area 34 is an area for controlling the execution of diagnosis. For example, a doctor checks the lesion area 32 and the extracted image 33, and if he or she decides not to execute a diagnosis, he or she gazes at the reject area 34. When the endoscopic examination support device 1 detects that the doctor has been gazing at the reject area 34 for a predetermined time TH1 or more, the endoscopic examination support device 1 terminates the process without executing a diagnosis by the lesion classification unit 116. On the other hand, if the doctor does not gaze at the reject area 34, the endoscopic examination support device 1 causes the lesion classification unit 116 to execute a diagnosis after a predetermined time TH2 has elapsed. The endoscopic examination support device 1 then displays the diagnosis result as shown in FIG. 7B on the display device 2. The doctor may also instruct the endoscopic examination support device 1 whether or not to execute a diagnosis by voice input.

[0061] 7B shows an example of the display of the diagnosis result. In FIG. 7B, the display area of ​​the display device 2 includes an endoscopic image 31, a lesion region 32, a cut-out image 33, and a diagnosis result 35. The diagnosis result 35 indicates the diagnosis result of the lesion region 32 by the lesion classification unit 116.

[0062] The process of generating the confirmation screen and the process of controlling the execution of the diagnosis can be performed, for example, by the lesion classification unit 116 of the endoscopic examination support device 1. In this case, the lesion classification unit 116 is an example of a confirmation screen data output means and an answer acquisition means.

[0063] Second Embodiment Next, a second embodiment will be described. The endoscopic examination system of the second embodiment includes an additional lesion area analyzer. A doctor can select a lesion area to be used for diagnosis from the lesion areas analyzed by each lesion area analyzer. Note that the system configuration and hardware configuration are the same as those of the first embodiment, and therefore description thereof will be omitted.

[0064] 8 is a block diagram showing the functional configuration of an endoscopic examination support device 1a according to the second embodiment. Functionally, the endoscopic examination support device 1a includes a gaze position identification unit 211, a coordinate conversion unit 212, a mask image generation unit 213, a first lesion area analysis unit 214, a background removal unit 215, a second lesion area analysis unit 216, a lesion area selection unit 217, a lesion classification unit 218, and an output unit 219. That is, the endoscopic examination support device 1a includes a second lesion analysis unit 216 in addition to the first lesion analysis unit 214 corresponding to the lesion area analysis unit 114 of the first embodiment.

[0065] The gaze position identification unit 211, coordinate conversion unit 212, mask image generation unit 213, first lesion area analysis unit 214, lesion classification unit 218, and output unit 219 have the same configuration and operate in the same manner as the gaze position identification unit 111, coordinate conversion unit 112, mask image generation unit 113, lesion area analysis unit 114, lesion classification unit 116, and output unit 117 of the endoscopic examination support device 1, and will not be described again.

[0066] The background removal unit 215 receives the segmentation result from the first lesion area analysis unit 214. The background removal unit 215 extracts only the segmented lesion area from the endoscopic image. The background removal unit 215 outputs the segmentation result and an image of the extracted lesion area to the lesion area selection unit 217.

[0067] An endoscopic image is input to second lesion area analysis unit 216. Second lesion area analysis unit 216 detects a lesion area contained in the endoscopic image using a pre-prepared image recognition model or the like. This image recognition model is a machine learning model that receives an endoscopic image as input and is pre-trained to estimate a lesion area contained in the endoscopic image, and is hereinafter also referred to as a "lesion detection model." The internal configuration of the machine learning model is arbitrary, but can be configured, for example, by a CNN or the like. When a lesion area is detected, second lesion area analysis unit 216 draws a rectangle surrounding the lesion area on the endoscopic image and outputs the rectangle to lesion area selection unit 217.

[0068] The lesion area estimated by first lesion area analysis unit 214 will hereinafter also be referred to as the "first lesion area." The lesion area detected by second lesion area analysis unit 216 will hereinafter also be referred to as the "second lesion area."

[0069] Lesion area selection unit 217 generates a selection screen for selecting a lesion area to be used for diagnosis and outputs it to display device 2. The doctor selects the lesion area to be used for diagnosis from the multiple lesion areas included in the selection screen. Then, lesion area selection unit 217 outputs the lesion area selected by the doctor to lesion classification unit 218.

[0070] Figure 9 shows an example of the display of a selection screen generated by lesion area selection unit 217. Figure 9 includes endoscopic image 41, lesion areas 42 to 44, and cropped images 42a to 44a. Endoscopic image 41 is an endoscopic image generated based on imaging instructions from a doctor. Lesion area 42 is a first lesion area. Cropped image 42a is an image cropped from lesion area 42. Lesion area 43 and lesion area 44 are second lesion areas. Cropped images 43a and 44a are images cropped from lesion area 43 and lesion area 44, respectively.

[0071] 9, the doctor selects which of the cut-out images 42a to 44a to use for diagnosis. For example, the doctor gazes at the cut-out image to be used for diagnosis from among the cut-out images 42a to 44a. The lesion area selector 217 outputs the cut-out image that the doctor gazes at for a predetermined time TH3 or more to the lesion classifier 218. The doctor may also select the cut-out image to be used for diagnosis by voice input.

[0072] 9 shows three lesion areas and cut-out images, but if four or more lesion areas are estimated for one endoscopic image, four or more lesion areas and their cut-out images may be displayed simultaneously depending on the display space of the display device 2. Furthermore, if four or more lesion areas are estimated for one endoscopic image and it is not possible to display all of the lesion areas and cut-out images, the lesion areas to be displayed may be determined depending on the importance of the lesion areas.

[0073] In the above configuration, the second lesion area analysis unit 216 is an example of a second lesion area estimation means and a second extraction means, and the lesion area selection unit 217 is an example of a lesion area selection screen data output means and a selection operation reception means.

[0074] [Endoscopic Examination Support Processing] Next, the endoscopic examination support processing that performs the above-mentioned processing will be described. Fig. 10 is a flowchart of the endoscopic examination support processing performed by the endoscopic examination support device 1a. This processing is realized by the processor 11 shown in Fig. 2 executing a program prepared in advance and operating as each element shown in Fig. 8. Note that the processing of steps S211 to S216 is similar to the processing of steps S111 to S116 of the endoscopic examination support processing shown in Fig. 4, and therefore description thereof will be omitted.

[0075] The endoscopic image is input to second lesion region analysis unit 216. Second lesion region analysis unit 216 uses a lesion detection model to detect a lesion region included in the endoscopic image (step S217).

[0076] Next, lesion area selection unit 217 generates a selection screen for selecting a lesion area to be used for diagnosis and outputs it to display device 2. Then, lesion area selection unit 217 outputs the lesion area selected by the doctor to lesion classification unit 218. Lesion classification unit 218 diagnoses the lesion area input from lesion area selection unit 217 (step S218). Next, output unit 219 outputs the diagnosis result to display device 2 (step S219). Then, the processing ends.

[0077] Third Embodiment Next, a third embodiment will be described. The endoscopic examination system of the third embodiment includes different lesion area analyzers, and each lesion area analyzer diagnoses the lesion area analyzed by the respective lesion area analyzers. The doctor then selects an appropriate diagnosis result from the multiple diagnosis results. Note that the system configuration and hardware configuration are the same as those of the first embodiment, and therefore will not be described here.

[0078] 11 is a block diagram showing the functional configuration of an endoscopic examination support device 1b according to the third embodiment. Functionally, the endoscopic examination support device 1b includes a gaze position identification unit 311, a coordinate conversion unit 312, a mask image generation unit 313, a first lesion area analysis unit 314, a background removal unit 315, a second lesion area analysis unit 316, a lesion classification unit 317, a classification result selection unit 318, and an output unit 319.

[0079] The gaze position identification unit 311, coordinate conversion unit 312, mask image generation unit 313, first lesion area analysis unit 314, background removal unit 215, and second lesion area analysis unit 316 have the same configuration and operate in the same manner as the gaze position identification unit 211, coordinate conversion unit 212, mask image generation unit 213, first lesion area analysis unit 214, background removal unit 215, and second lesion area analysis unit 216 of the endoscopic examination support device 1a of the second embodiment, and therefore will not be described again.

[0080] The lesion area estimated by first lesion area analysis unit 314 will hereinafter also be referred to as the "first lesion area." The lesion area detected by second lesion area analysis unit 316 will hereinafter also be referred to as the "second lesion area."

[0081] Lesion classification unit 317 diagnoses the first lesion area based on the image of the first lesion area cut out by background removal unit 315. Lesion classification unit 317 also diagnoses the second lesion area based on the endoscopic image input from second lesion area analysis unit 316. Specifically, lesion classification unit 317 diagnoses the second lesion area based on the image of the second lesion area enclosed by a rectangle on the endoscopic image. Lesion classification unit 317 outputs the diagnosis results of the first lesion area and the second lesion area to classification result selection unit 318.

[0082] The classification result selection unit 318 generates a selection screen for selecting a diagnostic result and outputs it to the display device 2. The doctor selects a diagnostic result to be adopted from the multiple diagnostic results included in the selection screen. The classification result selection unit 318 then outputs the diagnostic result selected by the doctor to the output unit 319.

[0083] FIG. 12 shows an example of the display of a selection screen generated by the classification result selection unit 318. FIG. 12 includes endoscopic image 51, lesion areas 52-54, cropped images 52a-54a, and diagnosis results 52b-54b. Endoscopic image 51 is an endoscopic image generated based on imaging instructions from a doctor. Lesion area 52 is a first lesion area. Cropped image 52a is an image cropped from lesion area 52. Diagnosis result 52b is the diagnosis result for cropped image 52a. Lesion area 53 and lesion area 54 are second lesion areas. Cropped images 53a and 54a are images cropped from lesion area 53 and lesion area 54, respectively. Diagnosis results 53b and 54b are the diagnosis results for cropped images 53a and 54a.

[0084] The doctor looks at the display as shown in Figure 12 and selects a diagnostic result to be adopted from among the diagnostic results 52b to 54b. For example, the doctor focuses on a diagnostic result to be adopted from among the diagnostic results 52b to 54b. The classification result selection unit 318 outputs the diagnostic result that the doctor has been focusing on for a predetermined time TH4 or more to the output unit 319. The doctor may also select a diagnostic result to be adopted by voice input.

[0085] 12 displays three lesion areas, cut-out images, and diagnostic results, but if four or more lesion areas are estimated for one endoscopic image, four or more lesion areas, their cut-out images, and their diagnostic results may be displayed simultaneously depending on the display space of the display device 2. Furthermore, if four or more lesion areas are estimated for one endoscopic image and it is not possible to display all of the lesion areas, cut-out images, and diagnostic results, the lesion areas to be displayed may be determined depending on the importance of the lesion areas.

[0086] The output unit 319 outputs the diagnosis results input from the classification result selection unit 318 to the display device 2 .

[0087] In the above configuration, the classification result selection unit 318 is an example of a diagnostic result selection screen data output means and a diagnostic result receiving means.

[0088] [Endoscopic examination support processing] Next, the endoscopic examination support processing that performs the above-mentioned processing will be described. Fig. 13 is a flowchart of the endoscopic examination support processing by the endoscopic examination support device 1b. This processing is realized by the processor 11 shown in Fig. 2 executing a program prepared in advance and operating as each element shown in Fig. 11. Note that the processing of steps S311 to S317 is similar to the processing of steps S211 to S217 of the endoscopic examination support processing shown in Fig. 10, and therefore description thereof will be omitted.

[0089] Lesion classification unit 317 diagnoses the first lesion area and the second lesion area (step S318). Specifically, lesion classification unit 317 diagnoses the first lesion area based on the image of the first lesion area cut out by background removal unit 315. Furthermore, lesion classification unit 317 diagnoses the second lesion area based on the endoscopic image input from second lesion area analysis unit 316. Specifically, lesion classification unit 317 diagnoses the second lesion area based on the image of the second lesion area surrounded by a rectangle on the endoscopic image.

[0090] Next, the classification result selection unit 318 generates a selection screen for selecting a diagnosis result and outputs it to the display device 2. Then, the classification result selection unit 318 outputs the diagnosis result selected by the doctor to the output unit 319. The output unit 319 outputs the diagnosis result input from the classification result selection unit 318 to the display device 2 (step S319). Then, the processing ends.

[0091] 14 is a block diagram showing the functional configuration of an endoscopic examination support device according to Embodiment 4. The endoscopic examination support device 500 includes an image acquisition unit 501, a gaze point detection unit 502, a mask image generation unit 503, a first lesion area estimation unit 504, and a first extraction unit 505.

[0092] 15 is a flowchart of processing by the endoscopic examination support device of the fourth embodiment. The image acquisition means 501 acquires an endoscopic image (step S501). The gaze point detection means 502 detects the user's gaze point on the endoscopic image (step S502). The mask image generation means 503 generates a mask image based on the gaze point (step S503). The first lesion area estimation means 504 estimates a first lesion area in the endoscopic image based on the endoscopic image and the mask image (step S504). The first extraction means 505 generates a first extracted image by extracting the first lesion area from the endoscopic image (step S505).

[0093] According to the endoscopic examination support device 500 of the fourth embodiment, it is possible to diagnose lesions with high accuracy. Furthermore, the endoscopic examination support device 500 can support user decision-making in the medical field.

[0094] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0095] (Supplementary Note 1) An endoscopic examination support device comprising: an image acquisition means for acquiring an endoscopic image; a gaze point detection means for detecting a user's gaze point on the endoscopic image; a mask image generation means for generating a mask image based on the gaze point; a first lesion area estimation means for estimating a first lesion area in the endoscopic image based on the endoscopic image and the mask image; and a first cut-out means for generating a first cut-out image by cutting out the first lesion area from the endoscopic image.

[0096] (Supplementary Note 2) The endoscopic examination support device according to Supplementary Note 1, wherein the first lesion area estimation means estimates the first lesion area using a first machine learning model that receives an endoscopic image and a mask image as input and segments a lesion area included in the endoscopic image.

[0097] (Supplementary Note 3) The endoscopic examination support device according to Supplementary Note 2, comprising: a diagnostic means for acquiring a diagnostic result of the first lesion area from the first extracted image using a second machine learning model that receives an image including a lesion area as an input and outputs a diagnostic result for the lesion area; and a diagnostic result output means for outputting the diagnostic result.

[0098] (Supplementary Note 4) The endoscopic examination support device according to Supplementary Note 1, comprising: an evaluation screen data output means for outputting evaluation screen data for evaluating the first lesion area and the first extracted image to the user; and an evaluation acquisition means for acquiring an evaluation of the first lesion area and the first extracted image from the user.

[0099] (Appendix 5) An endoscopic examination support device as described in Appendix 3, comprising: a confirmation screen data output means for outputting confirmation screen data to the user for confirming whether or not the first cut-out image will be used for diagnosis; and an answer acquisition means for acquiring an answer from the user as to whether or not the first cut-out image will be used for diagnosis, wherein the diagnosis means acquires a diagnosis result of the first lesion area from the first cut-out image when the answer is that it will be used for diagnosis.

[0100] (Supplementary Note 6) An endoscopic examination support device according to Supplementary Note 1, comprising: an editing screen data output means for outputting editing screen data for editing the first lesion area to the user; and an editing operation receiving means for receiving editing operations by the user on the first lesion area, wherein the editing operations include operations to move and resize the first lesion area.

[0101] (Supplementary Note 7) An endoscopic examination support device according to Supplementary Note 3, comprising: a second lesion area estimation means for estimating a second lesion area from an endoscopic image using a third machine learning model that receives an endoscopic image as an input and detects a lesion area included in the endoscopic image; a second cropping means for generating a second cropped image by cropping the second lesion area from the endoscopic image; a lesion area selection screen data output means for outputting to the user lesion area selection screen data for selecting a cropped image to be used for diagnosis; and a selection operation receiving means for receiving a selection operation of a cropped image by the user, wherein the lesion area selection screen data includes the first cropped image and the second cropped image, and the diagnosis means uses the second machine learning model to obtain a diagnosis result from the cropped image selected by the user.

[0102] (Supplementary Note 8) An endoscopic examination support device according to Supplementary Note 3, comprising: second lesion area estimation means for estimating a second lesion area from an endoscopic image using a third machine learning model that receives an endoscopic image as an input and detects a lesion area included in the endoscopic image; second cropping means for generating a second cropped image by cropping the second lesion area from the endoscopic image; diagnostic result selection screen data output means for outputting to the user diagnostic result selection screen data for selecting a diagnostic result to be adopted from a plurality of diagnostic results; and diagnostic result receiving means for receiving a diagnostic result selection operation by the user, wherein the diagnostic means uses the second machine learning model to obtain a diagnostic result of the first lesion area from the first cropped image and also obtains a diagnostic result of the second lesion area from the second cropped image, the diagnostic result selection screen data including the diagnostic result of the first lesion area and the diagnostic result of the second lesion area, and the diagnostic result output means outputs the diagnostic result selected by the user as a final result.

[0103] (Supplementary Note 9) The endoscopic examination support device according to Supplementary Note 5, wherein the answer acquisition means acquires the answer through an operation based on the user's line of sight or an operation based on the user's voice.

[0104] (Supplementary Note 10) The endoscopic examination support device according to Supplementary Note 6, wherein the editing operation receiving means receives an editing operation by voice of the user.

[0105] (Supplementary Note 11) The endoscopic examination support device according to Supplementary Note 7, wherein the selection operation receiving means receives a selection operation based on the user's line of sight.

[0106] (Supplementary Note 12) The endoscopic examination support device according to Supplementary Note 8, wherein the diagnosis result receiving means receives a selection operation based on the user's line of sight.

[0107] (Appendix 13) An endoscopic examination support device as described in Appendix 1, comprising an eyeball image acquisition means for acquiring an eyeball image including a user's eyeball, wherein the gaze point detection means detects the coordinates of the user's gaze point directed at a display device that displays the endoscopic image based on the eyeball image, and converts the coordinates of the user's gaze point on the display device into coordinates on the endoscopic image, thereby detecting the user's gaze point on the endoscopic image.

[0108] (Supplementary Note 14) The endoscopic examination support device according to Supplementary Note 1, wherein the mask image generating means generates a binarized mask image of a region within a predetermined range based on the fixation point and a region other than the predetermined range.

[0109] (Supplementary Note 15) The endoscopic examination support device according to Supplementary Note 1, wherein the mask image generating means generates a binarized mask image of a region within a predetermined range based on the fixation point where the dwell time is equal to or greater than a predetermined threshold, and the other region.

[0110] (Supplementary Note 16) The endoscopic examination support device according to Supplementary Note 1, wherein the mask image generating means generates a mask image binarized into an area where the fixation point has been detected a predetermined number of times or more and other areas.

[0111] (Supplementary Note 17) The endoscopic examination support device according to Supplementary Note 1, wherein the mask image generating means generates a mask image by representing an area of ​​a predetermined range based on the gaze point with shades of a single color based on a dwell time of the gaze point.

[0112] (Appendix 18) An endoscopic examination support method comprising: performing image acquisition to acquire an endoscopic image; performing gaze point detection to detect a user's gaze point on the endoscopic image; performing mask image generation to generate a mask image based on the gaze point; performing first lesion area estimation to estimate a first lesion area in the endoscopic image based on the endoscopic image and the mask image; and performing first cut-out to generate a first cut-out image by cutting out the first lesion area from the endoscopic image.

[0113] (Appendix 19) A recording medium having recorded thereon a program that causes a computer to execute the following processes: image acquisition for acquiring an endoscopic image; gaze point detection for detecting a user's gaze point on the endoscopic image; mask image generation for generating a mask image based on the gaze point; first lesion area estimation for estimating a first lesion area in the endoscopic image based on the endoscopic image and the mask image; and first cropping for generating a first cropped image by cropping the first lesion area from the endoscopic image.

[0114] Although the present disclosure has been described above with reference to the embodiments and examples, the present disclosure is not limited to the above-described embodiments and examples. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure.

[0115] DESCRIPTION OF SYMBOLS 1, 1a, 1b Endoscopy support device 2 Display device 3 Endoscope 4 Gaze tracking device 11 Processor 12 Memory 17 Database (DB) 100 Endoscopy system 111, 211, 311 Gaze position identification unit 112, 212, 312 Coordinate conversion unit 113, 213, 313 Mask image generation unit 114 Lesion area analysis unit 115, 215, 315 Background removal unit 116, 218, 317 Lesion classification unit 117, 219, 319 Output unit 214, 314 First lesion area analysis unit 216, 316 Second lesion area analysis unit 217 Lesion area selection unit 318 Classification result selection unit

Claims

1. An endoscopic examination support device comprising: an image acquisition means for acquiring an endoscopic image; a gaze point detection means for detecting a user's gaze point on the endoscopic image; a mask image generation means for generating a mask image based on the gaze point; a first lesion area estimation means for estimating a first lesion area in the endoscopic image based on the endoscopic image and the mask image; and a first cut-out means for generating a first cut-out image by cutting out the first lesion area from the endoscopic image.

2. The endoscopic examination support device described in claim 1, wherein the first lesion area estimation means estimates the first lesion area using a first machine learning model that receives an endoscopic image and a mask image as input and segments the lesion area contained in the endoscopic image.

3. An endoscopic examination support device as described in claim 2, comprising: a diagnostic means for acquiring a diagnostic result of the first lesion area from the first extracted image using a second machine learning model that receives an image including a lesion area as input and outputs a diagnostic result for the lesion area; and a diagnostic result output means for outputting the diagnostic result.

4. An endoscopic examination support device as described in claim 1, comprising: an evaluation screen data output means for outputting evaluation screen data for evaluating the first lesion area and the first cut-out image to the user; and an evaluation acquisition means for acquiring an evaluation of the first lesion area and the first cut-out image from the user.

5. An endoscopic examination support device as described in claim 3, comprising: a confirmation screen data output means for outputting confirmation screen data to the user for confirming whether or not the first cut-out image will be used for diagnosis; and an answer acquisition means for acquiring an answer from the user as to whether or not the first cut-out image will be used for diagnosis, wherein the diagnostic means acquires a diagnostic result of the first lesion area from the first cut-out image if the answer is that it will be used for diagnosis.

6. An endoscopic examination support device as described in claim 1, comprising: an editing screen data output means for outputting editing screen data for editing the first lesion area to the user; and an editing operation receiving means for receiving editing operations by the user on the first lesion area, wherein the editing operations include operations to move and resize the first lesion area.

7. An endoscopic examination support device as described in claim 3, comprising: a second lesion area estimation means for estimating a second lesion area from an endoscopic image using a third machine learning model that receives an endoscopic image as input and detects a lesion area contained in the endoscopic image; a second extraction means for generating a second extracted image by extracting the second lesion area from the endoscopic image; a lesion area selection screen data output means for outputting to the user lesion area selection screen data for selecting an extracted image to be used for diagnosis; and a selection operation receiving means for receiving a selection operation of an extracted image by the user, wherein the lesion area selection screen data includes the first extracted image and the second extracted image, and the diagnosis means uses the second machine learning model to obtain a diagnostic result from the extracted image selected by the user.

8. An endoscopic examination support device as described in claim 3, comprising: a second lesion area estimation means for estimating a second lesion area from an endoscopic image using a third machine learning model that receives an endoscopic image as input and detects a lesion area contained in the endoscopic image; a second extraction means for generating a second extracted image by extracting the second lesion area from the endoscopic image; a diagnostic result selection screen data output means for outputting to the user diagnostic result selection screen data for selecting a diagnostic result to be adopted from a plurality of diagnostic results; and a diagnostic result receiving means for receiving a diagnostic result selection operation by the user, wherein the diagnostic means uses the second machine learning model to obtain a diagnostic result of the first lesion area from the first extracted image and a diagnostic result of the second lesion area from the second extracted image, the diagnostic result selection screen data including a diagnostic result of the first lesion area and a diagnostic result of the second lesion area, and the diagnostic result output means outputs the diagnostic result selected by the user as a final result.

9. The endoscopic examination support device according to claim 5, wherein the answer acquisition means acquires the answer through an operation based on the user's line of sight or an operation based on the user's voice.

10. The endoscopic examination support device according to claim 6, wherein the editing operation receiving means receives editing operations by voice from the user.

11. The endoscopic examination support device according to claim 7, wherein the selection operation receiving means receives a selection operation based on the user's line of sight.

12. The endoscopic examination support device according to claim 8, wherein the diagnostic result receiving means receives a selection operation based on the user's line of sight.

13. An endoscopic examination support device as described in claim 1, further comprising an eyeball image acquisition means for acquiring an eyeball image including the user's eyeball, wherein the gaze point detection means detects the coordinates of the user's gaze point directed at a display device that displays the endoscopic image based on the eyeball image, and converts the coordinates of the user's gaze point on the display device into coordinates on the endoscopic image, thereby detecting the user's gaze point on the endoscopic image.

14. An endoscopic examination support device according to claim 1, wherein the mask image generating means generates a binarized mask image of a region within a predetermined range based on the gaze point and other regions.

15. An endoscopic examination support device as described in claim 1, wherein the mask image generating means generates a binarized mask image of a region within a predetermined range based on the gaze point where the dwell time is equal to or greater than a predetermined threshold, and other regions.

16. An endoscopic examination support device according to claim 1, wherein the mask image generating means generates a binarized mask image of an area where the fixation point has been detected a predetermined number of times or more and other areas.

17. An endoscopic examination support device as described in claim 1, wherein the mask image generating means generates a mask image by representing a predetermined range of area based on the gaze point with shades of a single color based on the duration of time the gaze point is maintained.

18. A method for supporting endoscopic examination, comprising: performing image acquisition to acquire an endoscopic image; performing gaze point detection to detect a user's gaze point on the endoscopic image; performing mask image generation to generate a mask image based on the gaze point; performing first lesion area estimation to estimate a first lesion area in the endoscopic image based on the endoscopic image and the mask image; and performing first cut-out to generate a first cut-out image by cutting out the first lesion area from the endoscopic image.

19. A recording medium having recorded thereon a program that causes a computer to execute the following processes: image acquisition to acquire an endoscopic image; gaze point detection to detect a user's gaze point on the endoscopic image; mask image generation to generate a mask image based on the gaze point; first lesion area estimation to estimate a first lesion area in the endoscopic image based on the endoscopic image and the mask image; and first cropping to generate a first cropped image by cropping the first lesion area from the endoscopic image.

Images