Endoscopic examination support device, endoscopic examination support method, and program

Abstract

Provided is an endoscopic examination assistance device, wherein a video acquisition means acquires an endoscopic video taken by an endoscope. A lesion detection means detects a lesion candidate from the endoscopic video. A heat map generation means generates a heat map expressing, with colors, the lesion possibility of the lesion candidate contained in the endoscopic video. A display control means superimposes and displays a region in which the lesion possibility is equal to or higher than a first threshold on the endoscopic video in a first display mode, and superimposes and displays a region in which the lesion possibility is equal to or higher than a second threshold on the endoscopic video in a second display mode.

Description

Endoscopic examination support device, endoscopic examination support method, and recording medium

[0001] The present disclosure relates to processing images related to endoscopy.

[0002] In endoscopic examinations, there are technologies that detect lesions from endoscopic images and display the detection results on the endoscopic images. For example, there is a technology that generates a heat map of endoscopic images containing lesions and displays the heat map superimposed on the endoscopic images to indicate the lesion area. Furthermore, Patent Document 1 proposes a method for quickly displaying a frame circumscribing the entire tumor on the endoscopic images.

[0003] International Publication No. WO2019 / 155628

[0004] When a heat map is superimposed on an endoscopic image, the original endoscopic image becomes difficult to see. Furthermore, the method of Patent Document 1 does not allow a user to grasp the degree of possibility that a detected lesion is a lesion.

[0005] One object of the present disclosure is to provide an endoscopic examination support device that can grasp the degree of possibility of a lesion while ensuring visibility of endoscopic images.

[0006] In one aspect of the present disclosure, an endoscopic examination support device comprises: an image acquisition means for acquiring endoscopic images captured by an endoscope; a lesion detection means for detecting lesion candidates from the endoscopic images; a heat map generation means for generating a heat map in which the lesion possibility of the lesion candidates contained in the endoscopic images is represented by color; and a display control means for superimposing and displaying, on the endoscopic images, areas where the lesion possibility is equal to or greater than a first threshold in a first display mode, and superimposing and displaying, on the endoscopic images, areas where the lesion possibility is equal to or greater than a second threshold in a second display mode, wherein the second threshold is greater than the first threshold.

[0007] In another aspect of the present disclosure, a method for supporting endoscopic examination includes acquiring an endoscopic image captured by an endoscope, detecting lesion candidates from the endoscopic image, generating a heat map in which the lesion possibility of the lesion candidates contained in the endoscopic image is represented by color, superimposing and displaying an area where the lesion possibility is equal to or greater than a first threshold on the endoscopic image in a first display mode, and superimposing and displaying an area where the lesion possibility is equal to or greater than a second threshold on the endoscopic image in a second display mode, wherein the second threshold is greater than the first threshold.

[0008] In yet another aspect of the present disclosure, a recording medium records a program that causes a computer to execute processing to acquire endoscopic video captured by an endoscope, detect lesion candidates from the endoscopic video, generate a heat map that uses colors to represent the lesion likelihood of the lesion candidates contained in the endoscopic video, superimpose areas where the lesion likelihood is equal to or greater than a first threshold on the endoscopic video in a first display mode, and superimpose areas where the lesion likelihood is equal to or greater than a second threshold on the endoscopic video in a second display mode, wherein the second threshold is a value greater than the first threshold.

[0009] According to the present disclosure, it is possible to grasp the degree of possibility of a lesion while ensuring visibility of the endoscopic image.

[0010] 1 is a block diagram showing a schematic configuration of an endoscopic examination system. FIG. 2 is a block diagram showing a hardware configuration of an endoscopic examination support device. FIG. 3 is a block diagram showing a functional configuration of the endoscopic examination support device. FIG. 4 is a diagram showing a display example of the endoscopic examination support device. FIG. 5 is a diagram showing another display example of the endoscopic examination support device. FIG. 6 is a diagram showing another display example of the endoscopic examination support device. FIG. 7 is a flowchart of display processing by the endoscopic examination support device. FIG. 8 is a block diagram showing the functional configuration of the endoscopic examination support device of a second embodiment. FIG. 9 is a flowchart of processing by the endoscopic examination support device of the second embodiment.

[0011] Preferred embodiments of the present disclosure will be described below with reference to the drawings. <First Embodiment> [System Configuration] Fig. 1 shows a schematic configuration of an endoscopic examination system 100. The endoscopic examination system 100 detects lesion candidates during an examination (including treatment) using an endoscope, and displays the lesion candidate detection results and the degree of likelihood that the lesion candidate is a lesion (hereinafter also referred to as "lesion probability") on the endoscopic video. This allows a doctor to understand the lesion candidates and lesion probability while viewing the endoscopic video.

[0012] As shown in FIG. 1 , the endoscopic examination system 100 mainly includes an endoscopic examination support device 1, a display device 2, and an endoscope 3 connected to the endoscopic examination support device 1.

[0013] The endoscopic examination support device 1 acquires images (i.e., moving images; hereinafter, also referred to as "endoscopic images Ic") captured by the endoscope 3 during endoscopic examination from the endoscope 3, and displays display data on the display device 2 for the examiner performing the endoscopic examination to confirm. Specifically, the endoscopic examination support device 1 acquires moving images of the inside of organs captured by the endoscope 3 during endoscopic examination as the endoscopic images Ic. The endoscopic examination support device 1 extracts still images (frame images) from the endoscopic images Ic and detects lesions using AI (Artificial Intelligence). When a lesion is detected from the frame images using AI, the endoscopic examination support device 1 generates a heat map based on the frame images. The endoscopic examination support device 1 then extracts the area with the highest lesion probability and the area with the second highest lesion probability from the heat map, and superimposes these two areas on the endoscopic images in different modes.

[0014] The display device 2 is a display or the like that displays an image based on a display signal supplied from the endoscopic examination support device 1 .

[0015] The endoscope 3 mainly has an operation unit 36 ​​that allows the examiner to input instructions such as air supply, water supply, angle adjustment, and photography instructions, a flexible shaft 37 that is inserted into the subject's organ to be examined, a tip 38 that incorporates an imaging unit such as a miniature imaging element, and a connection unit 39 for connecting to the endoscopic examination support device 1.

[0016] 2 shows the hardware configuration of the endoscopic examination support device 1. The endoscopic examination support device 1 mainly includes a processor 11, a memory 12, an interface 13, an input unit 14, a light source unit 15, a sound output unit 16, and a database (hereinafter referred to as "DB") 17. These elements are connected via a data bus 19.

[0017] The processor 11 performs predetermined processing by executing programs stored in the memory 12. The processor 11 is a processor such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a TPU (Tensor Processing Unit). The processor 11 may be composed of multiple processors. The processor 11 is an example of a computer.

[0018] The memory 12 is composed of various volatile memories used as working memories, such as RAM (Random Access Memory) and ROM (Read Only Memory), and nonvolatile memories that store information necessary for processing by the endoscopic examination support device 1. The memory 12 may include an external storage device such as a hard disk connected to or built into the endoscopic examination support device 1, or may include a storage medium such as a removable flash memory or disk medium. The memory 12 stores programs for the endoscopic examination support device 1 to execute each process in this embodiment.

[0019] The memory 12 also temporarily stores a series of endoscopic images Ic captured by the endoscope 3 during an endoscopic examination under the control of the processor 11. The memory 12 also temporarily stores still images acquired from the endoscopic images Ic during an endoscopic examination. These images are stored in the memory 12 in association with, for example, the subject's identification information (e.g., patient ID), timestamp information, and the like.

[0020] The interface 13 performs interface operations between the endoscopic examination assistance device 1 and an external device. For example, the interface 13 supplies the display data Id generated by the processor 11 to the display device 2. The interface 13 also supplies illumination light generated by the light source unit 15 to the endoscope 3. The interface 13 also supplies an electrical signal indicating the endoscopic video Ic supplied from the endoscope 3 to the processor 11. The interface 13 may be a communication interface such as a network adapter for wired or wireless communication with an external device, or may be a hardware interface compliant with USB (Universal Serial Bus), SATA (Serial AT Attachment), or the like.

[0021] The input unit 14 generates an input signal based on an operation by the examiner. The input unit 14 is, for example, a button, a touch panel, a remote controller, or a voice input device. The light source unit 15 generates light to be supplied to the tip 38 of the endoscope 3. The light source unit 15 may also incorporate a pump or the like for sending water or air to be supplied to the endoscope 3. The sound output unit 16 outputs sound based on the control of the processor 11.

[0022] The DB 17 stores endoscopic images acquired during past endoscopic examinations of the subject and lesion information. The lesion information includes lesion images and related information. The lesions include polyps (protruding lesions). The DB 17 may include an external storage device such as a hard disk connected to or built into the endoscopic examination support device 1, or may include a storage medium such as a removable flash memory. Instead of providing the DB 17 within the endoscopic examination system 100, the DB 17 may be provided on an external server, and the lesion information may be obtained from the server via communication.

[0023] 3 is a block diagram showing the functional configuration of the endoscopic examination support device 1. The endoscopic examination support device 1 functionally includes an interface 13, a lesion detection unit 21, a heat map generation unit 22, and a display control unit 23.

[0024] An endoscopic video Ic is input to the endoscopic examination support device 1 from the endoscope 3. The endoscopic video Ic is input to the interface 13. The interface 13 extracts frame images (hereinafter also referred to as "endoscopic images") from the input endoscopic video Ic and outputs them to the lesion detection unit 21 and the heat map generation unit 22. The interface 13 also outputs the input endoscopic video Ic to the display control unit 23.

[0025] The lesion detection unit 21 performs image analysis based on the endoscopic image input from the interface 13 and determines whether or not the endoscopic image contains a lesion. The lesion detection unit 21 detects lesion-like areas (hereinafter also referred to as "lesion candidates") contained in the endoscopic image using a previously prepared image recognition model or the like. This image recognition model is a model that has been trained in advance to estimate lesion candidates contained in the endoscopic image and is hereinafter also referred to as the "lesion detection model." If the lesion detection unit 21 detects a lesion candidate, it outputs a determination result that a lesion is present, together with information such as a timestamp, to the heat map generation unit 22 and the display control unit 23. On the other hand, if the lesion detection unit 21 does not detect a lesion candidate, it outputs a determination result that a lesion is not present to the heat map generation unit 22 and the display control unit 23.

[0026] The heat map generating unit 22 generates a heat map based on the endoscopic image input from the interface 13 and the determination result input from the lesion detecting unit 21 .

[0027] Specifically, when the lesion detection unit 21 inputs a determination result indicating the presence of a lesion, the heat map generation unit 22 acquires an endoscopic image including a lesion candidate from the endoscopic image input from the interface 13 based on information such as a timestamp. The heat map generation unit 22 then uses a pre-prepared image recognition model or the like to estimate whether each pixel in the endoscopic image is within a lesion candidate region (hereinafter also referred to as a "lesion region"). This image recognition model is a model that has been trained in advance to estimate whether each pixel in the endoscopic image is within a lesion region, and is hereinafter also referred to as a "lesion score estimation model."

[0028] The heat map generator 22 uses the lesion score estimation model to estimate for each pixel in the endoscopic image whether it is a pixel in a lesion area and calculates a score (hereinafter also referred to as the "lesion score") indicating the probability that the pixel is in a lesion area. The lesion score is, for example, a numerical value greater than or equal to 0 and less than or equal to 1, and the closer the lesion score is to 1, the more likely it is that the pixel is in a lesion area. The heat map generator 22 then generates a heat map based on a predetermined relationship between the lesion score and color. The heat map generator 22 outputs the generated heat map to the display controller 23.

[0029] In the above description, the heat map generator 22 generates a heat map when the lesion detector 21 determines that a lesion is present, but the timing for generating the heat map is not limited to this. For example, the heat map generator 22 may generate a heat map each time an endoscopic image is input from the interface 13 and output the heat map to the display controller 23.

[0030] The display control unit 23 generates display data based on the endoscopic image Ic input from the interface 13, the judgment result input from the lesion detection unit 21, and the heat map input from the heat map generation unit 22, and outputs it to the display device 2.

[0031] Specifically, when the display control unit 23 receives a lesion detection result from the lesion detection unit 21, it extracts a predetermined lesion area from the heat map. Based on the extracted lesion area, the display control unit 23 generates display data by superimposing the lesion area on the endoscopic image. For example, the display control unit 23 compares the lesion score of each pixel in the heat map with a predetermined threshold TH1 and extracts a region consisting of pixels whose lesion score is equal to or greater than the threshold TH1 (hereinafter also referred to as a "first threshold region"). The display control unit 23 also compares the lesion score of each pixel in the heat map with a predetermined threshold TH2 and extracts a region consisting of pixels whose lesion score is equal to or greater than the threshold TH2 (hereinafter also referred to as a "second threshold region"). Based on the first and second threshold regions extracted from the heat map, the display control unit 23 generates display data by superimposing each region on the endoscopic image in different display modes. The display control unit 23 then outputs the display data to the display device 2.

[0032] The threshold value TH2 is a value greater than the threshold value TH1, and the second threshold value region indicates the region with the highest lesion probability. The first threshold value region indicates the region with the second highest lesion probability. The values ​​of the threshold values ​​TH1 and TH2 can be set for each doctor.

[0033] When the display control unit 23 receives a predetermined number of consecutive inputs of a determination result indicating the presence of a lesion from the lesion detection unit 21, the display control unit 23 determines that a lesion candidate has been stably detected. Then, the display control unit 23 includes an endoscopic image and a heat map including the lesion candidate as display data, as a lesion history and a heat map corresponding to the lesion history, which will be described later, and outputs the display data to the display device 2. On the other hand, when the display control unit 23 receives a determination result indicating the absence of a lesion from the lesion detection unit 21, the display control unit 23 outputs the endoscopic video Ic as display data to the display device 2.

[0034] In the above configuration, the interface 13 is an example of an image acquisition means, the lesion detection unit 21 is an example of a lesion detection means, the heat map generation unit 22 is an example of a heat map generation means, and the display control unit 23 is an example of a display control means.

[0035] [Display Example] Next, a display example on the display device 2 will be described.

[0036] 4 shows an example of a display on the display device 2. In this example, the display device 2 displays an endoscopic image 41, a lesion history 42, a heat map 43, a first threshold region 44, and a second threshold region 45.

[0037] The endoscopic video 41 is the endoscopic video Ic during the examination and is updated as the endoscopic camera moves. The lesion history 42 is an endoscopic image including a lesion candidate detected during the endoscopic examination. If there are multiple endoscopic images including a lesion candidate, the lesion history 42 displays the endoscopic image including the most recent lesion candidate. The heat map 43 is a heat map of the endoscopic image corresponding to the lesion history 42.

[0038] The first threshold region 44 and the second threshold region 45 indicate lesion regions. Specifically, the first threshold region 44 is a region where the lesion score is equal to or greater than a predetermined threshold TH1. The second threshold region 45 is a region where the lesion score is equal to or greater than a predetermined threshold TH2. The threshold TH2 is a value greater than the threshold TH1, and the second threshold region 45 indicates a region with the highest lesion probability. Furthermore, different predetermined transparencies are set within the contours of the first threshold region 44 and the second threshold region 45 so that the original endoscopic image can be seen through them. In FIG. 4 , the transparency of the first threshold region 44 is set higher than the transparency of the second threshold region 45. This allows a doctor to identify regions with a high lesion probability while viewing the endoscopic image.

[0039] 5 shows another example of display by the display device 2. In FIG. 5, the display manner of the first threshold region and the second threshold region is different from that in FIG.

[0040] The first threshold region 44a is a region where the lesion score is equal to or greater than a predetermined threshold TH1. The second threshold region 45a is a region where the lesion probability is equal to or greater than a predetermined threshold TH2. In FIG. 5 , the transparency of the second threshold region 45a is set higher than the transparency of the first threshold region 44a. Note that in the example of FIG. 5 , since the second threshold region 45a is located inside the first threshold region 44a, the second threshold region 45a is displayed with a high transparency, and the region of the first threshold region 44a other than the second threshold region 45a is displayed with a low transparency. In this way, in the example of FIG. 5 , regions with high lesion probability are displayed with a high transparency, allowing doctors to easily observe regions with high lesion probability.

[0041] 6 shows another example of display by the display device 2. In FIG. 6, a center mark 46 is displayed within a second threshold region 45.

[0042] The center mark 46 indicates a suitable position for collecting a sample. The center mark 46 may be, for example, the center position of the circumscribing rectangle of the second threshold region 45, the center position of the inscribing circle of the second threshold region 45, or the center of gravity of the second threshold region 45. The display control unit 23 determines the center position or center of gravity as described above based on the second threshold region extracted from the heat map, and depicts the position as a graphic or the like on the endoscopic image. Displaying the center mark 46 on the endoscopic image allows the physician to grasp the suitable position for collecting a sample.

[0043] Each doctor may be allowed to select whether or not to display the first threshold region 44, the second threshold region 45, and the center mark 46. Doctors can freely combine and use the above information.

[0044] [Image Display Processing] Next, the display processing for performing the above-described display will be described. Fig. 7 is a flowchart of the processing performed by the endoscopic examination support device 1. This processing is realized by the processor 11 shown in Fig. 2 executing a program prepared in advance and operating as each element shown in Fig. 3.

[0045] First, an endoscopic video Ic is input from the endoscope 3 to the interface 13. The interface 13 acquires an endoscopic image from the input endoscopic video Ic (step S11). The interface 13 outputs the endoscopic image to the lesion detection unit 21 and the heat map generation unit 22. The interface 13 also outputs the endoscopic video Ic to the display control unit 23. Next, the lesion detection unit 21 detects a lesion from the endoscopic image (step S12). Specifically, the lesion detection unit 21 uses a lesion detection model to determine whether or not the endoscopic image contains a lesion. Then, the lesion detection unit 21 outputs the determination result to the heat map generation unit 22 and the display control unit 23.

[0046] Next, when a lesion is detected, the heat map generator 22 generates a heat map from the endoscopic image (step S13). Specifically, the heat map generator 22 estimates a lesion score for each pixel in the endoscopic image using a lesion score estimation model. The heat map generator 22 then generates a heat map based on a predetermined relationship between the score and color. The heat map generator 22 then outputs the generated heat map to the display controller 23.

[0047] Next, the display control unit 23 generates display data based on the endoscopic video input from the interface 13, the determination result input from the lesion detection unit 21, and the heat map input from the heat map generation unit 22, and outputs the generated display data to the display device 2 (step S14). Specifically, when the determination result indicating the presence of a lesion is input from the lesion detection unit 21, the display control unit 23 extracts a predetermined lesion area from the heat map. Then, based on the extracted lesion area, the display control unit 23 performs a depiction such that the lesion area is superimposed on the endoscopic video, and generates display data.

[0048] [Modifications] Next, a description will be given of modifications of the first embodiment. The following modifications can be applied to the first embodiment in appropriate combinations.

[0049] (Variation 1) In the first embodiment described above, the display control unit 23 displays the first threshold region and the second threshold region superimposed on the endoscopic image. However, the display control unit 23 may also display a third threshold region superimposed on the endoscopic image in addition to the first threshold region and the second threshold region. The display control unit 23 compares the lesion score of each pixel in the heat map with a predetermined threshold TH3 and defines a region consisting of pixels whose lesion score is equal to or greater than the threshold TH3 as the third threshold region. Then, the display control unit 23 depicts the first threshold region, the second threshold region, and the third threshold region extracted from the heat map so as to superimpose each region on the endoscopic image. By providing the third threshold region, the doctor can set more precise thresholds.

[0050] 8 is a block diagram showing the functional configuration of an endoscopic examination support device according to Embodiment 2. The endoscopic examination support device 70 includes an image acquisition unit 71, a lesion detection unit 72, a heat map generation unit 73, and a display control unit 74.

[0051] 9 is a flowchart of processing by the endoscopic examination support device of the second embodiment. The image acquisition means 71 acquires endoscopic images captured by an endoscope (step S71). The lesion detection means 72 detects lesion candidates from the endoscopic images (step S72). The heat map generation means 73 generates a heat map that uses colors to represent the lesion likelihood of the lesion candidates included in the endoscopic images (step S73). The display control means 74 superimposes and displays areas where the lesion likelihood is equal to or greater than a first threshold on the endoscopic images in a first display mode, and superimposes and displays areas where the lesion likelihood is equal to or greater than a second threshold on the endoscopic images in a second display mode (step S74).

[0052] Although the present disclosure has been described above with reference to the embodiments and examples, the present disclosure is not limited to the above-described embodiments and examples. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure.

[0053] (Supplementary Note 1) An endoscopic examination support device comprising: an image acquisition means for acquiring endoscopic images captured by an endoscope; a lesion detection means for detecting lesion candidates from the endoscopic images; a heat map generation means for generating a heat map in which the lesion possibility of the lesion candidates contained in the endoscopic images is represented by color; and a display control means for superimposing and displaying, on the endoscopic images, areas where the lesion possibility is equal to or greater than a first threshold in a first display mode, and superimposing and displaying, on the endoscopic images, areas where the lesion possibility is equal to or greater than a second threshold in a second display mode, wherein the second threshold is a value greater than the first threshold.

[0054] (Supplementary Note 2) The endoscopic examination support device according to Supplementary Note 1, wherein the second display mode has a transparency different from that of the first display mode.

[0055] (Supplementary Note 3) The endoscopic examination support device according to Supplementary Note 2, wherein the second display mode has a higher transparency than the first display mode.

[0056] (Supplementary Note 4) The endoscopic examination support device according to any one of Supplementary Notes 1 to 3, wherein a predetermined mark is superimposed on the center or center of gravity of an area most likely to have a lesion.

[0057] (Supplementary Note 5) The endoscopic examination support device according to Supplementary Note 1, wherein the first threshold value and the second threshold value can be set for each user.

[0058] (Supplementary Note 6) The endoscopic examination support device according to Supplementary Note 4, wherein whether or not the first display mode, the second display mode, and the predetermined mark are displayed can be set for each user.

[0059] (Appendix 7) A method for assisting endoscopic examination, comprising: acquiring endoscopic video captured by an endoscope; detecting lesion candidates from the endoscopic video; generating a heat map in which the lesion likelihood of the lesion candidates contained in the endoscopic video is represented by color; superimposing and displaying an area where the lesion likelihood is equal to or greater than a first threshold on the endoscopic video in a first display mode; and superimposing and displaying an area where the lesion likelihood is equal to or greater than a second threshold on the endoscopic video in a second display mode, wherein the second threshold is greater than the first threshold.

[0060] (Appendix 8) A recording medium having recorded thereon a program that causes a computer to execute processing of acquiring endoscopic video captured by an endoscope, detecting lesion candidates from the endoscopic video, generating a heat map that uses colors to represent the lesion likelihood of the lesion candidates contained in the endoscopic video, superimposing and displaying areas where the lesion likelihood is equal to or greater than a first threshold on the endoscopic video in a first display mode, and superimposing and displaying areas where the lesion likelihood is equal to or greater than a second threshold on the endoscopic video in a second display mode, wherein the second threshold is a value greater than the first threshold.

[0061] REFERENCE SIGNS LIST 1 Endoscopy support device 2 Display device 3 Endoscope 11 Processor 12 Memory 13 Interface 21 Lesion detection unit 22 Heat map generation unit 23 Display control unit 100 Endoscopy system

Claims

1. an image acquisition means for acquiring an endoscopic image captured by an endoscope; a lesion detection means for detecting lesion candidates from the endoscopic video; a heat map generating means for generating a heat map in which the lesion possibility of the lesion candidate included in the endoscopic image is expressed by color; a display control means for superimposing and displaying an area where the lesion possibility is equal to or greater than a first threshold on the endoscopic image in a first display mode, and for superimposing and displaying an area where the lesion possibility is equal to or greater than a second threshold on the endoscopic image in a second display mode; Equipped with The second threshold value is a value greater than the first threshold value.

2. The endoscopic examination support device according to claim 1 , wherein the second display mode has a transparency different from that of the first display mode.

3. The endoscopic examination support device according to claim 2 , wherein the second display mode has a higher transparency than the first display mode.

4. 4. The endoscopic examination support device according to claim 1, wherein a predetermined mark is displayed superimposed on the center or center of gravity of an area most likely to have a lesion.

5. The endoscopic examination support device according to claim 1 , wherein the first threshold value and the second threshold value can be set for each user.

6. The endoscopic examination support device according to claim 4, wherein the first display mode, the second display mode, and the predetermined mark can be set by each user as to whether or not to display them.

7. A computer-implemented method for assisting endoscopic examination, comprising: Acquire endoscopic images captured by the endoscope, detecting a lesion candidate from the endoscopic image; generating a heat map that represents the lesion possibility of the lesion candidate included in the endoscopic image using color; a region where the lesion possibility is equal to or greater than a first threshold is superimposed on the endoscopic image in a first display mode, and a region where the lesion possibility is equal to or greater than a second threshold is superimposed on the endoscopic image in a second display mode; An endoscopic examination assistance method, wherein the second threshold value is a value greater than the first threshold value.

8. Acquire endoscopic images captured by the endoscope, detecting a lesion candidate from the endoscopic image; generating a heat map that represents the lesion possibility of the lesion candidate included in the endoscopic image using color; a region where the lesion possibility is equal to or greater than a first threshold is superimposed on the endoscopic image in a first display mode, and a region where the lesion possibility is equal to or greater than a second threshold is superimposed on the endoscopic image in a second display mode; The second threshold is a value greater than the first threshold.