Endoscopy assistance device, endoscopy assistance method, and recording medium

Abstract

In this endoscopy assistance device, a lesion detection means detects a lesion included in an endoscopic image obtained by imaging the interior of a subject with an endoscopic camera. A reference position setting means sets in the endoscopic image a plurality of reference positions to be used to track the lesion, and sets in the endoscopic image a new reference position to replace a reference position among the plurality of reference positions that has exited the endoscopic image due to movement of the endoscopic camera. A display control means generates a display image by adding, to the endoscopic image, visual information indicating the direction of at least one lesion that has strayed outside the endoscopic image, on the basis of paths of the plurality of reference positions, including the new reference position. This endoscopy assistance device can be used to assist with decision-making by a user performing an endoscopy.

Description

Endoscopic examination support device, endoscopic examination support method, and recording medium

[0008] ,

[0001] The present disclosure relates to a technology that can be used when presenting information for supporting an endoscopic examination.

[0002] Technologies for presenting information for supporting an endoscopic examination are known.

[0003] Specifically, for example, in Patent Document 1, in an endoscopic device, when it is determined that a specific part selected by a user from within an image exceeds the display range of the display image, a technique is disclosed of inserting an arrow symbol indicating the direction of the specific part into the display image.

[0004] Japanese Patent Application Laid-Open No. 2011-10841

[0005] However, according to the technology disclosed in Patent Document 1, for example, among a plurality of lesions discovered at positions close to each other, there is a problem that it may take time to rediscover a lesion that has moved out of the display image as the observation range of the endoscope moves.

[0006] One object of the present disclosure is to provide an endoscopic examination support device capable of shortening the time required to rediscover a lesion when the discovered lesion has moved out of the screen as the observation range of the endoscope moves.

[0007] In one aspect of the present disclosure, an endoscopic examination support device includes a lesion detection means for detecting a lesion included in an endoscopic image obtained by imaging the inside of a subject with an endoscopic camera, a reference position setting means for setting a plurality of reference positions used for tracking the lesion in the endoscopic image, and setting a new reference position in the endoscopic image to replace a reference position that has moved outside the endoscopic image as the endoscopic camera moves among the plurality of reference positions, and a display control means for generating a display image by adding visual information indicating the direction of at least one of the lesions that has moved outside the endoscopic image based on the trajectory of the plurality of reference positions including the new reference position to the endoscopic image.

[0008] In another aspect of this disclosure, the endoscopic examination support method is a computer-operated endoscopic examination support method that detects a lesion contained in an endoscopic image obtained by imaging the inside of a subject with an endoscopic camera, sets a plurality of reference positions to be used for tracking the lesion in the endoscopic image, sets a new reference position in the endoscopic image to replace a reference position that has moved out of the endoscopic image due to the movement of the endoscopic camera, and generates a display image by adding visual information to the endoscopic image indicating the direction of at least one of the lesions that has moved out of the endoscopic image based on the trajectory of the plurality of reference positions including the new reference position.

[0009] In yet another aspect of this disclosure, the recording medium records a program that causes a computer to perform a process of detecting lesions contained in an endoscopic image obtained by imaging the inside of a subject with an endoscope camera, setting a plurality of reference positions in the endoscopic image for use in tracking the lesions, setting new reference positions in the endoscopic image to replace reference positions that have moved outside the endoscopic image due to the movement of the endoscope camera, and generating a display image by adding visual information to the endoscopic image indicating the direction of at least one of the lesions that has moved outside the endoscopic image based on the trajectory of the plurality of reference positions including the new reference position.

[0010] According to this disclosure, when a previously detected lesion moves out of the screen due to movement of the endoscope's observation range, the time required to rediscover the lesion can be shortened.

[0011] A diagram showing the general configuration of the endoscopic examination system related to this disclosure. A block diagram showing an example of the hardware configuration of the endoscopic examination support device related to this disclosure. A block diagram showing an example of the functional configuration of the endoscopic examination support device related to this disclosure. A diagram showing an example of an endoscopic image including a lesion. A diagram showing an example of a display image generated by the endoscopic examination support device related to this disclosure. A diagram showing an example of a method for setting a reference position used for tracking lesions. A diagram to explain a specific example of the process for setting a new reference position to replace a reference position that has gone off-screen. A diagram showing another example of a display image generated by the endoscopic examination support device related to this disclosure. A diagram showing another example of a display image generated by the endoscopic examination support device related to this disclosure. A diagram showing another example of a display image generated by the endoscopic examination support device related to this disclosure. A diagram showing another example of a display image generated by the endoscopic examination support device related to this disclosure. A diagram showing another example of a display image generated by the endoscopic examination support device related to this disclosure. A diagram showing another example of a display image generated by the endoscopic examination support device related to this disclosure. A diagram showing another example of a display image generated by the endoscopic examination support device related to this disclosure. A diagram showing another example of a display image generated by the endoscopic examination support device related to this disclosure. A flowchart showing an example of the processing performed in the endoscopic examination support device relating to this disclosure. A block diagram showing another example of the functional configuration of the endoscopic examination support device relating to this disclosure. A flowchart showing another example of the processing performed in the endoscopic examination support device relating to this disclosure.

[0012] Preferred embodiments of this disclosure will be described below with reference to the drawings.

[0013] <First Embodiment> [System Configuration] Figure 1 is a diagram showing the schematic configuration of an endoscopic examination system according to the present disclosure. As shown in Figure 1, the endoscopic examination system 100 comprises an endoscopic examination support device 1, a display device 2, and an endoscope scope 3 connected to the endoscopic examination support device 1.

[0014] The endoscopic examination support device 1 acquires video (hereinafter also referred to as endoscopic video) from the endoscope scope 3, which includes time-series images obtained by imaging the subject during the endoscopic examination, and displays the image on the display device 2 for confirmation by the user, such as a physician, performing the endoscopic examination. The endoscopic examination support device 1 also acquires images of the inside of the large intestine obtained during the endoscopic examination from the endoscope scope 3 as endoscopic video. Furthermore, the endoscopic examination support device 1 detects lesions contained in the endoscopic image obtained by imaging the inside of the subject with the endoscope camera, and displays information indicating the detected lesions on the display image.

[0015] Thus, when displaying lesions detected in endoscopic images, if the observation range of the endoscope changes as the user moves the endoscope, the detected lesion may move off-screen, requiring time to rediscover the lesion. Therefore, in this embodiment, the endoscopic examination support device 1 sets multiple reference positions used for tracking lesions and calculates the direction and distance of detected lesions that have moved off-screen by performing calculations using the trajectories of each reference position, and generates a display image that allows recognition of the direction and distance of detected lesions that have moved off-screen. As a result, according to this embodiment, when a detected lesion moves off-screen as the observation range of the endoscope changes, the time required to rediscover the lesion can be shortened. Furthermore, the endoscopic examination support device 1 can be used to support the decision-making of the user performing the endoscopic examination.

[0016] The display device 2 includes, for example, a liquid crystal monitor. The display device 2 also displays display images, etc., output from the endoscopy support device 1. According to this disclosure, for example, processing of the endoscopy support device 1 may be performed in response to operations performed on a touch panel provided on the display device 2.

[0017] The endoscope scope 3 mainly comprises an operating unit 36 ​​for the user to input commands such as air insufflation, water insufflation, angle adjustment, and imaging instructions; a flexible shaft 37 that is inserted into the organ being examined in the patient; a tip 38 that incorporates an endoscope camera such as an image sensor; and a connection unit 39 for connecting to the endoscopy support device 1. The endoscope camera can obtain an endoscopic image by imaging the inside of the large intestine, which is a tubular organ. The endoscope camera can also obtain an endoscopic image by imaging the inside of the patient into which the shaft 37 of the endoscope scope 3 is inserted.

[0018] [Hardware Configuration] Figure 2 is a block diagram showing an example of the hardware configuration of an endoscopy support device according to the present disclosure. The endoscopy support device 1 mainly includes a processor 11, a memory 12, an interface 13, an input unit 14, a light source unit 15, a sound output unit 16, and a database (hereinafter referred to as "DB") 17. Each of these elements is connected via a data bus 19.

[0019] The processor 11 executes predetermined processes by running programs stored in the memory 12. The processor 11 is a processor such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), or TPU (Tensor Processing Unit). The processor 11 may be composed of multiple processors. The processor 11 is an example of a computer. The processor 11 also performs processes such as generating a display image that can recognize the direction and distance of lesions that have moved off-screen.

[0020] Memory 12 is composed of various volatile memories used as working memory, such as RAM (Random Access Memory) and ROM (Read Only Memory), and non-volatile memory that stores information necessary for processing the endoscopic examination support device 1. Memory 12 may also include an external storage device such as a hard disk connected to or built into the endoscopic examination support device 1, or it may include a storage medium such as a removable flash memory or disk medium. Memory 12 stores a program for the endoscopic examination support device 1 to execute each of the processes related to this disclosure. In addition, memory 12 temporarily stores a series of endoscopic images obtained by the endoscope scope 3 during an endoscopic examination, based on the control of the processor 11.

[0021] Interface 13 performs interface operations between the endoscopic examination support device 1 and external devices. For example, interface 13 supplies display images generated by processor 11 to display device 2. Interface 13 also supplies illumination light generated by light source unit 15 to endoscope scope 3. Interface 13 also supplies electrical signals indicating endoscopic images supplied from endoscope scope 3 to processor 11. Interface 13 also supplies endoscopic images extracted from endoscopic images to processor 11. Interface 13 may be a communication interface such as a network adapter for wired or wireless communication with external devices, or it may be a hardware interface compliant with USB (Universal Serial Bus), SATA (Serial AT Attachment), etc.

[0022] The input unit 14 generates an input signal corresponding to user operation. The input unit 14 includes at least one device, such as a button, touch panel, remote controller, foot switch, and voice input device. The light source unit 15 generates light to be supplied to the tip 38 of the endoscope scope 3. The light source unit 15 may also incorporate a pump for supplying water or air to the endoscope scope 3. The sound output unit 16 outputs sound based on the control of the processor 11.

[0023] DB17 stores endoscopic images and other data acquired from the patient's past endoscopic examinations. DB17 may include an external storage device such as a hard disk connected to or built into the endoscopic examination support device 1, or it may include a storage medium such as a removable flash memory. Alternatively, instead of having DB17 within the endoscopic examination system 100, DB17 may be located on an external server, and relevant information may be obtained from the server via communication.

[0024] The endoscopic examination support device 1 may also be equipped with a sensor capable of measuring the rotation and translation of the endoscope camera, such as a magnetic sensor. Furthermore, the endoscopic examination support device 1 may have an endoscope insertion shape observation device that observes the shape of the shaft 37 inserted inside the patient, for example, based on the detection result of a magnetic field generated from a coil provided on the shaft 37 of the endoscope scope 3.

[0025] [Functional Configuration] Figure 3 is a block diagram showing an example of the functional configuration of an endoscopic examination support device according to the present disclosure. As shown in Figure 3, the endoscopic examination support device 1 includes a lesion detection unit 21, a trigger generation unit 22, a lesion tracking unit 23, a tracking information storage unit 24, and a display control unit 25.

[0026] The lesion detection unit 21 functions as a means for detecting lesions. The lesion detection unit 21 uses a trained image recognition model or a trained machine learning model to perform the process of detecting lesions LE from the endoscopic image EG. The lesion detection unit 21 also acquires information indicating the position and size of lesions LE within the endoscopic image EG as lesion detection information BJ, and outputs the acquired lesion detection information BJ to the lesion tracking unit 23 and the display control unit 25. If the lesion detection unit 21 is unable to detect a lesion from the endoscopic image EG, it acquires information indicating that no lesion exists within the endoscopic image EG as lesion detection information BJ, and outputs the acquired lesion detection information BJ to the lesion tracking unit 23 and the display control unit 25.

[0027] The trigger generation unit 22 determines whether the input signal NS generated by the input unit 14 or the operation unit 36 ​​is a signal indicating a predetermined instruction. If the trigger generation unit 22 determines that the input signal NS is a signal indicating a predetermined instruction, it generates a trigger signal TS and outputs the generated trigger signal TS to the lesion tracking unit 23.

[0028] The lesion tracking unit 23 starts processing related to the tracking of the lesion LE when the lesion detection information BJ obtained from the lesion detection unit 21 includes information indicating the location and size of the lesion LE, and when a trigger signal TS is input from the trigger generation unit 22. The lesion tracking unit 23 continues processing related to the tracking of the lesion LE, for example, until an instruction indicating the end of the endoscopic examination is given in the input unit 14 or the operation unit 36. The lesion tracking unit 23 also stores the lesion tracking information TJ obtained by the processing related to the tracking of the lesion LE in the tracking information storage unit 24. The lesion tracking unit 23 also has a reference position setting unit 23A and a position relationship acquisition unit 23B.

[0029] The reference position setting unit 23A functions as a reference position setting means. Furthermore, when the lesion detection information BJ obtained from the lesion detection unit 21 includes information indicating the position and size of the lesion LE, and a trigger signal TS is input from the trigger generation unit 22, the reference position setting unit 23A sets multiple reference positions around the lesion LE to be used for tracking the lesion LE. In this disclosure, a lesion for which multiple reference positions for tracking have been set will be referred to as a detected lesion. In this disclosure, for example, "detected lesion" can be replaced with "discovered lesion".

[0030] The reference position setting unit 23A, with multiple reference positions corresponding to detected lesions LE set, sets a new reference position to replace the reference position that has moved out of the screen while maintaining the existing reference positions remaining within the screen, if at least one of the multiple reference positions moves out of the screen due to the movement of the endoscope camera. In addition, the reference position setting unit 23A can acquire the in-screen coordinate position corresponding to each reference position set.

[0031] In this disclosure, the reference position is set as a position that moves within the screen and also moves from within the screen to outside the screen as the field of view of the endoscope camera moves. For example, if the field of view of the endoscope camera moves to the left, the reference position moves to the right within the screen. Furthermore, "within the screen" in this disclosure can be rephrased as, for example, within the field of view of the endoscope camera, within the display range of the endoscope image EG, or within the endoscope image EG. Furthermore, "outside the screen" in this disclosure can be rephrased as, for example, outside the field of view of the endoscope camera, outside the display range of the endoscope image EG, or outside the endoscope image EG.

[0032] The position relationship acquisition unit 23B acquires lesion tracking information TJ, which shows the positional relationship between the lesion LE and each of the multiple reference positions, when the lesion detection information BJ obtained from the lesion detection unit 21 includes information indicating the position and size of the lesion LE, and when a trigger signal TS is input from the trigger generation unit 22. Furthermore, if at least one of the multiple reference positions corresponding to the detected lesion LE moves off-screen, the position relationship acquisition unit 23B acquires lesion tracking information TJ, which shows the positional relationship between the existing reference position remaining on-screen and a new reference position that replaces the reference position that moved off-screen. The position relationship acquisition unit 23B also stores the acquired lesion tracking information TJ in the tracking information storage unit 24. Details of the information included in the lesion tracking information TJ will be explained later.

[0033] The tracking information storage unit 24 stores lesion tracking information TJ associated with each detected lesion LE.

[0034] The display control unit 25 functions as a display control means. The display control unit 25 uses lesion detection information BJ obtained from the lesion detection unit 21 to generate a display image HG having a display mode that allows the position of the lesion LE in the endoscopic image EG to be visually confirmed, and outputs the generated display image HG to the display device 2. The display control unit 25 also uses lesion tracking information TJ corresponding to the detected lesion LE stored in the tracking information storage unit 24 to determine whether the detected lesion LE has moved off-screen. The display control unit 25 also uses lesion tracking information TJ corresponding to the detected lesion LE to calculate the direction and distance of the detected lesion LE that is off-screen. The display control unit 25 then generates a display image HG by adding visual information corresponding to the calculated direction and distance of the detected lesion LE to the endoscopic image EG obtained after the detected lesion LE has moved off-screen, and outputs the generated display image HG to the display device 2.

[0035] [Specific Examples] Next, we will explain specific examples related to this disclosure.

[0036] Figure 4 shows an example of an endoscopic image containing a lesion. When an endoscopic image EG1, such as the one in Figure 4, is input to the lesion detection unit 21, it detects a lesion LE1 contained within the endoscopic image EG1. The lesion detection unit 21 also acquires information indicating the position and size of the lesion LE1 within the endoscopic image EG1 as lesion detection information BJ1, and outputs the acquired lesion detection information BJ1 to the lesion tracking unit 23 and the display control unit 25.

[0037] Figure 5 shows an example of a display image generated by the endoscopic examination support device according to this disclosure. The display control unit 25 uses the lesion detection information BJ1 obtained from the lesion detection unit 21 to generate a detection frame DW1 surrounding the lesion LE1 within the endoscopic image EG1. The display control unit 25 also generates a display image HG1 by superimposing the detection frame DW1 onto the endoscopic image EG1 and outputs the generated display image HG1 to the display device 2. Through this process, the display control unit 25 can display, for example, the display image HG1 shown in Figure 5 on the display device 2. Note that the detection frame DW1 may have a shape other than a rectangle.

[0038] The trigger generation unit 22 determines whether the input signal NS1 generated by the input unit 14 or the operation unit 36 ​​is a signal indicating a predetermined instruction.

[0039] Specifically, the trigger generation unit 22 determines, for example, whether the input signal NS1 corresponds to a signal indicating one of the following: an instruction related to still photography, an instruction related to illumination light switching, or an instruction related to lesion confirmation. An instruction related to still photography is given, for example, when recording an image including lesion LE1. An instruction related to illumination light switching is given, for example, when switching between an observation mode in which lesion LE1 is observed with white light and an observation mode in which lesion LE1 is observed with a special light different from white light. An instruction related to lesion confirmation is given, for example, when the user visually confirms that lesion LE1 is actually a lesion.

[0040] The trigger generation unit 22 generates a trigger signal TS1 when it determines that the input signal NS1 is a signal indicating a predetermined instruction, and outputs the generated trigger signal TS1 to the lesion tracking unit 23.

[0041] The reference position setting unit 23A sets a plurality of reference positions KP used for tracking the lesion LE1 around the lesion LE1 when the lesion detection information BJ1 includes information indicating the position and size of the lesion LE1 and a trigger signal TS1 is input.

[0042] Figure 6 shows an example of a method for setting reference positions used for tracking lesions. Specifically, the reference position setting unit 23A sets four reference positions KP around the lesion LE1 in the display image HG1, for example, as shown in Figure 6. The reference positions KP are set as invisible rectangular areas having a predetermined size and positioned at predetermined locations in the display image HG1. Hereafter, the four reference positions KP initially set around the lesion LE1 will be referred to as reference positions KP1, KP2, KP3, and KP4, as needed. In this disclosure, "reference position" may be replaced with "reference area." Furthermore, the reference positions KP1, KP2, KP3, and KP4 set around the lesion LE1 can be treated as the initial reference positions used for tracking the lesion LE1.

[0043] The reference position setting unit 23A is not limited to four, and a plurality of reference positions KP may be set at predetermined positions of the display image HG1 or the endoscopic image EG1. Further, it is desirable that the reference position setting unit 23A sets a plurality of reference positions KP at positions that do not touch the edge of the display image HG1 or the endoscopic image EG1, that is, inside the edge of the display image HG1 or the endoscopic image EG1. Further, it is desirable that the reference position setting unit 23A sets a reference position KP having a size larger than the size of one pixel in the display image HG1 or the endoscopic image EG1. Further, as long as the reference position KP has a predetermined size in the display image HG1 or the endoscopic image EG1, the reference position setting unit 23A may set a reference position KP having a shape different from a rectangular shape.

[0044] When the lesion detection information BJ1 includes information indicating the position and size of the lesion LE1 and the trigger signal TS1 is input, the position relationship acquisition unit 23B acquires lesion tracking information TJ1 indicating the position relationship between the lesion LE1 and each of the plurality of reference positions KP.

[0045] Here, a specific example of the process related to the acquisition of the lesion tracking information TJ1 will be described.

[0046] When the lesion detection information BJ1 includes information indicating the position and size of the lesion LE1 and the trigger signal TS1 is input, the position relationship acquisition unit 23B sets a coordinate system CS having the position of the lesion LE1 in the display image HG1 as the origin or the reference point.

[0047] The position relationship acquisition unit 23B acquires lesion tracking information TJ1 including distance information DJ1 indicating the distance from the lesion LE1 to each of the reference positions KP in the coordinate system CS and angle information KJ1 indicating the angle from the lesion LE1 to each of the reference positions KP in the coordinate system CS.

[0048] Specifically, for example, the position relationship acquisition unit 23B can acquire, as distance information DJ1, information indicating the distance D1 from the lesion LE1 to the reference position KP1 in the coordinate system CS, information indicating the distance D2 from the lesion LE1 to the reference position KP2 in the coordinate system CS, information indicating the distance D3 from the lesion LE1 to the reference position KP3 in the coordinate system CS, and information indicating the distance D4 from the lesion LE1 to the reference position KP4 in the coordinate system CS. Further, for example, the position relationship acquisition unit 23B can acquire, as angle information KJ1, information indicating the angle θ1 from the lesion LE1 to the reference position KP1 in the coordinate system CS, information indicating the angular distance θ2 from the lesion LE1 to the reference position KP2 in the coordinate system CS, information indicating the angle θ3 from the lesion LE1 to the reference position KP3 in the coordinate system CS, and information indicating the angle θ4 from the lesion LE1 to the reference position KP4 in the coordinate system CS.

[0049] When the reference positions KP1 to KP4 are set, the position relationship acquisition unit 23B starts acquiring lesion tracking information TJ1 including the distance information DJ1 and the angle information KJ1. Further, the position relationship acquisition unit 23B stores the lesion tracking information TJ1 including the distance information DJ1 and the angle information KJ1 in the tracking information storage unit 24.

[0050] In a state where a plurality of reference positions KP corresponding to the detected lesion LE1 are set, when at least any one of the plurality of reference positions KP goes out of the screen, the reference position setting unit 23A sets a new reference position KP to replace the reference position KP that has gone out of the screen while maintaining the existing reference positions KP remaining within the screen. Note that the reference position setting unit 23A can determine that the reference position KP has gone out of the screen, for example, when the coordinate positions of all the pixels included in the rectangular region set as the reference position KP indicate coordinate positions outside the screen.

[0051] Here, we will explain a specific example of the process for setting a new reference position KP to replace a reference position that has moved outside the screen. Figure 7 is a diagram illustrating a specific example of the process for setting a new reference position to replace a reference position that has moved outside the screen. Note that in Figure 7, for the sake of illustration and explanation, elements other than the reference position set within the screen have been omitted.

[0052] The reference position setting unit 23A detects, for example, that reference positions KP2 and KP4 have moved off-screen when the field of view of the endoscope camera moves parallel to the left while reference positions KP1 to KP4 are set. Furthermore, when the reference position setting unit 23A detects that reference positions KP2 and KP4 have moved off-screen, it maintains reference positions KP1 and KP3, which remain within the screen, and sets new reference positions KP5 and KP6 within the screen to replace those reference positions KP2 and KP4. For example, the reference position setting unit 23A can set reference position KP5 at a position within the screen corresponding to the position of reference position KP1 initially set around lesion LE1 (see Figure 7). Similarly, the reference position setting unit 23A can set reference position KP6 at a position within the screen corresponding to the position of reference position KP3 initially set around lesion LE1 (see Figure 7).

[0053] The reference position setting unit 23A detects, for example, that reference positions KP6 and KP3 have moved out of the screen when the field of view of the endoscope camera moves vertically upward while reference positions KP1, KP3, KP5, and KP6 are set. Furthermore, when the reference position setting unit 23A detects that reference positions KP6 and KP3 have moved out of the screen, it maintains reference positions KP1 and KP5, which remain within the screen, and sets new reference positions KP7 and KP8 within the screen to replace those reference positions KP6 and KP3. For example, the reference position setting unit 23A can set reference position KP7 at a position within the screen corresponding to the position of reference position KP1 initially set around lesion LE1 (see Figure 7). Similarly, the reference position setting unit 23A can set reference position KP8 at a position within the screen corresponding to the position of reference position KP2 initially set around lesion LE1 (see Figure 7).

[0054] According to the process described above, the reference position setting unit 23A can set a reference position to be used for tracking the lesion LE1 during the period from when the trigger signal TS1 is input while the lesion LE1 is detected until the end of the endoscopic examination.

[0055] The position relationship acquisition unit 23B acquires lesion tracking information TJ2, which shows the positional relationship between an existing reference position KP remaining on the screen and a new reference position KP that replaces the reference position KP that has moved off-screen, when the lesion detection information BJ1 obtained after the trigger signal TS1 is input no longer contains information related to the detected lesion LE1. In addition, the position relationship acquisition unit 23B acquires lesion tracking information TJ3, which shows the positional relationship between an existing reference position KP remaining on the screen and the new reference position KP, when a new reference position KP is set after the detected lesion LE1 has moved off-screen.

[0056] Here, we will explain a specific example of the process for acquiring lesion tracking information TJ2 and TJ3. In the following explanation, we will use as an example a case where reference positions KP1 to KP8, as shown in Figure 7, are set according to the movement of the field of view of the endoscope camera.

[0057] When reference positions KP5 and KP6 are set instead of reference positions KP2 and KP4, the position relationship acquisition unit 23B starts acquiring lesion tracking information TJ2, which includes distance information DJ2 indicating the distance from each of the existing reference positions to each of the new reference positions, and angle information KJ2 indicating the angle from each of the existing reference positions to each of the new reference positions. Furthermore, even after starting to acquire lesion tracking information TJ2, the position relationship acquisition unit 23B continues to acquire lesion tracking information TJ1.

[0058] Specifically, the position relationship acquisition unit 23B acquires, for example, information indicating the distance D15 from reference position KP1 to reference position KP5, information indicating the distance D16 from reference position KP1 to reference position KP6, information indicating the distance D35 from reference position KP3 to reference position KP5, and information indicating the distance D36 from reference position KP3 to reference position KP6 as distance information DJ2. In addition, the position relationship acquisition unit 23B acquires, for example, information indicating the angle θ15 from reference position KP1 to reference position KP5, information indicating the angle θ16 from reference position KP1 to reference position KP6, information indicating the angle θ35 from reference position KP3 to reference position KP5, and information indicating the angle θ36 from reference position KP3 to reference position KP6 as angle information KJ2. The positional relationship acquisition unit 23B then stores the lesion tracking information TJ2, which includes distance information DJ2 and angle information KJ2, in the tracking information storage unit 24.

[0059] When reference positions KP7 and KP8 are set instead of reference positions KP6 and KP3, the position relationship acquisition unit 23B starts acquiring lesion tracking information TJ3, which includes distance information DJ3 indicating the distance from each of the existing reference positions to each of the new reference positions, and angle information KJ3 indicating the angle from each of the existing reference positions to each of the new reference positions. Furthermore, even after starting to acquire lesion tracking information TJ3, the position relationship acquisition unit 23B continues to acquire lesion tracking information TJ1 and TJ2.

[0060] Specifically, the position relationship acquisition unit 23B acquires, for example, information indicating the distance D17 from reference position KP1 to reference position KP7, information indicating the distance D18 from reference position KP1 to reference position KP8, information indicating the distance D57 from reference position KP5 to reference position KP7, and information indicating the distance D58 from reference position KP5 to reference position KP8 as distance information DJ3. In addition, the position relationship acquisition unit 23B acquires, for example, information indicating the angle θ17 from reference position KP1 to reference position KP7, information indicating the angle θ18 from reference position KP1 to reference position KP8, information indicating the angle θ57 from reference position KP5 to reference position KP7, and information indicating the angle θ58 from reference position KP5 to reference position KP8 as angle information KJ3. The positional relationship acquisition unit 23B then stores the lesion tracking information TJ3, which includes distance information DJ3 and angle information KJ3, in the tracking information storage unit 24.

[0061] According to the process described above, the positional relationship acquisition unit 23B can acquire lesion tracking information TJ, which indicates the trajectory of the reference position KP that is displaced as the field of view of the endoscope camera moves, during the period from when the trigger signal TS1 is input while the lesion LE1 is detected until the end of the endoscopic examination. Furthermore, the positional relationship acquisition unit 23B can store the lesion tracking information TJ acquired during the period from when the trigger signal TS1 is input while the lesion LE1 is detected until the end of the endoscopic examination in the tracking information storage unit 24.

[0062] The display control unit 25 identifies the trajectory of the reference position KP using the distance information DJ and angle information KJ contained in the lesion tracking information TJ corresponding to the detected lesion LE1 stored in the tracking information storage unit 24. The display control unit 25 also determines whether the detected lesion LE1 has moved off-screen by performing calculations using the trajectory of the reference position KP. Furthermore, the display control unit 25 calculates the direction and distance of the detected lesion LE1 that is off-screen by performing calculations to trace the trajectory of the reference position KP.

[0063] The display control unit 25 can, for example, identify the trajectories of reference positions KP1 to KP6 using the distance information DJ and angle information KJ contained in the lesion tracking information TJ1 and TJ2 when lesion tracking information TJ1 and TJ2 are stored in the tracking information storage unit 24.

[0064] The display control unit 25 can, for example, identify the trajectories of reference positions KP1 to KP8 using the distance information DJ and angle information KJ contained in the lesion tracking information TJ1, TJ2, and TJ3 when lesion tracking information TJ1, TJ2, and TJ3 is stored in the tracking information storage unit 24.

[0065] The display control unit 25 generates a display image HG by adding an indicator IN corresponding to the calculated direction and distance of the detected lesion LE1 to the endoscopic image EG obtained after the detected lesion LE1 has moved off-screen, and outputs the generated display image HG to the display device 2.

[0066] Here, we will describe a specific example of a display image generated by the display control unit 25.

[0067] The display control unit 25 generates a display image HG by adding a band-shaped indicator IN to the outer edge of the endoscopic image EG obtained after the detected lesion LE1 has moved off-screen, at a position corresponding to the direction of the detected lesion LE1.

[0068] Specifically, in the following example, the display control unit 25 indicates the direction from the position corresponding to the current field of view of the endoscope camera to the detected lesion LE1 located outside the screen by the position of a band-shaped indicator IN attached to the outer edge of the endoscope image EG. The display control unit 25 also indicates the distance from the position corresponding to the current field of view of the endoscope camera to the detected lesion LE1 located outside the screen by the width of the band-shaped indicator IN attached to the outer edge of the endoscope image EG. In other words, the display control unit 25 can attach indicator INs with different widths to the endoscope image EG depending on the distance to the detected lesion LE1 located outside the endoscope image EG. In the following example, the display control unit 25 widens the band-shaped indicator IN attached to the outer edge of the endoscope image EG when the distance from the position corresponding to the current field of view of the endoscope camera to the detected lesion LE1 located outside the screen is short, and narrows the width of the indicator IN when the distance is long.

[0069] Figure 8 shows another example of a display image generated by the endoscopic examination support device according to the present disclosure. The display control unit 25 generates a display image HG11 as shown in Figure 8 when, for example, the calculated direction of a detected lesion LE1 located outside the screen is to the right, and the calculated distance corresponding to the detected lesion LE1 is less than the threshold TH1. The display image HG11 is generated, for example, as an image in which a band-shaped indicator IN11 with a width LW1 is added to the right outer edge of the endoscopic image EG11 obtained after the detected lesion LE1 has moved outside the screen.

[0070] Figure 9 shows another example of a display image generated by the endoscopic examination support device according to this disclosure. The display control unit 25 generates a display image HG12 as shown in Figure 9 when, for example, the calculated direction of a detected lesion LE1 located outside the screen is in the lower right direction, and the calculated distance corresponding to the detected lesion LE1 falls within the range from threshold TH1 to threshold TH2 (>TH1). The display image HG12 is generated as an image in which a band-shaped indicator IN12 with a width LW2 is added to the lower right outer edge of the endoscopic image EG12 obtained after the detected lesion LE1 has moved outside the screen. Note that the width LW2 only needs to be set to a value less than the width LW1.

[0071] Figure 10 shows another example of a display image generated by the endoscopic examination support device according to the present disclosure. The display control unit 25 generates a display image HG13 as shown in Figure 10 when, for example, the calculated direction of a detected lesion LE1 located outside the screen is to the upper right, and the calculated distance corresponding to the detected lesion LE1 is less than the threshold TH1. The display image HG13 is generated, for example, as an image in which a band-shaped indicator IN13 having a width LW1 is added to the upper right outer edge of the endoscopic image EG13 obtained after the detected lesion LE1 has moved outside the screen.

[0072] Figure 11 shows another example of a display image generated by the endoscopic examination support device according to the present disclosure. The display control unit 25 generates a display image HG14 as shown in Figure 11 when, for example, the calculated direction of a detected lesion LE1 located outside the screen is downward, and the calculated distance corresponding to the detected lesion LE1 falls within the range from threshold TH1 to threshold TH2. The display image HG14 is generated, for example, as an image in which a band-shaped indicator IN14 with a width LW2 is added to the lower outer edge of the endoscopic image EG14 obtained after the detected lesion LE1 has moved outside the screen.

[0073] Figure 12 shows another example of a display image generated by the endoscopic examination support device according to the present disclosure. The display control unit 25 generates a display image HG15 as shown in Figure 12 when, for example, the calculated direction of a detected lesion LE1 located outside the screen is upward and the calculated distance corresponding to the detected lesion LE1 is less than the threshold TH1. The display image HG15 is generated, for example, as an image in which a band-shaped indicator IN15 with a width LW1 is added to the upper outer edge of the endoscopic image EG15 obtained after the detected lesion LE1 has moved outside the screen.

[0074] Figure 13 shows another example of a display image generated by the endoscopic examination support device according to this disclosure. The display control unit 25 generates a display image HG16 as shown in Figure 13 when, for example, the calculated direction of a detected lesion LE1 located outside the screen is in the lower left direction, and the calculated distance corresponding to the detected lesion LE1 is greater than the threshold TH2. The display image HG16 is generated as an image in which a band-shaped indicator IN16 with a width LW3 is added to the lower left outer edge of the endoscopic image EG16 obtained after the detected lesion LE1 has moved outside the screen. Note that the width LW3 only needs to be set to a value less than the width LW2.

[0075] Figure 14 shows another example of a display image generated by the endoscopic examination support device according to the present disclosure. The display control unit 25 generates a display image HG17 as shown in Figure 14, for example, when the calculated direction of a detected lesion LE1 located outside the screen is to the left and the calculated distance corresponding to the detected lesion LE1 is greater than the threshold TH2. The display image HG17 is generated, for example, as an image in which a band-shaped indicator IN17 with a width LW3 is added to the left outer edge of the endoscopic image EG17 obtained after the detected lesion LE1 has moved outside the screen.

[0076] Figure 15 shows another example of a display image generated by the endoscopic examination support device according to the present disclosure. The display control unit 25 generates a display image HG18 as shown in Figure 15, for example, when the calculated direction of a detected lesion LE1 located outside the screen is in the upper left direction, and the calculated distance corresponding to the detected lesion LE1 is greater than the threshold TH2. The display image HG18 is generated as an image in which a band-shaped indicator IN18 with a width LW3 is added to the upper left outer edge of the endoscopic image EG18 obtained after the detected lesion LE1 has moved outside the screen.

[0077] According to the process described above, the display control unit 25 can indicate the direction from the position corresponding to the current field of view of the endoscope camera to the detected lesion LE1 located outside the screen by the position of the band-shaped indicator IN attached to the outer edge of the endoscope image EG. Furthermore, according to the process described above, the display control unit 25 can indicate the distance from the position corresponding to the current field of view of the endoscope camera to the detected lesion LE1 located outside the screen by the width of the band-shaped indicator IN attached to the outer edge of the endoscope image EG. In other words, the display control unit 25 can attach indicator INs with different widths to the endoscope image EG depending on the distance to the detected lesion LE1 that has moved outside the endoscope image EG.

[0078] Figure 16 shows another example of a display image generated by the endoscopic examination support device according to the present disclosure. The display control unit 25 can, for example, display the calculated direction and distance corresponding to each of the multiple detected lesions LE when lesion tracking information TJ associated with each of the multiple different detected lesions LE is stored in the tracking information storage unit 24. Through such processing, the display control unit 25 can generate a display image HG21, for example, as shown in Figure 16.

[0079] The displayed image HG21 is generated by adding two indicators IN21 and IN22, corresponding to each of the two detected lesions LE, to the endoscopic image EG21 obtained after the two detected lesions LE have moved off-screen. According to indicator IN21 of the displayed image HG21, for one of the two detected lesions LE that are off-screen, the calculated direction is upward and the calculated distance is less than the threshold TH1. According to indicator IN22 of the displayed image HG21, for the other of the two detected lesions LE that are off-screen, the calculated direction is to the left and the calculated distance is greater than the threshold TH2.

[0080] According to this disclosure, it is desirable that different colors and / or patterns be applied to indicators IN21 and IN22 for each lesion. Furthermore, according to this disclosure, the display control unit 25 can, for example, add indicators IN21 and IN22 to adjacent positions on the outer edge of the endoscopic image EG21 when two detected lesions LE are located outside the screen in the same direction.

[0081] [Processing Flow] Next, the processing flow performed in the endoscopic examination support device 1 will be explained. Figure 17 is a flowchart showing an example of processing performed in the endoscopic examination support device according to this disclosure.

[0082] First, the endoscopic examination support device 1 detects the lesion LE contained within the endoscopic image EG (step S11).

[0083] Next, the endoscopic examination support device 1 determines whether the input signal NS generated by the input unit 14 or the operation unit 36 ​​is a signal indicating a predetermined instruction, while the lesion LE detected in step S11 is included in the endoscopic image EG (step S12).

[0084] If the endoscopic examination support device 1 determines that the input signal NS generated by the input unit 14 or the operation unit 36 ​​is not a signal indicating a predetermined instruction (step S12: NO), it terminates processing without tracking the lesion LE detected in step S11. Alternatively, if the endoscopic examination support device 1 determines that the input signal NS generated by the input unit 14 or the operation unit 36 ​​is a signal indicating a predetermined instruction (step S12: YES), it sets a plurality of reference positions KP to be used for tracking the lesion LE detected in step S11 (step S13).

[0085] Next, the endoscopic examination support device 1 acquires information indicating the positional relationship between the lesion LE detected in step S11 and each of the multiple reference positions KP set in step S13 as lesion tracking information TJ (step S14).

[0086] Next, if at least one of the multiple reference positions KP set in step S13 moves outside the screen, the endoscopic examination support device 1 maintains the existing reference positions KP remaining within the screen and sets a new reference position KP to replace the reference position KP that moved outside the screen (step S15). The endoscopic examination support device 1 also acquires information indicating the positional relationship between the existing reference positions KP and the new reference positions KP as lesion tracking information TJ (step S16).

[0087] Furthermore, if all of the multiple reference positions KP set in step S13 remain within the screen, the endoscopic examination support device 1 continues to acquire lesion tracking information TJ in step S14, while skipping steps S15 and S16 and proceeding to step S17.

[0088] The endoscopic examination support device 1 identifies the trajectory of the reference position KP using the lesion tracking information TJ acquired in steps S14 and S16. The endoscopic examination support device 1 also determines whether the detected lesion LE has moved off-screen by performing calculations using the trajectory of the reference position KP (step S17).

[0089] If the endoscopy support device 1 determines that a detected lesion LE remains within the screen (step S17: NO), it returns to step S15. If the endoscopy support device 1 determines that a detected lesion LE has moved outside the screen (step S17: YES), it generates a display image HG that can recognize the direction and distance of the detected lesion LE that is outside the screen (step S18). For example, the endoscopy support device 1 can identify the trajectory of a reference position KP using the lesion tracking information TJ acquired before the processing in step S18, and can calculate the direction and distance of the detected lesion LE that is outside the screen by performing calculations using the identified trajectory of the reference position KP. Furthermore, by performing the processing in step S18, the endoscopy support device 1 can generate a display image HG having a display mode similar to the display modes shown in Figures 8 to 15, for example.

[0090] After generating the display image HG in step S18, the endoscopic examination support device 1 performs the same processing as in step S15 to set a new reference position KP that replaces the reference position KP that has moved outside the screen while maintaining the existing reference position KP that remains within the screen (step S19). Furthermore, after performing the processing in step S19, the endoscopic examination support device 1 performs the same processing as in step S16 to acquire information indicating the positional relationship between the existing reference position KP and the new reference position KP as lesion tracking information TJ (step S20).

[0091] The endoscopic examination support device 1 determines whether the input signal NS generated by the input unit 14 or the operation unit 36 ​​is a signal indicating the end of the endoscopic examination (step S21).

[0092] If the endoscopic examination support device 1 determines that the input signal NS generated by the input unit 14 or the operation unit 36 ​​is not a signal indicating the end of the endoscopic examination (step S21: NO), it returns to step S18. Alternatively, if the endoscopic examination support device 1 determines that the input signal NS generated by the input unit 14 or the operation unit 36 ​​is a signal indicating the end of the endoscopic examination (step S21: YES), it terminates the series of processes related to tracking the detected lesion LE.

[0093] Incidentally, the user can move the endoscope camera in the direction from which the detected lesion LE was first detected by, for example, checking the indicator IN on the displayed image HG. Also, in such a situation, it is possible that the detected lesion LE, which has moved out of the screen, may re-enter the screen. Therefore, as a process to deal with the aforementioned situation, the endoscopy support device 1 can, for example, if it detects that the detected lesion LE, which has moved out of the screen, has re-entered the screen during the period until the end of the endoscopy, it can perform a process to erase the indicator IN from the displayed image HG, and can also perform the same process as in step S15 and later.

[0094] The endoscopic examination support device 1 can track a single detected lesion LE that has gone off-screen by performing the processing shown in Figure 17. Furthermore, the endoscopic examination support device 1 can track multiple detected lesion LEs that have gone off-screen by performing the processing shown in Figure 17 in parallel according to the number of lesion LEs.

[0095] As described above, according to this embodiment, multiple reference positions used for tracking lesions can be set, and a new reference position can be set within the screen each time any of the multiple reference positions move outside the screen. Furthermore, according to this embodiment, the direction and distance of a detected lesion that has moved outside the screen can be calculated by performing calculations using the trajectory of each reference position. Furthermore, according to this embodiment, a display image can be generated that allows recognition of the direction and distance of a detected lesion that has moved outside the screen. Therefore, according to this embodiment, when a detected lesion moves outside the screen due to the movement of the endoscope's observation range, the time required to rediscover the lesion can be shortened. Furthermore, according to this embodiment, among multiple lesions found in close proximity to each other, a lesion that has moved out of the display image due to the movement of the endoscope's observation range can be easily rediscovered.

[0096] <Second Embodiment> Figure 18 is a block diagram showing another example of the functional configuration of the endoscopic examination support device according to the present disclosure.

[0097] The endoscopic examination support device 500 has the same hardware configuration as the endoscopic examination support device 1. The endoscopic examination support device 500 also includes a lesion detection means 511, a reference position setting means 512, and a display control means 513.

[0098] The lesion detection means 511 can be implemented, for example, using the functions of the lesion detection unit 21. The reference position setting means 512 can be implemented, for example, using the functions of the reference position setting unit 23A. The display control means 513 can be implemented, for example, using the functions of the display control unit 25.

[0099] Figure 19 is a flowchart showing another example of processing performed in the endoscopic examination support device according to this disclosure.

[0100] The lesion detection means 511 detects lesions contained in the endoscopic image obtained by imaging the inside of the subject with an endoscope camera (step S51).

[0101] The reference position setting means 512 sets a plurality of reference positions within the endoscopic image for tracking lesions (step S52), and sets a new reference position within the endoscopic image to replace a reference position that has moved out of the endoscopic image due to the movement of the endoscopic camera (step S53).

[0102] The display control means 513 generates a display image by adding visual information to the endoscopic image indicating the direction of at least one lesion that has moved outside the endoscopic image, based on the trajectories of a plurality of reference positions, including a new reference position (step S54).

[0103] According to this embodiment, if a previously detected lesion moves out of the screen due to movement of the endoscope's observation range, the time required to rediscover the lesion can be shortened.

[0104] Some or all of the above embodiments may also be described as follows, but are not limited to the following:

[0105] (Note 1) An endoscopic examination support device comprising: a lesion detection means for detecting lesions contained in an endoscopic image obtained by imaging the inside of a subject with an endoscope camera; a reference position setting means for setting a plurality of reference positions used for tracking the lesion within the endoscopic image, and setting a new reference position within the endoscopic image to replace a reference position that has moved outside the endoscopic image from among the plurality of reference positions; and a display control means for generating a display image by adding visual information indicating the direction of at least one of the lesions that has moved outside the endoscopic image to the endoscopic image based on the trajectory of the plurality of reference positions including the new reference position.

[0106] (Note 2) The endoscopic examination support device according to Note 1, wherein the display control means generates the display image by adding a band-shaped indicator at a position corresponding to the direction of the lesion that has moved outside the endoscopic image on the outer edge of the endoscopic image.

[0107] (Note 3) The endoscopic examination support device according to Note 2, wherein the display control means adds indicators having different widths to the endoscopic image according to the distance to the lesion that has moved outside the endoscopic image.

[0108] (Note 4) The endoscopic examination support device according to Note 2, wherein the display control means adds an indicator to the endoscopic image corresponding to each of the multiple lesions that have moved outside the endoscopic image.

[0109] (Note 5) The endoscopic examination support device according to Note 1, wherein the reference position setting means detects that a predetermined instruction has been given while the lesion included in the endoscopic image has been detected, and sets the plurality of reference positions around the lesion as the first reference position.

[0110] (Note 6) The endoscopic examination support device described in Note 5, wherein the prescribed instruction is one of the following: an instruction related to still photography, an instruction related to switching illumination light, or an instruction related to confirming a lesion.

[0111] (Note 7) The endoscopic examination support device according to Note 5, wherein the reference position setting means sets the new reference position while maintaining the existing reference position remaining in the endoscopic image, and the display control means identifies the trajectories of the plurality of reference positions based on the positional relationship between the lesion and the first reference position, and the positional relationship between the existing reference position and the new reference position.

[0112] (Note 8) The endoscopic examination support device according to Note 1, wherein the plurality of reference positions are set as a plurality of rectangular regions that are positioned so as not to come into contact with the edge of the endoscopic image.

[0113] (Note 9) The lesion detection means is an endoscopic examination support device according to Note 1 that detects the lesion using a trained machine learning model.

[0114] (Note 10) A computer-operated method for assisting endoscopic examinations, comprising: detecting a lesion contained in an endoscopic image obtained by imaging the inside of a subject with an endoscope camera; setting a plurality of reference positions for tracking the lesion within the endoscopic image; setting a new reference position within the endoscopic image to replace a reference position that has moved outside the endoscopic image due to the movement of the endoscope camera; and generating a display image by adding visual information to the endoscopic image indicating the direction of at least one of the lesions that has moved outside the endoscopic image, based on the trajectory of the plurality of reference positions including the new reference position.

[0115] (Note 11) A recording medium that records a program that causes a computer to perform a process of detecting lesions contained in an endoscopic image obtained by imaging the inside of a subject with an endoscope camera, setting a plurality of reference positions to be used for tracking the lesions in the endoscopic image, setting a new reference position in the endoscopic image to replace a reference position that has moved out of the endoscopic image due to the movement of the endoscopic camera, and generating a display image by adding visual information to the endoscopic image indicating the direction of at least one of the lesions that has moved out of the endoscopic image based on the trajectory of the plurality of reference positions including the new reference position.

[0116] Although the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the above embodiments and examples. Various modifications to the structure and details of the present disclosure are possible, as can be understood by those skilled in the art within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0117] 1 Endoscopy support device 2 Display device 3 Endoscope scope 11 Processor 12 Memory 13 Interface 21 Lesion detection unit 22 Trigger generation unit 23 Lesion tracking unit 24 Tracking information storage unit 25 Display control unit

Claims

1. An endoscopic examination support device comprising: a lesion detection means for detecting lesions contained in an endoscopic image obtained by imaging the inside of a subject with an endoscope camera; a reference position setting means for setting a plurality of reference positions used for tracking the lesion within the endoscopic image, and setting a new reference position within the endoscopic image to replace a reference position that has moved out of the endoscopic image due to the movement of the endoscopic camera; and a display control means for generating a display image by adding visual information indicating the direction of at least one of the lesions that has moved out of the endoscopic image to the endoscopic image based on the trajectory of the plurality of reference positions including the new reference position.

2. The endoscopic examination support device according to claim 1, wherein the display control means generates the display image by adding a band-shaped indicator at a position corresponding to the direction of the lesion that has deviated outside the endoscopic image on the outer edge of the endoscopic image.

3. The endoscopic examination support device according to claim 2, wherein the display control means adds indicators having different widths to the endoscopic image according to the distance to the lesion that has moved outside the endoscopic image.

4. The endoscopic examination support device according to claim 2, wherein the display control means adds an indicator to the endoscopic image corresponding to each of the plurality of lesions that have moved outside the endoscopic image.

5. The endoscopic examination support device according to claim 1, wherein the reference position setting means detects that a predetermined instruction has been given while the lesion included in the endoscopic image has been detected, and sets the plurality of reference positions around the lesion as the first reference position.

6. The endoscopic examination support device according to claim 5, wherein the predetermined instruction is one of the following: an instruction related to still photography, an instruction related to switching illumination light, or an instruction related to confirming a lesion.

7. The endoscopic examination support device according to claim 5, wherein the reference position setting means sets the new reference position while maintaining the existing reference position remaining in the endoscopic image, and the display control means identifies the trajectories of the plurality of reference positions based on the positional relationship between the lesion and the first reference position, and the positional relationship between the existing reference position and the new reference position.

8. The endoscopic examination support device according to claim 1, wherein the plurality of reference positions are set as a plurality of rectangular regions positioned so as not to come into contact with the edge of the endoscopic image.

9. The endoscopic examination support device according to claim 1, wherein the lesion detection means detects the lesion using a trained machine learning model.

10. A computer-operated endoscopic examination support method comprising: detecting a lesion contained in an endoscopic image obtained by imaging the inside of a subject with an endoscope camera; setting a plurality of reference positions for tracking the lesion within the endoscopic image; setting a new reference position within the endoscopic image to replace a reference position that has moved outside the endoscopic image due to the movement of the endoscopic camera; and generating a display image by adding visual information indicating the direction of at least one of the lesions that has moved outside the endoscopic image to the endoscopic image based on the trajectory of the plurality of reference positions including the new reference position.

11. A recording medium that records a program causing a computer to perform a process of detecting lesions contained in an endoscopic image obtained by imaging the inside of a subject with an endoscope camera, setting a plurality of reference positions within the endoscopic image for use in tracking the lesions, setting a new reference position within the endoscopic image to replace a reference position that has moved outside the endoscopic image due to the movement of the endoscopic camera, and generating a display image by adding visual information indicating the direction of at least one of the lesions that has moved outside the endoscopic image to the endoscopic image based on the trajectory of the plurality of reference positions including the new reference position.

Images