Endoscopic examination support system, operation method of endoscopy support system, and storage medium

Abstract

A display control unit 37 and a recording control unit 38 each acquire the following during an endoscopic examination: a first endoscope shape of when an endoscope distal end is located at a prescribed position; an endoscopy image captured during the endoscopy examination; and a second endoscope shape of when the endoscopy image was captured. The recording control unit 38 may also record the first endoscope shape, the endoscopy image, and the second endoscope shape in a storage device 43. The display control unit 37 may display the first endoscope shape and the second endoscope shape simultaneously on a display device 41.

Description

[Technical Field]

[0001] The present disclosure relates to an endoscopic examination support system for supporting an operator in an endoscopic examination, an endoscopic examination support System Operation The present invention relates to a method and a storage medium. [Background technology]

[0002] In colonoscopy, when a lesion (e.g., a polyp or cancer) is found, it is common to record the location of the lesion to facilitate subsequent treatment and follow-up after treatment. In relation to this, a method has been proposed in which lesions detected by AI during colonoscopy are marked at the corresponding location on a schematic diagram of the large intestine (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 18 / 179991 Summary of the Invention [Problem to be solved by the invention]

[0004] Because the large intestine is easily deformed, even if the location of a lesion is recorded using the above method, it may be difficult to identify the location of the lesion when the endoscope is reinserted at a later date.

[0005] The present disclosure has been made in light of these circumstances, and its purpose is to provide a technology that makes it easier to re-access a lesion or suspected lesion with an endoscope. [Means for solving the problem]

[0006] In order to solve the above problems, an endoscopic examination support system according to an embodiment of the present disclosure includes one or more processors having hardware, which acquire a first endoscopic shape when the distal end of the endoscope is positioned at a predetermined site during endoscopic examination, an endoscopic image captured during endoscopic examination, and a second endoscopic shape when the endoscopic image was captured.

[0007] Another aspect of the present disclosure is a method for supporting endoscopic examination, which acquires a first endoscopic shape when a distal end of an endoscope is positioned at a predetermined site during endoscopic examination, an endoscopic image captured during the endoscopic examination, and a second endoscopic shape when the endoscopic image was captured.

[0008] Any combination of the above components, and conversion of the expression of the present disclosure into a method, device, system, recording medium, computer program, etc., are also valid aspects of the present disclosure. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram showing the overall system configuration related to colonoscopy according to an embodiment; [Figure 2] 1 is a diagram showing an example of an endoscope used in the present embodiment. FIG. [Figure 3] 1 is a diagram illustrating an example of the configuration of an endoscopic examination support system according to an embodiment. [Figure 4] FIG. 1 is a diagram showing a first example of a screen displayed on a display device. [Figure 5] FIG. 10 is a diagram showing a second example of a screen displayed on the display device. [Figure 6] FIG. 10 is a diagram showing a third example of a screen displayed on a display device. [Figure 7] FIG. 10 is a diagram showing a fourth example screen displayed on the display device. [Figure 8] FIG. 10 is a diagram showing a fifth example of a screen displayed on a display device. [Figure 9] FIG. 6 is a diagram showing a screen example 6 displayed on the display device. [Figure 10]10(a)-(c) are diagrams showing examples of multiple endoscope shapes displayed in trigonometric projection views from three directions. [Figure 11] FIG. 10 is a diagram showing an example of a bird's-eye view of the three-dimensional shape of the endoscope when it reaches the cecum. [Figure 12] 10 is a flowchart showing an operation at the end of an examination of the endoscopic examination support system according to the embodiment. [Figure 13] 10 is a flowchart showing an operation of the endoscopic examination support system according to the embodiment when confirming examination information. DETAILED DESCRIPTION OF THE INVENTION

[0010] This embodiment relates to a colonoscopy. In a colonoscopy, an endoscope is inserted up to the cecum, and upon removal, lesion screening, detailed examination of the lesion, and treatment of the lesion are performed. When a lesion is confirmed, the location of the lesion is recorded to facilitate treatment to be performed at a later date and follow-up observation after treatment. Even if the examination reveals that no treatment is performed because it is not a lesion, follow-up observation may be performed to see if the area develops into a lesion, and the location of the area may be recorded as a lesion candidate.

[0011] It is common for lesions or potential lesions found during a previous colonoscopy to be revisited if a pathology diagnosis following a screening colonoscopy indicates the need for treatment. This may be done by the same physician, a different physician, or at another facility.

[0012] However, the large intestine is approximately 1 to 1.5 meters long and is easily deformed. In particular, the insertion method used can significantly affect the length of insertion from the anus to the same lesion. Even when the same physician attempts re-access, it may not be easy. While re-access may be performed at another facility for specialization reasons, re-access by a different physician is more difficult. While a referral letter may include a schematic diagram of the large intestine and the approximate location of the lesion, it is often the case that the lesion cannot be found even when re-accessing according to the schematic diagram. Therefore, the information on lesion location exchanged between physicians and facilities is merely for reference. Given these factors, it is necessary to accurately indicate the location of the lesion or suspected lesion.

[0013] 1 is a diagram showing the overall system configuration related to a colonoscopy according to an embodiment. This embodiment uses an endoscope system 10, an endoscope 11, a light source device 15, an endoscope insertion shape observation device (UPD: Endoscope Position Detecting Unit) 20, an endoscopic examination support system 30, a display device 41, an input device 42, and a storage device 43. The endoscope 11 according to this embodiment is a colonoscope to be inserted into the large intestine of a subject (patient).

[0014] The endoscope 11 includes a lens and a solid-state image sensor (for example, a CMOS image sensor, a CCD image sensor, or a CMD image sensor). The solid-state image sensor converts light collected by the lens into an electrical signal and outputs the signal as an endoscopic image (electrical signal) to the endoscope system 10. The endoscope 11 includes a forceps channel. An operator (doctor) can perform various procedures during an endoscopic examination by passing a treatment tool through the forceps channel.

[0015] The light source device 15 includes a light source such as a xenon lamp, and supplies observation light (white light, narrowband light, fluorescent light, near-infrared light, etc.) to the tip of the endoscope 11. The light source device 15 also includes a built-in pump that sends water or air to the endoscope 11.

[0016] The endoscope system 10 controls the light source device 15 and processes the endoscopic image input from the endoscope 11. The endoscope system 10 is equipped with functions such as narrow band imaging (NBI), red dichromatic imaging (RDI), texture and color enhancement imaging (TXI), and extended depth of field (EDOF).

[0017] Narrowband light observation uses specific wavelengths of violet (415 nm) and green (540 nm) light, which are strongly absorbed by hemoglobin in blood, to obtain endoscopic images that emphasize the capillaries and fine structures of the mucosal surface. Red light observation uses three specific wavelengths of light (green, amber, and red) to obtain endoscopic images that emphasize the contrast of deeper tissues. Structural color enhancement generates endoscopic images that optimize the three elements of the mucosal surface under normal light observation: structure, color, and brightness. Extended depth of field allows for the acquisition of endoscopic images with a wide focal range by combining two images focused on the near and far distances.

[0018] The endoscope system 10 outputs an endoscope image obtained by processing an endoscope image input from the endoscope 11, or an endoscope image input from the endoscope 11 as is, to the endoscope examination support system 30.

[0019] The endoscope insertion shape observation device 20 is a device for observing the three-dimensional shape of the endoscope 11 inserted into the lumen of the subject. A receiving antenna 20a is connected to the endoscope insertion shape observation device 20. The receiving antenna 20a is an antenna for detecting magnetic fields generated by multiple magnetic coils built into the endoscope 11.

[0020] 2 is a diagram showing an example of an endoscope 11 used in this embodiment. The endoscope 11 has an elongated tubular insertion section 11a made of a flexible member, and an operation section 11e connected to the base end of the insertion section 11a. The insertion section 11a has, from the tip end to the base end, a tip hard section 11b, a bending section 11c, and a flexible tubular section 11d. The base end of the tip hard section 11b is connected to the tip end of the bending section 11c, and the base end of the bending section 11c is connected to the base end of the flexible tubular section 11d.

[0021] The operation unit 11e has a main body 11f from which the flexible tube 11d extends and a grip 11g connected to the proximal end of the main body 11f. The grip 11g is held by the surgeon. A universal cord including an imaging electric cable, a light guide, etc., extends from the insertion unit 11a and is connected to the endoscope system 10 and the light source device 15.

[0022] The tip hard portion 11b is the tip of the insertion portion 11a and also the tip of the endoscope 11. The tip hard portion 11b houses a solid-state imaging element, an illumination optical system, an observation optical system, etc. Illumination light emitted from the light source device 15 is propagated along the light guide to the tip surface of the tip hard portion 11b and is irradiated from the tip surface of the tip hard portion 11b towards the object to be observed inside the lumen.

[0023] The bending section 11c is configured by connecting joint rings along the longitudinal axis of the insertion section 11a. The bending section 11c bends in a desired direction in response to an operation by the surgeon inputted to the operation section 11e, and the position and orientation of the distal end rigid section 11b change in response to the bending.

[0024] The flexible tube portion 11d is a tubular member extending from the main body portion 11f of the operation portion 11e, has a desired flexibility, and bends when subjected to an external force. The surgeon inserts the insertion portion 11a into the large intestine of the subject while bending the bending portion 11c and twisting the flexible tube portion 11d.

[0025] A plurality of magnetic coils 12 are arranged inside the insertion section 11a along the longitudinal direction at predetermined intervals (for example, 10 cm intervals). Each magnetic coil 12 generates a magnetic field when a current is supplied to it. The plurality of magnetic coils 12 function as position sensors for detecting each position of the insertion section 11a.

[0026] Returning to FIG. 1, the receiving antenna 20a receives magnetic fields emitted from the multiple magnetic coils 12 built into the insertion section 11a of the endoscope 11 and outputs them to the endoscope insertion shape observation device 20. The endoscope insertion shape observation device 20 applies the magnetic field strength of each of the multiple magnetic coils 12 received by the receiving antenna 20a to a predetermined position detection algorithm to estimate the three-dimensional position of each of the multiple magnetic coils 12. The endoscope insertion shape observation device 20 generates a three-dimensional endoscopic shape of the insertion section 11a of the endoscope 11 by curve interpolating the estimated three-dimensional positions of the multiple magnetic coils 12.

[0027] The reference plate 20b is attached to the subject (for example, the subject's abdomen). A posture sensor for detecting the subject's posture is disposed on the reference plate 20b. For example, a three-axis acceleration sensor or a gyro sensor can be used as the posture sensor. In FIG. 1, the reference plate 20b is connected to the endoscope insertion shape observation device 20 via a cable, and the reference plate 20b outputs three-dimensional posture information indicating the posture of the reference plate 20b (i.e., the posture of the subject) to the endoscope insertion shape observation device 20.

[0028] Note that a plurality of magnetic coils similar to the plurality of magnetic coils 12 built into the insertion section 11a of the endoscope 11 may be used as the body position sensor arranged on the reference plate 20b. In this case, the receiving antenna 20a receives magnetic fields emitted from the plurality of magnetic coils arranged on the reference plate 20b and outputs them to the endoscope insertion shape observation device 20. The endoscope insertion shape observation device 20 applies the magnetic field strength of each of the plurality of magnetic coils received by the receiving antenna 20a to a predetermined posture detection algorithm to generate three-dimensional posture information indicating the posture of the reference plate 20b (i.e., the posture of the subject).

[0029] The endoscope insertion shape observation device 20 changes the generated three-dimensional endoscope shape so as to follow changes in the three-dimensional posture information. Specifically, the endoscope insertion shape observation device 20 changes the three-dimensional endoscope shape so as to cancel out changes in the three-dimensional posture information. This makes it possible to always recognize the endoscope shape from a specific viewpoint (for example, a viewpoint that views the abdomen of the subject perpendicularly from the front side of the abdomen) even if the subject's position is changed during endoscopic examination.

[0030] The endoscope insertion shape observation device 20 can acquire the insertion length, which indicates the length of the portion of the endoscope 11 inserted into the large intestine, and the time elapsed since the endoscope 11 was inserted into the large intestine (hereinafter referred to as insertion time). For example, the endoscope insertion shape observation device 20 measures the insertion length using the position at the time when the operator inputs an operation to start the examination into the input device 42 as the base point, and measures the insertion time using that timing as the starting point. Note that the endoscope insertion shape observation device 20 may estimate the position of the anus from the generated three-dimensional endoscope shape and the difference in magnetic field strength between the magnetic coil inside the body and the magnetic field coil outside the body, and use the estimated position of the anus as the base point of the insertion length.

[0031] In order to measure the insertion length with high accuracy, an encoder may be installed near the subject's anus. The endoscope insertion shape observation device 20 detects the insertion length from the position of the anus as the base point based on the signal from the encoder.

[0032] The endoscope insertion shape observation device 20 adds the insertion length and insertion time to the three-dimensional endoscope shape after the position correction based on the three-dimensional posture information, and outputs it to the endoscopic examination support system 30.

[0033] The endoscopic examination support system 30 generates support information for endoscopic examination based on the endoscopic image input from the endoscopic system 10 and the endoscopic shape input from the endoscope insertion shape observation device 20, and presents it to the surgeon. In addition, the endoscopic examination support system 30 generates endoscopic examination history information based on the endoscopic image input from the endoscopic system 10 and the endoscopic shape input from the endoscope insertion shape observation device 20, and records the information in the storage device 43.

[0034] The display device 41 includes a liquid crystal monitor or an organic EL monitor, and displays images input from the endoscopic examination support system 30. The input device 42 includes a mouse, keyboard, touch panel, etc., and outputs operation information input by the surgeon, etc., to the endoscopic examination support system 30. The storage device 43 includes a storage medium such as an HDD or SSD, and stores the endoscopic examination history information generated by the endoscopic examination support system 30. The storage device 43 may be a dedicated storage device associated with the endoscopic system 10, or may be a database in an in-hospital server connected via an in-hospital network, or may be a database in a cloud server.

[0035] FIG. 3 shows an example of the configuration of an endoscopic examination support system 30 according to an embodiment. The endoscopic examination support system 30 may be constructed using a processing device dedicated to endoscopic examination support, or may be constructed using a general-purpose server (which may be a cloud server). The endoscopic examination support system 30 may also be constructed using any combination of a processing device dedicated to endoscopic examination support, a general-purpose server (which may be a cloud server), and a dedicated imaging diagnostic device. The endoscopic examination support system 30 may also be constructed integrally with the endoscope system 10.

[0036] The endoscopic examination support system 30 includes an endoscope shape acquisition unit 31, an endoscopic image acquisition unit 32, an operation information acquisition unit 33, an image recognition unit 34, a reference position determination unit 35, a recording timing determination unit 36, a display control unit 37, and a recording control unit 38. These components can be realized in hardware using at least one arbitrary processor (e.g., CPU, GPU), memory (e.g., DRAM), or other LSI (e.g., FPGA, ASIC), or in software using programs loaded into memory, but the functional blocks illustrated here are realized by cooperation between these components. Therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various forms using only hardware, only software, or a combination thereof.

[0037] The endoscope shape acquisition unit 31 acquires the endoscope shape from the endoscope insertion shape observation device 20. The endoscope shape also includes information on the insertion length and insertion time. The endoscope image acquisition unit 32 acquires the endoscope image from the endoscope system 10.

[0038] The image recognition unit 34 has multiple machine learning models for detecting colon regions, colon lumen conditions, and lesions from endoscopic images. The multiple machine learning models are generated by machine learning using a large number of endoscopic images, each annotated with various regions, conditions, and lesions, as supervised datasets. The annotations are added by annotators with specialized knowledge, such as doctors. CNN, RNN, LSTM, and other types of deep learning can be used for the machine learning.

[0039] The parts of the large intestine are roughly classified into the rectum, sigmoid colon, descending colon, transverse colon, ascending colon, and cecum, in order from the anal side. The image recognition unit 34 can input the endoscopic image into a part learning model and detect parts of the large intestine from the endoscopic image. In this case, the image recognition unit 34 may identify parts based on detection results of multiple endoscopic images that are consecutive in time series. For example, when the same part is detected in a set number of frames or more out of 30 or 60 consecutive frames of endoscopic images, the image recognition unit 34 identifies the part as the officially detected part.

[0040] The image recognition unit 34 may also identify the site by taking into account the anteroposterior relationship of the detected site or the endoscope shape acquired from the endoscope insertion shape observation device 20. For example, the image recognition unit 34 identifies whether the movement direction of the endoscope 11 is the insertion direction (anus → cecum) or the removal direction (cecum → anus). In the case of the insertion direction, if the left colonic flexure is detected, the image recognition unit 34 switches the detection site from the descending colon to the transverse colon, and if the right colonic flexure is detected, the image recognition unit 34 switches the detection site from the transverse colon to the ascending colon. In the case of the insertion direction, if the right colonic flexure is detected, the image recognition unit 34 switches the detection site from the ascending colon to the transverse colon, and if the left colonic flexure is detected, the image recognition unit 34 switches the detection site from the transverse colon to the descending colon.

[0041] Furthermore, the image recognition unit 34 may improve the accuracy of site detection by taking into consideration the three-dimensional position of the tip rigid portion 11b (hereinafter referred to as the endoscope tip) based on the endoscope shape acquired from the endoscope insertion shape observation device 20. For example, if there is a discrepancy between the position of the endoscope tip estimated from the endoscope shape and the position of the detected site based on image recognition, the image recognition unit 34 discards the detection result based on image recognition.

[0042] The image recognition unit 34 can input the endoscopic image into an intraluminal condition learning model and determine the intraluminal condition from the endoscopic image. The image recognition unit 34 can detect, for example, the presence or absence of folds above a predetermined height and the presence or absence of diverticula. The image recognition unit 34 can also input the endoscopic image into a lesion learning model and detect lesion candidates from the endoscopic image.

[0043] The image recognition unit 34 may check the image quality of the endoscopic image prior to image recognition of the detection target. The image recognition unit 34 excludes an endoscopic image that is determined to have poor image quality (for example, blur, out of focus, or abnormal brightness (for example, halation)) from the image recognition of the detection target.

[0044] The reference position determination unit 35 determines the endoscope shape to be used as the reference position from the endoscope shape continuously acquired from the endoscope insertion shape observation device 20. The reference position is determined to be the position at which the deepest part is reached during colonoscopy. Typically, the deepest part during colonoscopy is the cecum. Note that some surgeons insert the endoscope 11 as far as the ileum. Also, some subjects may not be able to insert the endoscope 11 as far as the cecum, and the ascending colon may become the deepest part during colonoscopy.

[0045] For example, the reference position determination unit 35 determines the position at which the insertion length obtained from the endoscope insertion shape observation device 20 is longest as the position at which the deepest part has been reached. Alternatively, the reference position determination unit 35 may determine the photographing position of the endoscopic image at which the cecum is detected by the image recognition unit 34 as the position at which the deepest part has been reached. Alternatively, the reference position determination unit 35 may determine the position at which the surgeon inputs an insertion completion operation to the input device 42 as the position at which the deepest part has been reached.

[0046] The recording timing determination unit 36 ​​determines the timing for recording the endoscopic image and the endoscopic shape. For example, the recording timing determination unit 36 ​​determines the timing when the surgeon presses the shooting button (release button) on the operation unit 11e as the recording timing. If a microphone is installed in the surgeon's pharynx, the surgeon can also instruct the recording timing by voice. The recording timing determination unit 36 ​​may also determine the timing for capturing the endoscopic image in which a lesion candidate is detected by the image recognition unit 34 as the recording timing.

[0047] The recording timing determination unit 36 ​​may also automatically determine the recording timing based on a predetermined rule. The automatic recording of the endoscopic image and the endoscopic shape is utilized to generate an examination digest. Generally, in a colonoscopy examination, the endoscope 11 is inserted up to the cecum, and then observation and treatment are performed while being withdrawn toward the anus. The recording timing determination unit 36 ​​may, for example, set the recording timing each time the insertion length increases by a predetermined interval (for example, several centimeters). The recording timing determination unit 36 ​​may also set the recording timing each time a predetermined amount of time has passed since the endoscope was removed.

[0048] The recording timing determination unit 36 ​​may change the frequency of automatic recording depending on the region or intraluminal condition detected by the image recognition unit 34. For example, the recording timing determination unit 36 ​​increases the frequency of automatic recording while passing through a region where lesions are likely to occur. The recording timing determination unit 36 ​​also increases the frequency of automatic recording while passing through a region where the intraluminal condition is poor. Note that the region for which the frequency of automatic recording is increased may be set in advance based on the medical history and epidemiological findings of the subject.

[0049] The recording timing determination unit 36 ​​may determine, as the recording timing, at least one or all of the timing based on the operation of the surgeon, the timing based on the detection of a lesion candidate by the image recognition unit 34, and the timing based on an automatic setting.

[0050] When displaying examination information including an endoscopic image, the display control unit 37 can simultaneously display two endoscopic shapes, namely, the endoscopic shape at the time of reaching the deepest part and the endoscopic shape at a specific recording timing, on the display device 41. The display control unit 37 may display the two endoscopic shapes when a predetermined operation is input to the input device 42. The display control unit 37 may display the two endoscopic shapes together in one graph, or may display them side by side in two graphs.

[0051] The display control unit 37 can display the two endoscopic shapes in real time during the examination. If a lesion candidate is detected by the image recognition unit 34 during the examination, the display control unit 37 superimposes a mark surrounding the lesion candidate on the endoscopic image in which the lesion candidate is detected. This reduces the risk of overlooking a lesion. An alert sound may also be output from the speaker.

[0052] During an endoscopic examination, the subject may change their position according to the surgeon's instructions or may move slightly on their own. The change in position is performed to make it easier to insert the endoscope 11 and make observations. As a result, the position and orientation of the endoscope shape may shift from the coordinate space based on the position and orientation of the examination room or examination table (bed).

[0053] In contrast, in the example shown in FIG. 1, a reference plate 20b is attached to the subject, the position and orientation of the subject are detected, and the position and orientation of the endoscope shape are corrected based on the detection results. In this regard, colonoscopy is often performed without using the reference plate 20b. Even in this case, if the insertion length is equal to or greater than a predetermined value, the position and orientation of the endoscope shape can be corrected by matching the endoscope shape with the arrangement model of the colon lumen. This is particularly effective at the time of removal, because the shape of the insertion section 11a generally matches the arrangement of the colon lumen. This correction may be performed by the endoscope insertion shape observation device 20 or the endoscopic examination support system 30.

[0054] The recording control unit 38 records examination information, including endoscopic images acquired during the examination, in the storage device 43, in association with the endoscope shape at the time of reaching the deepest point and the endoscope shape at at least one recording timing. Because the endoscope shape at the time of reaching the deepest point is common within a single case, it is sufficient to record the endoscope shape in association with the case. The format of the recorded endoscopic shape data is not important. For example, it may be a mathematical formula for calculating the shape, point cloud data for indicating the shape, or image data viewed from one or more directions.

[0055] When a doctor checks the examination information after an endoscopic examination, the examination information recorded in the storage device 43 is read out to the endoscopic examination support system 30. The display control unit 37 displays two types of endoscopic shapes on the display device 41: the endoscopic shape at the time of reaching the deepest part associated with the examination information, and the endoscopic shape at at least one recording timing. Note that the examination information may be appropriately selected or filtered, formatted, or transferred to another database before being displayed on the monitor of another PC. In this case, too, two types of endoscopic shapes, the endoscopic shape at the time of reaching the deepest part and the endoscopic shape at at least one recording timing, are simultaneously displayed on the monitor.

[0056] In this way, the display control unit 37 and the recording control unit 38 can each acquire the first endoscopic shape when the endoscope tip is located at a predetermined location during an endoscopic examination, an endoscopic image captured during the endoscopic examination, and the second endoscopic shape when the endoscopic image was captured. This makes it possible to simultaneously present the endoscopic image with the second endoscopic shape when the endoscopic image was captured and the first endoscopic shape when the endoscope tip is located at a predetermined location. The first endoscopic shape and the second endoscopic shape are different from each other. The predetermined location may be the deepest part during an endoscopic examination. In this case, the endoscopic shape when the endoscope tip is located at the deepest part during an endoscopic examination can be determined as the first endoscopic shape. The predetermined location may be the cecum. In this case, the first endoscopic shape can be determined when the deepest part during an endoscopic examination is the cecum.

[0057] During a colonoscopy, the insertion section 11a of the endoscope 11 is inserted into the winding intestinal tract of the large intestine. During this procedure, the position of the unfixed intestinal tract changes or expands and contracts within the body, which can cause the insertion shape of the endoscope 11 to vary depending on the physician, insertion technique, or each examination, even for the same patient. However, the positions of the parts of the intestine that are fixed to the patient's body, such as the ascending colon and descending colon, do not change significantly. The cecum, which is the end of the large intestine, is located at the end of these fixed parts and does not change its position within the body much.

[0058] The cecum is the deepest part of the large intestine and is the target area that doctors aim to reach during colonoscopy. In practice, the endoscope 11 may be inserted as far as the ileum, the end of the small intestine, but the cecum is one of the areas that can serve as a relatively stable reference point during examination, since it is located at the end of the ascending colon, which is a fixed part. Furthermore, in examination cases where it is not possible to insert the endoscope all the way to the cecum due to insertion difficulties or the like, the deepest part can serve as one of the areas that can serve as a reference point.

[0059] By knowing the insertion shape of the insertion portion 11a of the endoscope 11 up to the cecum or the deepest part, it is possible to check the position of the patient's large intestine tract, which varies from person to person, and the insertion state of the endoscope 11 during examination. For this reason, it is effective to set the endoscope shape when the tip of the endoscope is located in the cecum or the deepest part as the first endoscope shape.

[0060] On the other hand, when the endoscope 11 is removed from the deepest part, for example, the cecum, even if the intestinal tract is curved when inserted, the curved portion is straightened, reducing the degree of curvature. As a result, the intestinal tract at the time of removal assumes a shape with less curvature while remaining based on its original configuration. It is therefore considered that the shape and configuration of the intestinal tract approaches a more stable one compared to when it was inserted. At this time, it is considered that the tip of the insertion section 11a of the endoscope 11 follows a route or traces a trajectory with less variation compared to when it was inserted. Furthermore, the route or trace of the tip of the insertion section 11a of the endoscope 11 does not have to be a fixed, stable one, but is considered to be a route or trace that is generally determined by the shape of the insertion section 11a of the endoscope 11 when it reaches the cecum.

[0061] The reference position determination unit 35 determines that the endoscope tip is located at a predetermined location based on at least one of the endoscopic image, the endoscope shape, and the insertion length of the endoscope. The display control unit 37 and the recording control unit 38 each acquire the endoscope shape when the endoscope tip is located at a predetermined location as the first endoscope shape. In this case, the first endoscope shape can be acquired automatically. Note that the display control unit 37 and the recording control unit 38 may each acquire the endoscope shape at the timing when the reference position determination unit 35 acquires an endoscope insertion completion signal based on the surgeon's operation as the first endoscope shape. In this case, it is possible to acquire the first endoscope shape that matches the surgeon's intention.

[0062] The recording control unit 38 can associate the first endoscopic shape, the captured endoscopic image, and the second endoscopic shape corresponding to the endoscopic image, and record them in the storage device 43. This makes it possible to simultaneously present the endoscopic image after the examination with the second endoscopic shape when the endoscopic image was captured and the first endoscopic shape when the tip of the endoscope is located at a predetermined site.

[0063] The display control unit 37 can simultaneously display the first endoscopic shape and the second endoscopic shape on the display device 41. This allows the surgeon or doctor to intuitively grasp the relative position of the second endoscopic shape when the endoscopic image was captured. At that time, the display control unit 37 can display the first endoscopic shape and the second endoscopic shape on a single graph. This allows the surgeon or doctor to more accurately grasp the relative position of the second endoscopic shape when the endoscopic image was captured. Furthermore, the display control unit 37 can simultaneously display the first endoscopic shape, the second endoscopic shape, and an endoscopic image corresponding to the second endoscopic shape on the display device 41. This allows the surgeon or doctor to simultaneously grasp the endoscopic image and the relative position of the second endoscopic shape when the endoscopic image was captured.

[0064] As a result, the positions of the tips of the first and second endoscope shapes within the intestinal tract are displayed with relatively good reproducibility upon removal. Therefore, based on the first endoscope shape positioned deeper and its tip position, the position and placement of the tip of the second endoscope shape within the intestinal tract can be confirmed with relatively good reproducibility from the second endoscope shape at the position removed from that position. These results can then be confirmed during and after the examination. Therefore, they can serve as location identification information when recording the location of lesions, etc. during an examination and as highly reproducible guide information when approaching the same lesions, etc. during subsequent examinations or procedures. In particular, using images captured when the endoscope is in the second endoscope shape makes it easier to identify the location of lesions, etc. and re-approach, ensuring that the lesion is accurately located.

[0065] Furthermore, even when the second endoscope shape is obtained when the endoscope 11 is inserted, recording it as a record of insertion into difficult-to-insert areas, etc., by comparing it with the shape at the deepest part or when reaching the cecum, is useful as a record of individual examinations and patients in that it allows the tip position, insertion shape, and events at those positions to be confirmed together.

[0066] 4 is a diagram showing a screen example 1 displayed on the display device 41. The following screen example is assumed to be an example of a screen when a doctor checks examination information after an examination, but a similar screen display is also possible during an examination. During an examination, each time the surgeon takes an endoscopic image, the endoscopic image and the shape of the endoscope at the time of taking the image are added to the screen.

[0067] In screen example 1, the endoscope shape B1 when reaching the cecum and multiple endoscope shapes B2-B8 when photographing are simultaneously displayed on a graph located in the center. The display control unit 37 aligns and displays a specific portion of the endoscope shape B1 with specific portions of the multiple endoscope shapes B2-B8. The specific portion may be a portion corresponding to the insertion opening of the subject through which the endoscope 11 is inserted (the anus in the case of a colonoscopy). This allows the multiple endoscope shapes B2-B8 when photographing (second endoscope shape) to be positioned in accordance with the actual situation relative to the endoscope shape B1 when reaching the cecum (first endoscope shape). Marks C1-C8 indicating the photographing positions are added to the distal ends of the endoscope shape B1 when reaching the cecum and the multiple endoscope shapes B2-B8 when photographing, respectively.

[0068] Multiple endoscopic images A1-A8 are displayed surrounding a graph placed in the center. The endoscopic image A1 on the bottom left is the endoscopic image taken when the cecum is reached, and multiple endoscopic images A2-A8 are arranged clockwise in order of removal direction. Each endoscopic image A1-A8 displays the insertion length and insertion time (time elapsed since the start of insertion).

[0069] The display control unit 37 acquires a plurality of endoscopic images A2-A8 and a plurality of endoscopic shapes B2-B8 corresponding to the plurality of endoscopic images A2-A8, and displays the plurality of endoscopic images A2-A8 and the plurality of endoscopic shapes B2-B8 simultaneously with the endoscopic shape B1 on the display device 41. By displaying the plurality of endoscopic images A2-A8, the plurality of endoscopic shapes B2-B8 (second endoscopic shape), and the endoscopic shape B1 (first endoscopic shape) in a list, the operator or doctor can easily grasp the overall picture of the endoscopic examination.

[0070] When the endoscope insertion shape observation device 20 does not correct the change in the subject's position, the display control unit 37 estimates changes in the position and orientation of the subject from the acquired changes in the endoscope shape, and aligns and orients the endoscope shape B1 with the multiple endoscope shapes B2-B8 based on the estimation result. The display control unit 37 simultaneously displays the endoscope shape B1 and the multiple endoscope shapes B2-B8 after alignment and orientation adjustment on the display device 41. This allows the endoscope shape B1 (first endoscope shape) when reaching the cecum and the multiple endoscope shapes B2-B8 (second endoscope shape) at the time of imaging to be positioned in positions that match the actual situation, and can always be displayed as the endoscope shape from a specific viewpoint (for example, a viewpoint where the subject's abdomen is viewed perpendicularly from the front side of the abdomen).

[0071] In the example shown in Figure 4, lesion candidates are detected in three endoscopic images A2, A3, and A6. Marks D2, D3, and D6, which circle the lesion candidates, are superimposed on the three endoscopic images A2, A3, and A6. Of the marks C1-C8 indicating the shooting positions of multiple endoscopic shapes B1-B8, marks C1, C4-C5, and C7-C8 indicating the shooting positions of endoscopic shapes B1, B4-B5, and B7-B8 associated with endoscopic images A1, A4-A5, and A7-A8 in which no lesion candidates were detected are displayed as circles, and marks C2, C3, and C6 indicating the shooting positions of endoscopic shapes B2, B3, and B6 associated with endoscopic images A2, A3, and A6 in which lesion candidates were detected are displayed as stars.

[0072] The display control unit 37 acquires the endoscopic images A2, A3, and A6 in which lesion candidates are detected, and the endoscopic shapes B2, B3, and B6 at the time when the endoscopic images A2, A3, and A6 in which lesion candidates are detected were captured. This allows the endoscopic images of high importance to be acquired with the endoscopic shapes as the display target. The display control unit 37 simultaneously displays on the display device 41 the endoscopic shapes B2, B3, and B6, and marks C2, C3, and C6 that indicate that lesion candidates have been detected and are located at the distal ends of the endoscopic shapes B2, B3, and B6. This makes it easier for the surgeon or doctor to grasp the capture position of the endoscopic images in which lesion candidates are detected.

[0073] The example shown in Figure 4 shows the shape of the endoscope when the subject is lying supine on the examination table, as viewed from the ceiling. As will be described later, the shape of the endoscope may be displayed from viewpoints in two or three directions, or the shape of the endoscope may be displayed as a perspective view from an oblique direction. By displaying the shape of the endoscope when removed in this way, the location within the colon lumen where the tip of the endoscope is located can be clearly presented.

[0074] FIG. 5 is a diagram showing screen example 2 displayed on the display device 41. Screen example 2 is an example of a screen transitioned to when a user, such as a doctor, selects endoscopic image A7 by clicking or touching on screen example 1 shown in FIG. 4. The endoscopic image A7 that the user has focused on is displayed on the left side of the screen, and the right side of the screen displays a graph in which the endoscopic shape B7 at the time of capturing endoscopic image A7 and the endoscopic shape B1 at the time of reaching the cecum are simultaneously plotted. Below the graph, the insertion length (18 cm) at the time of capturing endoscopic image A7 and the insertion length (68 cm) at the time of reaching the cecum are displayed. Note that, in order to simplify the screen display, the graph and insertion length table on the right side may be displayed only when the user performs a predetermined operation on endoscopic image A7.

[0075] The display control unit 37 displays an endoscopic image A7 selected by the user from the multiple endoscopic images A2-A8 and an endoscopic shape B7 corresponding to the selected endoscopic image A7 on the display device 41 simultaneously with the endoscopic shape B1. This makes it possible to generate a screen that focuses on the information of the endoscopic image that the user is interested in. In this case, the display control unit 37 may display an insertion length corresponding to the endoscopic shape B1 and an insertion length corresponding to the endoscopic shape B7 on the display device 41 simultaneously with the endoscopic shape B1 and the endoscopic shape B7. By displaying the insertion lengths simultaneously, the amount of information presented to the user can be increased. The display control unit 37 may also display an insertion time corresponding to the endoscopic shape B1 and an insertion time corresponding to the endoscopic shape B7 on the display device 41 simultaneously with the endoscopic shape B1 and the endoscopic shape B7. By displaying the insertion times simultaneously, the amount of information presented to the user can be increased.

[0076] FIG. 6 is a diagram showing a screen example 3 displayed on the display device 41. In screen example 3, only the endoscope shape B11 at the time of reaching the cecum is displayed on a three-dimensional graph located at the center. A plurality of endoscopic images A11-A21 are displayed surrounding the graph located at the center. The plurality of endoscopic images A11-A21 may be thumbnail images. The endoscopic image A11 on the bottom left is the endoscopic image at the time of reaching the cecum, and a plurality of endoscopic images A12-A21 are arranged clockwise in the order of the removal direction. Marks C11-C21 indicating the respective photographing positions of the plurality of endoscopic images A11-A21 are added to the endoscope shape B11 at the time of reaching the cecum.

[0077] FIG. 7 is a diagram showing screen example 4 displayed on the display device 41. Screen example 4 is an example of a screen to which the user transitions when endoscopic image A18 is selected by the user on screen example 3 shown in FIG. 6. The left side of the screen displays the endoscopic image A18 that the user has focused on, and the right side of the screen displays a graph in which the endoscope shape B18 at the time of capturing the endoscopic image A18 and the endoscope shape B11 at the time of reaching the cecum are simultaneously plotted. Below the graph, the insertion length (23 [cm]) at the time of capturing the endoscopic image A18 is displayed. The screen examples shown in FIG. 6 and FIG. 7 may be displayed on a single screen.

[0078] The display control unit 37 can switch between a first display mode in which the endoscopic shape B11, the multiple endoscopic images A11-A21, and marks C11-C21 indicating the respective imaging positions of the multiple endoscopic images A11-A21 arranged on the endoscopic shape B11 are displayed on the display device 41, and a second display mode in which the endoscopic image A18 selected by the user, the endoscopic shape B18 corresponding to the selected endoscopic image A18, and the endoscopic shape B11 are simultaneously displayed on the display device 41. This can improve the visibility or operability for the user.

[0079] FIG. 8 is a diagram showing a screen example 5 displayed on the display device 41. In screen example 5, an insertion length straight line E1 is displayed, which is generated by linearizing the endoscope shape B11 at the time of reaching the cecum shown in FIG. 6. A plurality of endoscopic images A11-A21 are displayed above the insertion length straight line E1, parallel to the insertion length straight line E1. The plurality of endoscopic images A11-A21 may be thumbnail images. The leftmost endoscopic image A11 is an endoscopic image at the time of reaching the cecum, and a plurality of endoscopic images A12-A21 are arranged to the right in the order of the removal direction. Marks C11-C21 indicating the respective photographing positions of the plurality of endoscopic images A11-A21 are added to the insertion length straight line E1. When the user selects one of the endoscopic images A11-A21, the screen transitions to the screen example shown in FIG. 7.

[0080] The display control unit 37 can switch between the second display mode and a third display mode in which the endoscopic shape B11, the multiple endoscopic images A11-A21, and marks C11-C21 indicating the respective imaging positions of the multiple endoscopic images A11-A21 arranged on an insertion length straight line E1 generated by linearizing the endoscopic shape B11 are displayed on the display device 41. This can improve the visibility or operability for the user.

[0081] FIG. 9 is a diagram showing a screen example 6 displayed on the display device 41. In screen example 6, an insertion time line E2 is displayed, which is generated by linearizing the endoscope shape B11 at the time of reaching the cecum shown in FIG. 6. Above the insertion time line E2, multiple endoscopic images A11-A21 are displayed parallel to the insertion time line E2. The multiple endoscopic images A11-A21 may be thumbnail images. The leftmost endoscopic image A11 is the endoscopic image at the time of reaching the cecum, and multiple endoscopic images A12-A21 are arranged to the right in order of the removal direction. Marks C11-C21 indicating the capture timing of each of the multiple endoscopic images A11-A21 are added to the insertion time line E2. In the example shown in FIG. 9, the insertion time at the time of reaching the cecum is 4:26 elapsed from the start of insertion, and the insertion time is counted up from there until removal is complete. When the user selects one of the endoscopic images A11-A21, the screen transitions to the screen example shown in FIG. 7.

[0082] The display control unit 37 can switch between the second display mode and a fourth display mode in which the endoscopic shape B11, the multiple endoscopic images A11-A21, and marks C11-C21 indicating the respective capture timings of the multiple endoscopic images A11-A21 arranged on an insertion time line E2 generated by linearizing the endoscopic shape B11 are displayed on the display device 41. This can improve the visibility or operability for the user.

[0083] 10(a)-(c) are diagrams showing examples of multiple endoscope shapes displayed in trigonometric projections from three directions. In FIGS. 10(a)-(c), the subject is in a supine position, with the y direction representing the longitudinal direction, the x direction representing the lateral direction, and the z direction representing the thickness direction. FIG. 10(a) shows an example of multiple endoscope shapes plotted on the xy coordinate system. The left side of the x axis represents the direction of the right abdominal region, the right side of the x axis represents the direction of the left abdominal region, the upper side of the y axis represents the direction of the chest, and the lower side of the y axis represents the direction of the feet. FIG. 10(b) shows an example of multiple endoscope shapes plotted on the zy coordinate system. The left side of the z axis represents the direction of the abdomen, the right side of the z axis represents the direction of the back, the upper side of the y axis represents the direction of the chest, and the lower side of the y axis represents the direction of the feet. FIG. 10(c) shows an example of multiple endoscope shapes plotted on the xz coordinate system. The left side of the x-axis is the direction of the right flank, the right side of the x-axis is the direction of the left flank, the upper side of the z-axis is the direction of the abdomen, and the lower side of the z-axis is the direction of the back.

[0084] FIG. 11 shows an example of a bird's-eye view of a 3D endoscope shape B31 at the time of reaching the cecum. The figure is displayed using a coordinate system with the origin at a position corresponding to the anus, which is the starting point of insertion into the body. The coordinate system also has actual dimensions indicated by scales. The operator can confirm the 3D endoscope shape B31 by appropriately changing the viewpoint direction and scale to their desired direction and size. Note that multiple endoscope shapes during imaging, such as those shown in FIG. 4, may also be displayed simultaneously on the same 3D graph. The coordinate system and the dimensions may be displayed in any manner. For example, the coordinate system may not be displayed, the origin may not be set to the anus, or dimensions may be displayed without displaying the dimensions, or only a reduced scale may be displayed.

[0085] 12 is a flowchart showing an example of the operation of the endoscopic examination support system 30 according to the embodiment at the end of an examination. The recording control unit 38 acquires the endoscope shape when the endoscope reaches the deepest part (S10). The recording control unit 38 acquires multiple endoscopic images captured by the surgeon and multiple endoscopic shapes corresponding to the respective image capturing timings (S11). The recording control unit 38 associates the endoscope shape when the endoscope reaches the deepest part, the multiple endoscopic images captured by the surgeon, and the multiple endoscopic shapes corresponding to the respective image capturing timings, and records them in the storage device 43 (S12).

[0086] 13 is a flowchart showing an example of operation of the endoscopic examination support system 30 according to the embodiment when confirming examination information. The display control unit 37 reads out from the storage device 43 the endoscopic shape at the time of reaching the deepest part, multiple endoscopic images captured by the surgeon, and multiple endoscopic shapes corresponding to the respective capture timings, which are associated and stored in the storage device 43 (S20). The display control unit 37 displays a digest of the multiple read endoscopic images on the display device 41 (S21). The display control unit 37 displays the endoscopic image selected by the user, the endoscopic shape corresponding to the endoscopic image, and the endoscopic shape at the time of reaching the deepest part on the display device 41 (S22).

[0087] As described above, according to this embodiment, by simultaneously displaying the endoscope shape at the time of reaching the deepest part and the endoscope shape at the time of imaging, it becomes easier to re-access a lesion or lesion candidate with the endoscope 11. The surgeon can more accurately grasp the position in the large intestine lumen where the lesion or lesion candidate exists, making re-access easier. Furthermore, by employing the various display forms described above, it is possible to improve the visibility and operability for the user.

[0088] The present disclosure has been described above based on a number of embodiments. These embodiments are merely examples, and it will be understood by those skilled in the art that various modifications are possible in the combination of the respective components and processing steps, and that such modifications are also within the scope of the present disclosure.

[0089] In the above embodiment, an example has been described in which the endoscope shape is estimated by incorporating multiple magnetic coils in the endoscope 11. In this regard, the endoscope shape may also be estimated by incorporating multiple shape sensors in the endoscope 11. The shape sensor may be, for example, a fiber sensor that uses an optical fiber to detect the bending shape from the curvature of a specific location. The fiber sensor may have, for example, an optical fiber arranged along the longitudinal direction of the insertion section 11a, and the optical fiber has multiple photodetectors arranged along the longitudinal direction. Detection light is supplied from a detection light emitting device to the optical fiber, and the endoscope shape is estimated based on changes in the amount of light detected by each photodetector as the detection light propagates through the optical fiber. [Industrial Applicability]

[0090] The present disclosure can be used in colonoscopy. [Explanation of symbols]

[0091] 10 Endoscope system, 11 Endoscope, 11a Insertion section, 11b Hard tip section, 11c Bending section, 11d Flexible tube section, 11e Operation section, 11f Main body section, 11g Holding section, 12 Magnetic coil, 15 Light source device, 20 Endoscope insertion shape observation device, 20a Receiving antenna, 20b Reference plate, 30 Endoscope examination support system, 31 Endoscope shape acquisition section, 32 Endoscope image acquisition section, 33 Operation information acquisition section, 34 Image recognition section, 35 Reference position determination section, 36 Recording timing determination section, 37 Display control section, 38 Recording control section, 41 Display device, 42 Input device, 43 Storage device

Claims

1. An endoscopic examination support system, one or more processors having hardware; A storage medium storing a program is provided, The processor reads the program and executes the program, During an endoscopic examination, a curved shape of an insertion portion of a first endoscope when the distal end portion of the endoscope is positioned at a predetermined site, a plurality of endoscopic images taken during the endoscopic examination, and a plurality of curved shapes of an insertion portion of a second endoscope when the plurality of endoscopic images were taken are acquired, determining the curved shape of the insertion portion of the first endoscope as the endoscope shape to be used as a reference position; storing the curved shape of the first endoscope insertion portion and the curved shapes of the second endoscope insertion portions in association with each other in a storage device; acquiring a plurality of the endoscopic images and a plurality of curved shapes of the insertion portion of the second endoscope corresponding to the plurality of the endoscopic images, respectively; a plurality of the endoscope images and a plurality of the curved shapes of the insertion portion of the second endoscope are displayed on a monitor simultaneously with the curved shape of the insertion portion of the first endoscope; When one endoscopic image is selected from the plurality of endoscopic images by a user operation, the curved shape of the second endoscope insertion portion corresponding to the selected endoscopic image and the curved shape of the first endoscope insertion portion indicating the reference position are simultaneously displayed on the monitor. Endoscopy support system.

2. The endoscopic examination support system according to claim 1, the processor aligns and displays a specific portion of the curved shape of the insertion section of the first endoscope with a specific portion of the curved shape of the insertion section of the second endoscope. Endoscopy support system.

3. The endoscopic examination support system according to claim 1, the processor aligns and displays a portion of the curved shape of the insertion portion of the first endoscope corresponding to an insertion opening of the subject and a portion of the curved shape of the insertion portion of the second endoscope corresponding to the insertion opening. Endoscopy support system.

4. The endoscopic examination support system according to claim 1, The processor: Estimating changes in position and orientation of the subject; Based on the estimation result, the positioning and orientation of the curved shape of the first endoscope insertion portion and the curved shape of the second endoscope insertion portion are performed, and the curved shape of the first endoscope insertion portion and the curved shape of the second endoscope insertion portion are simultaneously displayed on the monitor. Endoscopy support system.

5. The endoscopic examination support system according to claim 1, the processor acquires the endoscopic image in which a lesion candidate is detected and a curved shape of the insertion section of the second endoscope at the time when the endoscopic image in which the lesion candidate is detected was captured; Endoscopy support system.

6. The endoscopic examination support system according to claim 1, the predetermined site is the deepest part during endoscopic examination, and the deepest part is set as the reference position. Endoscopy support system.

7. The endoscopic examination support system according to claim 1, the predetermined site is the cecum, and the cecum is set as the reference position. Endoscopy support system.

8. The endoscopic examination support system according to claim 1, The processor: determining whether the distal end of the endoscope is located at the predetermined site based on at least one of the endoscopic image, the curved shape of the endoscope insertion portion, and the insertion length of the endoscope; acquiring a curved shape of the endoscope insertion section when the distal end portion of the endoscope is positioned at the predetermined site as the curved shape of the first endoscope insertion section; Endoscopy support system.

9. The endoscopic examination support system according to claim 1, The processor: a curved shape of the endoscope insertion portion at a timing when an endoscope insertion completion signal based on an operation by an operator is acquired as the curved shape of the first endoscope insertion portion; Endoscopy support system.

10. The endoscopic examination support system according to claim 1, the processor displays, on the monitor, an insertion length corresponding to the curved shape of the first endoscope insertion portion and an insertion length corresponding to the curved shape of the second endoscope insertion portion, together with the curved shape of the first endoscope insertion portion and the curved shape of the second endoscope insertion portion. Endoscopy support system.

11. The endoscopic examination support system according to claim 1, the processor displays, on the monitor, an elapsed time from the start of insertion corresponding to the curved shape of the first endoscope insertion portion and an elapsed time from the start of insertion corresponding to the curved shape of the second endoscope insertion portion, simultaneously with the curved shape of the first endoscope insertion portion and the curved shape of the second endoscope insertion portion. Endoscopy support system.

12. The endoscopic examination support system according to claim 1, The processor: acquiring a plurality of the endoscopic images and a plurality of curved shapes of the insertion portion of the second endoscope corresponding to the plurality of the endoscopic images, respectively; an endoscopic image selected by a user from the plurality of endoscopic images and a curved shape of the insertion portion of the second endoscope corresponding to the selected endoscopic image are displayed on the monitor simultaneously with the curved shape of the insertion portion of the first endoscope. Endoscopy support system.

13. The endoscopic examination support system according to claim 1, The processor: a first display mode in which the monitor displays a curved shape of the first endoscope insertion section, a plurality of the endoscope images, and marks indicating the respective photographing positions of the plurality of the endoscope images arranged on the curved shape of the first endoscope insertion section; a second display mode in which an endoscopic image selected by a user, a bending shape of a second endoscope insertion portion corresponding to the selected endoscopic image, and a bending shape of the first endoscope insertion portion are simultaneously displayed on the monitor; Endoscopy support system.

14. The endoscopic examination support system according to claim 1, The processor: a third display mode in which the monitor displays the curved shape of the insertion portion of the first endoscope, a plurality of the endoscopic images, and marks indicating the respective photographing positions of the plurality of endoscopic images arranged on an insertion length line generated by linearizing the curved shape of the insertion portion of the first endoscope; a second display mode in which an endoscopic image selected by a user, a bending shape of a second endoscope insertion portion corresponding to the selected endoscopic image, and a bending shape of the first endoscope insertion portion are simultaneously displayed on the monitor; Endoscopy support system.

15. The endoscopic examination support system according to claim 1, The processor: a fourth display mode in which the monitor displays the curved shape of the insertion portion of the first endoscope, a plurality of the endoscopic images, and marks indicating the photographing timings of the plurality of endoscopic images arranged on an insertion time line generated by linearizing the curved shape of the insertion portion of the first endoscope; a second display mode in which an endoscopic image selected by a user, a bending shape of a second endoscope insertion portion corresponding to the selected endoscopic image, and a bending shape of the first endoscope insertion portion are simultaneously displayed on the monitor; Endoscopy support system.

16. A method for operating an endoscopy support system having a processor, comprising: the processor acquires a curved shape of the insertion portion of a first endoscope when the distal end portion of the endoscope is positioned at a predetermined site, a plurality of endoscopic images taken during the endoscopic examination, and a plurality of curved shapes of the insertion portion of a second endoscope when the plurality of endoscopic images were taken, the processor determines the curved shape of the insertion section of the first endoscope as the endoscope shape to be used as a reference position; the processor associates the curved shape of the first endoscope insertion section with the curved shapes of the plurality of second endoscope insertion sections and records them in a storage device; the processor acquires a plurality of the endoscopic images and a plurality of curved shapes of the insertion section of the second endoscope corresponding to the plurality of the endoscopic images, the processor displays a plurality of the endoscopic images and a plurality of the curved shapes of the insertion portion of the second endoscope on a monitor simultaneously with the curved shape of the insertion portion of the first endoscope; When one endoscopic image is selected from the plurality of endoscopic images by a user operation, the processor simultaneously displays on the monitor the curved shape of the second endoscope insertion portion corresponding to the selected endoscopic image and the curved shape of the first endoscope insertion portion indicating the reference position. A method for operating an endoscopy support system.

17. a process for acquiring, during an endoscopic examination, a curved shape of an insertion portion of a first endoscope when the distal end portion of the endoscope is positioned at a predetermined site, a plurality of endoscopic images taken during the endoscopic examination, and a plurality of curved shapes of an insertion portion of a second endoscope when the plurality of endoscopic images were taken; A process of determining the curved shape of the insertion section of the first endoscope as the endoscope shape to be used as a reference position; a process of associating the curved shape of the first endoscope insertion portion with the curved shapes of the plurality of second endoscope insertion portions and recording them in a storage device; A process of acquiring a plurality of the endoscopic images and a plurality of curved shapes of the insertion portion of the second endoscope corresponding to the plurality of the endoscopic images, respectively; a process of displaying a plurality of the endoscopic images and a plurality of the curved shapes of the insertion portion of the second endoscope on a monitor simultaneously with the curved shape of the insertion portion of the first endoscope; a process of simultaneously displaying on the monitor, when one endoscopic image is selected from the plurality of endoscopic images by a user operation, the curved shape of the second endoscope insertion portion corresponding to the selected endoscopic image and the curved shape of the first endoscope insertion portion indicating the reference position; A storage medium that stores a program that causes a computer to execute the above.

Images