Endoscopic examination support system, method for supporting endoscopic examinations using an endoscopic examination support system, and storage medium

Abstract

An examination time evaluation unit 37 acquires an evaluation index for the time associated with a specific examination step (e.g., a net observation step) in an endoscopic examination. An image classification unit 35 acquires an implementation record of the endoscopic examination. An examination time classification unit 36 calculates an actual time that is a time required for the specific examination step on the basis of the implementation record. A display control unit 38 may display the evaluation index and the actual time on a display device 41. The display control unit 38 may also display a result of the comparison between the evaluation index and the actual time on the display device 41.

Description

Technical Field

[0001] The present disclosure relates to an endoscopic examination support system, Endoscopy support system an endoscopic examination support method, and a storage medium.

Background Art

[0002] In major endoscopic societies, a removal time of 6 minutes or more in cases without treatment is recommended as an index of sufficient observation time. In relation to this, a method of classifying and recording the endoscopic examination time into non-observation time, normal observation time, detailed observation time, and treatment time during an endoscopic examination has been proposed (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The above method discloses a recording method for endoscopic examinations, but does not disclose a specific method for evaluating the quality of lesion screening in endoscopic examinations. There is a need for a mechanism that can evaluate whether sufficient lesion screening time is ensured.

[0005] The present disclosure has been made in view of such a situation, and an object thereof is to provide a technique capable of evaluating the quality of lesion screening in endoscopic examinations.

Means for Solving the Problems

[0006] To solve the above problems, an endoscopic examination support system according to one embodiment of the present disclosure comprises one or more processors having hardware. The processor acquires an evaluation index for the time required for a predetermined examination process in an endoscopic examination, acquires a record of the endoscopic examination, and calculates the actual time required for the predetermined examination process based on the record.

[0007] Another aspect of this disclosure is an endoscopic examination support method. This method obtains an evaluation index for the time required for a predetermined examination process in an endoscopic examination, obtains a record of the endoscopic examination, and calculates the actual time required for the predetermined examination process based on the record.

[0008] Furthermore, any combination of the above components, as well as any conversion of the expressions of this disclosure between methods, apparatus, systems, recording media, computer programs, etc., are also valid as aspects of this disclosure. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows the overall system configuration related to colonoscopy according to the embodiment. [Figure 2] This figure shows an example of an endoscope used in this embodiment. [Figure 3] This figure shows an example configuration of an endoscopic examination support system according to an embodiment. [Figure 4] This figure shows a table summarizing the results of the extraction time classification for specific example 1. [Figure 5] This figure shows a graph classifying the extraction time results for specific example 1 by insertion length. [Figure 6] This figure summarizes the results of the extraction time classification for specific example 2. [Figure 7] This figure shows a graph classifying the extraction time results for Specific Example 2 by the elapsed time at the time of extraction. [Figure 8] This figure shows a table summarizing the classification results of the net observation time related to specific example 1, categorized by the state of the intestinal tract. [Figure 9]Figures 9(a)-(b) show specific examples of endoscopic images of the intestinal tract with almost no folds. [Figure 10] Figures 10(a)-(c) show specific examples of endoscopic images of the intestinal tract, which has many deep folds. [Figure 11] Figures 11(a)-(d) show specific examples of endoscopic images of the intestinal tract containing diverticula. [Figure 12] This figure shows the classification results of net observation time shown in Figure 8, with evaluation results added and summarized in a table. [Figure 13] This figure shows the results of the extraction time classification for specific example 1, categorized by body part and summarized in a table. [Figure 14] Figure 13 shows a graph classifying the extraction time by body part. [Figure 15] This flowchart shows an example of the operation of the endoscopic examination support system according to the embodiment. [Modes for carrying out the invention]

[0010] This embodiment relates to colonoscopy. Generally, in colonoscopy, the endoscope is inserted to the cecum, and observation and treatment are performed while withdrawing it toward the anus. One of the QIs (Quality Indicators) in colonoscopy is the withdrawal time. To ensure the quality of lesion screening in colonoscopy, it is desirable to present a direct indicator (e.g., the rate of observation of the intestinal surface) that the colonic mucosa has been thoroughly and completely observed. However, from the standpoint of technical difficulty, major endoscopy societies recommend an indirect indicator of an withdrawal time of 6 minutes or more in cases without treatment.

[0011] However, limiting the screening to cases that do not require treatment results in a large number of excluded cases. In particular, the proportion of excluded cases is quite high when patients who test positive on a rapid test undergo a colonoscopy at a hospital for further examination. Furthermore, the proportion of eligible cases decreases in large hospitals with a high number of referrals and procedures. Therefore, it becomes difficult to ensure the quality of lesion screening for all cases.

[0012] On the other hand, the removal time includes, in addition to the observation time for lesion screening, an inspection time for examining a site suspected of having a lesion and a treatment time for excising the lesion, as well as a cleaning / suction time for cleaning and sucking residues and moisture. Just because the removal time is 6 minutes or more does not necessarily mean that the observation time for lesion screening has been sufficiently ensured. From the above, a mechanism is required to objectively and accurately determine that the observation time has been sufficiently ensured.

[0013] FIG. 1 is a diagram showing the overall system configuration related to colonoscopy according to an embodiment. In this embodiment, an endoscope system 10, an endoscope 11, a light source device 15, an endoscope insertion shape observing device (UPD: Endoscope Position Detecting Unit) 20, an endoscopy support system 30, a display device 41, an input device 42, and a storage device 43 are used. The endoscope 11 according to this embodiment is a colonoscope inserted into the large intestine of a subject (patient).

[0014] The endoscope 11 includes a lens and a solid-state imaging device (for example, a CMOS image sensor, a CCD image sensor, or a CMD image sensor). The solid-state imaging device converts the light collected by the lens into an electrical signal and outputs it to the endoscope system 10 as an endoscope image (electrical signal). The endoscope 11 includes a forceps channel. An operator (doctor) can perform various treatments during endoscopy by passing a treatment tool through the forceps channel.

[0015] The light source device 15 includes a light source such as a xenon lamp and supplies observation light (white light, narrow-band light, fluorescence, near-infrared light, etc.) to the tip of the endoscope 11. The light source device 15 also incorporates a pump for sending water and air to the endoscope 11.

[0016] The endoscope system 10 controls the light source device 15 and processes the endoscopic images input from the endoscope 11. The endoscope system 10 is equipped with functions such as narrow-band imaging (NBI), red dichromatic imaging (RDI), texture and color enhancement (TXI), and extended depth of field (EDOF).

[0017] Narrowband light observation allows for the acquisition of endoscopic images that highlight the capillaries and microstructures of the mucosal surface by irradiating with specific wavelengths of light, such as violet (415nm) and green (540nm), which are strongly absorbed by hemoglobin in the blood. Red light observation allows for the acquisition of endoscopic images that enhance the contrast of deep tissues by irradiating with specific wavelengths of light of three colors (green, amber, and red). Structural color enhancement generates endoscopic images in which the three elements of "structure," "color tone," and "brightness" of the mucosal surface are optimized under normal light observation. Depth of field expansion allows for the acquisition of endoscopic images with a wide focal range by combining two images that are focused on near and far distances, respectively.

[0018] The endoscope system 10 outputs either the processed endoscopic image input from the endoscope 11, or the endoscopic image input from the endoscope 11 as is, to the endoscope examination support system 30.

[0019] The endoscope insertion shape observation device 20 is a device for observing the three-dimensional shape of the endoscope 11 inserted into the lumen of a patient. A receiving antenna 20a is connected to the endoscope insertion shape observation device 20. The receiving antenna 20a is an antenna for detecting the magnetic field generated by multiple magnetic coils built into the endoscope 11.

[0020] Figure 2 shows an example of an endoscope 11 used in this embodiment. The endoscope 11 has an elongated tubular insertion section 11a made of a flexible material and an operating section 11e connected to the base end of the insertion section 11a. The insertion section 11a has a rigid tip section 11b, a curved section 11c, and a flexible tube section 11d, from the tip side to the base end side. The base end of the rigid tip section 11b is connected to the tip of the curved section 11c, and the base end of the curved section 11c is connected to the base end of the flexible tube section 11d.

[0021] The operating section 11e has a main body 11f from which a flexible tube section 11d extends, and a gripping section 11g connected to the base end of the main body section 11f. The gripping section 11g is grasped by the operator. A universal cord, including an imaging electrical cable and a light guide, extends from the operating section 11e from within the insertion section 11a and is connected to the endoscope system 10 and the light source device 15.

[0022] The rigid tip section 11b is the tip of the insertion section 11a and also the tip of the endoscope 11. The rigid tip section 11b houses a solid-state image sensor, illumination optics, observation optics, etc. Illumination light emitted from the light source device 15 propagates along the light guide to the tip surface of the rigid tip section 11b and is irradiated from the tip surface of the rigid tip section 11b toward the object to be observed inside the lumen.

[0023] The curved section 11c is formed by connecting nodal rings along the longitudinal axis of the insertion section 11a. The curved section 11c bends in a desired direction in response to the operator's operation input to the operating section 11e, and the position and orientation of the rigid tip section 11b change according to this bending.

[0024] The flexible tube section 11d is a tubular member extending from the main body section 11f of the operating section 11e, possessing the desired flexibility and bending in response to external force. The operator inserts the insertion section 11a into the subject's large intestine while bending the curved section 11c and twisting the flexible tube section 11d.

[0025] Multiple magnetic coils 12 are arranged inside the insertion section 11a along the longitudinal direction at predetermined intervals (for example, 10 cm intervals). Each magnetic coil 12 generates a magnetic field when current is supplied to it. The multiple magnetic coils 12 function as position sensors to detect the positions of each part of the insertion section 11a.

[0026] Returning to Figure 1, the receiving antenna 20a receives magnetic fields emitted from multiple magnetic coils 12 built into the insertion section 11a of the endoscope 11 and outputs them to the endoscope insertion shape observation device 20. The endoscope insertion shape observation device 20 applies the magnetic field strength of each of the multiple magnetic coils 12 received by the receiving antenna 20a to a predetermined position detection algorithm to estimate the three-dimensional position of each of the multiple magnetic coils 12. The endoscope insertion shape observation device 20 generates the three-dimensional endoscope shape of the insertion section 11a of the endoscope 11 by performing curve interpolation on the estimated three-dimensional positions of the multiple magnetic coils 12.

[0027] The reference plate 20b is attached to the subject (for example, the subject's abdomen). The reference plate 20b is equipped with a position sensor for detecting the subject's posture. For example, a 3-axis accelerometer or a gyroscope can be used as the position sensor. In Figure 1, the reference plate 20b is connected to the endoscope insertion shape observation device 20 by a cable, and the reference plate 20b outputs 3D posture information indicating the posture of the reference plate 20b (i.e., the posture of the subject) to the endoscope insertion shape observation device 20.

[0028] Furthermore, multiple magnetic coils, similar to the multiple magnetic coils 12 built into the insertion section 11a of the endoscope 11, may be used as the body position sensor placed on the reference plate 20b. In this case, the receiving antenna 20a receives the magnetic field emitted from the multiple magnetic coils placed on the reference plate 20b and outputs it to the endoscope insertion shape observation device 20. The endoscope insertion shape observation device 20 applies the magnetic field strength of each of the multiple magnetic coils received by the receiving antenna 20a to a predetermined posture detection algorithm to generate three-dimensional posture information indicating the posture of the reference plate 20b (i.e., the posture of the subject).

[0029] The endoscope insertion shape observation device 20 changes the generated 3D endoscope shape in accordance with changes in 3D posture information. Specifically, the endoscope insertion shape observation device 20 changes the 3D endoscope shape in a way that cancels out changes in 3D posture information. As a result, even if the subject's position is changed during the endoscopic examination, the endoscope shape can always be recognized from a specific viewpoint (for example, a viewpoint that looks at the subject's abdomen perpendicularly from the front of the abdomen).

[0030] The endoscope insertion shape observation device 20 can obtain the insertion length, which indicates the length of the portion of the endoscope 11 inserted into the large intestine, and the elapsed time since the endoscope 11 was inserted into the large intestine (hereinafter referred to as the insertion time). For example, the endoscope insertion shape observation device 20 measures the insertion length using the position at the time the operator inputs the examination start operation to the input device 42 as the starting point, and measures the insertion time from that timing. Alternatively, the endoscope insertion shape observation device 20 may estimate the position of the anus from the generated 3D endoscope shape and the difference in magnetic field strength between the magnetic coil inside the body and the magnetic field coil outside the body, and use the estimated position of the anus as the starting point for the insertion length.

[0031] Furthermore, an encoder may be installed near the anus of the subject in order to measure the insertion length with high precision. The endoscope insertion shape observation device 20 detects the insertion length based on the position of the anus, using the signal from the encoder as the reference point.

[0032] The endoscope insertion shape observation device 20 outputs the 3D endoscope shape after body position correction based on 3D posture information, along with the insertion length and insertion time, to the endoscopy support system 30.

[0033] The endoscopic examination support system 30 generates support information for endoscopic examinations based on the endoscopic images input from the endoscopic system 10 and the endoscopic shape input from the endoscopic insertion shape observation device 20, and presents it to the operator. The endoscopic examination support system 30 also generates endoscopic examination history information based on the endoscopic images input from the endoscopic system 10 and the endoscopic shape input from the endoscopic insertion shape observation device 20, and records it in the storage device 43.

[0034] The display device 41 is equipped with an LCD monitor or an OLED monitor and displays images input from the endoscopic examination support system 30. The input device 42 is equipped with a mouse, keyboard, touch panel, etc., and outputs operation information entered by the operator, etc., to the endoscopic examination support system 30. The storage device 43 is equipped with a storage medium such as an HDD or SSD and stores endoscopic examination history information generated by the endoscopic examination support system 30. The storage device 43 may be a dedicated storage device attached to the endoscopic system 10, a database on an in-hospital server connected via the in-hospital network, or a database on a cloud server.

[0035] In the system configuration shown in Figure 1, the reference plate 20b can be omitted. Furthermore, the endoscope insertion shape observation device 20 and the receiving antenna 20a can also be omitted. If the insertion length or insertion time is measured using a device other than the endoscope insertion shape observation device 20, and the endoscope shape is not used in the site identification process described later, the endoscope insertion shape observation device 20, the receiving antenna 20a, and the reference plate 20b can be omitted.

[0036] Figure 3 shows an example configuration of the endoscopic examination support system 30 according to an embodiment. The endoscopic examination support system 30 may be constructed using a dedicated processing unit for endoscopic examination support, or it may be constructed using a general-purpose server (which may be a cloud server). Furthermore, the endoscopic examination support system 30 may be constructed using any combination of a dedicated processing unit for endoscopic examination support, a general-purpose server (which may be a cloud server), and a dedicated image diagnostic device. In addition, the endoscopic examination support system 30 may be constructed integrally with the endoscopic system 10.

[0037] The endoscopic examination support system 30 includes an endoscope shape acquisition unit 31, an endoscope image acquisition unit 32, an operation information acquisition unit 33, an image recognition unit 34, an image classification unit 35, an examination time classification unit 36, an examination time evaluation unit 37, a display control unit 38, and a recording control unit 39. These components can be implemented in hardware terms by any at least one processor (e.g., CPU, GPU), memory (e.g., DRAM), or other LSI (e.g., FPGA, ASIC), and in software terms by a program loaded into memory, etc., but here we are describing functional blocks that are realized through the cooperation of these components. Therefore, it will be understood by those skilled in the art that these functional blocks can be implemented in various ways by hardware alone, software alone, or a combination thereof.

[0038] The endoscope shape acquisition unit 31 acquires the endoscope shape from the endoscope insertion shape observation device 20. The endoscope shape also includes information on insertion length and insertion time. The endoscope image acquisition unit 32 acquires the endoscope image from the endoscope system 10.

[0039] The image recognition unit 34 has multiple machine learning models for detecting the location and condition of the organ to be examined (large intestine in this embodiment), treatment instruments, observation conditions, lesions, etc., from endoscopic images. The multiple machine learning models are generated by machine learning using a supervised dataset of numerous endoscopic images each annotated with various locations and conditions, various treatment instruments, various observation conditions, various lesions, etc. The annotations are provided by annotators with specialized knowledge, such as physicians. Deep learning types such as CNN, RNN, and LSTM can be used for machine learning.

[0040] The parts of the large intestine can be broadly classified, in order from the anal side, into the rectum, sigmoid colon, descending colon, transverse colon, ascending colon, and cecum. The image recognition unit 34 can input endoscopic images into a site learning model to detect parts of the large intestine from the endoscopic images. In this case, the image recognition unit 34 may identify the parts based on the detection results of multiple endoscopic images that are consecutive in time series. For example, if the same part is detected in a set number of frames or more out of 30 or 60 consecutive endoscopic images, the image recognition unit 34 will identify that part as the officially detected part.

[0041] The image recognition unit 34 may also identify the location by considering the spatial relationship of the detected site or the shape of the endoscope obtained from the endoscope insertion shape observation device 20. For example, the image recognition unit 34 determines whether the direction of movement of the endoscope 11 is the insertion direction (anus → cecum) or the withdrawal direction (cecum → anus). In the case of insertion, if a left colic flexure is detected, the image recognition unit 34 switches the detection site from the descending colon to the transverse colon, and if a right colic flexure is detected, it switches the detection site from the transverse colon to the ascending colon. In the case of insertion, if a right colic flexure is detected, the image recognition unit 34 switches the detection site from the ascending colon to the transverse colon, and if a left colic flexure is detected, it switches the detection site from the transverse colon to the descending colon.

[0042] Furthermore, the image recognition unit 34 may improve the accuracy of site detection by considering the three-dimensional position of the hard tip portion 11b (hereinafter referred to as the endoscope tip) based on the endoscope shape obtained from the endoscope insertion shape observation device 20. For example, if the position of the endoscope tip estimated from the endoscope shape and the position of the detected site based on image recognition are inconsistent, the image recognition unit 34 discards the detection result based on image recognition.

[0043] Furthermore, the image recognition unit 34 can input endoscopic images into a learning model for treatment instruments and suction tubes to detect treatment instruments (e.g., biopsy forceps, snare) or suction tubes from the endoscopic images. The image recognition unit 34 can also input endoscopic images into a learning model for residue, irrigation foam, bleeding, and fluid to detect residue, irrigation foam, bleeding, or fluid from the endoscopic images. The image recognition unit 34 may also consider operator operation information input to the operation unit 11e or input device 42 of the endoscope 11 to identify treatment instruments, suction tubes, residue, irrigation foam, bleeding, and fluid, respectively.

[0044] The image recognition unit 34 can also input endoscopic images into an observation condition learning model to identify observation conditions from the endoscopic images. For example, the image recognition unit 34 can identify whether normal light is being used or special light (e.g., narrowband light, red light) is being used. For example, the image recognition unit 34 can identify whether staining solution is being used, and if so, the type of staining solution. For example, the image recognition unit 34 can identify whether image enhancement is being used, and if so, the method and intensity of image enhancement. For example, the image recognition unit 34 can identify whether zoom is being used, and if so, the zoom magnification. The image recognition unit 34 may also identify various observation conditions using or considering operator operation information input to the operation unit 11e or input device 42 of the endoscope 11.

[0045] Furthermore, the image recognition unit 34 can input endoscopic images into a lumen state learning model to determine the lumen state from the endoscopic images. For example, the image recognition unit 34 can detect the presence or absence of folds above a predetermined height and the presence or absence of diverticula. The image recognition unit 34 can also input endoscopic images into a lesion learning model to detect candidate lesions from the endoscopic images.

[0046] The image recognition unit 34 may also check the image quality of the endoscopic image prior to recognizing the target image. The image recognition unit 34 excludes endoscopic images that it determines to have poor image quality (e.g., blur, out of focus, brightness abnormalities (e.g., halation)) from the target image recognition.

[0047] The image classification unit 35 identifies the endoscopic images at the time of reaching the deepest point and at the time of completion of withdrawal. The deepest point in a colonoscopy is usually the cecum. For example, the image classification unit 35 determines the endoscopic image at the time of reaching the deepest point as the endoscopic image when the cecum is detected by the image recognition unit 34. Alternatively, the image classification unit 35 may determine the endoscopic image at the time of reaching the deepest point as the endoscopic image when the insertion length acquired from the endoscopic insertion shape observation device 20 is at its longest. Alternatively, the image classification unit 35 may determine the endoscopic image at the time of reaching the deepest point as the endoscopic image at the time the operator inputs the insertion completion operation to the input device 42. Note that some operators may insert the endoscope 11 all the way to the ileum. Also, depending on the subject, it may not be possible to insert the endoscope 11 all the way to the cecum, and the ascending colon may be the deepest point in a colonoscopy.

[0048] The image classification unit 35 determines the endoscopic image at the moment of switching from an internal image to an external image as the endoscopic image at the time of successful withdrawal. Alternatively, the image classification unit 35 may determine the endoscopic image at the moment of successful withdrawal as the endoscopic image at the time the operator inputs the withdrawal completion operation to the input device 42.

[0049] The image classification unit 35 classifies multiple endoscopic images taken from the time the endoscopic device reaches its deepest point until its removal is complete, based on various conditions. The image classification unit 35 performs lesion screening classification, detailed examination classification, treatment classification, cleaning / suction classification, observation condition classification, intraluminal state classification, image quality classification, etc., on the multiple endoscopic images.

[0050] In lesion screening classification, the image classification unit 35 classifies endoscopic images where the operator is presumed to be performing tasks (procedures) other than detailed examination and treatment as endoscopic images used for lesion screening. Tasks other than treatment include washing and aspiration. Endoscopic images where lesion screening is not performed other than detailed examination, treatment, washing and aspiration are also excluded from endoscopic images used for lesion screening. The image classification unit 35 may also consider the use of special light observation when identifying endoscopic images used for lesion screening.

[0051] In the detailed examination classification, the image classification unit 35 classifies endoscopic images of the area containing the candidate lesion, where the lesion remains for a predetermined period of time or longer, into endoscopic images for detailed examination. When identifying endoscopic images for detailed examination, the image classification unit 35 may consider improving the zoom magnification, using special light observation, or using stained observation.

[0052] In procedure classification, the image classification unit 35 classifies endoscopic images in which a procedure instrument is detected as endoscopic images taken during a procedure. In cleaning / suction classification, the image classification unit 35 classifies endoscopic images in which a suction tube, residue, cleaning foam, bleeding, or moisture is detected as endoscopic images taken during cleaning / suction. Cleaning / suction during observation is performed to clean and aspirate residue and moisture to make the lumen surface easier to observe. It is also performed to wash away staining solution with water. Cleaning / suction after procedure is performed to wash away bleeding and dirt.

[0053] In the observation condition classification, the image classification unit 35 identifies the light source setting conditions, image enhancement setting conditions, etc., for each endoscopic image. Observation condition classification is performed independently of other image classifications. In the intraluminal condition classification, the image classification unit 35 identifies, for each endoscopic image, the presence or absence of folds above a predetermined height, the presence or absence of diverticula, and the presence or absence of other conditions or physical characteristics that affect the difficulty of observation. Intraluminal condition classification is also performed independently of other image classifications.

[0054] In image quality classification, the image classification unit 35 identifies the presence or absence of blur, out-of-focus images, and brightness abnormalities (e.g., halation) for each endoscopic image, and excludes endoscopic images in which these image quality defects are detected from the classification target for each examination process.

[0055] The image classification unit 35 can also classify multiple endoscopic images by region. For example, the image classification unit 35 classifies multiple endoscopic images by anatomically defined regions of the large intestine (e.g., rectum, sigmoid colon, descending colon, transverse colon, ascending colon, and cecum). The image classification unit 35 identifies the region of each endoscopic image based on at least one of the region detected by image recognition and the positional information of the endoscope tip of the endoscope shape supplied from the endoscope insertion shape observation device 20. Classification by region is performed independently of other image classifications.

[0056] The image classification unit 35 can also classify insertion lengths in predetermined lengths (for example, 5 cm). The image classification unit 35 can obtain insertion lengths from the endoscope insertion shape observation device 20. Classification by insertion length is performed independently of other image classifications. This makes it possible to check the time spent on lesion screening, etc., for each specific site or location. In addition, it is possible to check the time ratio of each item that constitutes the working state, observation conditions, intraluminal state, etc., for each specific site or location.

[0057] Users, such as physicians, can set the image classification method from the input device 42 to the image classification unit 35. Users can change the image classification method to meet the needs of the surgeon or facility, recommendations from academic societies, or legal requirements.

[0058] In this way, the image classification unit 35 acquires the record of the endoscopic examination. The record of the endoscopic examination includes multiple endoscopic images that are consecutive in time during the endoscopic examination. The classification method of the endoscopic examination includes classification by examination process. The image classification unit 35 classifies which examination process each endoscopic image included in the multiple endoscopic images in time series corresponds to, based on the image recognition results, etc.

[0059] The examination time classification unit 36 ​​calculates the actual time, which is the time taken for each examination step, based on the results of the classification of the endoscopic examination record by the image classification unit 35. Among the multiple examination steps that constitute the endoscopic examination, a predetermined examination step is included. The predetermined examination step may be an observation step obtained by excluding at least the detailed examination step and the treatment step from the overall endoscopic examination steps. The predetermined examination step may be a net observation step obtained by further excluding the cleaning step and the aspiration step from the observation step.

[0060] The image classification unit 35 can classify multiple endoscopic images using various criteria in addition to classification by the inspection process. The inspection time classification unit 36 ​​can aggregate the actual time for each item constituting each classification. Furthermore, the inspection time classification unit 36 ​​can also calculate the time ratio between the items constituting each classification. The user can set the calculation method for the actual time of the evaluation target item to the inspection time classification unit 36 ​​from the input device 42. The calculation method may differ for display and recording. For example, the calculation method for display may be simpler than that for recording.

[0061] The examination time evaluation unit 37 acquires the actual time for each examination process calculated by the examination time classification unit 36. Users such as doctors can set evaluation indicators (e.g., appropriate observation time) for the actual time of a predetermined examination process to the examination time evaluation unit 37 via the input device 42. The examination time evaluation unit 37 acquires the evaluation indicators for the actual time of the predetermined examination process.

[0062] The inspection time evaluation unit 37 compares the actual time spent on a predetermined inspection process with an evaluation index for that process, and evaluates the actual time spent on the predetermined inspection process based on the comparison result. The inspection time evaluation unit 37 can also evaluate actual time spent on processes other than the predetermined inspection process. Evaluation indexes for determining whether the actual time being evaluated is sufficient, insufficient, or excessive can be set for each actual time being evaluated. The evaluation index is expected to be a fixed numerical value or a numerical value calculated from a mathematical formula. It is desirable that the evaluation index be one whose effectiveness has been statistically confirmed or recommended by an academic society, but other values, such as a value whose effectiveness is being verified or a provisionally set value, are also acceptable. It is desirable that it be possible to leave the index unset when it is not desirable to set a value, and that it be possible to change the value to be set when it becomes desirable to set one.

[0063] The display control unit 38 displays the actual time for a predetermined inspection process and the evaluation index for that predetermined inspection process on the display device 41. The display control unit 38 can also display the comparison result of the two as an evaluation result on the display device 41. The display control unit 38 can also display the actual time for each of the multiple inspection processes that constitute the endoscopic examination on the display device 41.

[0064] The display control unit 38 can display the actual time and evaluation results of predetermined examination items on the display device 41 during or after the examination. By displaying the actual time and evaluation results of predetermined examination items in real time or near real time (with a delay of a few seconds) during the examination, the operator can determine in the examination room whether the lesion screening is sufficient. If the lesion screening is insufficient, the operator can repeat at least some of the lesion screening.

[0065] The recording control unit 39 records examination time record data in the storage device 43, which comprehensively records the actual time of each examination process and the evaluation results of at least a predetermined examination process. The recording of examination time record data in the storage device 43 may be temporary or long-term. The examination time record data recorded in the storage device 43 can be transferred to another database within the hospital, with the data format appropriately modified. The examination time record data recorded in the storage device 43 can also be displayed on the referencing display device in response to external access. Through this external transfer or display, the operator or other medical personnel can refer to the examination time record data after the examination.

[0066] Figure 4 is a table summarizing the extraction time classification results for specific example 1. Extraction time is defined as the elapsed time from reaching the deepest point to the completion of extraction. The examination time classification unit 36 ​​broadly classifies the extraction time into three categories: observation time, detailed examination time, and treatment time. The examination time classification unit 36 ​​further subdivides the observation time into three categories: natural light observation (WLI) time, special light observation (NBI) time, and washing / suction time. The examination time classification unit 36 ​​further subdivides the treatment time into three categories: biopsy forceps usage time, snare usage time, and washing / suction time.

[0067] The examination time classification unit 36 ​​calculates the net observation time (8:57) by subtracting the treatment time (5:31), the detailed examination time (1:40), and the cleaning / suction time for observation (1:13) from the extraction time (17:21 = 17 minutes 21 seconds, and so on). The net observation time (8:57) can also be calculated from the sum of the observation time under natural light (WLI) (3:09) and the observation time under special light (NBI) (5:48). The examination time classification unit 36 ​​can also calculate the non-treatment extraction time (11:50) by subtracting the treatment time (5:31) from the extraction time (17:21). Note that the calculation of the non-treatment extraction time is optional.

[0068] In this way, the net observation time used for lesion screening itself can be measured, excluding the time spent on incidental tasks such as cleaning and suction. Net observation time can be measured in all cases. In addition, the time spent removing instruments other than for treatment can also be measured in all cases. The actual time to be evaluated can be selected as appropriate, and multiple actual time periods may be selected for evaluation.

[0069] The display control unit 38 can classify and display on the display device 41 the actual time for each examination process constituting the endoscopic examination, according to divisions obtained by dividing the insertion length of the endoscope 11 during the endoscopic examination into predetermined length intervals. The insertion length is defined as ranging from 0 [cm] to the length [cm] at which the endoscope 11 reaches its deepest point.

[0070] Figure 5 shows a graph classifying the withdrawal time results for Specific Example 1 by insertion length. The horizontal axis represents the insertion length, and the breakdown of withdrawal time in 5cm increments is shown as a bar graph. In the graph shown in Figure 5, the examination time is further classified into examination time using staining solution and examination time with magnified observation. Users such as physicians can refer to the graph shown in Figure 5 to check the work content and observation time for each position of the endoscope 11.

[0071] Figure 6 is a table summarizing the results of the extraction time classification for specific example 2. The display control unit 38 can classify the actual time of each examination process constituting the endoscopic examination into categories obtained by dividing the elapsed time at the time of extraction in the endoscopic examination into predetermined time intervals, and display them on the display device 41.

[0072] Figure 7 is a graph showing the results of the extraction time classification for Specific Example 2, classified by the elapsed time at the time of extraction. The elapsed time at the time of extraction corresponds to the video recording time by the endoscope 11 from the time it reached the deepest point until the extraction was completed. The horizontal axis represents the elapsed time, and the breakdown of the extraction time in 1-minute increments is shown as a bar graph. In the example shown in Figure 7, it can be seen that the examination was performed between 0:17:00 and 0:18:00 and between 0:24:00 and 0:25:00. In Specific Example 2, 2:20 was allocated to the washing and suction time, and 9:58 was allocated to the natural light (WLI) observation time.

[0073] Incidentally, the intestinal tract has areas where lesion screening is easy and areas where it is difficult. For example, areas with few folds are easy to observe and require relatively little observation time. On the other hand, areas with many deep folds or diverticulous ridges require relatively long observation times to check for lesions in the shaded areas. Therefore, checking the observation time for areas with few folds, areas with many deep folds, and areas with diverticula is useful for confirming whether areas where lesion screening is difficult have been observed for a time commensurate with the difficulty.

[0074] Figure 8 is a table summarizing the classification results of the net observation time related to Specific Example 1, categorized by the state of the intestinal tract. The example shown in Figure 8 shows a classification of the net observation time (8:57) into four categories: observation time for areas with few folds (0:36), observation time for areas with many deep folds (5:44), observation time for areas with diverticula (0:07), and observation time for other areas (normal areas) (2:30).

[0075] Figures 9(a)-(b) show specific examples of endoscopic images A11 and A12 of the intestinal tract with almost no folds. Figures 10(a)-(c) show specific examples of endoscopic images A21-A23 of the intestinal tract with many deep folds. Figures 11(a)-(d) show specific examples of endoscopic images A31-A34 of the intestinal tract with diverticula D31-D34.

[0076] The image classification unit 35 classifies endoscopic images corresponding to net observation steps included in a series of endoscopic images according to the state of the organ to be examined (in this embodiment, the intestine) based on the image recognition results from the image recognition unit 34. The examination time classification unit 36 ​​calculates the actual time for each state of the net observation step based on the results of the state classification. The state of the intestine is classified according to the difficulty of observation. The state of the intestine may include the state of the folds of the intestine. The state of the intestine may include the presence or absence of diverticula.

[0077] The inspection time evaluation unit 37 acquires the net actual time for each state of the observation process calculated by the inspection time classification unit 36. The inspection time evaluation unit 37 acquires the net evaluation index for each state of the observation process. The inspection time evaluation unit 37 compares the net actual time for each state of the observation process with the evaluation index for each state, and evaluates the net actual time for each state of the observation process based on the results of this comparison.

[0078] The display control unit 38 displays the actual time for each state of the net observation process and the evaluation index for each state of the net observation process on the display device 41. The display control unit 38 can also display the comparison result of the two as the evaluation result for each state on the display device 41.

[0079] Figure 12 is a table summarizing the classification results of net observation time shown in Figure 8, with evaluation results added. The cumulative length corresponding to the net observation time (8:57) is 73 cm, the cumulative length in the area with almost no folds is 23 cm, the cumulative length in the area with many deep folds is 29 cm, the cumulative length in the area with diverticula is 1 cm, and the cumulative length in the other areas (normal areas) is 20 cm.

[0080] The examination time classification unit 36 ​​calculates the cumulative length of the net observation process by state based on the results of classification according to the state of the endoscopic image corresponding to the net observation process. The examination time evaluation unit 37 sets evaluation indicators for the net observation process by state according to the cumulative length of the net observation process by state, or according to the cumulative length of the net observation process by state and the type of intestinal state. For example, the examination time evaluation unit 37 sets the appropriate observation time for each state as an evaluation indicator for each state.

[0081] The examination time evaluation unit 37 sets the appropriate observation time per unit length for each condition of the intestinal tract in areas with deep folds of a certain depth or greater, or in other words, areas with folds of a certain height or greater, or areas with more than a set value of folds of a certain height or greater, to be longer than the appropriate observation time per unit length for each condition of the normal intestinal tract. The examination time evaluation unit 37 also sets the appropriate observation time per unit length for each condition of the intestinal tract in areas with diverticula to be longer than the appropriate observation time per unit length for each condition of the intestinal tract in areas without diverticula.

[0082] In the example shown in Figure 12, the standard appropriate observation time is set to 4 minutes (approximately 3.43 seconds per cm) for observing a normal 70 cm section. The appropriate observation time for each section classified by condition is determined by multiplying the standard appropriate observation time by a predetermined coefficient. For sections with a higher observation difficulty than the normal section, the coefficient is set to a value greater than 1.0, and for sections with a lower observation difficulty than the normal section, the coefficient is set to a value less than 1.0. In the example shown in Figure 12, the coefficient for sections with almost no folds is set to 0.25, and the coefficient for sections with undulations (sections with many deep folds, sections with diverticula) is set to 2.5.

[0083] The final optimal observation time for each part can be determined by multiplying the optimal observation time per unit length (adjusted based on observation difficulty) by the cumulative length of each part. If no adjustment based on observation difficulty is made, the final optimal observation time for each part can be determined by multiplying the standard optimal observation time by the cumulative length of each part.

[0084] In the case shown in Figure 12, the appropriate observation time for the total net observation time is 5:24 or more. Furthermore, while the appropriate observation time for areas with almost no folds is 0:39 or more, the actual observation time for areas with almost no folds is 0:36, indicating that the actual observation time for areas with almost no folds is slightly shorter. In the example shown in Figure 12, items where the actual observation time is equal to or greater than the appropriate observation time are marked OK, and items where the actual observation time is less than the appropriate observation time are marked NG. The examination time evaluation unit 37 may also determine whether the actual observation time for each part is excessive.

[0085] Figure 13 is a table summarizing the results of the extraction time classification for Specific Example 1, categorized by body part. As shown in Figure 4, the extraction time for Specific Example 1 was 17:21, and the net observation time was 8:57. When classified by body part, as shown in Figure 13, the extraction time for the ascending colon was 8:13, with a net observation time of 2:57; the extraction time for the transverse colon was 0:55, with a net observation time of 0:55; the extraction time for the descending colon was 4:30, with a net observation time of 1:54; the extraction time for the sigmoid colon was 1:48, with a net observation time of 1:21; and the extraction time for the rectum was 1:55, with a net observation time of 1:50.

[0086] Figure 14 shows a graph classifying the extraction time results shown in Figure 13 by body part. The horizontal axis represents the body part classification, and the breakdown of extraction time by body part is displayed as a bar graph. Furthermore, as shown in Figure 5, it is also possible to classify the work content for each body part.

[0087] The image classification unit 35 classifies the endoscopic images corresponding to the net observation process included in multiple endoscopic images in a time series by part of the organ to be examined (intestinal tract in this embodiment), based on the image recognition results from the image recognition unit 34. The examination time classification unit 36 ​​calculates the actual time for each part of the net observation process based on the results of the classification by part.

[0088] The inspection time evaluation unit 37 obtains the actual time for each part of the net observation process calculated by the inspection time classification unit 36. The inspection time evaluation unit 37 obtains the evaluation index for each part of the net observation process. The inspection time evaluation unit 37 compares the actual time for each part of the net observation process with the evaluation index for each part, and evaluates the actual time for each part of the net observation process based on the comparison results.

[0089] The display control unit 38 displays the actual time for each part of the net observation process and the evaluation index for each part of the net observation process on the display device 41. The display control unit 38 can also display the comparison result of the two as the evaluation result for each part on the display device 41.

[0090] By generating graphs that further classify the extraction time classification results by body part, it becomes possible to check the extraction time and net observation time for each body part, enabling more detailed analysis and judgment, such as whether there are inconsistencies in observation. In other words, it becomes possible to analyze and judge where cleaning and suction were performed for a long time, where treatment of lesions was performed for a long time, and whether the observation time for each body part was sufficient.

[0091] Figure 15 is a flowchart illustrating an example of the operation of the endoscopic examination support system 30 according to the embodiment. The endoscopic image acquisition unit 32 acquires multiple endoscopic images in a time series from the endoscopic system 10 (S10). The image classification unit 35 classifies the acquired multiple endoscopic images into each examination process (S11). The examination time classification unit 36 ​​calculates the actual time of the net observation process based on the endoscopic images classified into net observation processes (S12). The examination time evaluation unit 37 compares the actual time of the net observation process with the appropriate observation time and evaluates the observation time of the net observation process based on the comparison result (S13).

[0092] As explained above, this embodiment makes it possible to properly evaluate the quality of lesion screening by measuring the net observation time during endoscopic examination. Major endoscopic societies recommend a withdrawal time of 6 minutes or more as an indicator of sufficient observation time in cases without procedures. However, since the withdrawal time in endoscopic examination includes procedure time and cleaning / suction time, the indicator of a withdrawal time of 6 minutes or more does not guarantee that sufficient observation time has been secured. Even if the withdrawal time is 6 minutes or more, if the procedure time or cleaning / suction time is long, sufficient observation time may not have been secured.

[0093] In contrast, this embodiment measures the net observation time, which is the time taken after subtracting the treatment time and the time taken for washing and suctioning from the time taken for removal. This allows for a proper evaluation of whether sufficient observation time has been secured. Furthermore, by measuring the observation time for each condition or location of the intestinal tract and comparing it with the appropriate observation time for each condition or location, the quality of lesion screening can be evaluated in more detail.

[0094] The present disclosure has been described above based on several embodiments. These embodiments are illustrative, and it will be understood by those skilled in the art that various modifications are possible in combinations of their components and processing processes, and that such modifications are also within the scope of the present disclosure.

[0095] In the above embodiment, an example was described in which an endoscope insertion shape observation device 20 is connected to the endoscope examination support system 30, and the endoscope examination support system 30 acquires the endoscope shape from the endoscope insertion shape observation device 20. In this regard, the present disclosure is also applicable to systems in which the endoscope insertion shape observation device 20 is omitted. In that case, the image classification unit 35 will not be able to use information related to the endoscope shape, and will classify the endoscope image based on the image recognition result and operation information. Furthermore, if an encoder is installed near the anus of the subject and the encoder is connected to the endoscope examination support system 30, the endoscope examination support system 30 can acquire the insertion length from the encoder. [Industrial applicability]

[0096] This disclosure can be used for colonoscopy. [Explanation of Symbols]

[0097] 10...Endoscope system, 11...Endoscope, 11a...Insertion section, 11b...Hard tip section, 11c...Bending section, 11d...Flexible tube section, 11e...Operation section, 11f...Main body section, 11g...Gripping section, 12...Magnetic coil, 15...Light source device, 20...Endoscope insertion shape observation device, 20a...Receiving antenna, 20b...Reference plate, 30...Endoscope examination support system, 31...Endoscope shape acquisition unit, 32...Endoscope image acquisition unit, 33...Operation information acquisition unit, 34...Image recognition unit, 35...Image classification unit, 36...Examination time classification unit, 37...Examination time evaluation unit, 38...Display control unit, 39...Recording control unit, 41...Display device, 42...Input device, 43...Storage device.

Claims

1. This is an endoscopic examination support system, Equipped with one or more processors having hardware, The aforementioned processor, An examination time evaluation device (37) that acquires condition-specific evaluation indices defined according to the condition of the organ being examined in the endoscopic examination, as an evaluation index for the time related to a predetermined examination process in the endoscopic examination, As a record of the endoscopic examination, an endoscopic image acquisition device (32) acquires time-series endoscopic images of the endoscopic examination, An image recognition device (34) that detects the state of the organ to be examined based on the endoscopic images corresponding to the predetermined examination step included in the time-series endoscopic images, An inspection time classification device (36) classifies the state of the organ to be inspected detected by the image recognition device (34) and calculates the actual time for each state of the predetermined inspection process based on the results of the classification by state, A feature having Endoscopy support system.

2. In the endoscopic examination support system according to claim 1, The inspection time evaluation device (37) compares the state-specific evaluation index with the state-specific actual time and evaluates the state-specific actual time. Endoscopy support system.

3. The endoscopic examination support system according to claim 2 is The system further includes a display control unit (38) that displays on a display device (41) any of the following: the state-specific evaluation index, the state-specific actual time, or the comparison result between the state-specific evaluation index and the state-specific actual time obtained by the inspection time evaluation device (37). Endoscopy support system.

4. In the endoscopic examination support system according to claim 1, The examination time classification device (36) calculates the cumulative length of the examination target organ by state based on the results of classification by state, The inspection time evaluation device (37) sets the state-specific evaluation index according to the cumulative length for each state. Endoscopy support system.

5. In the endoscopic examination support system according to claim 1, The examination time classification device (36) calculates the cumulative length of the examination target organ by state based on the results of classification by state, The examination time evaluation device (37) sets the condition-specific evaluation index according to the cumulative length for each condition and the type of condition of the organ to be examined. Endoscopy support system.

6. In the endoscopic examination support system according to claim 1, The state of the organ to be examined detected by the image recognition device (34) includes the state of the folds of the intestinal tract. Endoscopy support system.

7. In the endoscopic examination support system according to claim 1, The state of the organ to be examined detected by the image recognition device (34) includes the presence or absence of diverticula in the intestinal tract. Endoscopy support system.

8. A method for supporting an endoscopy using an endoscopy support system, The examination time evaluation device (37) that constructs the processor of the endoscopic examination support system takes the step of acquiring a state-specific evaluation index defined according to the state of the organ to be examined in the endoscopic examination, as an evaluation index of the time related to a predetermined examination process in the endoscopic examination, Based on the endoscopic images corresponding to the predetermined examination steps included in the time-series endoscopic images of the endoscopic examination acquired by the endoscopic image acquisition device (32) that constitutes the processor, the image recognition device (34) that constitutes the processor performs the step of detecting the state of the organ to be examined, The inspection time classification device (36) that constructs the processor classifies the state of the organ to be inspected detected by the image recognition device (34), and calculates the actual time for each state of the predetermined inspection process based on the results of the classification by state. A feature having Endoscopic examination support methods.

9. In the endoscopic examination support method according to claim 8, The inspection time evaluation device (37) compares the state-specific evaluation index and the state-specific actual time, and evaluates the state-specific actual time. Endoscopic examination support methods.

10. In the endoscopic examination support method according to claim 9, The display control unit (38) that constructs the processor displays on the display device (41) any of the following: the state-specific evaluation index, the actual time for each state, or the comparison result between the state-specific evaluation index and the actual time for each state by the inspection time evaluation device (37). Endoscopic examination support methods.

11. In the endoscopic examination support method according to claim 8, Based on the results of the classification by state, the examination time classification device (36) calculates the cumulative length of the examination target organ by state, The inspection time evaluation device (37) sets the state-specific evaluation index according to the cumulative length for each state. Endoscopic examination support methods.

12. In the endoscopic examination support method according to claim 8, Based on the results of the classification by state, the examination time classification device (36) calculates the cumulative length of the examination target organ by state, The examination time evaluation device (37) sets the condition-specific evaluation index according to the cumulative length for each condition and the type of condition of the organ to be examined. Endoscopic examination support methods.

13. In the endoscopic examination support method according to claim 8, The state of the organ to be examined detected by the image recognition device (34) includes the state of the folds of the intestinal tract. Endoscopic examination support methods.

14. In the endoscopic examination support method according to claim 8, The state of the organ to be examined detected by the image recognition device (34) includes the presence or absence of diverticula in the intestinal tract. Endoscopic examination support methods.

15. As an evaluation index for the time required for a predetermined examination process in an endoscopic examination, the examination time evaluation device (37) is used to obtain a state-specific evaluation index defined according to the state of the organ being examined in the endoscopic examination, As a record of the endoscopic examination, the process involves acquiring time-series endoscopic images of the endoscopic examination using an endoscopic image acquisition device (32), Based on the endoscopic images corresponding to the predetermined examination steps included in the time-series endoscopic images, the image recognition device (34) performs a process to detect the state of the organ to be examined. The inspection time classification device (36) classifies the state of the organ to be inspected detected by the image recognition device (34), and based on the results of the classification by state, the process of calculating the actual time for each state of the predetermined inspection process, A storage medium that stores a program that causes a computer to execute a command.

16. The storage medium described in claim 15 is The system stores a program that compares the aforementioned state-specific evaluation index and the aforementioned state-specific actual time using the inspection time evaluation device (37) and executes a process to evaluate the aforementioned state-specific actual time. storage medium.

17. The storage medium described in claim 16 is The system stores a program that causes the computer to perform a process of displaying on the display device (41) by the display control unit (38) any of the following: the state-specific evaluation index, the state-specific actual time, or the comparison result between the state-specific evaluation index and the state-specific actual time obtained by the inspection time evaluation device (37). storage medium.

18. The storage medium described in claim 15 is Based on the results of the classification by condition using the examination time classification device (36), the cumulative length of the examination target organ by condition is calculated. The inspection time evaluation device (37) stores a program to be executed by a computer that sets the state-specific evaluation index according to the cumulative length for each state. storage medium.

19. The storage medium described in claim 15 is Based on the results of the classification by condition using the examination time classification device (36), the cumulative length of the examination target organ by condition is calculated. The examination time evaluation device (37) stores a program to be executed by a computer that sets the condition-specific evaluation index according to the cumulative length for each condition and the type of condition of the organ being examined. storage medium.

Images