Endoscopy assistance system, endoscopy assistance method, and storage medium
Through the endoscopy auxiliary system to evaluate and measure net observation time, the problem of the inability to accurately evaluate the quality of lesion screening in the prior art is solved, ensuring sufficient observation time in large intestinal endoscopy, and improving the quality of lesion screening .
Patent Information
- Application Number
- CN202280093572.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-15
- Publication Date
- 2024-11-12
AI Technical Summary
The prior art has failed to effectively evaluate the quality of lesion screening in endoscopy, and it is difficult to ensure sufficient observation time. Especially in large intestinal endoscopy, the extraction time includes multiple time periods, and it is impossible to accurately determine whether sufficient completion is completed. Lesion screening.
An endoscopy auxiliary system was designed to evaluate the time indicators of the inspection steps through a hardware processor, and combine the implementation records of the endoscopy to calculate the actual time of each inspection step, especially by measuring the net observation time to remove the disposal and cleaning. time provides an objective evaluation mechanism.
Accurate evaluation of the quality of lesion screening in endoscopy, ensure the adequacy of observation time in large intestinal endoscopy, can carefully adjust the observation time according to the intestinal state and location, and improve the lesion screening. quality.
Smart Images

Figure CN118946299A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an endoscope inspection assisting system, an endoscope inspection assisting method and a storage medium. Background Art
[0002] The main endoscopy societies recommend that the removal time of more than 6 minutes in cases without treatment be used as an indicator of sufficient observation time. In connection with this, a method has been proposed for classifying the endoscopic examination time into non-observation time, normal observation time, detailed observation time, and treatment time during endoscopic examination and recording them (for example, see Patent Document 1).
[0003] Prior art literature
[0004] Patent Literature
[0005] Patent Document 1: International Publication No. 18 / 179991 Summary of the invention
[0006] Problems to be solved by the invention
[0007] The above method discloses a method for recording endoscopic examinations, but does not disclose a specific method for evaluating the quality of lesion screening during endoscopic examinations. A mechanism is required that can evaluate whether sufficient lesion screening time is ensured.
[0008] The present invention has been made in view of such circumstances, and an object of the present invention is to provide a technology capable of evaluating the quality of lesion screening in endoscopic examination.
[0009] Means for solving problems
[0010] In order to solve the above problems, an endoscope inspection support system of one embodiment of the present invention has one or more processors with hardware. The processor obtains an evaluation index of the time involved in a predetermined inspection step in the endoscope inspection, obtains an implementation record of the endoscope inspection, and calculates the time taken for the predetermined inspection step, that is, the actual time, based on the implementation record.
[0011] Another aspect of the present invention is an endoscope inspection support method that obtains an evaluation index of time involved in a predetermined inspection step in an endoscope inspection, obtains an implementation record of the endoscope inspection, and calculates the actual time taken for the predetermined inspection step based on the implementation record.
[0012] In addition, arbitrary combinations of the above-described constituting elements and forms in which the expression of the present invention is converted between methods, apparatuses, systems, recording media, computer programs, and the like may also be practiced as additional modes of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Figure 1It is a diagram showing the overall system configuration related to colon endoscopy according to the embodiment.
[0014] Figure 2 This is a diagram showing an example of an endoscope used in this embodiment.
[0015] Figure 3 It is a diagram showing a configuration example of an endoscope inspection support system according to an embodiment.
[0016] Figure 4 This is a diagram that summarizes the results of the extraction time classification of Specific Example 1 in a table.
[0017] Figure 5 This is a diagram showing a graph in which the results of the removal time classification in Specific Example 1 are classified according to the insertion length.
[0018] Figure 6 This is a diagram that summarizes the results of the extraction time classification of Specific Example 2 in a table.
[0019] Figure 7 This is a diagram showing a graph in which the results of the extraction time classification of the specific example 2 are classified according to the elapsed time at the time of extraction.
[0020] Figure 8 This is a diagram that summarizes the classification results of the net observation time of Specific Example 1 according to the intestinal state.
[0021] Fig. 9 (a)-(b) are figures showing specific examples of endoscopic images of an intestinal tract with almost no wrinkles.
[0022] Fig.10 (a) to (c) are figures showing specific examples of endoscopic images of an intestinal tract with many deep folds.
[0023] Fig.11 (a) to (d) are diagrams showing specific examples of endoscopic images of the intestine in which a diverticulum is present.
[0024] Fig.12 is Figure 8 The evaluation results are added to the classification results of the net observation time shown and summarized in the table.
[0025] Fig.13 This is a diagram that summarizes the extraction time classification results of Specific Example 1 by site.
[0026] Fig.14 It shows that Fig.13 The figure shown is a graph of the extraction time classification result classified by body part.
[0027] Fig.15This is a flowchart showing an operation example of the endoscope inspection support system according to the embodiment. DETAILED DESCRIPTION
[0028] The present embodiment relates to colon endoscopy. Usually, in colon endoscopy, after the endoscope is inserted into the cecum, it is pulled out toward the anus while observation / treatment is performed. One of the QIs (Quality Indicator) in colon endoscopy is the withdrawal time. In order to ensure the quality of lesion screening by colon endoscopy, it is preferable to present a direct indicator (for example, the observation rate of the intestinal surface) that firmly and completely observes the colon mucosa. Among them, in major endoscopy societies, from the perspective of technical difficulty, as an indirect indicator, it is recommended that the withdrawal time be more than 6 minutes in cases without treatment.
[0029] However, if the method is limited to cases without treatment, more cases will be excluded. In particular, if a simple test is positive and a colon endoscopy is performed in a hospital for a detailed examination, the proportion of cases that are excluded becomes quite high. Moreover, the larger the hospital, the smaller the proportion of cases that are covered. Therefore, it is difficult to ensure the quality of lesion screening for all cases.
[0030] On the other hand, the extraction time includes, in addition to the observation time for lesion screening, the time for detailed inspection of suspected lesions, the time for excision of lesions, and the time for cleaning / suctioning of residues or moisture. It cannot be concluded that sufficient observation time for lesion screening is ensured simply because the extraction time is more than 6 minutes. Based on the above, a mechanism for objectively and accurately judging whether sufficient observation time is ensured is required.
[0031] Figure 1 1 is a diagram showing the overall system configuration associated with colon endoscopy according to the embodiment. In the present embodiment, an endoscope system 10, an endoscope 11, a light source device 15, an endoscope insertion shape observation device (UPD: Endoscope Position Detecting Unit) 20, an endoscope inspection auxiliary system 30, a display device 41, an input device 42, and a storage device 43 are used. The endoscope 11 of the present embodiment is a colon endoscope inserted into the colon of a subject (patient).
[0032] The endoscope 11 includes a lens and a solid-state imaging element (for example, a CMOS image sensor, a CCD image sensor, or a CMD image sensor). The solid-state imaging element converts the light focused by the lens into an electrical signal and outputs it as an endoscopic image (electrical signal) to the endoscope system 10. The endoscope 11 includes a forceps channel. The operator (doctor) can perform various treatments during endoscopic examination by passing a treatment instrument through the forceps channel.
[0033] The light source device 15 includes a light source such as a xenon lamp, and supplies observation light (white light, narrowband light, fluorescence, near-infrared light, etc.) to the distal end of the endoscope 11. The light source device 15 also has a built-in pump for sending water or air to the endoscope 11.
[0034] The endoscope system 10 controls the light source device 15 and processes the endoscope image input from the endoscope 11. The endoscope system 10 is equipped with functions such as narrow band imaging (NBI), red dichromatic imaging (RDI), texture and color enhancement imaging (TXI), and extended depth of field (EDOF).
[0035] In narrow-band light observation, by irradiating light of specific wavelengths of purple (415nm) and green (540nm) that are strongly absorbed by hemoglobin in the blood, an endoscopic image that emphasizes the capillaries or fine structures on the surface of the mucosa can be obtained. In red light observation, by irradiating light of specific wavelengths of three colors (green, amber, and red), an endoscopic image that emphasizes the contrast of deep tissues can be obtained. In structural color emphasis, an endoscopic image is generated that optimizes the three elements of "structure", "hue", and "brightness" of the mucosal surface under normal light observation. In depth of field expansion, by synthesizing two images that are focused at close distances and long distances, an endoscopic image with a wide focus range can be obtained.
[0036] The endoscope system 10 outputs an endoscopic image obtained by processing the endoscopic image input from the endoscope 11 or the endoscopic image itself input from the endoscope 11 to the endoscope inspection support system 30 .
[0037] The endoscope insertion shape observation device 20 is a device for observing the three-dimensional shape of the endoscope 11 inserted into the lumen of the subject. A receiving antenna 20a is connected to the endoscope insertion shape observation device 20. The receiving antenna 20a is an antenna for detecting magnetic fields generated by a plurality of magnetic coils built into the endoscope 11.
[0038] Figure 21 is a diagram showing an example of an endoscope 11 used in the present embodiment. The endoscope 11 includes an elongated tubular insertion portion 11a formed of a flexible member and an operation portion 11e connected to the base end of the insertion portion 11a. The insertion portion 11a includes a front end hard portion 11b, a curved portion 11c, and a flexible tube portion 11d from the front end side toward the base end side. The base end of the front end hard portion 11b is connected to the front end of the curved portion 11c, and the base end of the curved portion 11c is connected to the base end of the flexible tube portion 11d.
[0039] The operation part 11e includes a main body 11f extending from a flexible tube part 11d and a grip part 11g connected to the base end of the main body 11f. The grip part 11g is gripped by the operator. A universal cable extends from the operation part 11e and is connected to the endoscope system 10 and the light source device 15. The universal cable includes an imaging cable, a light guide, etc. extending from the insertion part 11a.
[0040] The distal end rigid portion 11b is the distal end portion of the insertion portion 11a and is also the distal end portion of the endoscope 11. The distal end rigid portion 11b has built-in solid-state imaging elements, an illumination optical system, an observation optical system, etc. The illumination light emitted from the light source device 15 propagates along the light guide to the distal end surface of the distal end rigid portion 11b, and irradiates the observation object in the lumen from the distal end surface of the distal end rigid portion 11b.
[0041] The bending portion 11c is formed by connecting node rings along the longitudinal axis of the insertion portion 11a. The bending portion 11c bends in a desired direction according to an operation input to the operation portion 11e by an operator, and the position and orientation of the distal rigid portion 11b change according to the bending.
[0042] The flexible tube portion 11d is a tubular member extending from the main body portion 11f of the operation portion 11e, has desired flexibility, and is bent by an external force. The operator inserts the insertion portion 11a into the large intestine of the subject while bending the bending portion 11c or twisting the flexible tube portion 11d.
[0043] A plurality of magnetic coils 12 are arranged at predetermined intervals (eg, 10 cm intervals) along the longitudinal direction inside the insertion portion 11a. Each magnetic coil 12 generates a magnetic field when current is supplied. The plurality of magnetic coils 12 function as position sensors for detecting each position of the insertion portion 11a.
[0044] return Figure 1The receiving antenna 20a receives the magnetic field emitted from the plurality of magnetic coils 12 built into the insertion portion 11a of the endoscope 11, and outputs it to the endoscope insertion shape observation device 20. The endoscope insertion shape observation device 20 applies the magnetic field strength of each of the plurality of magnetic coils 12 received by the receiving antenna 20a to a predetermined position detection algorithm to estimate the three-dimensional position of each of the plurality of magnetic coils 12. The endoscope insertion shape observation device 20 generates a three-dimensional endoscope shape of the insertion portion 11a of the endoscope 11 by performing curve interpolation on the estimated three-dimensional positions of the plurality of magnetic coils 12.
[0045] The reference plate 20b is attached to the subject (for example, the abdomen of the subject). A body position sensor for detecting the body position of the subject is arranged on the reference plate 20b. As the body position sensor, for example, a three-axis acceleration sensor or a gyro sensor can be used. Figure 1 In the embodiment, the reference plate 20 b is connected to the endoscope insertion shape observation device 20 via a cable, and the reference plate 20 b outputs three-dimensional posture information indicating the posture of the reference plate 20 b (ie, the posture of the subject) to the endoscope insertion shape observation device 20 .
[0046] In addition, as the body position sensor arranged on the reference plate 20b, a plurality of magnetic coils similar to the plurality of magnetic coils 12 built into the insertion portion 11a of the endoscope 11 may be used. In this case, the receiving antenna 20a receives the magnetic field emitted from the plurality of magnetic coils arranged on the reference plate 20b, and outputs it to the endoscope insertion shape observation device 20. The endoscope insertion shape observation device 20 applies the magnetic field strength of each of the plurality of magnetic coils received by the receiving antenna 20a to a predetermined posture detection algorithm, and generates three-dimensional posture information representing the posture of the reference plate 20b (that is, the posture of the subject).
[0047] The endoscope insertion shape observation device 20 changes the generated three-dimensional endoscope shape in accordance with the change of the three-dimensional posture information. Specifically, the endoscope insertion shape observation device 20 changes the three-dimensional endoscope shape so as to eliminate the change of the three-dimensional posture information. Thus, even when the body position of the subject is changed during the endoscopic examination, the endoscope shape from a specific viewpoint (for example, a viewpoint from which the abdomen of the subject is observed vertically from the front side of the abdomen) can always be recognized.
[0048] The endoscope insertion shape observation device 20 can obtain the insertion length and the time elapsed from the insertion of the endoscope 11 into the large intestine (hereinafter referred to as the insertion time), wherein the insertion length indicates the length of the portion of the endoscope 11 inserted into the large intestine. For example, the endoscope insertion shape observation device 20 measures the insertion length based on the position at which the operator inputs the inspection start operation into the input device 42 as a base point, and measures the insertion time based on this time as a starting point. In addition, the endoscope insertion shape observation device 20 can also estimate the position of the anus based on the generated three-dimensional endoscope shape and the difference in magnetic field strength between the magnetic coil located inside the body and the magnetic field coil located outside the body, and use the estimated position of the anus as the base point of the insertion length.
[0049] In order to measure the insertion length with high accuracy, an encoder may be provided near the anus of the subject. The endoscope insertion shape observation device 20 detects the insertion length based on the position of the anus based on the signal from the encoder.
[0050] The endoscope insertion shape observation device 20 adds the insertion length and the insertion time to the three-dimensional endoscope shape after the body position correction based on the three-dimensional posture information, and outputs the result to the endoscope inspection support system 30 .
[0051] The endoscope inspection support system 30 generates support information for endoscope inspection based on the endoscope image input from the endoscope system 10 and the endoscope shape input from the endoscope insertion shape observation device 20, and presents it to the operator. In addition, the endoscope inspection support system 30 generates endoscope inspection history information based on the endoscope image input from the endoscope system 10 and the endoscope shape input from the endoscope insertion shape observation device 20, and records it in the storage device 43.
[0052] The display device 41 includes a liquid crystal monitor or an organic EL monitor, and displays images input from the endoscope inspection supporting system 30. The input device 42 includes a mouse, a keyboard, a touch panel, etc., and outputs operation information input by the operator to the endoscope inspection supporting system 30. The storage device 43 includes a storage medium such as an HDD and an SSD, and stores endoscope inspection history information generated by the endoscope inspection supporting system 30. The storage device 43 may be a dedicated storage device attached to the endoscope system 10, or a database in an in-hospital server connected via an in-hospital network, or a database in a cloud server.
[0053] exist Figure 1In the system configuration shown, the reference plate 20b can be omitted. In addition, the endoscope insertion shape observation device 20 and the receiving antenna 20a can also be omitted. When the insertion length or insertion time is measured outside the endoscope insertion shape observation device 20 and the endoscope shape is not used in the part identification process described later, the endoscope insertion shape observation device 20, the receiving antenna 20a and the reference plate 20b can be omitted.
[0054] Figure 3 The following is a structural example of an endoscope inspection assistance system 30 according to an embodiment. The endoscope inspection assistance system 30 can be constructed by a dedicated processing device for endoscope inspection assistance, or by a general-purpose server (which can be a cloud server). Furthermore, the endoscope inspection assistance system 30 can be constructed by any combination of a dedicated processing device for endoscope inspection assistance, a general-purpose server (which can be a cloud server), and a dedicated image diagnosis device. In addition, the endoscope inspection assistance system 30 can also be constructed integrally with the endoscope system 10.
[0055] The endoscope inspection support system 30 includes an endoscope shape acquisition unit 31, an endoscope image acquisition unit 32, an operation information acquisition unit 33, an image recognition unit 34, an image classification unit 35, an inspection time classification unit 36, an inspection time evaluation unit 37, a display control unit 38, and a recording control unit 39. These components can be implemented in hardware by any at least one processor (such as a CPU, a GPU), a memory (such as a DRAM), or other LSIs (such as an FPGA, an ASIC), and in software by a program loaded into the memory, etc., but the functional blocks implemented by their cooperation are described here. Therefore, it should be understood by those skilled in the art that these functional blocks can be implemented in various forms by only hardware, only software, or a combination thereof.
[0056] The endoscope shape acquisition unit 31 acquires the endoscope shape from the endoscope insertion shape observation device 20 . The endoscope shape also includes information on the insertion length and insertion time. The endoscope image acquisition unit 32 acquires an endoscope image from the endoscope system 10 .
[0057] The image recognition unit 34 has a plurality of machine learning models for detecting the position / state, treatment instruments, observation conditions, lesions, etc. of the inspection object organ (in this embodiment, the large intestine) from the endoscopic image. The plurality of machine learning models are generated by machine learning, which uses a plurality of endoscopic images annotated with various positions / states, various treatment instruments, various observation conditions, various lesions, etc. as training data sets. The annotations are given by annotators with professional knowledge, such as doctors. In machine learning, CNN, RNN, LSTM, etc., which are a type of deep learning, can be used.
[0058] The parts of the large intestine are roughly classified into the rectum, sigmoid colon, descending colon, transverse colon, ascending colon and cecum, starting from the anus side. The image recognition unit 34 can input the endoscopic image into the part learning model and detect the part of the large intestine based on the endoscopic image. At this time, the image recognition unit 34 can also determine the part based on the detection results of multiple endoscopic images that are continuous in time series. For example, in 30 consecutive frames or 60 frames of endoscopic images, when the image recognition unit 34 detects the same part in more than a set number of frames, it determines the part as the formal detection part.
[0059] In addition, the image recognition unit 34 may also determine the location by considering the front-to-back relationship of the detection location or the endoscope shape obtained from the endoscope insertion shape observation device 20. The image recognition unit 34 determines, for example, whether the moving direction of the endoscope 11 is the insertion direction (anus→cecum) or the removal direction (cecum→anus). In the case of the insertion direction, when the left colic flexure is detected, the image recognition unit 34 switches the detection location from the descending colon to the transverse colon, and when the right colic flexure is detected, the detection location is switched from the transverse colon to the ascending colon. In the case of the insertion direction, when the right colic flexure is detected, the image recognition unit 34 switches the detection location from the ascending colon to the transverse colon, and when the left colic flexure is detected, the detection location is switched from the transverse colon to the descending colon.
[0060] Furthermore, the image recognition unit 34 may improve the accuracy of part detection by considering the three-dimensional position of the distal end hard portion 11b (hereinafter referred to as the endoscope distal end portion) based on the endoscope shape acquired from the endoscope insertion shape observation device 20. For example, when the position of the endoscope distal end portion estimated based on the endoscope shape is inconsistent with the position of the detected part based on the image recognition, the image recognition unit 34 discards the detection result based on the image recognition.
[0061] Furthermore, the image recognition unit 34 can input the endoscopic image into the treatment instrument / suction tube learning model, and detect the treatment instrument (e.g., biopsy forceps, snare) or suction tube based on the endoscopic image. Furthermore, the image recognition unit 34 can input the endoscopic image into the residue / washing bubble / bleeding / water learning model, and detect the residue, washing bubble, bleeding, or water based on the endoscopic image. In addition, the image recognition unit 34 can also consider the operator's operation information input to the operation unit 11e of the endoscope 11 or the input device 42, and determine the treatment instrument, suction tube, residue, washing bubble, bleeding, and water, respectively.
[0062] Furthermore, the image recognition unit 34 can input the endoscopic image into the observation condition learning model and recognize the observation condition based on the endoscopic image. For example, the image recognition unit 34 can recognize whether normal light or special light (e.g., narrow-band light, red light) is used. For example, the image recognition unit 34 can recognize the presence or absence of a dye, and when a dye is used, the image recognition unit 34 can recognize the type of dye. For example, the image recognition unit 34 can recognize the presence or absence of image enhancement, and when image enhancement is used, the image enhancement method and intensity can be recognized. For example, the image recognition unit 34 can recognize the presence or absence of zoom, and when zoom is used, the image recognition unit 34 can recognize the zoom ratio. In addition, the image recognition unit 34 can also use the operator's operation information input to the operation unit 11e of the endoscope 11 or the input device 42 or consider the operation information to recognize various observation conditions.
[0063] Furthermore, the image recognition unit 34 can input the endoscopic image into the intraluminal state learning model and determine the intraluminal state based on the endoscopic image. For example, the image recognition unit 34 can detect the presence or absence of wrinkles above a specified height and the presence or absence of diverticula. Furthermore, the image recognition unit 34 can input the endoscopic image into the lesion learning model and detect lesion candidates based on the endoscopic image.
[0064] The image recognition unit 34 may also check the image quality of the endoscopic image before recognizing the detection target image. The image recognition unit 34 excludes endoscopic images that are judged to have poor image quality (e.g., shaking, out of focus, abnormal brightness (e.g., halation)) from the detection target image recognition object.
[0065] The image classification unit 35 determines the endoscopic image when the deepest part is reached and when the removal is completed. The deepest part in the colon endoscopy is usually the cecum. For example, the image classification unit 35 determines the endoscopic image when the cecum is detected by the image recognition unit 34 as the endoscopic image when the deepest part is reached. In addition, the image classification unit 35 can also determine the endoscopic image when the insertion length obtained from the endoscope insertion shape observation device 20 is the longest as the endoscopic image when the deepest part is reached. In addition, the image classification unit 35 can also determine the endoscopic image at the moment when the operator inputs the insertion completion operation to the input device 42 as the endoscopic image when the deepest part is reached. In addition, depending on the operator, the endoscope 11 is sometimes inserted into the ileum. In addition, depending on the subject, the endoscope 11 cannot be inserted into the cecum, and the ascending colon becomes the deepest part in the colon endoscopy.
[0066] The image classifying unit 35 identifies the endoscopic image at the time when the in-vivo image is switched to the out-vivo image as the endoscopic image when the extraction is completed. Alternatively, the image classifying unit 35 may identify the endoscopic image at the time when the operator inputs the extraction completion operation to the input device 42 as the endoscopic image when the extraction is completed.
[0067] The image classification unit 35 classifies the multiple endoscopic images captured from the time of reaching the deepest part to the time of complete extraction according to various conditions. The image classification unit 35 performs lesion screening classification, detailed inspection classification, treatment classification, cleaning / suction classification, observation condition classification, intraluminal state classification, image quality classification, etc. on the multiple endoscopic images.
[0068] In the lesion screening classification, the image classification unit 35 classifies the endoscopic images that are estimated to be performed by the operator for operations (surgery) other than detailed inspection and treatment as endoscopic images for lesion screening. Operations other than treatment also include cleaning / suction. Endoscopic images that are estimated to be performed for operations other than detailed inspection, treatment, cleaning / suction and not for lesion screening are excluded from the endoscopic images for lesion screening. In addition, when determining the endoscopic images for lesion screening, the image classification unit 35 may also consider the use of special light observation.
[0069] In the detailed inspection classification, the image classification unit 35 classifies the endoscopic image during the period of staying at the part containing the lesion candidate for more than a predetermined time as the endoscopic image for detailed inspection. In addition, when determining the endoscopic image for detailed inspection, the image classification unit 35 may also consider the increase of the zoom factor, the use of special light observation, or the use of staining observation.
[0070] In the treatment classification, the image classification unit 35 classifies the endoscopic image in which the treatment instrument is detected as the endoscopic image during treatment. In the cleaning / suction classification, the image classification unit 35 classifies the endoscopic image in which the suction tube, residue, cleaning bubble, bleeding or water is detected as the endoscopic image during cleaning / suction. The cleaning / suction during observation is performed to clean / suction the residue and water so that the lumen surface is easy to observe. In addition, the cleaning / suction is performed to rinse the dye solution with water. The cleaning / suction after treatment is performed to rinse bleeding and dirt.
[0071] In the observation condition classification, the image classification unit 35 determines the setting conditions of the light source, the setting conditions of the image enhancement, etc. according to each endoscopic image. The observation condition classification is implemented independently of other image classifications. In the intraluminal state classification, the image classification unit 35 determines the presence or absence of wrinkles above a specified height, the presence or absence of diverticula, and the presence or absence of other states or constitutions related to the difficulty of observation according to each endoscopic image. The intraluminal state classification is also implemented independently of other image classifications.
[0072] In the image quality classification, the image classification unit 35 determines the presence or absence of blurring, defocusing, or abnormal brightness (eg, halation) for each endoscopic image, and excludes endoscopic images in which these image quality defects are detected from the classification targets for each inspection step.
[0073] Furthermore, the image classification unit 35 can also classify the plurality of endoscopic images according to each part. For example, the image classification unit 35 classifies the plurality of endoscopic images according to the parts of the large intestine anatomically classified (e.g., rectum, sigmoid colon, descending colon, transverse colon, ascending colon, and cecum). The image classification unit 35 determines the part of each endoscopic image based on at least one of the part detected by image recognition and the position information of the endoscope front end of the endoscope shape provided by the endoscope insertion shape observation device 20. The classification based on the part is implemented independently of other image classifications.
[0074] In addition, the image classification unit 35 can classify the insertion length according to each specified length (for example, 5 cm). The image classification unit 35 can obtain the insertion length from the endoscope insertion shape observation device 20. The classification based on the insertion length is implemented independently of other image classifications. Thus, the time spent on lesion screening, etc. can be confirmed according to each specific part or each specific position. In addition, the time ratio of each item that constitutes the operating state, observation conditions, intraluminal state, etc. can be confirmed in units of specific parts or specific positions.
[0075] A user such as a doctor can set the image classification method in the image classification unit 35 through the input device 42. The user can set and change the image classification method in order to meet the needs of operators or facilities, recommendations of academic societies, etc., or legal requirements.
[0076] In this way, the image classification unit 35 obtains the implementation record of the endoscopic examination. The implementation record of the endoscopic examination includes a plurality of endoscopic images that are continuous in time series during the endoscopic examination. The classification method of the endoscopic examination includes classification based on the inspection steps. The image classification unit 35 classifies which inspection step each endoscopic image included in the plurality of endoscopic images in time series corresponds to based on the image recognition result and the like.
[0077] The inspection time classification unit 36 calculates the time required for each inspection step, i.e., the actual time, based on the result of the classification of the implementation record of the endoscope inspection by the image classification unit 35. The plurality of inspection steps constituting the endoscope inspection include a predetermined inspection step. The predetermined inspection step may be an observation step after at least the detailed inspection step and the treatment step are removed from all the steps of the endoscope inspection. The predetermined inspection step may also be a net observation step after further removing the washing step and the suction step from the observation step.
[0078] The image classification unit 35 can classify a plurality of endoscopic images according to various incisions in addition to the classification based on the inspection steps. The inspection time classification unit 36 can summarize the actual time for each item constituting each classification. Furthermore, the inspection time classification unit 36 can also calculate the time ratio between the items constituting each classification. The user can set the calculation method of the actual time of the evaluation object item to the inspection time classification unit 36 from the input device 42. The calculation method can be different for display and recording. For example, the calculation method for display can be simpler than the calculation method for recording.
[0079] The examination time evaluation unit 37 obtains the actual time involved in each examination step calculated by the examination time classification unit 36. A user such as a doctor can set an evaluation index (for example, appropriate observation time) of the actual time involved in a predetermined examination step to the examination time evaluation unit 37 through the input device 42. The examination time evaluation unit 37 obtains the evaluation index of the actual time involved in the predetermined examination step.
[0080] The inspection time evaluation unit 37 compares the actual time involved in the prescribed inspection steps with the evaluation index involved in the prescribed inspection steps, and evaluates the actual time involved in the prescribed inspection steps based on the comparison result. The inspection time evaluation unit 37 can also use the actual time other than the prescribed inspection steps as the evaluation object. The evaluation index for determining whether the actual time of the evaluation object is sufficient, too little or too much can be set according to the actual time of each evaluation object. The evaluation index is assumed to be a fixed value or a value calculated according to a mathematical formula. The preferred evaluation index is an index whose validity has been statistically confirmed or an index recommended by a society, but it can also be a value other than these, such as a value in verifying validity or a temporarily set value. It is expected that the following changes can be made: when it is not desired to set a value, it can be possible to not set it, and to set a value when it is desired to set it.
[0081] The display control unit 38 displays the actual time related to the predetermined inspection step and the evaluation index related to the predetermined inspection step on the display device 41. The display control unit 38 can also display the comparison result of the two as the evaluation result on the display device 41. The display control unit 38 can also display the actual time of each of the plurality of inspection steps constituting the endoscopic inspection on the display device 41.
[0082] The display control unit 38 can display the actual time and evaluation results of the specified inspection items on the display device 41 during or after the inspection. By displaying the actual time and evaluation results of the specified inspection items in real time or approximately in real time (with a delay of about several seconds) during the inspection, the operator can determine whether the lesion screening is sufficient in the inspection room. If the lesion screening is insufficient, the operator can re-perform at least a part of the lesion screening.
[0083] The recording control unit 39 records the examination time record data which comprehensively records the actual time of each examination step and the evaluation result of at least the prescribed examination step in the storage device 43. The recording of the examination time record data in the storage device 43 may be temporary or long-term. The examination time record data recorded in the storage device 43 can be transmitted to other databases in the hospital by appropriately changing the data format. In addition, the examination time record data recorded in the storage device 43 can be displayed on the display device of the reference source based on the reference from the outside. Through these transmission or display to the outside, the operator or other medical-related personnel can refer to the examination time record data after the examination.
[0084] Figure 4 This is a diagram that summarizes the extraction time classification results of specific example 1 in a table. The extraction time is defined as the time elapsed from the time when the deepest part is reached to the time when the extraction is completed. The inspection time classification unit 36 classifies the extraction time into three broad categories: observation time, detailed inspection time, and treatment time. The inspection time classification unit 36 subclassifies the observation time into three categories: natural light observation (WLI: White Light Imaging) time, special light observation (NBI) time, and cleaning / suction time. The inspection time classification unit 36 subclassifies the treatment time into three categories: biopsy forceps use time, snare use time, and cleaning / suction time.
[0085] The inspection time classification unit 36 calculates the net observation time (8:57) by excluding the treatment time (5:31), the detailed inspection time (1:40), and the washing / suction time (1:13) of the observation from the extraction time (17:21=17 minutes and 21 seconds, the same below). The net observation time (8:57) can also be obtained from the total of the natural light observation (WLI) time (3:09) and the special light observation (NBI) time (5:48). The inspection time classification unit 36 can calculate the extraction time (11:50) other than the treatment by excluding the treatment time (5:31) from the extraction time (17:21). In addition, the calculation of the extraction time other than the treatment is optional.
[0086] In this way, the time for auxiliary operations such as cleaning / suction can be saved, and the net observation time used for lesion screening itself can be measured. The net observation time can be measured in all cases. In addition, the extraction time outside of treatment can also be measured in all cases. In addition, the actual time to be evaluated can be appropriately selected, and multiple actual times can be selected as evaluation objects.
[0087] The display control unit 38 can classify the actual time of each inspection step constituting the endoscopic inspection into each section obtained by dividing the insertion length of the endoscope 11 during the endoscopic inspection into predetermined length intervals, and display the result on the display device 41. The insertion length is specified in a range from 0 [cm] to the length [cm] of the endoscope 11 when it reaches the deepest part.
[0088] Figure 5 FIG. 1 is a diagram showing a graph of the results of the extraction time classification in Example 1 classified by the insertion length. The horizontal axis is the insertion length, and the details of the extraction time are shown in a bar graph in units of 5 cm. Figure 5 In the graph shown, the close examination time is further classified into the close examination time using a staining solution and the close examination time under magnification observation. Doctors and other users can refer to Figure 5 The operation details and observation time at each position of the endoscope 11 can be confirmed by referring to the graph shown.
[0089] Figure 6 The table shows the results of the extraction time classification of Example 2. The display control unit 38 can classify the actual time of each inspection step constituting the endoscope inspection into each section obtained by dividing the elapsed time of the extraction in the endoscope inspection into predetermined time intervals, and display the result on the display device 41 .
[0090] Figure 7 This is a diagram showing a chart of the extraction time classification results of Specific Example 2 classified according to the elapsed time during extraction. The elapsed time during extraction corresponds to the dynamic image shooting time of the endoscope 11 from the time of reaching the deepest part to the time of completion of extraction. The horizontal axis is the elapsed time, and the details of the extraction time are shown in a bar graph in units of 1 minute. Figure 7 In the example shown, it can be seen that the close inspection was performed between 0:17:00 and 0:18:00 and between 0:24:00 and 0:25:00. In specific example 2, the cleaning / suction time was 2:20 and the natural light observation (WLI) time was 9:58.
[0091] However, in the intestine, there are parts where lesion screening is easy and parts where it is difficult. For example, parts with almost no wrinkles are easy to observe and a short observation time is sufficient. On the other hand, for parts with many deep wrinkles and undulating parts with diverticula, a longer observation time is required to confirm whether there is a lesion in the shadowed part. Therefore, confirming the observation time of parts with almost no wrinkles, parts with many deep wrinkles, and parts with diverticula is effective in confirming whether the parts that are difficult to screen for lesions have been observed for a time commensurate with the difficulty.
[0092] Figure 8 This is a graph that summarizes the classification results of the net observation time of Specific Example 1 by intestinal state. Figure 8 In the example shown, the net observation time (8:57) is divided into four, namely, the observation time of the part with almost no wrinkles (0:36), the observation time of the part with many deep wrinkles (5:44), the observation time of the part with diverticula (0:07), and the observation time of other parts (normal parts) (2:30).
[0093] Fig. 9 (a)-(b) are diagrams showing specific examples of endoscopic images A11 and A12 of an intestinal tract that has almost no wrinkles. Fig.10 (a) to (c) are diagrams showing specific examples of endoscopic images A21 to A23 of an intestinal tract with many deep folds. Fig.11 (a) to (d) are diagrams showing specific examples of endoscopic images A31 to A34 of the intestine in which diverticula D31 to D34 are present.
[0094] The image classification unit 35 classifies the endoscopic images corresponding to the net observation step included in the multiple endoscopic images of the time series according to the state of the inspection object organ (the intestine in this embodiment) based on the image recognition result of the image recognition unit 34. The inspection time classification unit 36 calculates the actual time of the net observation step by state based on the result of the classification by state. The state of the intestine is classified according to the difficulty of observation. The state of the intestine may also include the state of the folds of the intestine. The state of the intestine may include whether there is a diverticulum.
[0095] The inspection time evaluation unit 37 obtains the actual time by state of the net observation step calculated by the inspection time classification unit 36. The inspection time evaluation unit 37 obtains the evaluation index by state of the net observation step. The inspection time evaluation unit 37 compares the actual time by state of the net observation step and the evaluation index by state, and evaluates the actual time by state of the net observation step based on the comparison result.
[0096] The display control unit 38 displays the actual time for each state of the net observation step and the evaluation index for each state of the net observation step on the display device 41. The display control unit 38 can also display the comparison result of the two on the display device 41 as the evaluation result for each state.
[0097] Fig.12 is Figure 8 The figure shows the classification results of the net observation time, and the evaluation results are added to the table. The cumulative length corresponding to the actual observation time (8:57) is 73 cm, the cumulative length of the part with almost no wrinkles is 23 cm, the cumulative length of the part with many deep wrinkles is 29 cm, the cumulative length of the part with diverticula is 1 cm, and the cumulative length of other parts (normal parts) is 20 cm.
[0098] The inspection time classification unit 36 calculates the state-by-state cumulative length of the net observation step based on the state-by-state classification result of the endoscopic image corresponding to the net observation step. The inspection time evaluation unit 37 sets the state-by-state evaluation index of the net observation step based on the state-by-state cumulative length of the net observation step, or based on the state-by-state cumulative length of the net observation step and the type of intestinal state. The inspection time evaluation unit 37 sets, for example, the state-by-state appropriate observation time as the state-by-state evaluation index.
[0099] The inspection time evaluation unit 37 sets the appropriate state-based observation time per unit length of the intestine having deep wrinkles of a certain depth or more, in other words, wrinkles of a certain height or more, or a portion where the number of wrinkles of a certain height or more is greater than a set value, longer than the appropriate state-based observation time per unit length of the intestine of a normal portion. In addition, the inspection time evaluation unit 37 sets the appropriate state-based observation time per unit length of the intestine of a portion having a diverticulum to be longer than the appropriate state-based observation time per unit length of the intestine of a portion without a diverticulum.
[0100] exist Fig.12 In the example shown, 70 cm of the normal part is observed for 4 minutes (about 3.43 seconds per 1 cm) as the standard appropriate observation time. The appropriate observation time for each part classified by state is calculated by multiplying the standard appropriate observation time by a specified coefficient. The coefficient of the part with a higher observation difficulty than the normal part is set to a value greater than 1.0, and the coefficient of the part with a lower observation difficulty than the normal part is set to a value less than 1.0. Fig.12 In the example shown, the coefficient of a portion having almost no wrinkles is set to 0.25, and the coefficient of a portion having undulations (a portion having many deep wrinkles, a portion having a diverticulum) is set to 2.5.
[0101] The final appropriate observation time for each part can be obtained by multiplying the appropriate observation time per unit length corrected based on the observation difficulty by the cumulative length of each part. In addition, without performing correction based on the observation difficulty, the final appropriate observation time for each part can be obtained by multiplying the standard appropriate observation time by the cumulative length of each part.
[0102] exist Fig.12 In the case shown, the appropriate observation time for the entire net observation time is 5:24 or more. In addition, the appropriate observation time for the part with almost no wrinkles is 0:39 or more, while the actual observation time for the part with almost no wrinkles is 0:36, so it can be seen that the actual observation time for the part with almost no wrinkles is slightly shorter. Fig.12In the example shown, OK is displayed for items where the actual observation time is longer than the appropriate observation time, and NG is displayed for items where the actual observation time is shorter than the appropriate observation time. The inspection time evaluation unit 37 may also determine whether the actual observation time of each part is excessive.
[0103] Fig.13 This is a diagram that summarizes the extraction time classification results of Specific Example 1 by classifying them for each part. Figure 4 As shown in the figure, the extraction time of Example 1 is 17:21 and the net observation time is 8:57. Fig.13 As shown, the extraction time for the ascending colon was 8:13 and the net observation time was 2:57, the extraction time for the transverse colon was 0:55 and the net observation time was 0:55, the extraction time for the descending colon was 4:30 and the net observation time was 1:54, the extraction time for the sigmoid colon was 1:48 and the net observation time was 1:21, and the extraction time for the rectum was 1:55 and the net observation time was 1:50.
[0104] Fig.14 It shows that Fig.13 The figure shows the extraction time classification results by body part. The horizontal axis is the classification of body parts, and the details of the extraction time by body part are shown in a bar graph. Figure 5 As shown, the work content of each part can also be classified.
[0105] The image classification unit 35 classifies the endoscopic images corresponding to the net observation step included in the multiple endoscopic images of the time series according to the parts of the inspection object organ (in this embodiment, the intestine) based on the image recognition result of the image recognition unit 34. The inspection time classification unit 36 calculates the actual time of the net observation step by part based on the result of the classification by part.
[0106] The inspection time evaluation unit 37 obtains the actual time by part of the net observation step calculated by the inspection time classification unit 36. The inspection time evaluation unit 37 obtains the evaluation index by part of the net observation step. The inspection time evaluation unit 37 compares the actual time by part of the net observation step and the evaluation index by part, and evaluates the actual time by part of the net observation step based on the comparison results.
[0107] The display control unit 38 displays the actual time for each part of the net observation step and the evaluation index for each part of the net observation step on the display device 41. The display control unit 38 can also display the comparison result of the two on the display device 41 as the evaluation result for each part.
[0108] By generating a graph in which the extraction time classification results are further classified by site, the extraction time and net observation time can be confirmed for each site, and a more detailed analysis and judgment on whether there is observation inequality can be performed. In other words, it is possible to analyze and judge where cleaning / suction was performed for a long time, where lesion treatment was performed for a long time, whether the observation time for each site is sufficient, etc.
[0109] Fig.15 1 is a flowchart showing an example of the operation of the endoscopic inspection support system 30 according to the embodiment. The endoscopic image acquisition unit 32 acquires a plurality of endoscopic images in time series from the endoscopic system 10 (S10). The image classification unit 35 classifies the acquired plurality of endoscopic images into each inspection step (S11). The inspection time classification unit 36 calculates the actual time of the net observation step based on the endoscopic image classified as the net observation step (S12). The inspection time evaluation unit 37 compares the actual time of the net observation step with the appropriate observation time, and evaluates the observation time of the net observation step based on the comparison result (S13).
[0110] As described above, according to the present embodiment, by measuring the net observation time in endoscopic examination, the quality of lesion screening can be appropriately evaluated. In major endoscopy societies, a removal time of more than 6 minutes in cases without treatment is recommended as an indicator of sufficient observation time. However, the removal time in endoscopic examination also includes the treatment time or the cleaning / suction time. Therefore, in the indicator of a removal time of more than 6 minutes, it cannot be guaranteed that sufficient observation time is ensured. Even if the removal time is more than 6 minutes, if the treatment time or the cleaning / suction time is long, sufficient observation time cannot be ensured.
[0111] In contrast, in the present embodiment, the net observation time after removing the treatment time and the cleaning / suction time from the extraction time is measured, so it is possible to appropriately evaluate whether sufficient observation time is ensured. In addition, by measuring the observation time according to the state or part of the intestine and comparing it with the appropriate observation time according to the state or part, the quality of lesion screening can be evaluated more finely.
[0112] The present invention has been described above based on a plurality of embodiments. These embodiments are merely illustrative, and those skilled in the art will appreciate that various modifications may be made to the combination of these components and processing steps, and such modifications are also within the scope of the present invention.
[0113] In the above embodiment, an example is described in which the endoscope insertion shape observation device 20 is connected to the endoscope inspection auxiliary system 30, and the endoscope inspection auxiliary system 30 obtains the endoscope shape from the endoscope insertion shape observation device 20. In this regard, the present invention can also be applied to a system in which the endoscope insertion shape observation device 20 is omitted. In this case, the image classification unit 35 cannot use information related to the endoscope shape, and classifies the endoscope image based on the image recognition result and the operation information. In addition, if an encoder is set near the anus of the subject and the encoder is connected to the endoscope inspection auxiliary system 30, the endoscope inspection auxiliary system 30 can obtain the insertion length from the encoder.
[0114] Possibility of industrial application
[0115] The present invention can be used for colon endoscopy.
[0116] Description of Reference Numerals
[0117] 10···Endoscope system, 11···Endoscope, 11a···Insertion portion, 11b···Front end hard portion, 11c···Bending portion, 11d···Flexible tube portion, 11e···Operation portion, 11f···Main body portion, 11g···Grip portion, 12···Magnetic coil, 15···Light source device, 20···Endoscope insertion shape observation device, 20a···Receiving antenna, 20b···Reference plate, 30 ···Endoscopic inspection assisting system, 31···Endoscopic shape acquisition unit, 32···Endoscopic image acquisition unit, 33···Operation information acquisition unit, 34···Image recognition unit, 35···Image classification unit, 36···Inspection time classification unit, 37··Inspection time evaluation unit, 38···Display control unit, 39···Recording control unit, 41···Display device, 42···Input device, 43··Storage device.
Claims
1. An endoscope inspection auxiliary system, wherein: The endoscope inspection support system includes one or more processors having hardware. The processor performs the following processing: Obtaining evaluation indicators of the time involved in the prescribed inspection steps in the endoscopy; obtaining a record of the performance of the endoscopic examination; and Based on the implementation record, the time taken for the prescribed inspection steps, ie, the actual time, is calculated.
2. The endoscope inspection assisting system according to claim 1, wherein: The processor displays the evaluation index and the actual time on a monitor.
3. The endoscope inspection assisting system according to claim 1, wherein: The processor displays a comparison result between the evaluation index and the actual time on a monitor.
4. The endoscope inspection assisting system according to claim 1, wherein: The processor performs the following processing: recording the actual time in a storage device; and After the inspection, the actual time is read from the storage device, and the read actual time and the evaluation index are displayed on a monitor.
5. The endoscope inspection assisting system according to claim 1, wherein: The predetermined inspection step is a step obtained by removing a treatment step from the overall steps of the endoscopic inspection.
6. The endoscope inspection assisting system according to claim 1, wherein: The predetermined inspection step is an observation step obtained by removing at least a detailed inspection step and a treatment step from the overall steps of the endoscopic inspection.
7. The endoscope inspection assisting system according to claim 6, wherein: The predetermined inspection step is a net observation step obtained by removing the cleaning step and the suction step from the observation step.
8. The endoscope inspection assisting system according to claim 1, wherein: The processor performs the following processing: Acquiring time-series endoscopic images of the endoscopic examination as the implementation record; classifying to which inspection step each endoscopic image included in the time series endoscopic images corresponds; and The actual time of each inspection step is calculated based on the classification results.
9. The endoscope inspection assisting system according to claim 1, wherein: The processor performs the following processing: calculating the actual time of each inspection step constituting the endoscopic inspection; and The actual time of each inspection step is displayed on the monitor.
10. The endoscope inspection assisting system according to claim 1, wherein: The processor performs the following processing: calculating the actual time of each inspection step constituting the endoscopic inspection; and The actual time of each inspection step is classified into each section obtained by dividing the insertion length of the endoscope during the endoscopic inspection into predetermined length intervals, and is displayed on a monitor.
11. The endoscope inspection assisting system according to claim 1, wherein: The processor performs the following processing: calculating the actual time of each inspection step constituting the endoscopic inspection; and The actual time of each inspection step is classified into each section obtained by dividing the elapsed time of the removal of the endoscope in the inspection at predetermined time intervals and displayed on the monitor.
12. The endoscope inspection assisting system according to claim 1, wherein: The processor performs the following processing: obtaining a state-by-state evaluation index, the state-by-state evaluation index defining an evaluation index of time involved in the predetermined inspection step according to a state of an inspection target organ in the endoscopic inspection; Acquiring time-series endoscopic images of the endoscopic examination as the implementation record; classifying the endoscopic images corresponding to the predetermined inspection step included in the time series endoscopic images according to the state of the inspection object organ; and According to the result of classification by status, the actual time by status of the specified inspection step is calculated.
13. The endoscope inspection assisting system according to claim 12, wherein: The processor displays the status-based evaluation index and the status-based actual time on a monitor.
14. The endoscope inspection assisting system according to claim 12, wherein: The processor displays a comparison result between the state-based evaluation index and the state-based actual time on a monitor.
15. The endoscope inspection assisting system according to claim 12, wherein: The processor performs the following processing: Calculating the cumulative length of the organ under examination by state according to the result of classification by state; and The state-by-state evaluation index is set according to the state-by-state cumulative length.
16. The endoscope inspection assisting system according to claim 15, wherein: The processor sets the state-by-state evaluation index based on the state-by-state cumulative length and the type of the state of the examination target organ.
17. The endoscope inspection assisting system according to claim 12, wherein: The state of the organ to be examined includes the state of intestinal folds.
18. The endoscope inspection assisting system according to claim 17, wherein: The state-based evaluation indicator is the appropriate observation time according to the state. When the depth of the wrinkles is greater than or equal to a predetermined depth, the processor sets the proper state observation time per unit length of the intestinal tract to be longer than the proper state observation time per unit length of the intestinal tract in a normal portion.
19. The endoscope inspection assisting system according to claim 12, wherein: The state of the organ to be examined includes whether there is a diverticula in the intestine.
20. The endoscope inspection assisting system according to claim 19, wherein: The state-based evaluation indicator is the appropriate observation time according to the state. The processor sets a state-appropriate observation time per unit length of the portion of the intestine having the diverticula to be longer than a state-appropriate observation time per unit length of the portion of the intestine not having the diverticula.
21. The endoscope inspection assisting system according to claim 1, wherein: The processor performs the following processing: obtaining a site-specific evaluation index that specifies an evaluation index of time involved in the specified inspection step according to a site of an inspection target organ in the endoscopic inspection; Acquiring time-series endoscopic images of the endoscopic examination as the implementation record; classifying the endoscopic images corresponding to the predetermined inspection step included in the time series endoscopic images according to the parts of the inspection object organ; and According to the result of the classification by part, the actual time by part of the prescribed inspection step is calculated.
22. The endoscope inspection assisting system according to claim 1, wherein: The evaluation index is the appropriate observation time.
23. An endoscopy auxiliary method, comprising: Obtaining evaluation indicators of the time involved in the prescribed inspection steps in the endoscopy; Obtaining a record of the implementation of the endoscopic examination; as well as The time taken for the specified inspection steps, ie, the actual time, is calculated based on the implementation record.
24. A storage medium storing a program, wherein: The program causes the computer to execute the following processing: Obtaining evaluation indicators of the time involved in the prescribed inspection steps in the endoscopy; Obtaining a record of the implementation of the endoscopic examination; Based on the implementation record, the time taken for the prescribed inspection steps, ie, the actual time, is calculated.
