Endoscope insertion assist device, method, and non-transitory computer readable medium having a program stored thereon

Abstract

An object of the present application is to provide an endoscope insertion assisting apparatus, method, and program which allow an operator of an endoscope to easily grasp an insertion state of the endoscope, thereby performing effective insertion assistance. An endoscope insertion assisting apparatus (100) according to the present application includes an acquisition unit (110) which acquires shape data to identify an insertion shape of an endoscope inserted into a lumen, an estimation unit (120) which estimates any one of a plurality of shape categories of the insertion shape from the shape data, and an output unit (130) which outputs time-sequenced display information of the estimated shape category.

Description

TECHNICAL FIELD

[0001] The present invention relates to an endoscope insertion assisting apparatus, method, and program, and more particularly, to an endoscope insertion assisting apparatus, method, and program for assisting insertion of an endoscope. BACKGROUND

[0002] Insertion of an endoscope to observe the inside of a body such as a large intestine or a small intestine has become common. In this case, an operator (a doctor) needs to know what shape an insertion portion of the endoscope takes inside the body.

[0003] Therefore, Patent Literature 1 discloses a technology related to an analysis apparatus that analyzes an insertion shape of an endoscope. The analysis apparatus acquires insertion shape data from an endoscope insertion shape observation apparatus, and detects a shape of an insertion portion of the endoscope from the acquired insertion shape data. The analysis apparatus analyzes and obtains a specific position or a specific portion of the detected shape, and classifies a static shape of the insertion portion of the endoscope into a pattern based on an analysis result thereof. Then, the analysis apparatus displays the classified pattern on a screen.

[0004] PRIOR ART DOCUMENTS

[0005] PATENT LITERATURE

[0006] Patent Literature 1: Japanese Patent Application Publication No. 2007-054401 SUMMARY

[0007] PROBLEMS TO BE SOLVED BY THE INVENTION

[0008] Here, actual endoscopic examination needs to be performed by a doctor skilled in operation, and there is a problem that the operation of the endoscope is difficult. That is, it is necessary to provide a mechanism that performs insertion assistance for an operator who operates the endoscope. Note that in the technology related to the above-described Patent Literature 1, there is room for improvement in insertion assistance.

[0009] The present disclosure has been achieved to solve the above-described problems, and an object of the present disclosure is to provide an endoscope insertion assisting apparatus, method, and program that allow an operator of an endoscope to easily grasp an insertion situation of the endoscope and perform effective insertion assistance.

[0010] SOLUTION TO PROBLEM

[0011] The endoscope insertion assisting apparatus according to a first aspect of the present disclosure includes an acquisition unit that acquires shape data to identify an insertion shape of an endoscope inserted into a lumen, an estimation unit that estimates any one of a plurality of shape categories of the insertion shape from the shape data, and an output unit that outputs time-sequenced display information of the estimated shape category.

[0012] The endoscope insertion assistance method according to the second aspect of the present disclosure includes: a computer acquiring shape data to identify an insertion shape of an endoscope inserted into a lumen; estimating, from the shape data, any one of a plurality of shape categories of the insertion shape; and outputting time-sequenced display information of the estimated shape category.

[0013] The endoscope insertion assistance program according to the third aspect of the present disclosure causes a computer to execute: an acquisition process for acquiring shape data to identify an insertion shape of an endoscope inserted into a lumen; an estimation process for estimating, from the shape data, any one of a plurality of shape categories of the insertion shape; and an output process for outputting time-sequenced display information of the estimated shape category.

[0014] Advantages of the Invention

[0015] According to the present disclosure, it is possible to provide an endoscope insertion assistance device, method, and program that allow an operator of an endoscope to easily grasp the insertion situation of the endoscope, thereby performing effective insertion assistance. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a block diagram illustrating a configuration of an endoscope insertion assistance device according to a first example embodiment;

[0017] Figure 2 is a flowchart illustrating a flow of an endoscope insertion assistance method according to the first example embodiment;

[0018] Figure 3 is a block diagram illustrating an overall configuration of an endoscope insertion assistance system according to a second example embodiment;

[0019] Figure 4 is a block diagram illustrating a configuration of an endoscope insertion assistance device according to the second example embodiment;

[0020] Figure 5 is a diagram illustrating an example of shape data according to the second example embodiment;

[0021] Figure 6 is a diagram illustrating an example of an endoscope image according to the second example embodiment;

[0022] Figure 7 is a diagram illustrating an example of time-sequenced conversion display information of a shape category according to the second example embodiment;

[0023] Figure 8 is a flowchart illustrating a flow of an endoscope insertion assistance method according to the second example embodiment;

[0024] Figure 9is a block diagram illustrating a configuration of an endoscope insertion assisting apparatus according to a third example embodiment;

[0025] Figure 10 is a flowchart illustrating a flow of a comparison process according to the third example embodiment;

[0026] Figure 11 is a graph illustrating an example of a comparison result of a time-sequence conversion of a shape category according to the third example embodiment;

[0027] Figure 12 is a block diagram illustrating a configuration of an endoscope insertion assisting apparatus according to a fourth example embodiment; and

[0028] Figure 13 is a flowchart illustrating a flow of a search process according to the fourth example embodiment. DETAILED DESCRIPTION

[0029] Hereinafter, example embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the respective drawings, the same or corresponding elements are given the same reference numerals, and overlapping description will be omitted as necessary in order to clarify the description.

[0030] <First Example Embodiment>

[0031] Figure 1 is a block diagram illustrating a configuration of an endoscope insertion assisting apparatus 100 according to the first example embodiment. The endoscope insertion assisting apparatus 100 is an information processing apparatus for assisting an examiner (doctor) in an operation of inserting an endoscope, which the examiner (doctor) uses for an examination of a subject (object) inside a body. The endoscope insertion assisting apparatus 100 includes an acquisition unit 110, an estimation unit 120, and an output unit 130.

[0032] The acquisition unit 110 acquires shape data for identifying an insertion shape of an endoscope inserted into a body lumen (lumen). Here, the shape data is information for identifying a shape of an optical fiber cable included in the endoscope when the endoscope is inserted into the body lumen, and is, for example, data such as image data for expressing a three-dimensional position coordinate and a shape in two dimensions. The acquisition unit 110 can also receive insertion shape data generated by a shape processing apparatus of an endoscope insertion shape observation apparatus illustrated in, for example, Patent Literature 1, as the shape data.

[0033] The estimation unit 120 estimates any one of a plurality of shape categories of the insertion shape from the shape data. Here, the shape category is a pattern defined by combining a plurality of shapes having similar characteristics of the insertion shape. Two or more shape categories are defined, and the estimation unit 120 performs estimation by classifying the shape data acquired by the acquisition unit 110 into any one of the plurality of shape categories. Note that the estimation unit 120 can also perform estimation using a trained model trained from training data labeled with the shape category for each of the plurality of shape data.

[0034] The output unit 130 outputs the display information visualized in the time-sequenced form for the plurality of estimated shape categories according to the insertion situation of the endoscope. That is, the output unit 130 sequentially generates and outputs the display information visualized in the time-sequenced form for the plurality of estimated shape categories according to the insertion situation of the endoscope.

[0035] Figure 2 is a flowchart illustrating a flow of an endoscope insertion assistance method according to the first example embodiment. First, the acquisition unit 110 acquires shape data for identifying an insertion shape of an endoscope inserted into a body cavity (S11). Next, the estimation unit 120 estimates any one of a plurality of shape categories in the insertion shape from the shape data (S12). Then, the output unit 130 outputs display information visualized in a time-sequenced form for the estimated shape category (S13).

[0036] Thus, in the present example embodiment, the acquired shape data is classified (estimated) into a predetermined shape category indicating characteristics of the insertion shape, the acquired and estimated shape category is sequentially output as display information in a time-sequenced form, and presented to an operator of the endoscope. Thus, the operator of the endoscope can easily grasp the insertion situation of the endoscope. Therefore, according to the present example embodiment, effective insertion assistance can be performed on the endoscope.

[0037] Note that the endoscope insertion assistance apparatus 100 includes a processor, a memory, and a storage device as not-shown components. The storage device stores a computer program in which processes of the endoscope insertion assistance method according to the present example embodiment are implemented. The processor reads the computer program from the storage device into the memory and executes the computer program. In this way, the processor implements the functions of the acquisition unit 110, the estimation unit 120, and the output unit 130.

[0038] Alternatively, the acquisition unit 110, the estimation unit 120, and the output unit 130 can also be implemented by a dedicated hardware. Some components or all components of each device can be implemented by a general-purpose or a dedicated circuit device, a processor, or a combination thereof. The components can be constituted by a single chip or a plurality of chips connected via a bus. Some components or all components of each device can also be implemented by a combination of the above-described circuit device and a program. A CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field-Programmable Gate Array), or the like can be used as the processor.

[0039] When some components or all components of the endoscope insertion assisting device 100 are implemented by a plurality of information processing devices or circuits, the plurality of information processing devices and circuits can be arranged centrally or can be distributed. For example, the information processing devices or circuits and the like can also be implemented in a mode connected via a communication network such as a client-server system and a cloud computing system. Furthermore, the functions of the endoscope insertion assisting device 100 can be provided in a SaaS (Software as a Service) format.

[0040] <Second Example Embodiment>

[0041] The second example embodiment is a specific example of the above-described first example embodiment. Figure 3 is a block diagram illustrating the overall configuration of an endoscope insertion assisting system 2000 according to the second example embodiment. The endoscope insertion assisting system 2000 is a system that assists a doctor U1 in inserting an electronic endoscope 11 when the doctor U1 performs an examination on a subject U2 using the electronic endoscope 11. The endoscope insertion assisting system 2000 includes an endoscope device 10, an endoscope insertion shape observation device 20, an endoscope insertion assisting device 30, a display device 41, a speaker 42, and an input device 43.

[0042] The endoscope device 10 includes the electronic endoscope 11 connected to the main body of the endoscope device 10 via a cable. The electronic endoscope 11 includes an insertion portion 11a that is a portion to be inserted into a body lumen of the subject U2. The insertion portion 11a includes a distal end portion and an optical fiber cable and a light guide cable connected to the distal end portion. One end of the insertion portion 11a is connected to the proximal end portion and the main body of the endoscope device 10 via the above-described cable. The distal end portion includes an electronic image pickup device, an observation light irradiation portion, and a bending portion, and the like. The electronic image pickup device corresponds to a camera of the electronic endoscope 11 and is, for example, a CCD (Charge-Coupled Device). The irradiation portion emits observation light from the light guide cable. The bending portion bends the distal end portion in response to a control signal from an operation portion of the proximal end portion.

[0043] An optical fiber cable and a light guide cable are also connected to the main body of the endoscope device 10. The optical fiber cable transmits / receives various signals to / from the main body of the endoscope device 10, and particularly transmits an image (endoscope image) captured by an electronic image pickup device to the main body of the endoscope device 10. The main body of the endoscope device 10 outputs the endoscope image to the endoscope insertion assisting device 30. The light guide cable guides light from a light source of the endoscope device 10 to the irradiation portion.

[0044] Here, the insertion portion 11a includes source coils (not shown) at a plurality of positions. The source coils are arranged, for example, at predetermined intervals. The source coils generate a magnetic field in response to a drive signal from the endoscope insertion shape observation device 20. Thus, the electronic endoscope 11 only needs to include magnetic coils to identify the shape at the plurality of positions of the insertion portion 11a. Note that since disclosed known components can be used for other components of the endoscope device 10 and the electronic endoscope 11, the illustration and description of the other components will be omitted. Alternatively, an instrument of a built-in coil inserted into the endoscope can also be used as the magnetic coil (not a coil included in the endoscope).

[0045] The endoscope insertion shape observation device 20 includes at least an induction coil unit 21 and a shape processing device 22. The induction coil unit 21 is a unit that detects a magnetic field generated from a plurality of source coils provided in the insertion portion 11a of the electronic endoscope 11.

[0046] The shape processing device 22 outputs a drive signal to the electronic endoscope 11. When the subject U2 wears a magnetic coil, the shape processing device 22 should also output a drive signal to the magnetic coil. In this case, the induction coil unit 21 further detects a magnetic field generated from the magnetic coil attached to the subject U2.

[0047] The shape processing device 22 obtains a three-dimensional shape of the insertion portion 11a based on the magnetic field detected by the induction coil unit 21. For example, the shape processing device 22 calculates a three-dimensional coordinate of each source coil based on the detected magnetic field, and uses a set of the three-dimensional coordinates as shape data. Alternatively, the shape processing device 22 generates image data (resulting from projecting the calculated three-dimensional coordinates to two-dimensional coordinates), and uses the image data as the shape data.

[0048] Further, when the subject U2 wears a magnetic coil, the shape processing device 22 can obtain a posture of the subject U2 based on the detected magnetic field. More specifically, the shape processing device 22 calculates a three-dimensional coordinate indicating a position of the magnetic coil worn by the subject U2 with respect to the source coils, and uses the three-dimensional coordinate as posture data.

[0049] The shape processing apparatus 22 outputs the shape data and the posture data to the endoscope insertion assistance apparatus 30. Note that since a publicly known component can be used as other components of the endoscope insertion shape observation apparatus 20, the illustration and the description of the other components will be omitted. For example, the components described in the aforementioned Patent Literature 1 can be used for the endoscope apparatus 10 and the endoscope insertion shape observation apparatus 20.

[0050] The endoscope insertion assistance apparatus 30 is connected to the endoscope apparatus 10, the endoscope insertion shape observation apparatus 20, the display apparatus 41, the speaker 42, and the input apparatus 43. The endoscope insertion assistance apparatus 30 is one example of the aforementioned endoscope insertion assistance apparatus 100. The endoscope insertion assistance apparatus 30 acquires the endoscope image from the endoscope apparatus 10, and acquires the shape data and the posture data from the endoscope insertion shape observation apparatus 20.

[0051] The endoscope insertion assistance apparatus 30 estimates a shape category of the insertion shape of the insertion portion 11a of the electronic endoscope 11 based on the acquired shape data, the endoscope image, and the posture data, and records the estimated shape category with time information. The endoscope insertion assistance apparatus 30 generates display information to display the recorded shape categories in a time-sequenced format, and outputs the display information to the display apparatus 41. The endoscope insertion assistance apparatus 30 outputs a voice corresponding to the estimated shape category to the speaker 42.

[0052] The display apparatus 41 displays the display information received from the endoscope insertion assistance apparatus 30 on a screen. The speaker 42 outputs the voice received from the endoscope insertion assistance apparatus 30. The input apparatus 43 receives an input operation from an examiner (operator) such as the doctor U1 or an examination assistant, and outputs a control signal corresponding to the input operation to the endoscope insertion assistance apparatus 30. The input apparatus 43 is, for example, a mouse or a keyboard. Note that when the display apparatus 41 is a touch panel, the display apparatus 41 and the input apparatus 43 are integral. Note that some or all of the display apparatus 41, the speaker 42, and the input apparatus 43 can be incorporated in the endoscope insertion assistance apparatus 30.

[0053] Figure 4 is a block diagram illustrating a configuration of the endoscope insertion assistance apparatus 30 according to the second example embodiment. The endoscope insertion assistance apparatus 30 includes a storage apparatus 31, a memory 32, an IF (interface) unit 33, and a control unit 34. The storage apparatus 31 is a storage apparatus such as a hard disk or a flash memory. The storage apparatus 31 stores a shape category estimation model 311, history information 312, and an endoscope insertion assistance program 313.

[0054] The shape category estimation model 311 is a program module or model expression in which the logic of estimating the shape category from the shape data (or the shape data as normalized data) is implemented. The shape category estimation model 311 is a model that receives a set of three-dimensional coordinates or image data of shape data as input, estimates a shape category to which the shape of the insertion portion 11a represented by the shape data is more likely to correspond, and outputs the shape category as an estimation result. Note that the shape category estimation model 311 can be said to be a trained model trained in advance from training data labeled with "shape category" for each of a plurality of shape data.

[0055] Here, examples of the shape category include "straight line", "small curve", "large curve", "abdominal bulge", but the shape category is not limited to these examples. The shape category does not indicate the shape of the insertion portion 11a alone, but includes the condition of the body cavity (the shape of the organ). For example, the shape category can be considered as different shape categories depending on whether the curved portion of the distal end portion of the insertion portion 11a is caught in the fold of the lumen or the curved portion fails to be caught in the fold of the lumen. That is, the shape category can represent the shape of the insertion portion 11a, and can be classified in consideration of the shape of the organ to be examined. In this case, the shape category estimation model 311 further receives the operation content of the electronic endoscope 11 as input, which will be described later.

[0056] The history information 312 is information in which the shape category 3121, the time information 3122, and the additional information 3123 are associated with each other. Note that the additional information 3123 is not essential to the history information 312. The shape category 3121 is information indicating each of the above-described categories, for example, identification information or a character string. The time information 3122 is the time at which the shape data is acquired, the time at which the shape category is estimated, and the like. The additional information 3123 is additional information on the estimated shape category. For example, the additional information 3123 is a region in the body cavity, a voice uttered by the doctor U1, and the like. With the history information 312, it is possible to efficiently generate the time-sequenced transition information between the shape categories (which will be described later).

[0057] The endoscope insertion assistance program 313 is a computer program in which the procedure of the endoscope insertion assistance method according to the present example embodiment is implemented.

[0058] The memory 32 is a volatile storage device such as a RAM (Random Access Memory), and is a storage area for temporarily retaining information during the operation of the control unit 34. The IF unit 33 is an interface that provides input / output to / from external devices for the endoscope insertion assisting apparatus 30. For example, the IF unit 33 receives the user's operation via the input device 43, and outputs the received operation content to the control unit 34. The IF unit 33 receives the endoscope image or the like from the endoscope apparatus 10, receives the shape data and the posture data from the endoscope insertion shape observation apparatus 20, stores the received shape data and posture data such as the endoscope image in the memory 32, and notifies the control unit 34 of such data. The IF unit 33 provides output for the display device 41 or the speaker 42 in response to the instruction from the control unit 34.

[0059] The control unit 34 is a processor, that is, a control device that controls each component of the endoscope insertion assisting apparatus 30. The control unit 34 reads the endoscope insertion assistance program 313 from the storage device 31 into the memory 32, and executes the endoscope insertion assistance program 313. Thus, the control unit 34 realizes the functions of the acquisition unit 341, the estimation unit 342, the recording unit 343, the output unit 344, and the registration unit 345.

[0060] The acquisition unit 341 is one example of the above-described acquisition unit 110. The acquisition unit 341 acquires the endoscope image from the endoscope apparatus 10 via the IF unit 33, and acquires the shape data and the posture data from the endoscope insertion shape observation apparatus 20 via the IF unit 33. Figure 5 is a diagram illustrating one example of the shape data 314 (image data) according to the second example embodiment. Figure 6 is a diagram illustrating one example of the endoscope image 315 according to the second example embodiment.

[0061] The estimation unit 342 is one example of the above-described estimation unit 120. The estimation unit 342 includes the in-vivo insertion detection unit 3421, the operation estimation unit 3422, the normalization unit 3423, and the shape category estimation unit 3424. The in-vivo insertion detection unit 3421 identifies the endoscope image acquired by the acquisition unit 341, and determines whether the electronic endoscope 11 is inserted into the body (for example, the mouth, the nose, or the anus) of the subject U2. That is, using the endoscope image, the in-vivo insertion detection unit 3421 detects that the electronic endoscope 11 is inserted into the body. Here, the in-vivo insertion detection unit 3421 can determine the mouth, the nose, or the anus by identifying the endoscope image. The in-vivo insertion detection unit 3421 notifies the normalization unit 3423, the shape category estimation unit 3424, and the recording unit 343 of this detection.

[0062] The operation estimation unit 3422 estimates the operation content of the electronic endoscope 11 in the body cavity based on the change in the endoscope image and the shape data, and notifies the shape category estimation unit 3424 of the estimated operation content. Here, the operation content of the electronic endoscope 11 includes a state in which the bent portion is bent and pinched in a fold of the lumen, and the like. For example, generally, if the electronic endoscope 11 moves back and forth within the body cavity, the endoscope image also changes. In this case, the shape data can change, but the shape category can not change. However, although the shape data is similar between a state in which the bent portion is pinched in a fold of the lumen and a state in which the bent portion fails to be pinched in a fold of the lumen, it can be said that there is a clear difference in the state of the body cavity (the shape of the organ). Therefore, in this case, the shape category is divided. When the bent portion is pinched in a fold of the lumen, the position of the distal end portion of the insertion portion 11a changes, but the surrounding environment of the camera at the distal end portion does not change. That is, it can be said that the endoscope image does not change much. On the other hand, when the bent portion is not pinched on the fold, the position of the distal end portion changes, and the distance between the camera and the membrane increases, and thus it can be said that the endoscope image also changes. Therefore, the operation estimation unit 3422 preferably estimates the operation content of the electronic endoscope 11 and the shape of the organ while taking into account the change in the endoscope image and the change in the shape data. Therefore, the estimation accuracy is improved, and the insertion assistance can be performed more effectively.

[0063] In response to receiving the detection notification from the in-vivo insertion detection unit 3421, the normalization unit 3423 identifies that the latest shape data of the current time acquired by the acquisition unit 341 is the examination start point. More specifically, the normalization unit 3423 identifies the three-dimensional coordinates of the distal end portion of the three-dimensional coordinate set of the latest shape data of the current time as the examination start point (origin). Thereafter, the normalization unit 3423 converts the three-dimensional coordinates of the shape data acquired by the acquisition unit 341 to a coordinate system with the examination start point as the origin and performs normalization. The normalized shape data is output by the normalization unit 3423 to the shape category estimation unit 3424. The accuracy of the normalization is thus improved.

[0064] The normalization unit 3423 can normalize the shape data based on the posture data (three-dimensional coordinates) acquired by the acquisition unit 341, and output the normalized shape data to the shape category estimation unit 3424. This further improves the accuracy of the normalization.

[0065] In response to receiving the detection notification from the in-vivo insertion detection unit 3421, the shape category estimation unit 3424 starts a process of estimation of the shape category. More specifically, the shape category estimation unit 3424 inputs the normalized shape data received from the normalization unit 3423 to the shape category estimation model 311, and acquires the shape category (estimation result) as an output. That is, the shape category estimation unit 3424 estimates any one of a plurality of shape categories from the shape data acquired by the acquisition unit 341 using the trained model described above. In this way, the accuracy of estimation can be improved by continuing training using accumulated training data. The shape category estimation unit 3424 outputs the received shape category to the recording unit 343. Further, the shape category estimation unit 3424 can input the operation content received from the operation estimation unit 3422 to the shape category estimation model 311 together with the shape data, and acquire the shape category (estimation result) as an output. In this way, the estimation accuracy can be further improved due to an increase in the number of types of input data.

[0066] Note that the estimation unit 342 can not include the normalization unit 3423. In this case, the shape category estimation unit 3424 inputs the shape data acquired by the acquisition unit 341 to the shape category estimation model 311 as it is.

[0067] The recording unit 343 identifies the examination start time point based on the endoscope image, and records the time-sequenced conversion information of the estimated shape category in the storage 31 as the history information 312 based on the examination start time point. More specifically, the recording unit 343 identifies the current time as the examination start time point in response to receiving the detection notification from the in-vivo insertion detection unit 3421. Each time the shape category 3121 is received from the estimation unit 342 after the examination start time point, the recording unit 343 associates the current time as the time information 3122, and stores the current time in the storage 31 as the history information 312. That is, the recording unit 343 starts a process of recording of the shape category in response to receiving the detection notification from the in-vivo insertion detection unit 3421. Since the time information is associated with each shape category, the history information 312 can be said to be the time-sequenced conversion information of the shape category.

[0068] The output unit 344 is one example of the above-described output unit 130. The output unit 344 reads one record of the history information 312 from the storage device 31 at a time, and generates display information indicating a time-sequenced transition among a plurality of estimated shape categories. The output unit 344 outputs the generated display information to the display device 41 via the IF unit 33. Here, it can be assumed that the display information is represented, for example, on a two-dimensional graph in which one axis represents time and the other axis represents a shape category. That is, the output unit 344 plots at a point on the graph corresponding to the read shape category 3121 and the associated time information 3122. The output unit 344 plots so as to connect between adjacent time information in the time information. Thus, a time-sequenced transition among shape categories is realized, which allows an examiner or the like to easily grasp the transition of shape categories. Note that the display information is not limited to a two-dimensional graph, and the display information can be a three-dimensional graph or other information in which the transition among shape categories is time-sequenced.

[0069] Further, the shape category can be subdivided into a plurality of levels. For example, assume that a plurality of shape subcategories belong to a particular shape category. In this case, the shape category estimation model 311 estimates the shape category down to the shape subcategory in addition to estimating the shape category. The recording unit 343 stores the estimated shape category 3121 and the shape subcategory for the same shape data associated with the history information 312 in the storage device 31. Thereafter, the output unit 344 plots at positions on the graph corresponding to the shape subcategory in the read record of the history information 312 and the associated time information 3122. The output unit 344 plots so as to connect the transition among different shape subcategories (even within the same shape category) with a line. This makes it possible to easily grasp a more accurate transition.

[0070] Figure 7 is a graph illustrating one example of display information 316 of a time-sequenced transition of shape categories according to a second example embodiment. Here, one example of a two-dimensional graph is shown in which a horizontal axis represents a shape category and a vertical axis represents time information. Figure 7 The shape category C1 "straight line", the shape category C2 "small curve", the shape category C3 "large curve", and the shape category C4 "abdominal bulge" are illustrated. Figure 7 Further illustrated are that three shape subcategories C11 to C13 are classified within the shape category C1, two shape subcategories C21 and C22 are classified within the shape category C2, three shape subcategories C31 to C33 are classified within the shape category C3, and one shape subcategory C41 is further classified within the shape category C4. In Figure 7In this case, the transition (change in shape category) between the shape category C1 and the shape category C2 can be visually recognized. Further, even within the shape category C1, the transition between the shape subcategory C11 and C12 can be visually recognized. Note that the numerical value on the horizontal axis can be a shape subcategory number.

[0071] Returning to Figure 4 To further explain, the output unit 344 can output a voice corresponding to the estimated shape category. In this case, it is assumed that the storage 31 pre-stores voice data corresponding to each shape category. When the history information 312 is read from the storage 31, the output unit 344 reads the voice data corresponding to the shape category 3121 together. The output unit 344 outputs the read voice data to the speaker 42 via the IF unit 33. This allows the examiner to grasp the shape category of the current electronic endoscope 11 without looking at the display 41, and thus the insertion assistance can be performed more efficiently.

[0072] The registration unit 345 receives an input of additional information regarding the estimated shape category via the input 43 and the IF unit 33, further associates the additional information 3123 with the shape category 3121, and registers the additional information 3123 in the storage 31 as the history information 312. Thus, the registration unit 345 can update the registered history information 312. When the output unit 344 reads the history information 312, the output unit 344 reads the additional information 3123 together with the shape category 3121, and can display the additional information 3123 included in the display information.

[0073] Note that the estimation unit 342 can identify the position in the body from the estimated shape category, and the output unit 344 can further output the identified position. For example, when the examination target is the large intestine, examples of the internal position include the sigmoid colon, descending colon, transverse colon, and ascending colon. In this case, the training data labeled with the internal position is created for the combination of the transitions between the shape categories, and the internal position estimation model is trained using the training data. This allows the estimation unit 342 to input the combination of the transitions between the shape categories of the history information 312 to the internal position estimation model at any time, and obtain the estimated internal position as the output. Thus, the internal position estimation model can be used to verify the examination result after the examination. Alternatively, the estimation unit 342 can obtain the estimated value of the internal position by inputting the estimation result (combination of the transitions between the shape categories) from the examination start time point to each time point before the examination period to the internal position estimation model. Thus, the output unit 344 outputs the estimated internal position, and thereby the examiner can grasp the internal position where the distal portion of the electronic endoscope 11 is currently located in real time, and the insertion assistance can be performed more efficiently.

[0074] Figure 8 This is a flowchart illustrating an endoscopic insertion assistance method according to a second exemplary embodiment. First, the acquisition unit 341 acquires shape data from the endoscopic insertion shape observation device 20 (S201). The acquisition unit 341 outputs the acquired shape data to the normalization unit 3423 and the operation estimation unit 3422. The acquisition unit 341 acquires an endoscopic image from the endoscopic device 10 (S202). The acquisition unit 341 outputs the acquired endoscopic image to the in vivo insertion detection unit 3421 and the operation estimation unit 3422. The acquisition unit 341 acquires posture data from the endoscopic insertion shape observation device 20 (S203). The acquisition unit 341 outputs the acquired posture data to the normalization unit 3423.

[0075] After steps S201 and S202, the operation estimation unit 3422 estimates the operation content of the electronic endoscope 11 based on the changes in shape data and endoscopic images received from the acquisition unit 341 (S204). The operation estimation unit 3422 then notifies the shape category estimation unit 3424 of the estimated operation content.

[0076] After step S202, the in vivo insertion detection unit 3421 detects that the electronic endoscope 11 has been inserted into the subject U2 based on the endoscopic image received from the acquisition unit 341 (S205). When the insertion of the electronic endoscope 11 is detected, the in vivo insertion detection unit 3421 notifies the normalization unit 3423, the shape category estimation unit 3424, and the recording unit 343 of the detection.

[0077] Following steps S201, S203, and S205, the normalization unit 3423, in response to a detection notification from the in vivo insertion detection unit 3421, identifies the latest shape data at the current time as the inspection start point. Thereafter, the normalization unit 3423 performs normalization on the received shape data based on the pose data acquired by the acquisition unit 341 and the inspection start point (S206). The normalization unit 3423 outputs the normalized shape data to the shape category estimation unit 3424.

[0078] After steps S204, S205, and S206, in response to a notification of detection of in-vivo insertion from the in-vivo insertion detection unit 3421, the shape category estimation unit 3424 starts estimation of the shape category. More specifically, the shape category estimation unit 3424 estimates the shape category from the normalized shape data received from the shape category estimation unit 3424 and the operation content of the electronic endoscope 11 received from the operation estimation unit 3422 (S207). That is, the shape category estimation unit 3424 inputs the normalized shape data and the operation content to the shape category estimation model 311, and acquires the shape category as an estimation result. Then, the shape category estimation unit 3424 outputs the acquired shape category to the recording unit 343.

[0079] After steps S205 and S207, in response to a notification of detection of in-vivo insertion from the in-vivo insertion detection unit 3421, the recording unit 343 identifies the current time as an examination start time point, and starts recording the shape category on the history information. More specifically, the recording unit 343 associates the current time with the shape category 3121 received from the shape category estimation unit 3424 as time information 3122, and records (stores) the current time in the storage device 31 as the history information 312 (S208).

[0080] After step S208, the output unit 344 reads one record of the history information 312 from the storage device 31 at a time, and generates display information indicating a time-sequenced transition between a plurality of estimated shape categories (S209). Then, the output unit 344 outputs the generated display information to the screen of the display device 41 (S210). After step S208, the output unit 344 outputs a voice corresponding to the estimated shape category to the speaker 42 (S211).

[0081] After steps S210 and S211, the process returns to steps S201, S202, and S203 and repeats the subsequent steps. Note that the endoscope insertion assistance method can be completed at a predetermined time. For example, when the in-vivo insertion detection unit 3421 detects that the electronic endoscope 11 has been removed from the in-vivo, the process can be completed.

[0082] In this way, the estimation unit 342 according to the present example embodiment further uses the endoscope image to estimate any one of the plurality of shape categories, and thus the estimation accuracy is improved compared to a case where the shape category is estimated using only the shape data. The estimation using the operation content of the electronic endoscope 11 and the posture data of the subject U2 can further improve the estimation accuracy. By recording the estimated shape category as the history information 312, the estimated shape category can not only be effectively used during the examination, but also be used for post-examination analysis and the like. By displaying the temporal sequential conversion between the shape categories on the screen, the physician can easily grasp the state of the electronic endoscope 11 during the examination or the condition of the body cavity. Thus, the insertion assistance of the endoscope can be more effectively achieved.

[0083] <Third Example Embodiment>

[0084] The third example embodiment is a modified example of the above-described second example embodiment. In the endoscope insertion assistance system according to the third example embodiment, the endoscope insertion assistance device 30 is replaced by an endoscope insertion assistance device 30a compared to the above-described endoscope insertion assistance system 2000. Thus, the description thereof will be omitted, and the following description will focus on the changed parts.

[0085] Figure 9 is a block diagram illustrating a configuration of the endoscope insertion assistance device 30a according to the third example embodiment. Compared to the above-described endoscope insertion assistance device 30, the endoscope insertion assistance program 313 is replaced by an endoscope insertion assistance program 313a, and a comparison unit 346 is added in the endoscope insertion assistance device 30a. Note that, for the control unit 34, the endoscope insertion assistance device 30a only needs to include at least the acquisition unit 341, the shape category estimation unit 3424, the recording unit 343, the output unit 344, and the comparison unit 346, and can not include other components. The endoscope insertion assistance program 313a is a computer program in which a comparison process in the endoscope insertion assistance method according to the present example embodiment is implemented.

[0086] The comparison unit 346 outputs a comparison result of two or more of the history information 312. This allows the physician to easily compare past endoscope insertion histories, and more effectively perform analysis. By visually recognizing the comparison result, the technique of the endoscope insertion operation can be more effectively improved. Note that it is assumed that the output of the comparison unit 346 is output using the output unit 344. That is, the comparison unit 346 performs a comparison process on two or more of the history information 312, outputs the comparison result to the output unit 344, and the output unit 344 instructs output of the comparison result to the display device 41 or the speaker 42.

[0087] Further, the comparison unit 346 can evaluate the comparison destination in the history information 312 with respect to the comparison source, and output the evaluation result as a comparison result. For example, by using the history information of a skilled doctor as the comparison source, and using the history information of a relatively inexperienced doctor as the comparison destination, an objective evaluation of the endoscope insertion operation can be obtained. This allows the doctor to objectively grasp the problems in his or her endoscope insertion operation, and to improve the technique of the endoscope insertion operation in a short period of time.

[0088] When the comparison result shows that the duration of a specific shape category is a predetermined time or more, the comparison unit 346 can output a warning. This allows the doctor to easily grasp the potential problem location in the endoscope insertion operation at the comparison destination.

[0089] Figure 10 is a flowchart illustrating a flow of a comparison process according to the third example embodiment. First, the comparison unit 346 receives the specification of the conversion information of the shape categories of the comparison source via the input device 43 and the IF unit 33 (S31). For example, the input device 43 receives an input of the date and time (examination time zone) corresponding to the past endoscopic examination of a skilled doctor in accordance with the operation of the doctor. The input device 43 transmits the received date and time information to the endoscope insertion assistance device 30a. The comparison unit 346 of the endoscope insertion assistance device 30a acquires a set of shape categories 3121 associated with the time information 3122 (included in the time zone indicated by the date and time information received from the storage device 31), and stores the set of shape categories 3121 in the storage 32 as the comparison source.

[0090] Next, the comparison unit 346 receives the specification of the conversion information of the shape categories at the comparison destination via the input device 43 and the IF unit 33 (S32). For example, the input device 43 receives an input of the date and time (examination time zone) corresponding to the past endoscopic examination of a relatively inexperienced doctor in accordance with the operation of the doctor. Below, a process similar to that in step S31 will be performed, and the comparison unit 346 of the endoscope insertion assistance device 30a retains the history information acquired from the storage device 31 in the storage 32 as the comparison destination.

[0091] Next, the comparison unit 346 generates a comparison result between the comparison source and the comparison destination (S33). For example, the comparison unit 346 compares the shape categories corresponding to the relative elapsed times from the check start time point between the comparison source and the comparison destination, and calculates the presence or absence of a difference in time order. Alternatively, the comparison unit 346 generates a comparison result between the conversion information of the comparison source and the conversion information of the comparison destination so as to be plotted on a two-dimensional graph by aligning their check start time points. Furthermore, the comparison unit 346 can evaluate the comparison destination with respect to the comparison source. For example, the comparison unit 346 can judge the superiority of the comparison destination with respect to the comparison source, and use the judgment result as an evaluation result. For example, when the duration of a certain shape category at the comparison source is longer than a predetermined time, the comparison unit 346 can judge the evaluation in the time region as low. Alternatively, when the duration of a certain shape category at the comparison source or the comparison destination is longer than a predetermined time, the comparison unit 346 can output a warning in the time region with or without evaluation.

[0092] After that, the comparison unit 346 outputs the comparison result to the screen of the display device 41 via the IF unit 33 (S34). Figure 11 is a diagram illustrating an example of a comparison result 316a of the time-ordered conversion of the shape categories according to the third example embodiment. Here, Figure 11 illustrates an example in which the check start time points are aligned so that the conversion information of the comparison source and the conversion information of the comparison destination are overlapped.

[0093] Therefore, according to the present example embodiment, the history information 312 accumulated in the second example embodiment can be effectively used, and further improvement of the endoscope insertion technique is promoted.

[0094] Note that if the doctor ID is associated with the history information 312, the input device 43 can receive the doctor ID, and the comparison unit 346 can search the history information 312 by the doctor ID, and acquire the comparison source and the comparison destination.

[0095] <Fourth Example Embodiment>.

[0096] The fourth example embodiment is an improved example of the above-described second example embodiment. In the endoscope insertion assisting system according to the fourth example embodiment, the endoscope insertion assisting device 30 is replaced by the endoscope insertion assisting device 30b, compared to the above-described endoscope insertion assisting system 2000. Therefore, the description thereof will be omitted, and the following description will focus on the changed parts.

[0097] Figure 12is a block diagram illustrating a configuration of an endoscope insertion assisting apparatus 30b according to a fourth example embodiment. Compared with the above-described endoscope insertion assisting apparatus 30a, the history information 312 is replaced by history information 312a, the endoscope insertion assisting program 313a is replaced by an endoscope insertion assisting program 313b, and a search unit 347 is added in the endoscope insertion assisting apparatus 30b. Further, medical information 3124 is added to the history information 312a. Note that, for the control unit 34, the endoscope insertion assisting apparatus 30b only needs to include at least the acquisition unit 341, the shape category estimation unit 3424, the recording unit 343, the output unit 344, and the search unit 347, and can not include other components. The endoscope insertion assisting program 313b is a computer program in which a search process is implemented in the endoscope insertion assisting method according to the present example embodiment.

[0098] The search unit 347 searches the history information 312 based on the conversion information indicating the time-sequenced conversion of the estimated shape category, and outputs a search result. Thereby, it is possible to browse past history information similar to the specific conversion information. Thus, in addition to the conversion information, it is possible to confirm a difference in additional information or similar information, and to facilitate the examination. Note that it is also possible to assume that the output of the search unit 347 is output using the output unit 344. That is, the search unit 347 searches the history information 312 using the conversion information as a search condition, and outputs the search result to the output unit 344, and the output unit 344 instructs output of the search result to the display device 41 or the speaker 42.

[0099] In the history information 312a, the medical information 3124 is further associated with the shape category 3121, the time information 3122, and the additional information 3123. Here, the medical information 3124 is information indicating body information, a clinical history, or a consultation history, or the like of the examinee. Thus, the search unit 347 can search the history information 312 based on the medical information 3124, and output a search result. In this way, before the examination is implemented, it is possible to acquire and confirm past history information of an examinee similar to the medical information of this examinee. Thus, it is possible to assist in inserting the endoscope into a patient having similar body information or a clinical history with a more appropriate operation.

[0100] Figure 13is a flowchart illustrating a search process according to the fourth example embodiment. First, the search unit 347 receives a search condition of the conversion information (S41). For example, the input device 43 receives an input of the date and time (examination time region) corresponding to a past endoscopy or medical information in accordance with an operation of a doctor. The input device 43 sends the received date and time information or medical information to the endoscope insertion assistance device 30b. The search unit 347 of the endoscope insertion assistance device 30b retains the received date and time information or medical information in the storage 32 as a search condition.

[0101] Next, the search unit 347 searches the history information 312 based on the search condition (S42). For example, when the search condition is the date and time information, the search unit 347 acquires a set of the shape categories 3121 associated with the time information 3122 (included in the time region indicated by the date and time information) from the storage 31. For example, when the search condition is the medical information, the search unit 347 acquires a set of the shape categories 3121 associated with the medical information 3124 from the storage 31.

[0102] The search unit 347 generates display information based on the search result (S43). For example, with respect to the conversion information as the search result, the search unit 347 generates the conversion information as the display information as in the case of the above-described step S209.

[0103] Thereafter, the search unit 347 outputs the generated display information to the screen of the display device 41 via the IF unit 33 (S44).

[0104] Accordingly, the present example embodiment can further contribute to improvement of the endoscope insertion technique by effectively using the accumulated history information 312 of the second example embodiment.

[0105] <Other Example Embodiments>

[0106] Note that the above-described example embodiments are applicable to an examination of a body lumen such as a large intestine, a small intestine, a stomach, or a bronchus (lung) by an endoscope.

[0107] Note that although the present disclosure is described as a hardware configuration in the above-described example embodiments, the present disclosure is not limited to this. According to the present disclosure, any process can be implemented by causing a CPU to execute a computer program.

[0108] In the above-described example, the program can be stored in the computer and supplied to the computer using various types of non-transitory computer-readable media. The non-transitory computer-readable media include various types of tangible recording media. Examples of the non-transitory computer-readable media include a magnetic recording medium (e.g., a floppy disk, a magnetic tape, a hard disk), a magneto-optical recording medium (e.g., a magneto-optical disk), a CD-ROM (Read Only Memory), a CD-R, a CD-R / W, a DVD (Digital Versatile Disc), and a semiconductor memory (e.g., a mask ROM, a PROM (Programmable ROM), an EPROM (Erasable PROM), a flash memory, a RAM (Random Access Memory)). The program can be supplied to the computer using various types of transitory computer-readable media. Examples of the transitory computer-readable media include an electrical signal, an optical signal, and an electromagnetic wave. The transitory computer-readable media can supply the program to the computer through a wired communication channel or a wireless communication channel such as electric wires and optical fibers.

[0109] Note that the present disclosure is not limited to the above-described example embodiments, but can be appropriately changed without departing from the spirit of the present disclosure. The present disclosure can be implemented by appropriately combining the respective example embodiments.

[0110] Although some or all of the above-described example embodiments can also be described as the following supplementary notes, the present disclosure is not limited to the following supplementary notes.

[0111] (Supplementary Note A1)

[0112] An endoscope insertion assisting apparatus comprising:

[0113] acquiring means for acquiring shape data to identify an insertion shape of an endoscope inserted into a lumen;

[0114] estimating means for estimating, from the shape data, any one of a plurality of shape categories in the insertion shape; and

[0115] outputting means for outputting display information in the estimated shape category sequenced in time.

[0116] (Supplementary Note A2)

[0117] The endoscope insertion assisting apparatus according to Clause 1, wherein the outputting means outputs display information indicating a transition between a plurality of the estimated shape categories sequenced in time.

[0118] (Supplementary Note A3)

[0119] The endoscope insertion assisting apparatus according to Clause 1 or 2, wherein the outputting means further outputs a voice corresponding to the estimated shape category.

[0120] (Supplementary Note A4)

[0121] The endoscope insertion assisting apparatus according to any one of clauses 1 to 3, wherein

[0122] The acquisition component further acquires an endoscope image captured by the endoscope, and

[0123] The estimation component further estimates any one of a plurality of shape categories using the endoscope image.

[0124] (Supplementary note A5)

[0125] The endoscope insertion assisting apparatus according to clause 4, wherein the estimation component

[0126] estimates a content of operation of the endoscope in the lumen based on the change in the endoscope image and the shape data, and

[0127] estimates any one of a plurality of shape categories with further consideration of the content of operation of the endoscope.

[0128] (Supplementary note A6)

[0129] The endoscope insertion assisting apparatus according to clause 4 or 5, wherein the estimation component

[0130] identifies a check start point based on the endoscope image,

[0131] normalizes the shape data based on the check start point, and

[0132] estimates any one of a plurality of shape categories for the normalized shape data.

[0133] (Supplementary note A7)

[0134] The endoscope insertion assisting apparatus according to any one of clauses 4 to 6, further comprising a recording component for identifying a check start time point based on the endoscope image, and recording time-sequenced transition information of the estimated shape categories based on the check start time point.

[0135] (Supplementary note A8)

[0136] The endoscope insertion assisting apparatus according to any one of clauses 1 to 7, wherein the estimation component normalizes the shape data based on a posture detected from the subject, and estimates any one of a plurality of shape categories for the normalized shape data.

[0137] (Supplementary note A9)

[0138] The endoscope insertion assisting apparatus according to any one of clauses 1 to 8, further comprising a registration component for receiving input of additional information related to the estimated shape categories, and registering the additional information in association with the shape categories.

[0139] (Supplementary note A10)

[0140] The endoscope insertion assisting apparatus according to any one of clauses 1 to 9, further comprising a storage means for storing history information in which the estimated shape category is associated with time information.

[0141] (Supplementary note A11)

[0142] The endoscope insertion assisting apparatus according to clause 10, further comprising a comparison means for outputting a comparison result of two or more of the history information.

[0143] (Supplementary note A12)

[0144] The endoscope insertion assisting apparatus according to clause 11, wherein the comparison means

[0145] evaluates a comparison destination with respect to a comparison source in the history information, and

[0146] outputs the evaluation result as the comparison result.

[0147] (Supplementary note A13)

[0148] The endoscope insertion assisting apparatus according to clause 11 or 12, wherein

[0149] the comparison means outputs a warning when the comparison result shows that a duration of a specific shape category is above a predetermined time.

[0150] (Supplementary note A14)

[0151] The endoscope insertion assisting apparatus according to any one of clauses 10 to 13, further comprising a search means for searching the history information based on conversion information indicating a time-sequenced conversion of the estimated shape category, and outputting a search result.

[0152] (Supplementary note A15)

[0153] The endoscope insertion assisting apparatus according to clause 14, wherein

[0154] the history information is further associated with medical information of the examinee, and

[0155] the search means searches the history information based on the medical information, and outputs the search result.

[0156] (Supplementary note A16)

[0157] The endoscope insertion assisting apparatus according to any one of clauses 1 to 15, wherein

[0158] The estimating component identifies a position in the body from the estimated shape category, and

[0159] The output component also outputs the identified position.

[0160] (Supplementary note A17)

[0161] The endoscope insertion assisting apparatus according to any one of clauses 1 to 16, wherein

[0162] The estimating component estimates, using a trained model, any one of a plurality of shape categories from the acquired shape data, the trained model being trained from training data that is labeled with a shape category for each of the plurality of shape data.

[0163] (Supplementary note B1)

[0164] An endoscope insertion assisting method by a computer:

[0165] acquiring shape data to identify an insertion shape of an endoscope inserted into a lumen;

[0166] estimating, from the shape data, any one of a plurality of shape categories in the insertion shape; and

[0167] outputting time-sequenced display information of the estimated shape category.

[0168] (Supplementary note C1)

[0169] A non-transitory computer-readable medium storing an endoscope insertion assisting program that causes a computer to execute:

[0170] an acquiring process that acquires shape data to identify an insertion shape of an endoscope inserted into a lumen;

[0171] an estimating process that estimates, from the shape data, any one of a plurality of shape categories in the insertion shape; and

[0172] an outputting process that outputs time-sequenced display information of the estimated shape category.

[0173] Although the present application has been described so far with reference to the example embodiments (and examples), the present application is not limited to the above-described example embodiments (and examples). Various changes can be made to the configurations and details of the present application in a manner that can be understood by those skilled in the art without departing from the scope of the present application.

[0174] This application claims priority to Japanese Patent Application No. 2020-070370, filed April 9, 2020, the disclosure of which is incorporated by reference herein in its entirety.

[0175] List of reference signs

[0176] 100 endoscope insertion assisting device

[0177] 110 acquisition unit

[0178] 120 estimation unit

[0179] 130 output unit

[0180] 2000 endoscope insertion assisting system

[0181] 10 endoscope device

[0182] 11 electronic endoscope

[0183] 11a insertion portion

[0184] 20 endoscope insertion shape observation device

[0185] 21 induction coil unit

[0186] 22 shape processing device

[0187] 30 endoscope insertion assisting device

[0188] 30a endoscope insertion assisting device

[0189] 30b endoscope insertion assisting device

[0190] 31 storage device

[0191] 311 shape category estimation model

[0192] 312 history information

[0193] 312a history information

[0194] 3121 shape category

[0195] 3122 time information

[0196] 3123 additional information

[0197] 3124 medical information

[0198] 313 endoscope insertion assisting program

[0199] 313a endoscope insertion assisting program

[0200] 313b endoscope insertion assisting program

[0201] 314 shape data

[0202] 315 endoscope image

[0203] 316 display information

[0204] 316a comparison result

[0205] 32 memory

[0206] 33 IF unit

[0207] 34 control unit

[0208] 341 acquisition unit

[0209] 342 estimation unit

[0210] 3421 in-vivo insertion detection unit

[0211] 3422 operation estimation unit

[0212] 3423 normalization unit

[0213] 3424 shape category estimation unit

[0214] 343 recording unit

[0215] 344 output unit

[0216] 345 registration unit

[0217] 346 comparison unit

[0218] 347 search unit

[0219] 41 display device

[0220] 42 speaker

[0221] 43 input device

[0222] U1 doctor

[0223] U2 examinee

[0224] C1 shape category

[0225] C11 shape subcategory

[0226] C12 shape subcategory

[0227] C13 shape subcategory

[0228] C2 shape category

[0229] C21 shape subcategory

[0230] C22 shape subcategory

[0231] C3 shape category

[0232] C31 shape subcategory

[0233] C32 shape subcategory

[0234] C33 shape subcategory

[0235] C4 shape category

[0236] C41 shape subcategory

Claims

1. An endoscope insertion assisting apparatus comprising: an acquisition section for acquiring shape data to identify an insertion shape of an endoscope inserted into a lumen; an estimation section for estimating a plurality of shape categories from the shape data, each shape category being a pattern grouping a plurality of shapes having similar insertion shape characteristics; a recording section for recording transition information of the shape categories as history information in time order by associating time information with a shape category from among the estimated plurality of shape categories; and an output section for outputting display information indicating the transition information in time order between the estimated plurality of shape categories from among the history information.

2. The endoscope insertion assisting apparatus according to claim 1, wherein the output section further outputs speech corresponding to the estimated plurality of shape categories.

3. The endoscope insertion assisting apparatus according to claim 1, wherein the acquisition section further acquires an endoscope image captured by the endoscope, the endoscope insertion assisting apparatus further comprises a normalization unit for detecting that the endoscope has been inserted into a subject and identifying a start point of the shape data; and the estimation section further estimates any one of a plurality of shape categories using the endoscope image.

4. The endoscope insertion assisting apparatus according to claim 3, wherein the estimation section estimates an operation content of the endoscope in the lumen based on a change in the endoscope image and the shape data, and estimates a first shape category when a position of a distal end portion of the endoscope changes and a state of a body cavity reflected in the endoscope image does not change, and a second shape category when the shape data changes and the state of the body cavity reflected in the endoscope image changes.

5. The endoscope insertion assisting apparatus according to claim 1, further comprising: a storage section for storing history information in which the estimated plurality of shape categories are associated with time information, and a search section for searching the history information based on transition information indicating a transition in time order of the estimated plurality of shape categories, and outputting a search result.

6. The endoscope insertion assisting apparatus according to claim 1, further comprising: a comparison section for outputting a comparison result of two or more pieces of history information in the history information, and an output section for simultaneously outputting display information in the estimated plurality of shape categories in time order.

7. The endoscope insertion assisting apparatus according to claim 6, wherein the comparison section compares the history information of a skilled physician with the history information of an inexperienced physician.

8. The endoscope insertion assisting apparatus according to claim 6, wherein the comparison section outputs a warning when a duration of a specific shape category in the comparison result is equal to or greater than a predetermined time.

9. An endoscope insertion assisting method by a computer: acquiring shape data to identify an insertion shape of an endoscope inserted into a lumen; estimating a plurality of shape categories from the shape data, each shape category being a pattern that groups a plurality of shapes having similar insertion shape characteristics; recording transition information of the shape categories in time order as history information by associating time information with shape categories from the estimated plurality of the shape categories; and outputting display information indicating the transition information in time order between the estimated plurality of shape categories from the history information.

10. A non-transitory computer readable medium storing an endoscope insertion assistance program that causes a computer to execute: an acquiring process that acquires shape data to identify an insertion shape of an endoscope inserted into a lumen; an estimating process that estimates a plurality of shape categories from the shape data, each shape category being a pattern that groups a plurality of shapes having similar insertion shape characteristics; a recording process that records transition information of the shape categories in time order as history information by associating time information with shape categories from the estimated plurality of the shape categories; and an outputting process that outputs display information indicating the transition information in time order between the estimated plurality of shape categories from the history information.

Images